A convex approach to validation-based learning of the regularization constant

IEEE Transactions on Neural Networks

Volumn 18, Issue 3, 2007, Pages 917-920

  Pelckmans, Kristiaan ^a   Suykens, J A K ^a   De Moor, B ^a  

^a UNIVERSITY OF LEUVEN   (Belgium)

Author keywords

Convex optimization; Model selection; Regularization

Indexed keywords

ALGORITHMS; LEAST SQUARES APPROXIMATIONS; NEURAL NETWORKS; OPTIMIZATION; REGRESSION ANALYSIS; SUPPORT VECTOR MACHINES;

CONVEX OPTIMIZATION; MODEL SELECTION; REGULARIZATION;

LEARNING SYSTEMS;

ALGORITHM; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; DECISION SUPPORT SYSTEM; EVALUATION; INFORMATION RETRIEVAL; LETTER; METHODOLOGY; THEORETICAL MODEL; VALIDATION STUDY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; DECISION SUPPORT TECHNIQUES; INFORMATION STORAGE AND RETRIEVAL; MODELS, THEORETICAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED;

EID: 34248670883     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2007.891187     Document Type: Article

References (22)

1. G. C. Cawley and N. L. C. Talbot, "Fast exact leave-one-out cross-validation of sparse least-squares support vector machines," Neural Netw., vol. 17, no. 10, pp. 1467-1475, 2004.
2. O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee, "Choosing multiple parameters for support vector machines," Mach. Learn., vol. 46, no. 1-3, pp. 131-159, 2002.
3. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag, 1996.
4. R. M. Dudley, "Universal donsker classes and metric entropy," Ann. Probability, vol. 15, no. 4, pp. 1306-1326, 1987.
5. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, "Least angle regression," Ann. Statist., vol. 32, no. 2, pp. 407-499, 2004.
6. F. Girosi, M. Jones, and T. Poggio, "Regularization theory and neural networks architectures," Neural Comput., vol. 7, pp. 219-269, 1995.
7. G. H. Golub, M. Heath, and G. Wahba, "Generalized cross-validation as a method for choosing a good ridge parameter," Technometrics, vol. 21, pp. 215-223, 1979.
8. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning. Heidelberg, Germany: Springer-Verlag, 2001.
9. A. E. Hoerl and R. W. Kennard, "Ridge regression: Biased estimation for nonorthogonal problems," Technometrics, vol. 12, no. 1, pp. 55-82, 1970.
10. G. R. G. Lanckriet, N. Cristianini, P. Bartlett, M. I. Jordan, and L. El Ghaoui, "Learning the kernel matrix with semidefinite programming," J. Mach. Learn. Res., vol. 5, pp. 27-72, 2004.
11. K. Pelckmans, "Primal-dual kernel machines," Ph.D. dissertation, Faculty Eng., K. U. Leuven, Leuven, Belgium, 2005.
12. K. Pelckmans, J. A. K. Suykens, and B. De Moor, "Additive regularization trade-off: Fusion of training and validation levels in kernel methods," Mach. Learn., vol. 62, no. 3, pp. 217-252, Mar. 2006.
13. K. Pelckmans, J. A. K. Suykens, and B. De Moor, ESAT-SISTA, K. U. Leuven, "A convex approach to learning the ridge based on CV," Leuven, Belgium, Tech. Rep. 05-216, 2005.
14. C. Saunders, A. Gammerman, and V. Vovk, "Ridge regression learning algorithm in dual variables," in Proc. 15th Int. Conf. Mach. Learn. (ICML), 1998, pp. 515-521.
15. J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis. Cambridge, U.K.: Cambridge Univ. Press, 2004.
16. J. A. K. Suykens, J. De Brabanter, L. Lukas, and J. Vandewalle, "Weighted least squares support vector machines: Robustness and sparse approximation," Neurocomput., vol. 48, no. 1-4, pp. 85-105, 2002.
17. J. A. K. Suykens, T. Van Gestel, J. De Brabanter, B. De Moor, and J. Vandewalle, Least Squares Support Vector Machines. Singapore: World Scientific, 2002.
18. R. J. Tibshirani, "Regression shrinkage and selection via the LASSO," J. Roy. Statist. Soc., vol. 58, pp. 267-288, 1996.
19. A. N. Tikhonov and V. Y. Arsenin, Solution of Ill-Posed Problems. Washington, DC: Winston, 1977.
20. V. N. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
21. U. von Luxburg, O. Bousquet, and M. Belkin, "Limits of spectral clustering," in Advances in Neural Information Processing Systems (NIPS), L. K. Saul, Y. Weiss, and L. Bottou, Eds. Cambridge, MA: MIT Press, 2005, vol. 17.
22. G. Wahba, Spline Models for Observational Data. Philadelphia, PA: SIAM, 1990.