Model selection for regularized least-squares algorithm in learning theory

Foundations of Computational Mathematics

Volumn 5, Issue 1, 2005, Pages 59-85

  De Vito, E ^a,b   Caponnetto, A ^b,c   Rosasco, L ^b,c  

^a UNIVERSITY OF MODENA AND REGGIO EMILIA   (Italy)

^b INFN SEZIONE DI GENOVA   (Italy)

^c UNIVERSITY OF GENOA   (Italy)

Author keywords

Model selection; Optimal choice of parameters; Regularized least squares algorithm

Indexed keywords

CONTINUOUS PARAMETERS; GENERALIZATION ERROR; LEAST-SQUARES ALGORITHMS; MODEL SELECTION; MODEL SELECTION PROCEDURES; OPTIMAL CHOICE; REGRESSION FUNCTION; SMOOTHNESS CONDITIONS;

ALGORITHMS;

PARAMETER ESTIMATION;

EID: 24944432318     PISSN: 16153375     EISSN: 16153383     Source Type: Journal    
DOI: 10.1007/s10208-004-0134-1     Document Type: Article

References (24)

1. N. Alon, S. Ben-David, N. Cesa-Bianchi, and D. Haussler, Scale-sensitive dimensions, uniform convergence, and learnability, J. ACM 44 (1997), 615-631.
2. N. Aronszajn, Theory of reproducing kernels, Trans. Amer. Math. Soc. 68 (1950), 337-404.
3. P. Bartlett, S. Boucheron, and G. Lugosi, Model selection and error estimation, Machine Learn-ing 48 (2002), 85-113.
4. O. Bousquet and A. Elisseeff, Stability and generalization, J. Mach. Learn. Res. 2 (2002), 499-526.
5. N. Cristianini and J. Shawe Taylor, An Introduction to Support Vector Machines, Cambridge University Press, Cambridge, 2000.
6. F. Cucker and S. Smale, Best choices for regularization parameters in learning theory: On the bias-variance problem, Found. Comput. Math. 2 (2002), 413-428.
7. F. Cucker and S. Smale, On the mathematical foundations of learning, Bull. Amer. Math. Soc. (N.S.) 39 (2002), 1-49.
8. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition, Springer-Verlag, New York, 1996.
9. R. Dudley, Real Analysis and Probability, Cambridge University Press, Cambridge, 2002.
10. T. Evgeniou, M. Pontil, and T. Poggio, Regularization networks and support vector machines, Adv. Comput. Math. 13 (2000), 1-50.
11. F. Girosi, M. Jones, and T. Poggio, Regularization theory and neural networks architectures, Neural Comput. 7 (1995), 219-269.
12. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning, Springer-Verlag, New York, 2001.
13. C. McDiarmid, On the method of bounded differences, in Surveys in Combinatorics, 1989 (Norwich, 1989), Cambridge University Press, Cambridge, 1989, pp. 148-188.
14. S. Mendelson, A few notes on statistical learning theory, in Advanced Lectures in Machine Learning (S. Mendelson and A. Smola, eds.), Springer-Verlag, 2003, pp. 1-40.
15. P. Niyogi and F. Girosi, Generalization bounds for function approximation from scattered noisy data, Adv. Comput. Math. 10 (1999), 51-80.
16. T. Poggio and F. Girosi, Networks for approximation and learning, in Proceedings of the IEEE, Vol. 78, 1990, pp. 1481-1497.
17. T. Poggio, R. Rifkin, S. Mukherjee, and P. Niyogi, General conditions for predictivity in learning theory, Nature 428 (2004), 419-422.
18. L. Rosasco, E. De Vito, A. Caponnetto, M. Piana, and A. Verri, Are loss functions all the same?, Neural Comput. 16 (2004), 1063-1076.
19. S. Smale and D.-X. Zhou, Estimating the approximation error in learning theory, Anal. Appl. (Singap.) 1 (2003), 17-41.
20. I. Steinwart, Consistency of support vector machines and other regularized kernel machines, Technical Report 02-03, University of Jena, Department of Mathematics and Computer Science, 2002.
21. V. Vapnik, Statistical learning theory, Wiley, New York, 1998.
22. G. Wahba, Spline Models for Observational Data, SIAM, Philadelphia, PA, 1990.
23. T. Zhang, Leave-one-out bounds for kernel methods, Neural Comput. 15 (2003), 1397-1437.
24. D.-X. Zhou, The covering number in learning theory, J. Complexity 18 (2002), 739-767.