Optimal rates for the regularized least-squares algorithm

Foundations of Computational Mathematics

Volumn 7, Issue 3, 2007, Pages 331-368

  Caponnetto, A ^a,b   De Vito, E ^c,d  

^a THE UNIVERSITY OF CHICAGO   (United States)

^b UNIVERSITY OF GENOA   (Italy)

^c UNIVERSITY OF MODENA AND REGGIO EMILIA   (Italy)

^d INFN SEZIONE DI GENOVA   (Italy)

Author keywords

Learning theory; Least squares; Model selection; Optimal rates

Indexed keywords

LEARNING THEORY; LEAST SQUARE; LEAST-SQUARES ALGORITHMS; LEAST-SQUARES ESTIMATOR; MODEL SELECTION; OPTIMAL RATE; REGULARIZATION PARAMETERS; REPRODUCING KERNEL HILBERT SPACES;

ALGORITHMS;

OPTIMIZATION;

EID: 34548537866     PISSN: 16153375     EISSN: 16153383     Source Type: Journal    
DOI: 10.1007/s10208-006-0196-8     Document Type: Article

References (37)

1. Y. Altun, I. Tsochantaridis, and T. Hofmann, Hidden Markov support vector machines, in 20th International Conference on Machine Learning ICML-2004, Washington, DC, 2003.
2. N. Aronszajn, Theory of reproducing kernels, Trans. Amer. Math. Soc. 68 (1950), 337-404.
3. J. Burbea and P. Masani, Banach and Hilbert Spaces of Vector-Valued Functions, Vol. 90, Pitman, Boston, MA, 1984.
4. C. Carmeli, A. De Vito, and E. Toigo, Reproducing kernel Hilbert spaces and the Mercer theorem, Anal. Appl. (2006), to appear.
5. F. Cucker and S. Smale, Best choices for regularization parameters in learning theory: On the bias-variance problem, Found. Comput. Math. 2 (2002), 413-428.
6. F. Cucker and S. Smale, On the mathematical foundations of learning, Bull. Amer. Math. Soc. (N.S.) 39(1) (2002), 1-49.
7. E. De Vito and A. Caponnetto, Risk bounds for the regularized least-squares algorithm with operator-valued kernels, Technical report, Massachusetts Institute of Technology, Cambridge, MA, May 2005. CBCL Paper #249/AI Memo #2005-015.
8. E. De Vito, A. Caponnetto, and L. Rosasco, Model selection for the regularized least-squares algorithm in learning theory, Found. Comput. Math. 5(1) (2005), 59-85.
9. E. De Vito, L Rosasco, A. Caponnetto, U. De Giovannini, and F. Odone, Learning from examples as an inverse problem, J. Mach. Learn. Res. 6 (2005), 883-904.
10. R. DeVore, G. Kerkyacharian, D. Picard, and V. Temlyakov, Mathematical methods for supervised learning, IMI Preprints 22 (2004), 1-51.
11. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition, Applications of Mathematics, Vol. 31, Springer-Verlag, New York, 1996.
12. R. M. Dudley, Real Analysis and Probability, Cambridge Studies in Advanced Mathematics, Vol. 74, Cambridge University Press, Cambridge, UK, 2002. Revised reprint of the 1989 original.
13. H. W. Engl, M. Hanke, and A. Neubauer, Regularization of Inverse Problems, Mathematics and its Applications, Vol. 375, Kluwer Academic, Dordrecht, 1996.
14. T. Evgeniou, C. A. Micchelli, and M. Pontil, Learning multiple tasks with kernel methods, J. Mach. Learn. Res. 6 (2005), 615-637.
15. T. Evgeniou, M. Pontil, and T. Poggio, Regularization networks and support vector machines, Adv. Comput. Math. 13 (2000), 1-50.
16. C. W. Groetsch, The Theory of Tikhonov Regularization for Fredholm Equations of the First Kind, Research Notes in Mathematics, Vol. 105, Pitman, Boston, MA, 1984.
17. L. Györfi, M. Kohler, A. Krzyzak, and H. Walk, A Distribution-Free Theory of Non-parametric Regression, Springer Series in Statistics, Springer-Verlag, New York, 2002.
18. M. Kohler and A. Krzyżak, Nonparametric regression estimation using penalized least squares, IEEE Trans. Inform. Theory 47(7) (2001), 3054-3058.
19. S. Mendelson, On the performance of kernel classes, J. Mach. Learn. Res. 4 (2003), 759-771.
20. C. A. Micchelli and M. Pontil, On learning vector-valued functions, Neural Comput. 17 (2005), 177-204.
21. I. F. Pinelis and A. I. Sakhanenko, Remarks on inequalities for probabilities of large deviations, Theory Probab. Appl. 30(1) (1985), 143-148.
22. T. Poggio and F. Girosi, A theory of networks for approximation and learning, in C. Lau, ed., Foundation of Neural Networks, pp. 91-106, IEEE Press, Piscataway, NJ, 1992.
23. T. Poggio and S. Smale, The mathematics of learning: Dealing with data, Notices Amer. Math. Soc. 50(5) (2003), 537-544.
24. B. Schölkopf and A. J. Smola, Learning with Kernels, MIT Press, Cambridge, MA, 2002.
25. L. Schwartz, Sous-espaces Hilbertiens d'espaces vectoriels topologiques et noyaux associés (noyaux reproduisants), J. Analyse Math. 13 (1964), 115-256.
26. S. Smale and D. Zhou, Shannon sampling II: Connections to learning theory, to appear in Appl. Comput. Harmon. Anal. 19 (2005), 285-302.
27. S. Smale and D. Zhou, Learning theory estimates via integral operators and their approximations, Preprint, Toyota Technological Institute, Chicago, 2005.
28. V. N. Temlyakov, Nonlinear methods of approximation, Found. Comput. Math. 3 (2003), 33-107.
29. V. N. Temlyakov, Approximation in learning theory, IMI Preprints 5 (2005), 1-42.
30. S. A. van de Geer, Applications of the Empirical Process Theory, Cambridge Series in Statistical and Probabilistic Mathematics, Cambridge University Press, Cambridge, UK, 2000.
31. A. W. van der Vaart and J. A. Wellner, Weak Convergence and Empirical Processes, Springer Series in Statistics, Springer-Verlag, New York, 1996. With Applications to Statistics.
32. V. N. Vapnik, Statistical Learning Theory, Adaptive and Learning Systems for Signal Processing, Communications, and Control, Wiley, New York, 1998. A Wiley-Interscience Publication.
33. G. Wahba, Spline Models for Observational Data, CBMS-NSF Regional Conference Series in Applied Mathematics, Vol. 59, SIAM, Philadelphia, PA, 1990.
34. J. Weston, O. Chapelle, A. Elisseeff, B. Schoelkopf, and V. Vapnik, Kernel dependency estimation, in S. Thrun, S. Becker, and K. Obermayer, eds., Advances in Neural Information Processing Systems, Vol. 15, pp. 873-880, MIT Press, Cambridge, MA, 2003.
35. V. Yurinsky, Sums and Gaussian Vectors, Lecture Notes in Mathematics, Vol. 1617, Springer-Verlag, Berlin, 1995.
36. T. Zhang, Effective dimension and generalization, of kernel learning, NIPS 2002, pp. 454-461.
37. T. Zhang, Leave-one-out bounds for kernel methods, Neural Comput. 13 (2003), 1397-1437.