Learning rates of least-square regularized regression

Foundations of Computational Mathematics

Volumn 6, Issue 2, 2006, Pages 171-192

  Wu, Qiang ^a   Ying, Yiming ^a   Zhou, Ding Xuan ^a  

^a CITY UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Covering number; Learning theory; Regularization error; Regularization scheme; Reproducing kernel Hilbert space

Indexed keywords

COVERING NUMBERS; LEARNING THEORY; REGULARIZATION ERRORS; REGULARIZATION SCHEMES; REPRODUCING KERNEL HILBERT SPACES;

COMPUTATION THEORY; PROBABILITY DISTRIBUTIONS; REGRESSION ANALYSIS;

LEARNING ALGORITHMS;

EID: 33744772341     PISSN: 16153375     EISSN: 16153383     Source Type: Journal    
DOI: 10.1007/s10208-004-0155-9     Document Type: Article

References (32)

1. S. Andonova, Generalization bounds and complexities based on sparsity and clustering for convex combinations of functions from random classes, J. Mach. Learn. Res. 6 (2005), 307-340.
2. M. Anthony and P. L. Bartlett, Neural Network Learning; Theoretical Foundations, Cambridge University Press, Cambridge, 1999.
3. N. Aronszajn, Theory of reproducing kernels, Trans. Amer. Math. Soc. 68 (1950), 337-404.
4. A. R. Barron, Complexity regularization with applications to artificial neural networks, in Non-parametric Functional Estimation (G. Roussa, ed.), Kluwer Academic, Dortrecht, 1990, pp. 561-576.
5. P. L. Bartlett, The sample complexity of pattern classification with neural networks: The size of the weights is more important than the size of the network, IEEE Trans. Inform. Theory 44 (1998), 525-536.
6. O. Bousquet and A. Elisseeff, Stability and generalization, J. Mach. Learn. Res. 2 (2002), 499-526.
7. D. R. Chen, Q. Wu, Y. Ying, and D.-X. Zhou, Support vector machine soft margin classifiers: Error analysis, J. Mach. Learn. Res. 5 (2004), 1143-1175.
8. F. Cucker and S. Smale, On the mathematical foundations of learning, Bull. Amer. Math. Soc. 39 (2001), 1-49.
9. F. Cucker and S. Smale, Best choices for regularization parameters in learning theory: On the bias-variance problem, Found. Comput. Math. 2 (2002), 413-428.
10. F. Cucker and D.-X. Zhou, Learning Theory: An Approximation Theory Viewpoint, monograph manuscript in preparation for Cambridge University Press, Cambridge.
11. E. De Vito, A. Caponnetto, and L. Rosasco, Model selection for regularized least-squares algorithm in learning theory, Found. Comput. Math. 5 (2005), 59-85.
12. T. Evgeniou, M. Pontil, and T. Poggio, Regularization networks and support vector machines, Adv. Comput. Math. 13 (2000), 1-50.
13. V. Koltchinskii and D. Panchenko, Rademacher processes and bounding the risk of function learning, in High Dimensional Probability II (E. Gine, D. M. Mason, and J. A. Wellner, eds.), Birkhäuser, Boston, 2000, pp. 443-459.
14. W. S. Lee, P. Bartlett, and R. Williamson, The importance of convexity in learning with least square loss, IEEE Trans. Inform. Theory 44 (1998), 1974-1980.
15. G. Lugosi and N. Vayatis, On the Bayes-risk consistency of regularized boosting methods, Ann. Statist. 32 (2004), 30-55.
16. J. Shawe-Taylor, P. L. Bartlett, R. C. Williamson, and M. Anthony, Structural risk minimization over data-dependent hierarchies, IEEE Trans. Inform. Theory 44 (1998), 1926-1940.
17. S. Smale and D.-X. Zhou, Estimating the approximation error in learning theory, Anal. Appl. 1 (2003), 17-41.
18. S. Smale and D.-X. Zhou, Shannon sampling and function reconstruction from point values, Bull. Amer. Math. Soc. 41 (2004), 279-305.
19. S. Smale and D.-X. Zhou, Shannon sampling II. Connections to learning theory, Appl. Comput. Harmonic Anal., in press.
20. I. Steinwart and C. Scovel, Fast rates for support vector machines, in Proceedings of the Conference on Learning Theory (COLT-2005), to appear.
21. A. W. van der Vaart and J. A. Wellner, Weak Convergence and Empirical Processes, Springer-Verlag, New York, 1996.
22. V. Vapnik, Statistical Learning Theory, Wiley, New York, 1998.
23. G. Wahba, Spline Models for Observational Data, SIAM, Philadelphia, PA, 1990.
24. Q. Wu, Y. Ying, and D.-X. Zhou, Multikernel regularized classifiers, submitted, 2004.
25. Q. Wu and D.-X. Zhou, Analysis of support vector machine classification, submitted, 2004.
26. Q. Wu and D.-X. Zhou, SVM soft margin classifiers: Linear programming versus quadratic programming, Neural Comput. 17 (2005), 1160-1187.
27. Y. Ying and D.-X. Zhou, Learnability of Gaussians with flexible variances, submitted, 2004.
28. T. Zhang, Leave-one-out bounds for kernel methods, Neural Comput. 15 (2003), 1397-1437.
29. D.-X. Zhou, The covering number in learning theory, J. Complexity 18 (2002), 739-767.
30. D.-X. Zhou, Capacity of reproducing kernel spaces in learning theory, IEEE Trans. Inform. Theory 49 (2003), 1743-1752.
31. D.-X. Zhou, Density problem and approximation error in learning theory, preprint 2003.
32. D.-X. Zhou and K. Jetter, Approximation with polynomial kernels and SVM classifiers, Adv. Comput. Math., in press.