Regularization in kernel learning

Annals of Statistics

Volumn 38, Issue 1, 2010, Pages 526-565

  Mendelson, Shahar ^a,b,c   Neeman, Joseph ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

^c TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Least squares; Model selection; Regression; Regulation; Reproducing kernel Hilbert space

Indexed keywords

EID: 73949116964     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/09-AOS728     Document Type: Article

References (37)

1. BARTLETT, P. L. (2008). Fast rates for estimation error and oracle inequalities for model selection. Econometric Theory 24 545-552. MR2490397
2. BARTLETT, P. L., BOUSQUET, O. andMENDELSON, S. (2005). Local rademacher complexities. Ann. Statist. 33 1497-1537. MR2166554
3. BARTLETT, P. L. and MENDELSON, S. (2006). Empirical minimization. Probab. Theory Related Fields 135 311-344. MR2240689
4. BARTLETT, P. L.,MENDELSON, S. andNEEMAN, J. (2009).ℓ1-regularized linear regression: Persistence and oracle inequalities. Submitted.
5. BLANCHARD, G., BOUSQUET, O. and MASSART, P. (2008). Statistical performance of support vector machines. Ann. Statist. 36 489-531. MR2396805
6. BIRMAN, M. Š. and SOLOMYAK, M. Z. (1977). Estimates of singular numbers of integral operators. Uspehi Mat. Nauk 32 15-89. MR0438186
7. CAPONNETTO, A. and DE VITO, E. (2007). Optimal rates for regularized least-squares algorithm. Found. Comput. Math. 7 331-368. MR2335249
8. CUCKER, F. and SMALE, S. (2002). On the mathematical foundations of learning. Bull. Amer. Math. Soc. (N.S.) 39 1-49. MR1864085
9. CUCKER, F. and SMALE, S. (2002). Best choices for regularization parameters in learning theory: On the Bias-variance problem. Found. Comput. Math. 2 413-428. MR1930945
10. CUCKER, F. and ZHOU, D. X. (2007). Learning Theory: An Approximation Theory Viewpoint. Cambridge Univ. Press, Cambridge. MR2354721
11. DUDLEY, R. M. (1999). Uniform Central Limit Theorems. Cambridge Studies in Advanced Mathematics 63. Cambridge Univ. Press, Cambridge. MR1720712
12. FERNIQUE, X. (1975). Régularité des trajectoires des fonctiones aléatoires Gaussiennes. In Ecole d'Eté de Probabilités de St-Flour 1974. Lecture Notes in Mathematics 480 1-96. Springer, Berlin. MR0413238
13. GINÉ, E. and ZINN, J. (1984). Some limit theorems for empirical processes. Ann. Probab. 12 929-989. MR0757767
14. GUEDON, O., MENDELSON, S., PAJOR, A. and TOMCZAK-JAEGERMANN, N. (2007). Subspaces and orthogonal decompositions generated by bounded orthogonal systems. Positivity 11 269-283. MR2321621
15. p moments of random vectors via majorizing measures. Adv. Math. 208 798-823. MR2304336
16. KOLTCHINSKII, V. (2006). Local Rademacher complexities and oracle inequalities in risk minimization. Ann. Statist. 34 2593-2656. MR2329442
17. KONIG, H. (1986). Eigenvalue Distribution of Compact Operators. Birkhäuser, Basel. MR0889455
18. LEDOUX, M. (2001). The Concentration of Measure Phenomenon. Amer. Math. Soc., Providence, RI. MR1849347
19. LEE, W. S., BARLETT, P. L. andWILLIAMSON, R. C. (1996). The importance of convexity in learning with squared loss. IEEE Trans. Inform. Theory 44 1974-1980. MR1664079
20. MASSART, P. (2000). About the constants in Talagrand's concentration inequality for empirical processes. Ann. Probab. 28 863-884. MR1782276
21. MASSART, P. (2007). Concentration Inequalities and Model Selection. Lecture Notes in Mathematics 1896. Springer, Berlin. MR2319879
22. MENDELSON, S. (2003). Estimating the performance of kernel classes. J. Mach. Learn. Res. 4 759-771. MR2075996
23. MENDELSON, S., PAJOR, A. and TOMCZAK-JAEGERMANN, N. (2007). Reconstruction and subgaussian operators in asymptotic geometric analysis. Geom. Funct. Anal. 17 1248-1282. MR2373017
24. MENDELSON, S. (2009). Obtaining fast error rates in nonconvex situations. J. Complexity 24 380-397. MR2426759
25. MENDELSON, S. (2008). On weakly bounded empirical processes. Math. Ann. 340 293-314. MR2368981
26. PAJOR, A. and TOMCZAK-JAEGERMANN, N. (1985). Remarques sur les nombres d'entropie d'un opérateur et de son transposé. C. R. Acad. Sci. Paris Ser. I Math. 301 743-746. MR0817602
27. PISIER, G. (1989). The Volume of Convex Bodies and Banach Space Geometry. Cambridge Univ. Press, Cambridge. MR1036275
28. RUDELSON, M. (1999). Random vectors in the isotropic position. J. Funct. Anal. 164 60-72. MR1694526
29. STEINWART, I. and SCOVEL, C. (2007). Fast rates for support vector machines using Gaussian kernels. Ann. Statist. 35 575-607. MR2336860
30. SMALE, S. and ZHOU, D. X. (2003). Estimating the approximation error in learning theory. Anal. Appl. 1 17-41. MR1959283
31. SMALE, S. and ZHOU, D. X. (2007). Learning theory estimates via integral operators and their approximations. Constr. Approx. 26 153-172. MR2327597
32. TALAGRAND, M. (1987). Regularity of Gaussian processes. Acta Math. 159 99-149. MR0906527
33. TALAGRAND, M. (1994). Sharper bounds for Gaussian and empirical processes. Ann. Probab. 22 28-76. MR1258865
34. TALAGRAND, M. (2005). The Generic Chaining. Springer, Berlin. MR2133757
35. WILLIAMSON, R. C., SMOLA, A. J. and SCHÖLKOPF, B. (2001). Generalization performance of regularization networks and support vector machines via entropy numbers of compact operators. IEEE Trans. Inform. Theory 47 2516-2532. MR1873936
36. WU, Q., YING, Y. and ZHOU, D. (2006). Learning rates of least-square regularized regression. Found. Comput. Math. 6 171-192. MR2228738
37. ZHOU, D. (2002). The covering number in learning theory. J. Complexity 18 739-767. MR1928805