Sparsity in multiple kernel learning

Annals of Statistics

Volumn 38, Issue 6, 2010, Pages 3660-3695

  Koltchinskii, Vladimir ^a   Yuan, Ming ^b  

^a GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

^b Georgia Institute of Technology   (United States)

Author keywords

High dimensionality; Multiple kernel learning; Oracle inequality; Reproducing kernel Hilbert spaces; Restricted isometry; Sparsity

Indexed keywords

EID: 78650166948     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/10-AOS825     Document Type: Article

References (25)

1. ARONSZAJN, N. (1950). Theory of reproducing kernels. Trans. Amer. Math. Soc. 68 337-404. MR0051437
2. BACH, F. (2008). Consistency of the group lasso and multiple kernel learning. J.Mach. Learn. Res. 9 1179-1225. MR2417268
3. BICKEL, P., RITOV, Y. and TSYBAKOV, A. (2009). Simultaneous analysis of lasso and Dantzig selector. Ann. Statist. 37 1705-1732. MR2533469
4. BOUSQUET, O. and HERRMANN, D. (2003). On the complexity of learning the kernel matrix. In Advances in Neural Information Processing Systems 15 415-422. MIT Press, Cambridge.
5. BLANCHARD, G., BOUSQUET, O. and MASSART, P. (2008). Statistical performance of support vector machines. Ann. Statist. 36 489-531. MR2396805
6. BOUSQUET, O. (2002). A Bennett concentration inequality and its applications to suprema of empirical processes. C. R. Math. Acad. Sci. Paris 334 495-500. MR1890640
7. CANDES, E. and TAO, T. (2007). The Dantzig selector: Statistical estimation when p is much larger than n. Ann. Statist. 35 2313-2351.
8. CRAMMER, K., KESHET, J. and SINGER, Y. (2003). Kernel design using boosting. In Advances in Neural Information Processing Systems 15 553-560. MIT Press, Cambridge.
9. KOLTCHINSKII, V. (2008). Oracle inequalities in empirical risk minimization and sparse recovery problems. Ecole d'Eté de Probabilités de Saint-Flour, Lecture Notes. Preprint.
10. KOLTCHINSKII, V. (2009a). Sparsity in penalized empirical risk minimization. Ann. Inst. H. Poincaré Probab. Statist. 45 7-57. MR2500227
11. KOLTCHINSKII, V. (2009b). The Dantzig selector and sparsity oracle inequalities. Bernoulli 15 799-828. MR2555200
12. KOLTCHINSKII, V. (2009c). Sparse recovery in convex hulls via entropy penalization. Ann. Statist. 37 1332-1359. MR2509076
13. KOLTCHINSKII, V. and YUAN, M. (2008). Sparse recovery in large ensembles of kernel machines. In Proceedings of 19th Annual Conference on Learning Theory 229-238. Omnipress, Madison, WI.
14. LANCKRIET, G., CRISTIANINI, N., BARTLETT, P., GHAOUI, L. and JORDAN, M. (2004). Learning the kernel matrix with semidefinite programming. J.Mach. Learn. Res. 5 27-72. MR2247973
15. LEDOUX, M. and TALAGRAND, M. (1991). Probability in Banach Spaces. Springer, New York. MR1102015
16. LIN, Y. and ZHANG, H. (2006). Component selection and smoothing in multivariate nonparametric regression. Ann. Statist. 34 2272-2297. MR2291500
17. MEIER, L., VAN DE GEER, S. and BÜHLMANN, P. (2009). High-dimensional additive modeling. Ann. Statist. 37 3779-3821. MR2572443
18. MENDELSON, S. (2002). Geometric parameters of kernel machines. In COLT 2002. Lecture Notes in Artificial Intelligence 2375 29-43. Springer, Berlin. MR2040403
19. MICCHELLI, C. and PONTIL, M. (2005). Learning the kernel function via regularization. J. Mach. Learn. Res. 6 1099-1125. MR2249850
20. RASKUTTI, G., WAINWRIGHT, M. and YU, B. (2009). Lower bounds on minimax rates for nonparametric regression with additive sparsity and smoothness. In Advances in Neural Information Processing Systems (NIPS 22) 1563-1570. Curran Associates, Red Hook, NY.
21. RAVIKUMAR, P., LIU, H., LAFFERTY, J. and WASSERMAN, L. (2008). SpAM: Sparse additive models. In Advances in Neural Information Processing Systems (NIPS 20) 1201-1208. Curran Associates, Red Hook, NY.
22. SREBRO, N. and BEN-DAVID, S. (2006). Learning bounds for support vector machines with learned kernels. In Learning Theory. Lecture Notes in Comput. Sci. 4005 169-183. Springer, Berlin. MR2280605
23. TALAGRAND, M. (1996). New concentration inequalities for product measures. Invent. Math. 126 505-563. MR1419006 (Pubitemid 126323770)
24. TSYBAKOV, A. B. (2009). Introduction to Nonparametric Estimation. Springer, New York. MR2013911
25. VAN DER VAART, A.W. and WELLNER, J. A. (1996). Weak Convergence and Empirical Processes. With Applications to Statistics. Springer, New York. MR1385671