On the convergence rate of lp-norm multiple kernel learning

Journal of Machine Learning Research

Volumn 13, Issue , 2012, Pages 2465-2501

  Kloft, Marius ^a,c   Blanchard, Gilles ^b  

^a TECHNISCHE UNIVERSITÄT BERLIN   (Germany)

^b UNIVERSITY OF POTSDAM   (Germany)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Generalization bounds; Learning kernels; Local Rademacher complexity; Multiple kernel learning

Indexed keywords

CONVERGENCE RATES; DECAY RATE; EIGEN-VALUE; EXCESS LOSS; FAST CONVERGENCE RATE; FEATURE MAPPING; GENERALIZATION BOUND; LEARNING KERNELS; LOCAL APPROACHES; LOWER BOUNDS; MULTIPLE KERNEL LEARNING; RADEMACHER COMPLEXITY; UPPER BOUND;

EIGENVALUES AND EIGENFUNCTIONS;

DECAY (ORGANIC);

EID: 84869166429     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (49)

1. J. Aflalo, A. Ben-Tal, C. Bhattacharyya, J. S. Nath, and S. Raman. Variable sparsity kernel learning algorithms and applications. Journal of Machine Learning Research, 12:565-592, Feb 2011.
2. F. R. Bach. Consistency of the group lasso and multiple kernel learning. J. Mach. Learn. Res., 9: 1179-1225, 2008.
3. F. R. Bach, G. R. G. Lanckriet, and M. I. Jordan. Multiple kernel learning, conic duality, and the SMO algorithm. In Proc. 21st ICML. ACM, 2004.
4. P. Bartlett and S. Mendelson. Rademacher and gaussian complexities: Risk bounds and structural results. Journal of Machine Learning Research, 3:463-482, Nov. 2002.
5. P. L. Bartlett, O. Bousquet, and S.Mendelson. Local Rademacher complexities. Annals of Statistics, 33(4):1497-1537, 2005. (Pubitemid 41423979)
6. P. L. Bartlett, M. I. Jordan, and J. D. McAuliffe. Convexity, classification, and risk bounds. Journal of the American Statistical Association, 101(473):138-156, 2006. (Was Department of Statistics, U.C. Berkeley Technical Report number 638, 2003). (Pubitemid 43500031)
7. G. Blanchard, O. Bousquet, and P. Massart. Statistical performance of support vector machines. Annals of Statistics, 36(2):489-531, 2008.
8. R. R. Bouckaert, E. Frank, M. A. Hall, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten. WEKA experiences with a java open-source project. Journal of Machine Learning Research, 11:2533-2541, 2010.
9. O. Bousquet and A. Elisseeff. Stability and generalization. J. Mach. Learn. Res., 2:499-526, March 2002. ISSN 1532-4435. (Pubitemid 135712570)
10. O. Bousquet, S. Boucheron, and G. Lugosi. Introduction to statistical learning theory. In O. Bousquet, U. von Luxburg, and G. Rätsch, editors, Advanced Lectures on Machine Learning, volume 3176 of Lecture Notes in Computer Science, pages 169-207. Springer Berlin /Heidelberg, 2004. (Pubitemid 39741632)
11. C. Cortes. Invited talk: Can learning kernels help performance? In Proceedings of the 26th Annual International Conference on Machine Learning, ICML '09, pages 1:1-1:1, New York, NY, USA, 2009. ACM. ISBN 978-1-60558-516-1.
12. C. Cortes, A. Gretton, G. Lanckriet, M. Mohri, and A. Rostamizadeh. Proceedings of the NIPS Workshop on Kernel Learning: Automatic Selection of Optimal Kernels, 2008. URL http: //www.cs.nyu.edu/learning-kernels.
13. C. Cortes, M. Mohri, and A. Rostamizadeh. L2 regularization for learning kernels. In Proceedings of the International Conference on Uncertainty in Artificial Intelligence, 2009.
14. C. Cortes, M. Mohri, and A. Rostamizadeh. Generalization bounds for learning kernels. In Proceedings, 27th ICML, 2010.
15. P. V. Gehler and S. Nowozin. Let the kernel figure it out: Principled learning of pre-processing for kernel classifiers. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 06 2009.
16. A. Gelman. Causality and statistical learning. American Journal of Sociology, 0, 2010.
17. R. Ibragimov and S. Sharakhmetov. The best constant in the rosenthal inequality for nonnegative random variables. Statistics & Probability Letters, 55(4):367 376, 2001. ISSN 0167-7152. (Pubitemid 33619150)
18. J.-P. Kahane. Some Random Series of Functions. Cambridge University Press, 2nd edition, 1985.
19. M. Kloft. ℓp-Norm Multiple Kernel Learning. PhD thesis, Berlin Institute of Technology, Oct 2011. URL http://opus.kobv.de/tuberlin/volltexte/ 2011/3239/.
20. M. Kloft, U. Brefeld, P. Laskov, and S. Sonnenburg. Non-sparse multiple kernel learning. In Proc. of the NIPS Workshop on Kernel Learning: Automatic Selection of Kernels, dec 2008.
21. M. Kloft, U. Brefeld, S. Sonnenburg, P. Laskov, K.-R. Müller, and A. Zien. Efficient and accurate lp-norm multiple kernel learning. In Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, editors, Advances in Neural Information Processing Systems 22, pages 997-1005. MIT Press, 2009.
22. M. Kloft, U. Brefeld, S. Sonnenburg, and A. Zien. ℓp-norm multiple kernel learning. Journal of Machine Learning Research, 12:953-997, Mar 2011.
23. V. Koltchinskii. Rademacher penalties and structural risk minimization. IEEE Transactions on Information Theory, 47(5):1902-1914, 2001. (Pubitemid 32644704)
24. V. Koltchinskii. Local Rademacher complexities and oracle inequalities in risk minimization. Annals of Statistics, 34(6):2593-2656, 2006.
25. V. Koltchinskii. Sparsity in penalized empirical risk minimization. Annales de l'Institut Henri Poincaŕe, Probabilit́es et Statistiques, 45(1):7-57, 2009.
26. V. Koltchinskii and M. Yuan. Sparsity in multiple kernel learning. Annals of Statistics, 38(6): 3660-3695, 2010.
27. S. Kwapi'en and W. A. Woyczy'nski. Random Series and Stochastic Integrals: Single and Multiple. Birkhäuser, Basel and Boston, M.A., 1992.
28. G. Lanckriet, N. Cristianini, L. E. Ghaoui, P. Bartlett, and M. I. Jordan. Learning the kernel matrix with semi-definite programming. JMLR, 5:27-72, 2004.
29. H. Leeb and B. M. Pötscher. Sparse estimators and the oracle property, or the return of Hodges' estimator. Journal of Econometrics, 142:201-211, 2008. (Pubitemid 350117135)
30. C.McDiarmid. On the method of bounded differences. In Surveys in combinatorics, 1989 (Norwich, 1989), volume 141 of London Math. Soc. Lecture Note Ser., pages 148-188. Cambridge Univ. Press, Cambridge, 1989.
31. S. Mendelson. On the performance of kernel classes. J. Mach. Learn. Res., 4:759-771, December 2003.
32. C. A. Micchelli and M. Pontil. Learning the kernel function via regularization. Journal of Machine Learning Research, 6:1099-1125, 2005.
33. K.-R. Müller, S. Mika, G. Rätsch, K. Tsuda, and B. Schölkopf. An introduction to kernel-based learning algorithms. IEEE Neural Networks, 12(2):181-201, May 2001. (Pubitemid 32371478)
34. G. Raskutti, M. J. Wainwright, and B. Yu. Minimax-optimal rates for sparse additive models over kernel classes via convex programming. CoRR, abs/1008.3654, 2010.
35. H. Rosenthal. On the subspaces of Lp (p > 2) spanned by sequences of independent random variables. Israel J. Math., 8:273-303, 1970.
36. B. Schölkopf, A. Smola, and K.-R. Müller. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10:1299-1319, 1998. (Pubitemid 128463674)
37. J. R. Searle. Minds, brains, and programs. Behavioral and Brain Sciences, 3(03):417424, 1980. doi: 10.1017/S0140525X00005756. URL http://dx.doi.org/10. 1017/S0140525X00005756.
38. S. Sonnenburg, G. Rätsch, C. Schäfer, and B. Schölkopf. Large scale multiple kernel learning. Journal of Machine Learning Research, 7:1531-1565, July 2006. (Pubitemid 44373694)
39. N. Srebro and S. Ben-David. Learning bounds for support vector machines with learned kernels. In G. Lugosi and H.-U. Simon, editors, COLT, volume 4005 of Lecture Notes in Computer Science, pages 169-183. Springer, 2006. (Pubitemid 44072193)
40. J. M. Steele. The Cauchy-Schwarz Master Class: An Introduction to the Art of Mathematical Inequalities. Cambridge University Press, New York, NY, USA, 2004. ISBN 052154677X.
41. I. Steinwart and A. Christmann. Support Vector Machines. Springer, New York, 2008.
42. M. Stone. Cross-validatory choice and assessment of statistical predictors (with discussion). Journal of the Royal Statistical Society, B36:111-147, 1974.
43. T. Suzuki. Unifying framework for fast learning rate of non-sparse multiple kernel learning. In J. Shawe-Taylor, R. Zemel, P. L. Bartlett, F. Pereira, and K. Weinberger, editors, Advances in Neural Information Processing Systems 24. 2011. To appear.
44. M. Talagrand. Concentration of measure and isoperimetric inequalities in product spaces. Publications Mathmatiques de L'IHS, 81:73-205, 1995.
45. A. Tsybakov. Optimal rates of aggregation. In B. Schölkopf and M. Warmuth, editors, Computational Learning Theory and Kernel Machines (COLT-2003), volume 2777 of Lecture Notes in Artificial Intelligence, pages 303-313. Springer, 2003. (Pubitemid 37053212)
46. S. van de Geer. Empirical Processes in M-estimation. Cambridge University Press, 2000.
47. R. C. Williamson, A. J. Smola, and B. Schölkopf. Generalization performance of regularization networks and support vector machines via entropy numbers of compact operators. IEEE Transactions on Information Theory, 47(6):2516-2532, 2001. (Pubitemid 32949899)
48. H. Xu, S. Mannor, and C. Caramanis. Sparse algorithms are not stable: A no-free-lunch theorem. In Proceedings of the 46th Annual Allerton Conference on Communication, Control, and Computing, pages 1299 -1303, 2008.
49. Y. Ying and C. Campbell. Generalization bounds for learning the kernel problem. In COLT, 2009.