Minimax-optimal rates for sparse additive models over kernel classes via convex programming

Journal of Machine Learning Research

Volumn 13, Issue , 2012, Pages 389-427

  Raskutti, Garvesh ^a   Wainwright, Martin J ^a,b   Yu, Bin ^a,b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b Department of Elecrical Engineering and Computer Science ^*  (United States)

Author keywords

Convex; Kernel; Minimax; Non parametric; Sparsity

Indexed keywords

CONVEX; KERNEL; MINIMAX; NON-PARAMETRIC; SPARSITY;

ARTIFICIAL INTELLIGENCE; SOFTWARE ENGINEERING;

OPTIMIZATION;

EID: 84857824105     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (40)

1. K. S. Alexander. Rates of Growth and Sample Moduli for Weighted Empirical Processes Indexed by Sets. Probability Theory and Related Fields, 75:379-423, 1987.
2. N. Aronszajn. Theory of Reproducing Kernels. Transactions of the AmericanMathematical Society, 68:337-404, 1950.
3. F. Bach. Consistency of the Group Lasso and Multiple Kernel Learning. Journal of Machine Learning Research, 9:1179-1225, 2008.
4. P. Bartlett, O. Bousquet, and S. Mendelson. Local Rademacher Complexities. Annals of Statistics, 33:1497-1537, 2005. (Pubitemid 41423979)
5. P. Bickel, Y. Ritov, and A. Tsybakov. Simultaneous Analysis of Lasso and Dantzig Selector. Annals of Statistics, 37(4):1705-1732, 2009.
6. α. Math. USSR-Sbornik, 2(3):295-317, 1967.
7. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, Cambridge, UK, 2004.
8. L. Breiman. Better Subset Regression Using the Nonnegative Garrote. Technometrics, 37:373-384, 1995.
9. V. V. Buldygin and Y. V. Kozachenko. Metric Characterization of Random Variables and Random Processes. American Mathematical Society, Providence, RI, 2000.
10. B. Carl and I. Stephani. Entropy, Compactness and the Approximation of Operators. Cambridge Tracts in Mathematics. Cambridge University Press, Cambridge, UK, 1990.
11. B. Carl and H. Triebel. Inequalities Between Eigenvalues, Entropy Numbers and Related Quantities of Compact Operators in Banach Spaces. Annals of Mathematics, 251:129-133, 1980.
12. F. Chung and L. Lu. Concentration Inequalities and Martingale Inequalities. Internet Mathematics, 3:79-127, 2006.
13. T.M. Cover and J.A. Thomas. Elements of Information Theory. John Wiley and Sons, New York, 1991.
14. U. Einmahl and D. M. Mason. Some Universal Results on the Behavior of the Increments of Partial Sums. Annals of Probability, 24:1388-1407, 1996.
15. C. Gu. Smoothing Spline ANOVA Models. Springer Series in Statistics. Springer, New York, NY, 2002.
16. R. Z. Has'minskii. A Lower Bound on the Risks of Nonparametric Estimates of Densities in the Uniform Metric. Theory Prob. Appl., 23:794-798, 1978.
17. T. Hastie and R. Tibshirani. Generalized Additive Models. Statistical Science, 1(3):297-310, 1986.
18. G. Kimeldorf and G. Wahba. Some Results on Tchebycheffian Spline Functions. Jour. Math. Anal. Appl., 33:82-95, 1971.
19. V. Koltchinskii and M. Yuan. Sparse Recovery in Large Ensembles of Kernel Machines. In Proceedings of COLT, 2008.
20. V. Koltchinskii and M. Yuan. Sparsity in Multiple Kernel Learning. Annals of Statistics, 38:3660-3695, 2010.
21. M. Ledoux. The Concentration of Measure Phenomenon. Mathematical Surveys and Monographs. American Mathematical Society, Providence, RI, 2001.
22. Y. Lin and H. H. Zhang. Component Selection and Smoothing in Multivariate Nonparametric Regression. Annals of Statistics, 34:2272-2297, 2006.
23. P. Massart. About the Constants in Talagrand's Concentration Inequalities for Empirical Processes. Annals of Probability, 28(2):863-884, 2000.
24. L. Meier, S. van de Geer, and P. Buhlmann. High-dimensional Additive Modeling. Annals of Statistics, 37:3779-3821, 2009.
25. S. Mendelson. Geometric Parameters of Kernel Machines. In Proceedings of COLT, pages 29-43, 2002.
26. J. Mercer. Functions of Positive and Negative Type and Their Connection With the Theory of Integral Equations. Philosophical Transactions of the Royal Society A, 209:415-446, 1909.
27. S. Negahban, P. Ravikumar, M. J. Wainwright, and B. Yu. A Unified Framework for Highdimensional Analysis of M-estimators with Decomposable Regularizers. In NIPS Conference, 2009.
28. G. Pisier. The Volume of Convex Bodies and Banach Space Geometry, volume 94 of Cambridge Tracts in Mathematics. Cambridge University Press, Cambridge, UK, 1989.
29. G. Raskutti, M. J. Wainwright, and B. Yu. Minimax Rates of Estimation for High-dimensional Linear Regression Over ℓq-balls. IEEE Trans. Information Theory, 57(10):6976-6994, October 2011.
30. P. Ravikumar, H. Liu, J. Lafferty, and L. Wasserman. SpAM: Sparse Additive Models. Journal of the Royal Statistical Society, Series B, 71(5):1009-1030, 2009.
31. S. Saitoh. Theory of Reproducing Kernels and its Applications. Longman Scientific & Technical, Harlow, UK, 1988.
32. B. Schölkopf and A. Smola. Learning with Kernels. MIT Press, Cambridge, MA, 2002.
33. C. J. Stone. Additive Regression and Other Nonparametric Models. Annals of Statistics, 13(2): 689-705, 1985.
34. T. Suzuki and M. Sugiyama. Fast Learning Rate of Multiple Kernel Learning: Trade-off Between Sparsity and Smoothness. In AISTATS Conference, 2012.
35. S. van de Geer. Empirical Processes in M-Estimation. Cambridge University Press, 2000.
36. A. W. van der Vaart and J. Wellner. Weak Convergence and Empirical Processes. Springer-Verlag, New York, NY, 1996.
37. G. Wahba. Spline Models for Observational Data. CBMS-NSF Regional Conference Series in Applied Mathematics. SIAM, Philadelphia, PN, 1990.
38. Y. Yang and A. Barron. Information-theoretic Determination of Minimax Rates of Convergence. Annals of Statistics, 27(5):1564-1599, 1999.
39. B. Yu. Assouad, Fano and Le Cam. Research Papers in Probability and Statistics: Festschrift in Honor of Lucien Le Cam, pages 423-435, 1996.
40. M. Yuan. Nonnegative Garrote Component Selection in Functional ANOVAModels. In Conference on Artificial Intelligence and Statistics, pages 660-666, 2007.