ℓ1-regularized linear regression: Persistence and oracle inequalities

Probability Theory and Related Fields

Volumn 154, Issue 1-2, 2012, Pages 193-224

  Bartlett, Peter L ^a,b   Mendelson, Shahar ^c   Neeman, Joseph ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84867330878     PISSN: 01788051     EISSN: None     Source Type: Journal    
DOI: 10.1007/s00440-011-0367-2     Document Type: Article

References (42)

1. Bartlett P. L.: Fast rates for estimation error and oracle inequalities for model selection. Econom. Theory 24(02), 545-552 (2008).
2. Bartlett P. L., Jordan M. I., McAuliffe J. D.: Convexity, classification, and risk bounds. J. Am. Stat. Assoc. 101(473), 138-156 (2006).
3. Bartlett P. L., Mendelson S.: Empirical minimization. Probab. Theory Relat. Fields 135(3), 311-334 (2006).
4. Bickel P. J., Ritov Y., Tsybakov A. B.: Simultaneous analysis of LASSO and Dantzig selector. Ann. Stat. 39(4), 1705-1732 (2009).
5. Bunea F., Tsybakov A., Wegkamp M.: Sparsity oracle inequalities for the Lasso. Electron. J. Stat. 1, 169-194 (2007) (electronic).
6. 1 penalized least squares. In: Proceedings of the 19th Annual Conference on Learning Theory (COLT 2006). Lecture Notes in Artificial Intelligence, vol. 4005, pp. 379-391. Springer, Berlin (2006).
7. Candès E. J., Plan Y.: Near-ideal model selection by l1 minimization. Ann. Stat. 37(5A), 2145-2177 (2009).
8. Carl B.: Inequalities of Bernstein-Jackson-type and the degree of compactness of operators in Banach spaces. Ann. Inst. Fourier (Grenoble) 35(3), 79-118 (1985).
9. Catoni, O.: Statistical learning theory and stochastic optimization. Ecole d'Eté de Probabilités de Saint-Flour 2001. Lecture Notes in Mathematics, vol. 1851. Springer, Berlin (2004).
10. de la Peña V. H., Giné E.: Decoupling: From Dependence to Independence, Probability and its Applications (New York). Springer, New York (1999).
11. Donoho D. L., Elad M., Temlyakov V. N.: Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Trans. Inform. Theory 52(1), 6-18 (2006).
12. Donoho D. L., Johnstone I. M.: Ideal spatial adaptation by wavelet shrinkage. Biometrika 81(3), 425-455 (1994).
13. Dudley R. M.: The sizes of compact subsets of Hilbert space and continuity of Gaussian processes. In: Giné, E., Koltchinskii, V., Norvaisa, R. (eds) Selected Works of R. M. Dudley, Selected Works in Probability and Statistics, pp. 125-165. Springer, New York (2010).
14. Giné E., Zinn J.: Some limit theorems for empirical processes. Ann. Probab. 12(4), 929-998 (1984) (with discussion).
15. Gordon Y., Litvak A. E., Mendelson S., Pajor A.: Gaussian averages of interpolated bodies and applications to approximate reconstruction. J. Approx. Theory 149(1), 59-73 (2007).
16. 1 constraint. Ann. Stat. 34(5), 2367-2386 (2006).
17. Greenshtein E., Ritov Y.: Persistence in high-dimensional linear predictor selection and the virtue of overparametrization. Bernoulli 10(6), 971-988 (2004).
18. Guédon O., Mendelson S., Pajor A., Tomczak-Jaegermann N.: Subspaces and orthogonal decompositions generated by bounded orthogonal systems. Positivity 11(2), 269-283 (2007).
19. Guédon O., Mendelson S., Pajor A., Tomczak-Jaegermann N.: measures and proportional subsets of bounded orthonormal systems. Rev. Mat. Iberoamericana 24(3), 1075-1095 (2008).
20. Hoeffding W.: Probability inequalities for sums of bounded random variables. J. Am. Stat. Assoc. 58, 13-30 (1963).
21. Koltchinskii V.: Sparsity in penalized empirical risk minimization. Annales de l'Institut Henri Poincaré-Probabilités et Statistiques 45(1), 7-57 (2009).
22. Lecué, G., Mendelson, S.: General oracle inequalities and applications to high dimensional data analysis (preprint).
23. Leng C., Lin Y., Wahba G.: A note on the lasso and related procedures in model selection. Stat. Sinica 16(4), 1273-1284 (2006).
24. Lounici K.: Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators. Electron. J. Stat. 2, 90-102 (2008).
25. Meinshausen N., Bühlmann P.: High-dimensional graphs and variable selection with the lasso. Ann. Stat. 34(3), 1436-1462 (2006).
26. Meinshausen N., Yu B.: Lasso-type recovery of sparse representations for high-dimensional data. Ann. Stat. 37(1), 246-270 (2009).
27. Mendelson S.: Improving the sample complexity using global data. IEEE Trans. Inform. Theory 48(7), 1977-1991 (2002).
28. Mendelson S.: On the performance of kernel classes. J. Mach. Learn. Res. 4(5), 759-771 (2004).
29. Mendelson S., Neeman J.: Regularization in Kernel learning. Ann. Stat. 30(1), 526-565 (2010).
30. Milman V. D., Schechtman G.: Asymptotic theory of finite-dimensional normed spaces. Lecture Notes in Mathematics, vol. 1200. Springer, Berlin (1986).
31. Pajor A., Tomczak-Jaegermann N.: Remarques sur les nombres d'entropie d'un opérateur et de son transposé. C. R. Acad. Sci. Paris Sér. I Math. 301(15), 743-746 (1985).
32. Paouris G.: Concentration of mass on convex bodies. Geom. Funct. Anal. 16(5), 1021-1049 (2006).
33. Pisier, G.: Some applications of the metric entropy condition to harmonic analysis. In: Banach Spaces, Harmonic Analysis, and Probability Theory, pp. 123-154 (1983).
34. Pisier, G.: The volume of convex bodies and Banach space geometry. In: Cambridge Tracts in Mathematics, vol. 94. Cambridge University Press, Cambridge (1989).
35. Talagrand M.: Regularity of gaussian processes. Acta Mathematica 159, 99-149 (1987). doi: 10. 1007/BF02392556.
36. Talagrand M.: The generic chaining. Springer Monographs in Mathematics Upper and lower bounds of stochastic processes. Springer, Berlin (2005).
37. Tibshirani R.: Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc. Ser. B 58(1), 267-288 (1996).
38. Tsybakov, A. B.: Optimal rates of aggregation. In: Computational Learning Theory 2003. Lecture Notes in Artificial Intelligence. vol. 2777, pp. 303-313. Springer, Berlin (2003).
39. van de Geer S. A.: High-dimensional generalized linear models and the lasso. Ann. Stat. 36(2), 614-645 (2008).
40. van der Vaart, A. W., Wellner, J. A.: Weak convergence and empirical processes. Springer Series in Statistics. Springer, New York (with applications to statistics, 1996).
41. Zhang C. H., Huang J.: The sparsity and bias of the LASSO selection in high-dimensional linear regression. Ann. Stat. 36(4), 1567-1594 (2008).
42. 1 regularization. Ann. Stat. 37(5A), 2109-2144 (2009).