Sparsity oracle inequalities for the Lasso

Electronic Journal of Statistics

Volumn 1, Issue , 2007, Pages 169-194

  Bunea, Florentina ^a   Tsybakov, Alexandre ^b   Wegkamp, Marten ^a  

^a FLORIDA STATE UNIVERSITY   (United States)

^b SORBONNE UNIVERSITÉ   (France)

Author keywords

Adaptive estimation; Aggregation; Dimension reduction; Lasso; Mutual coherence; Nonparametric regression; Oracle inequalities; Penalized least squares; Sparsity

Indexed keywords

EID: 50849114939     PISSN: 19357524     EISSN: None     Source Type: Journal    
DOI: 10.1214/07-EJS008     Document Type: Article

References (29)

1. Abramovich, F., Benjamini, Y., Donoho, D.L. and Johnstone, I.M., (2006). Adapting to unknown sparsity by controlling the False Discovery Rate. Annals of Statistics 34(2) 584-653.
2. Antoniadis, A. and Fan, J. (2001). Regularized wavelet approximations (with discussion). Journal of American Statistical Association 96 939-967.
3. Baraud, Y. (2002). Model selection for regression on a random design. ESAIM Probability & Statistics 7 127-146.
4. Birgé, L. (2004). Model selection for Gaussian regression with random design. Bernoulli 10 1039-1051.
5. Bunea, F., Tsybakov, A.B. and Wegkamp, M.H. (2005) Aggregation for Gaussian regression. Preprint Department of Statistics, Florida state University.
6. 1-penalized least squares. Proceedings of 19th Annual Conference on Learning Theory, COLT 2006. Lecture Notes in Artificial Intelligence 4005 379-391. Springer-Verlag, Heidelberg.
7. Candes, E. and Tao, T. (2005). The Dantzig selector: statistical estimation when p is much larger than n. Manuscript.
8. Donoho, D.L., Elad, M. and Temlyakov, V. (2004). Stable Recovery of Sparse Overcomplete Representations in the Presence of Noise. Manuscript.
9. Donoho, D.L. and Johnstone, I.M. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika 81 425-455.
10. Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R. (2004). Least angle regression. Annals of Statistics 32(2) 407-451.
11. Golubev, G.K. (2002). Reconstruction of sparse vectors in white Gaussian noise. Problems of Information Transmission 38(1) 65-79.
12. Greenshtein, E. and Ritov, Y. (2004). Persistency in high dimensional linear predictor-selection and the virtue of over-parametrization. Bernoulli 10971-988.
13. Györfi, L., Kohler, M., Krzyżak, A. and Walk, H. (2002). A Distribution-Free Theory of Nonparametric Regression. Springer: New York.
14. Juditsky, A. and Nemirovski, A. (2000). Functional aggregation for nonparametric estimation. Annals of Statistics 28 681-712.
15. Kerkyacharian, G. and Picard, D. (2005). Tresholding in learning theory. Prépublication n.1017, Laboratoire de Probabilités et Modèles Aléatoires, Universités Paris 6 -Paris 7.
16. Knight, K. and Fu, W. (2000). Asymptotics for lasso-type estimators. Annals of Statistics 28(5) 1356-1378.
17. Koltchinskii, V. (2005). Model selection and aggregation in sparse classification problems. Oberwolfach Reports: Meeting on Statistical and Probabilistic Methods of Model Selection, October 2005 (to appear).
18. Koltchinskii, V. (2006). Sparsity in penalized empirical risk minimization. Manuscript.
19. Loubes, J.-M. and van de Geer, S.A. (2002). Adaptive estimation in regression, using soft thresholding type penalties. Statistica Neerlandica 56453-478.
20. Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the Lasso. Annals of Statistics 34 (3) 1436-1462.
21. Nemirovski, A. (2000). Topics in Non-parametric Statistics. Ecole d’Eté de Probabilités de Saint-Flour XXVIII -1998, Lecture Notes in Mathematics, v. 1738, Springer: New York.
22. Osborne, M.R., Presnell, B. and Turlach, B.A (2000a). On the lasso and its dual. Journal of Computational and Graphical Statistics 9 319-337.
23. Osborne, M.R., Presnell, B. and Turlach, B.A (2000b). A new approach to variable selection in least squares problems. IMA Journal of Numerical Analysis 20(3) 389-404.
24. Tibshirani, R. (1996). Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society, Series B. 58 267-288.
25. Tsybakov, A.B. (2003). Optimal rates of aggregation. Proceedings of 16th Annual Conference on Learning Theory (COLT) and 7th Annual Workshop on Kernel Machines. Lecture Notes in Artificial Intelligence 2777 303-313. Springer-Verlag, Heidelberg.
26. 1 constraint. 2004 Proceedings of the American Statistical Association, Statistical Computing Section [CD-ROM], American Statistical Association, Alexandria, VA, pp. 2572-2577.
27. van de Geer, S.A. (2006). High dimensional generalized linear models and the Lasso. Research report No.133. Seminar für Statistik, ETH, Zürich.
28. 1-constrained quadratic programming. Technical report 709, Department of Statistics, UC Berkeley.
29. Zhao, P. and Yu, B. (2005). On model selection consistency of Lasso. Manuscript.