Sparsity in penalized empirical risk minimization

Annales de l'institut Henri Poincare (B) Probability and Statistics

Volumn 45, Issue 1, 2009, Pages 7-57

  Koltchinskii, Vladimir ^a  

^a GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

p penalty; Empirical risk; Oracle inequalities; Penalized empirical risk; Sparsity

Indexed keywords

EID: 62549157465     PISSN: 02460203     EISSN: 02460203     Source Type: Journal    
DOI: 10.1214/07-AIHP146     Document Type: Article

References (30)

1. A. Barron, L. Birgé and P. Massart. Risk bounds for model selection via penalization. Probab. Theory Related Fields 113 (1999) 301-413. MR1679028
2. P. Bartlett, O. Bousquet and S. Mendelson. Local Rademacher complexities. Ann. Statist. 33 (2005) 1497-1537. MR2166554
3. A. Ben-Tal and A. Nemirovski. Lectures on Modern Convex Optimization. Analysis, Algorithms and Engineering Applications. MPS/SIAM, Series on Optimization, Philadelphia, 2001. MR1857264
4. F. Bunea, A. Tsybakov and M. Wegkamp. Aggregation for Gaussian regression. Ann. Statist. 35 (2007) 1674-1697. MR2351101
5. F. Bunea, A. Tsybakov and M. Wegkamp. Sparsity oracle inequalities for the LASSO. Electron. J. Statist. 1 (2007) 169-194. MR2312149
6. E. Candes and T. Tao. The Dantzig selector statistical estimation when p is much larger than n. Ann. Statist. 35 (2007) 2313-2351.
7. E. Candes, M. Rudelson, T. Tao and R. Vershynin. Error correction via linear programming. In Proc. 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS05) 295-308. IEEE, Pittsburgh, PA, 2005.
8. E. Candes, J. Romberg and T. Tao. Stable signal recovery from incomplete and inaccurate measurements. Comm. Pure Appl. Math. 59 (2006) 1207-1223. MR2230846
9. O. Catoni. Statistical Learning Theory and Stochastic Optimization. Springer, New York, 2004. MR2163920
10. 1-norm near-solution approximates the sparsest nearsolution. Preprint, 2004.
11. 1-norm solution is also the sparsest solution. Comm. Pure Appl. Math. 59 (2006) 797-829. MR2217606
12. D. L. Donoho. Compressed sensing. IEEE Trans. Inform. Theory 52 (2006) 1289-1306. MR2241189
13. D. L. Donoho, M. Elad and V. Temlyakov. Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Trans. Inform. Theory 52 (2006) 6-18. MR2237332
14. van de S. Geer. High-dimensional generalized linear models and the Lasso. Ann. Statist. 36 (2008) 614-645.
15. V. Koltchinskii. Model selection and aggregation in sparse classification problems. Oberwolfach Reports Meeting on Statistical and Probabilistic Methods of Model Selection, October, 2005.
16. V. Koltchinskii. Local Rademacher complexities and oracle inequalities in risk mnimization. Ann. Statist. 34 (2006) 2593-2656. MR2329442
17. V. Koltchinskii and D. Panchenko. Complexities of convex combinations and bounding the generalization error in classification. Ann. Statist. 33 (2005) 1455-1496. MR2166553
18. M. Ledoux and M. Talagrand. Probability in Banach Spaces. Springer, New York, 1991. MR1102015
19. P. Massart. Some applications of concentration inequalities to statistics. Ann. Fac. Sci. Tolouse (IX) 9 (2000) 245-303. MR1813803
20. P. Massart. Concentration Inequalities and Model Selection. Springer, Berlin, 2007. MR2319879
21. S. Mendelson, A. Pajor and N. Tomczak-Jaegermann. Reconstruction and subgaussian operators in Asymptotic Geometric Analysis. Geom. Funct. Anal. 17 (2007) 1248-1282.
22. N. Meinshausen and P. Bühlmann. High-dimensional graphs and variable selection with the LASSO. Ann. Statist. 34 (2006) 1436-1462. MR2278363
23. A. Nemirovski. Topics in non-parametric statistics. In Ecole d'Et'e de Probabilités de Saint-Flour XXVIII, 1998 85-277. P. Bernard (Ed). Springer, New York, 2000. MR1775640
24. M. Rudelson and R. Vershynin. Geometric approach to error correcting codes and reconstruction of signals. Int. Math. Res. Not. 64 (2005) 4019-4041. MR2206919
25. R. Tibshirani. Regression shrinkage and selection via Lasso. J. Royal Statist. Soc. Ser. B 58 (1996) 267-288. MR1379242
26. A. Tsybakov. Optimal rates of aggregation. In Proc. 16th Annual Conference on Learning Theory (COLT) and 7th Annual Workshop on Kernel Machines, 303-313. Lecture Notes in Artificial Intelligence 2777. Springer, New York, 2003.
27. van der A. Vaart and J. Wellner. Weak Convergence and Empirical Processes. Springer, New York, 1996. MR1385671
28. Y. Yang. Mixing strategies for density estimation. Ann. Statist. 28 (2000) 75-87. MR1762904
29. Y. Yang. Aggregating regression procedures for a better performance. Bernoulli 10 (2004) 25-47. MR2044592
30. P. Zhao and B. Yu. On model selection consistency of LASSO. J.Mach. Learn. Res. 7 (2006) 2541-2563. MR2274449