Best subset selection, persistence in high-dimensional statistical learning and optimization under l1 constraint

Annals of Statistics

Volumn 34, Issue 5, 2006, Pages 2367-2386

  Greenshtein, Eitan ^a,b  

^a PURDUE UNIVERSITY   (United States)

^b PURDUE UNIVERSITY   (United States)

Author keywords

Persistence; Variable selection

Indexed keywords

EID: 33845612972     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/009053606000000768     Document Type: Article

References (26)

1. BICKEL, P. and LEVINA, E. (2004). Some theory of Fisher's linear discriminant function, "naive Bayes," and some alternatives where there are many more variables than observations. Bernoulli 10 989-1010. MR2108040
2. BREIMAN, L. (2001). Statistical modeling: The two cultures (with discussion). Statist. Sci. 16 199-231. MR1874152
3. BREIMAN, L. (2004). Population theory for boosting ensembles. Ann. Statist. 32 1-11. MR2050998
4. BÜHLMANN, P. and BIN, Y. (2004). Discussion of boosting papers. Ann. Statist. 32 96-101.
5. CHEN, S., DONOHO, D. and SAUNDERS, M. (2001). Atomic decomposition by basis pursuit. SIAM Rev. 43 129-159. MR1854649
6. 1-norm solution is also the sparsest solution. Technical Report 2004-9, Dept. Statistics, Stanford Univ.
7. 1-norm near-solution approximates the sparsest near-solution. Technical Report 2004-10, Dept. Statistics, Stanford Univ.
8. EFRON, B., JOHNSTONE, I., HASTIE, T. and TIBSHIRANI, R. (2004). Least angle regression (with discussion). Ann. Statist. 32 407-499. MR2060166
9. FAN, J. and LI, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348-1360. MR1946581
10. FAN, J. and PENG, H. (2004). Nonconcave penalized likelihood with a diverging number of parameters. Ann. Statist. 32 928-961. MR2065194
11. FRIEDMAN, J., HASTIE, T., ROSSET, S., TIBSHIRANI, R. and ZHU, J. (2004). Discussion of boosting papers. Ann. Statist. 32 102-107.
12. GREENSHTEIN, E. (2005). Prediction, model selection and random dimension penalties. Sankhyā 67 46-73. MR2203888
13. GREENSHTEIN, E. and RITOV, Y. (2004). Persistence in high-dimensional predictor selection and the virtue of overparametrization. Bernoulli 10 971-988. MR2108039
14. HASTIE, T., TIBSHIRANI, R. and FRIEDMAN, J. (2001). The Elements of Statistical Learning. Data Mining, Inference and Prediction. Springer, New York. MR1851606
15. HUBER, P. (1973). Robust regression: Asymptotics, conjectures, and Monte Carlo. Ann. Statist. 1 799-821. MR0356373
16. JUDITSKY, A. and NEMIROVSKI, A. (2000). Functional aggregation for nonparametric regression. Ann. Statist. 28681-712. MR1792783
17. LEE, W. S., BARTLETT, P. L. and WILLIAMSON, R. C. (1996). Efficient agnostic learning of neural networks with bounded fan-in. IEEE Trans. Inform. Theory 42 2118-2132. MR1447518
18. LUGOSI, G. and VAYATIS, N. (2004). On the Bayes-risk consistency of regularized boosting methods. Ann. Statist. 32 30-55. MR2051000
19. MEINSHAUSEN, N. and BÜHLMANN, P. (2006). High-dimensional graphs and variable selection with the Lasso. Ann. Statist. 34 1436-1462.
20. NEMIROVSKI, A. and YUDIN, D. (1983). Problem Complexity and Method Efficiency in Optimization. Wiley, New York. MR0702836
21. NGUYEN, D. V., ARPAT, A. B., WANG, N. and CARROLL, R. J. (2002). DNA microarray experiments: Biological and technological aspects. Biometrics 58 701-717. MR1939398
22. PISIER, G. (1981). Remarques sur un résultat non publié de B, Maurey. In Seminaire d'Analyse Fonctionelle 112. École Polytechnique, Palaiseau. MR0659306
23. 2/n is large. I. Consistency. Ann. Statist. 12 1298-1309. MR0760690
24. TIBSHIRANI, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58 267-288. MR1379242
25. VAPNIK, N. V. (1998). Statistical Learning Theory. Wiley, New York. MR1641250
26. YOHAI, V. J. and MARONNA, R. A. (1979). Asymptotic behavior of M-estimators for the linear model. Ann. Statist. 7 258-268. MR0520237