Persistence in high-dimensional linear predictor selection and the virtue of overparametrization

Bernoulli

Volumn 10, Issue 6, 2004, Pages 971-988

  Greenshtein, Eitan ^a   Ritov, Ya'acov ^b  

^a UNIVERSITY OF HAIFA   (Israel)

^b HEBREW UNIVERSITY OF JERUSALEM   (Israel)

Author keywords

Consistency; Lasso; Regression; Variable selection

Indexed keywords

EID: 31344454903     PISSN: 13507265     EISSN: None     Source Type: Journal    
DOI: 10.3150/bj/1106314846     Document Type: Article

References (17)

1. Bickel, P. and Levina, E. (2004) Some theory for Fisher's linear discriminant function, 'naive Bayes', and some alternatives where there are more variables than observations. Bernoulli, 10, 989-1010.
2. Billingsley, P. (1995) Probability and Measure. (3rd edition). New York: Wiley.
3. Breiman, L. (2001) Statistical modeling: the two cultures. Statist. Sci., 16, 199-231.
4. Breiman, L. and Freedman, D. (1983) How many variables should be entered in a regression equation? J. Amer. Statist. Assoc., 78, 131-136.
5. Chen, S., Donoho, D. and Saunders, M. (2001) Atomic decomposition by basis pursuit. SIAM Rev., 43, 129-159.
6. Donoho, D.L. and Johnstone, I.M. (1994) Ideal spatial adaptation by wavelet shrinkage. Biometrika, 81, 425-455.
7. Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R. (2004) Least angle regression. Ann. Statist., 32, 407-499.
8. Emery, M., Nemirovski, A. and Voiculescu, D. (2000) Lectures on Probability Theory and Statistics. Éole d'Été de Probabilités de Saint-Flour XXVIII - 1998 (P. Bernard, ed.), Lecture Notes in Math, 1738. Berlin: Springer-Verlag.
9. Foster, D.P. and George, E.L. (1994) The risk inflation criterion for multiple regression. Ann. Statist., 22, 1947-1975.
10. Huber, P. (1973) Robust regression: asymptotics, conjectures, and Monte Carlo. Ann. Statist., 1, 799-821.
11. Juditsky, A. and Nemirovski, A. (2000) Functional aggregation for nonparametric regression. Ann. Statist., 28, 681-712.
12. Le Cam, L. and Yang, G.L. (1990) Asymptotics in Statistics. New York: Springer-Verlag.
13. Lee, W.S., Bartlett, P.L. and Williamson, R.C. (1996) Efficient agnostics learning of neural networks with bounded fan in. IEEE Trans. Inform. Theory, 42, 2118-2132.
14. Nemirovski, A. and Yudin, D. (1983) Problem Complexity and Method Efficiency in Optimization. Chichester: Wiley.
15. 2/n is large, I. Consistency. Ann. Statist., 12, 1298-1309.
16. Silverstein, J.W. (1985) The smallest eigen value of a large dimensional Wishart matrix. Ann. Probab., 13, 1364-1368.
17. Tibshirani, R. (1996) Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B, 58, 267-288.