Nonconcave penalized likelihood with NP-dimensionality

IEEE Transactions on Information Theory

Volumn 57, Issue 8, 2011, Pages 5467-5484

  Fan, Jianqing ^a   Lv, Jinchi ^b  

^a Princeton University   (United States)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

Coordinate optimization; folded concave penalty; high dimensionality; Lasso; nonconcave penalized likelihood; oracle property; SCAD; variable selection; weak oracle property

Indexed keywords

COORDINATE OPTIMIZATION; FOLDED-CONCAVE PENALTY; HIGH DIMENSIONALITY; LASSO; NONCONCAVE PENALIZED LIKELIHOODS; ORACLE PROPERTY; SCAD; VARIABLE SELECTION; WEAK ORACLE PROPERTY;

DATA REDUCTION;

OPTIMIZATION;

EID: 79960979995     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2011.2158486     Document Type: Review

References (40)

1. A. Antoniadis and J. Fan, "Regularization of wavelets approximations (with discussion)," J. Amer. Statist. Assoc., vol. 96, pp. 939-967, 2001.
2. G. Bennett, "Probability inequalities for the sum of independent random variables," J. Amer. Statist. Assoc., vol. 57, pp. 33-45, 1962.
3. P. J. Bickel, Y. Ritov, and A. Tsybakov, "Simultaneous analysis of Lasso and Dantzig selector," Ann. Statist., vol. 37, pp. 1705-1732, 2009.
4. L. Breiman, "Better subset regression using the non-negative garrote," Technometrics, vol. 37, pp. 373-384, 1995.
5. F. Bunea, A. Tsybakov, and M. H. Wegkamp, "Sparsity oracle inequalities for the Lasso," Elec. Jour. Statist., vol. 1, pp. 169-194, 2007.
6. E. Candes and T. Tao, "The Dantzig selector: Statistical estimation when n is much larger than - (with discussion)," Ann. Statist., vol. 35, pp. 2313-2404, 2007.
7. I. Daubechies, M. Defrise, and C. De Mol, "An iterative thresholding algorithm for linear inverse problems with a sparsity constraint," Comm. Pure Appl. Math., vol. 57, pp. 1413-1457, 2004. (Pubitemid 39427442)
8. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, "Least angle regression (with discussion)," Ann. Statist., vol. 32, pp. 407-499, 2004.
9. J. Fan, "Comments on "Wavelets in statistics: A review" by A. Antoniadis," J. Italian Statist. Assoc., vol. 6, pp. 131-138, 1997.
10. J. Fan and Y. Fan, "High-dimensional classification using features annealed independence rules," Ann. Statist., vol. 36, pp. 2605-2637, 2008.
11. J. Fan and R. Li, "Variable selection via nonconcave penalized likelihood and its oracle properties," J. Amer. Statist. Assoc., vol. 96, pp. 1348-1360, 2001. (Pubitemid 33695585)
12. J. Fan and J. Lv, "Sure independence screening for ultrahigh dimensional feature space (with discussion)," J. Roy. Statist. Soc. Ser. B, vol. 70, pp. 849-911, 2008.
13. J. Fan and J. Lv, "A selective overview of variable selection in high dimensional feature space (invited review article)," Statistica Sinica, vol. 20, pp. 101-148, 2010.
14. J. Fan and H. Peng, "Nonconcave penalized likelihood with diverging number of parameters," Ann. Statist., vol. 32, pp. 928-961, 2004. (Pubitemid 41250289)
15. J. Fan, R. Samworth, and Y. Wu, "Ultrahigh dimensional variable selection: Beyond the linear model," J. Machine Learning Res., vol. 10, pp. 1829-1853, 2009.
16. I. E. Frank and J. H. Friedman, "A statistical view of some chemometrics regression tools (with discussion)," Technometrics, vol. 35, pp. 109-148, 1993.
17. J. Friedman, T. Hastie, H. Höfling, and R. Tibshirani, "Pathwise coordinate optimization," Ann. Appl. Statist., vol. 1, pp. 302-332, 2007.
18. W. J. Fu, "Penalized regression: The bridge versus the LASSO," Journal of Computational and Graphical Statistics, vol. 7, pp. 397-416, 1998.
19. P. Hall and H. Miller, "Using generalized correlation to effect variable selection in very high dimensional problems," J. Comput. Graph. Statist., vol. 18, pp. 533-550, 2009.
20. P. Hall, D. M. Titterington, and J.-H. Xue, "Tilting methods for assessing the influence of components in a classifier," J. Roy. Statist. Soc. Ser. B, vol. 71, pp. 783-803, 2009.
21. W. Hoeffding, "Probability inequalities for sums of bounded random variables," J. Amer. Statist. Assoc., vol. 58, pp. 13-30, 1963.
22. J. Huang, J. Horowitz, and S. Ma, "Asymptotic properties of bridge estimators in sparse high-dimensional regression models," Ann. Statist., vol. 36, pp. 587-613, 2008.
23. V. Koltchinskii, "Sparse recovery in convex hulls via entropy penalization," Ann. Statist., vol. 37, pp. 1332-1359, 2009.
24. J. Lv and Y. Fan, "A unified approach to model selection and sparse recovery using regularized least squares," Ann. Statist., vol. 37, pp. 3498-3528, 2009.
25. J. Lv and J. S. Liu, "Model selection principles in misspecified models," Journal of the Royal Statistical Society Series B, 2011, under revision.
26. P. McCullagh and J. A. Nelder, Generalized Linear Models. London: Chapman and Hall, 1989.
27. L. Meier, S. van de Geer, and P. Bühlmann, "The group lasso for logistic regression," J. R. Statist. Soc. B, vol. 70, pp. 53-71, 2008.
28. N. Meinshausen and P. Bühlmann, "High dimensional graphs and variable selection with the Lasso," Ann. Statist., vol. 34, pp. 1436-1462, 2006. (Pubitemid 44231168)
29. A. Oberthuer, F. Berthold, P.Warnat, B. Hero, Y. Kahlert, R. Spitz, K. Ernestus, R. König, S. Haas, R. Eils, M. Schwab, B. Brors, F. Westermann, and M. Fischer, "Customized oligonucleotide microarray gene expression-based classification of neuroblastoma patients outperforms current clinical risk stratification," Journal of Clinical Oncology, vol. 24, pp. 5070-5078, 2006. (Pubitemid 46631411)
30. R. Tibshirani, "Regression shrinkage and selection via the Lasso," J. Roy. Statist. Soc. Ser. B, vol. 58, pp. 267-288, 1996.
31. S. van de Geer, "High-dimensional generalized linear models and the lasso," Ann. Statist., vol. 36, pp. 614-645, 2008.
32. A. W. van der Vaart and J. A. Wellner, Weak Convergence and Empirical Processes. New York: Springer, 1996.
33. M. N. Vrahatis, "A short proof and a generalization of Miranda's existence theorem," in Proceedings of American Mathematical Society, 1989, vol. 107, pp. 701-703.
34. W. J. Wainwright, "Sharp thresholds for noisy and high-dimensional recovery of sparsity using α -constrained quadratic programming," IEEE Transactions on Information Theory, vol. 55, pp. 2183-2202, 2009.
35. M. Yuan and Y. Lin, "Model selection and estimation in regression with grouped variables," J. Roy. Statist. Soc. Ser. B, vol. 68, pp. 49-67, 2006. (Pubitemid 43415335)
36. C.-H. Zhang, "Nearly unbiased variable selection under minimax concave penalty," Ann. Statist., vol. 38, pp. 894-942, 2010.
37. C.-H. Zhang and J. Huang, "The sparsity and bias of the LASSO selection in high-dimensional linear regression," Ann. Statist., vol. 36, pp. 1567-1594, 2008.
38. P. Zhao and B. Yu, "On model selection consistency of Lasso," J. Machine Learning Res., vol. 7, pp. 2541-2567, 2006.
39. H. Zou, "The adaptive Lasso and its oracle properties," J. Amer. Statist. Assoc., vol. 101, pp. 1418-1429, 2006.
40. H. Zou and R. Li, "One-step sparse estimates in nonconcave penalized likelihood models (with discussion)," Ann. Statist., vol. 36, pp. 1509-1566, 2008.