Estimation and selection via absolute penalized convex minimization and its multistage adaptive applications

Journal of Machine Learning Research

Volumn 13, Issue , 2012, Pages 1839-1864

  Huang, Jian ^a   Zhang, Cun Hui ^b  

^a UNIVERSITY OF IOWA   (United States)

^b RUTGERS UNIVERSITY   (United States)

Author keywords

Generalized linear models; Oracle inequality; Penalized estimation; Selection consistency; Sparsity; Variable selection

Indexed keywords

GENERALIZED LINEAR MODEL; ORACLE INEQUALITY; PENALIZED ESTIMATION; SELECTION CONSISTENCY; SPARSITY; VARIABLE SELECTION;

LOGISTICS; REGRESSION ANALYSIS;

ESTIMATION;

EID: 84864948194     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (38)

1. P. J. Bickel, Y. Ritov, and A. Tsybakov. Simultaneous analysis of Lasso and Dantzig selector. Annals of Statistics, 37(4):1705-1732, 2009.
2. L. M. Bregman. The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Computational Mathematics and Mathematical Physics, 7:200-217, 1967.
3. F. Bunea, A. Tsybakov, and M. H. Wegkamp. Sparsity oracle inequalities for the Lasso. Electronic Journal of Statistics, 1:169-194, 2007.
4. E. J. Candes and T. Tao. Decoding by linear programming. IEEE Trans. on Information Theory, 51:4203-4215, 2005. (Pubitemid 41800353)
5. E. J. Candes and T. Tao. The dantzig selector: statistical estimation when p is much larger than n (with discussion). Annals of Statistics, 35:2313-2404, 2007.
6. S. Chen, D. L. Donoho, and M. A. Saunders. Atomic decomposition by basis pursuit. SIAM J. Sci. Comput., 20:33-61, 1998. (Pubitemid 128689501)
7. J. Fan and R. Li. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96:1348-1360, 2001. (Pubitemid 33695585)
8. E. Greenshtein and Y. Ritov. Persistence in high-dimensional linear predictor selection and the virtue of overparametrization. Bernoulli, 10:971-988, 2004. (Pubitemid 44242744)
9. J. Huang, S. Ma, and C.-H. Zhang. Adaptive lasso for sparse high-dimensional regression models. Statistica Sinica, 18:1603-1618, 2008.
10. D. R. Hunter and R. Li. Variable selection using mm algorithms. Annals of Statistics, 33:1617-1642, 2005. (Pubitemid 41423982)
11. V. Koltchinskii. The dantzig selector and sparsity oracle inequalities. Bernoulli, 15:799-828, 2009.
12. P. McCullagh and J. A. Nelder. Generalized Linear Models. Chapmann & Hall, 1989.
13. L. Meier and P. B̈uhlmann. Smoothing l1-penalized estimators for high-dimensional time-course data. Electronic Journal of Statistics, 1:597-615, 2007.
14. N. Meinshausen and P. B̈uhlmann. High-dimensional graphs and variable selection with the lasso. Annals of Statistics, 34:1436-1462, 2006. (Pubitemid 44231168)
15. N. Meinshausen and B. Yu. Lasso-type recovery of sparse representations for high-dimensional data. Annals of Statistics, 37:246-270, 2009.
16. S. Negahban, P. Ravikumar, M. J. Wainwright, and B. Yu. A unified framework for highdimensional analysis of m-estimators with decomposable regularizer. Technical Report arXiv:1010.2731, arXiv, 2010.
17. F. Nielsen and R. Nock. On the centroids of symmetrized bregman divergences. CoRR, abs/0711.3242, 2007.
18. P. Ravikumar, M. J. Wainwright, G. Raskutti, and B. Yu. Model selection in gaussian graphical models: High-dimensional consistency of l1-regularized mle. In Advances in Neural Information Processing Systems (NIPS), volume 21, 2008.
19. A. J. Rothman, P. J. Bickel, E. Levina, and J. Zhu. Sparse permutation invariant covariance estimation. Electronic Journal of Statistics, 2:494-515, 2008.
20. N. Sẗadler, P. B̈uhlmann, and S. van de Geer. l1-penalization for mixture regression models (with discussion). Test, 19(2):209-285, 2010.
21. T. Sun and C.-H. Zhang. Scaled sparse linear regression. Technical Report arXiv:1104.4595, arXiv, 2011.
22. R. Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 58:267-288, 1996.
23. R. Tibshirani and J. Taylor. The solution path of the generalized lasso. The Annals of Statistics, 39: 1335-1371, 2011.
24. J. A. Tropp. Just relax: convex programming methods for identifying sparse signals in noise. IEEE Transactions on Information Theory, 52:1030-1051, 2006. (Pubitemid 46444890)
25. S. van de Geer. The deterministic lasso. Technical Report 140, ETH Zurich, Switzerland, 2007.
26. S. van de Geer. High-dimensional generalized linear models and the lasso. Annals of Statistics, 36: 614-645, 2008.
27. S. van de Geer and P. B̈uhlmann. On the conditions used to prove oracle results for the lasso. Electronic Journal of Statistics, 3:1360-1392, 2009.
28. M. J. Wainwright. Sharp thresholds for noisy and high-dimensional recovery of sparsity using l1-constrained quadratic programming (lasso). IEEE Transactions on Information Theory, 55: 2183-2202, 2009.
29. F. Ye and C.-H. Zhang. Rate minimaxity of the lasso and dantzig selector for the lq loss in lr balls. Journal of Machine Learning Research, 11:3481-3502, 2010.
30. C.-H. Zhang. Least squares estimation and variable selection under minimax concave penalty. In Mathematisches Forschungsintitut Oberwolfach: Sparse Recovery Problems in High Dimensions, 3 2009.
31. C.-H. Zhang. Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 38:894-942, 2010a.
32. C.-H. Zhang and J. Huang. The sparsity and bias of the Lasso selection in high-dimensional linear regression. Annals of Statistics, 36(4):1567-1594, 2008.
33. T. Zhang. Analysis of multi-stage convex relaxation for sparse regularization. Journal of Machine Learning Research, 11:1087-1107, 2010b.
34. T. Zhang. Adaptive forward-backward greedy algorithm for learning sparse representations. IEEE Transactions on Information Theory, 57:4689-4708, 2011a.
35. T. Zhang. Multi-stage convex relaxation for feature selection. Technical Report arXiv:1106.0565, arXiv, 2011b.
36. P. Zhao and B. Yu. On model selection consistency of Lasso. Journal of Machine Learning Research, 7:2541-2567, 2006.
37. H. Zou. The adaptive lasso and its oracle properties. Journal of the American Statistical Association, 101:1418-1429, 2006.
38. H. Zou and R. Li. One-step sparse estimates in nonconcave penalized likelihood models. Annals of Statistics, 36(4):1509-1533, 2008.