A comparison of the lasso and marginal regression

Journal of Machine Learning Research

Volumn 13, Issue , 2012, Pages 2107-2143

  Genovese, Christopher R ^a   Jin, Jiashun ^a   Wasserman, Larry ^a   Yao, Zhigang ^b  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b UNIVERSITY OF PITTSBURGH   (United States)

Author keywords

High dimensional regression; Lasso; Phase diagram; Regularization

Indexed keywords

COMPUTATIONAL ADVANTAGES; COVARIATES; DEPENDENT VARIABLES; HIGH PROBABILITY; HIGH-DIMENSIONAL; HIGH-DIMENSIONAL PROBLEMS; LASSO; NOISE FREE CASE; NUMBER OF DATUM; PARAMETER SPACES; RANDOM COEFFICIENTS; RATES OF CONVERGENCE; REGRESSION PROBLEM; REGULARIZATION; STATISTICAL PERFORMANCE; THEORETICAL RESULT;

ALGORITHMS; PHASE DIAGRAMS; RECOVERY;

REGRESSION ANALYSIS;

EID: 84864959435     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (31)

1. P. Bühlmann, M. Kalisch, and M. H. Maathuis. Variable selection in high-dimensional linear models: Partially faithful distributions and the PC-simple algorith. Biometrika, 97:261-278, 2009.
2. T. Cai, L. Wang, and G. Xu. Shifting inequality and recovery of sparse signals. IEEE Transactions on Signal Processing, 59(3):1300-1308, 2010.
3. E. J. Candès and Y. Plan. Near-ideal model selection by l1 minimization. The Annals of Statistics, 37:2145-2177, 2009.
4. E. J. Candés and T. Tao. The Dantzig selector: Statistical estimation when p is much larger than n. The Annals of Statistics, 35:2313-2351, 2007.
5. S. Chen, D. Donoho, and M. Saunders. Atomic decomposition by basis pursuit. SIAM Journal on Scientific Computing, 20(1):33-61, 1998. (Pubitemid 128689501)
6. D. Donoho. For most large underdetermined systems of equations, the minimal l1-norm nearsolution approximates the sparsest near-solution. Communications on Pure and Applied Mathematics, 59(7):907-934, 2006.
7. D. Donoho and M. Elad. Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization. Proceedings of the National Academy of Sciences of the United States of America, 100(5):2197-2202, 2003. (Pubitemid 36297476)
8. D. Donoho and X. Huo. Uncertainty principles and ideal atomic decomposition. IEEE Transactions on Information Theory, 47(7):2845-2862, 2001. (Pubitemid 33053488)
9. D. Donoho and J. Jin. Higher criticism for detecting sparse heterogeneous mixtures. The Annals of Statistics, 32(3):962-994, 2004. (Pubitemid 41250290)
10. B. Efron, R. Tibshirani, J. Storey, and V. Tusher. Empirical Bayes analysis of a microarray experiment. Journal of the American Statistical Association, 96:1151-1160, 2001. (Pubitemid 33695570)
11. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani. Least angle regression. The Annals of Statistics, 32(2):407-499, 2004. (Pubitemid 41250302)
12. J. Fan and J. Lv. Sure independence screening for ultrahigh dimensional feature space. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70(5):849-911, 2008.
13. J.J. Fuchs. Recovery of exact sparse representations in the presence of noise. IEEE Transactions on Information Theory, 51(10):3601-3608, 2005. (Pubitemid 41448600)
14. P. Ji and J. Jin. UPS delivers optimal phase diagram in high dimensional variable selection. The Annals of Statistics, 40(1):73-103, 2012.
15. J. Jin. Proportion of nonzero normal means: Oracle equivalence and uniformly consistent estimators. Journal of the Royal Statistical Society: Series B (StatisticalMethodology), 70(3):461-493, 2007.
16. K. Knight and W. J. Fu. Asymptotics for lasso-type estimators. The Annals of Statistics, 28:1356-1378, 2000. (Pubitemid 33244917)
17. N. Meinshausen and P. Bühlmann. High-dimensional graphs and variable selection with the lasso. The Annals of Statistics, 34(3):1436-1462, 2006. (Pubitemid 44231168)
18. N. Meinshausen and J. Rice. Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses. The Annals of Statistics, 34(1):373-393, 2006. (Pubitemid 43836805)
19. P. Ravikumar. Personal Communication, 2007.
20. J. M. Robins, R. Scheines, P. Spirtes, and L. Wasserman. Uniform consistency in causal inference. Biometrika, 90(3):491-515, 2003. (Pubitemid 39047137)
21. G. R. Shorack and J. A. Wellner. Empirical Processes with Applications to Statistics. John Wiley & Sons, NY, 1986.
22. P. Spirtes, C. Glymour, and R. Scheines. Causation, Prediction, and Search (Lecture Notes in Statistics). Springer-Verlag, NY, 1993.
23. T. Sun and C.-H. Zhang. Scaled sparse linear regression. 2011. Manuscript available at http://arxiv.org/abs/1104.4595.
24. R. Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 58(1):267-288, 1996.
25. J. Tropp. Greed is good: Algorithic results for sparse approximation. IEEE Transactions on Information Theory, 50(10):2231-2242, 2004.
26. R. Vershynin. Introduction to the Non-asymptotic Analysis of Random Matrices. Lecture notes, Department of Mathematics, University of Michigan, 2010. Available electronically via wwwpersonal. umich.edu/eromanv/teaching/2006- 07/280/course.html.
27. M. Wainwright. Sharp Threshold for High-dimensional and Noisy Recovery of Sparsity. Technical report, Department of Statistics, University of Berkeley, 2006.
28. L. Wasserman. All of Nonparametric Statistics. Springer Texts in Statistics. Springer, New York, 2006.
29. L. Wasserman and K. Roeder. High-dimensional variable selection. The Annals of Statistics, 37(5): 2178-2201, 2009.
30. P. Zhao and B. Yu. On model selection consistency of lasso. Journal ofMachine Learning Research, 7:2541-2563, 2006. (Pubitemid 44866738)
31. H. Zou. The adaptive lasso and its oracle properties. Journal of the American Statistical Association, 101(476):1418-1429, 2006.