Confidence intervals and hypothesis testing for high-dimensional regression

Journal of Machine Learning Research

Volumn 15, Issue , 2014, Pages 2869-2909

  Javanmard, Adel ^a   Montanari, Andrea ^a  

^a Stanford University   (United States)

Author keywords

Bias of an estimator; Confidence intervals; High dimensional models; Hypothesis testing; LASSO

Indexed keywords

ALGORITHMS; PROBABILITY DISTRIBUTIONS; STATISTICAL TESTS; THROUGHPUT; UNCERTAINTY ANALYSIS;

BIAS OF AN ESTIMATOR; CONFIDENCE INTERVAL; HIGH-DIMENSIONAL MODELS; HYPOTHESIS TESTING; LASSO;

PARAMETER ESTIMATION;

EID: 84919709419     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (50)

1. M. Bayati, M. A. Erdogdu, and A. Montanari. Estimating Lasso risk and noise level. In Advances in Neural Information Processing Systems, pages 944-952, 2013.
2. A. Belloni and V. Chernozhukov. Least squares after model selection in high-dimensional sparse models. Bernoulli, 19(2):521-547, 2013.
3. A. Belloni, V. Chernozhukov, and Y. Wei. Honest confidence regions for a regression parameter in logistic regression with a large number of controls. arXiv Preprint arXiv:1304.3969, 2013.
4. A. Belloni, V. Chernozhukov, and C. Hansen. Inference on treatment effects after selection amongst high-dimensional controls. arXiv Preprint arXiv:1201.0224, The Review of Economic Studies, 2014.
5. P. J. Bickel, Y. Ritov, and A. B. Tsybakov. Simultaneous analysis of Lasso and Dantzig selector. The Annals of Statistics, 37:1705-1732, 2009.
6. P. Bühlmann. Statistical significance in high-dimensional linear models. Bernoulli, 19(4): 1212-1242, 2013.
7. P. Bühlmann and S. van de Geer. Statistics for High-Dimensional Data. Springer-Verlag, 2011.
8. P. Bühlmann, M. Kalisch, and L. Meier. High-dimensional statistics with a view toward applications in biology. Annual Review of Statistics and Its Application, 1(1):255-278, 2014.
9. E. Candès and T. Tao. The Dantzig selector: statistical estimation when p is much larger than n. The Annals of Statistics, 35:2313-2351, 2007.
10. 1 minimization. The Annals of Statistics, 37(5A):2145-2177, 2009.
11. E. J. Candès and T. Tao. Decoding by linear programming. IEEE Trans. on Inform. Theory, 51:4203-4215, 2005.
12. S. S. Chen and D. L. Donoho. Examples of basis pursuit. In Proceedings of Wavelet Applications in Signal and Image Processing III, San Diego, CA, 1995.
13. R. Dezeure and P. Bühlmann. Private Communication, 2013.
14. L. H. Dicker. Residual variance and the signal-to-noise ratio in high-dimensional linear models. arXiv Preprint arXiv:1209.0012, 2012.
15. D. L. Donoho and X. Huo. Uncertainty principles and ideal atomic decomposition. IEEE Transactions on Information Theory, 47(7):2845-2862, 2001.
16. B. Efron. Large-Scale Inference: Empirical Bayes Methods for Estimation, Testing, and Prediction, volume 1. Cambridge University Press, 2010.
17. J. Fan and R. Li. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456):1348-1360, 2001.
18. J. Fan and J. Lv. Sure independence screening for ultrahigh dimensional feature space. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70(5):849-911, 2008.
19. J. Fan, R. Samworth, and Y. Wu. Ultrahigh dimensional feature selection: beyond the linear model. The Journal of Machine Learning Research, 10:2013-2038, 2009.
20. J. Fan, S. Guo, and N. Hao. Variance estimation using refitted cross-validation in ultrahigh dimensional regression. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 74(1):37-65, 2012.
21. J. Friedman, T. Hastie, and R. Tibshirani. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1):1-22, 2010. URL http://www.jstatsoft.org/v33/i01/.
22. E. Greenshtein and Y. Ritov. Persistence in high-dimensional predictor selection and the virtue of over-parametrization. Bernoulli, 10:971-988, 2004.
23. A. Javanmard and A. Montanari. Confidence intervals and hypothesis testing for high-dimensional statistical models. In Advances in Neural Information Processing Systems, pages 1187-1195, 2013a.
24. A. Javanmard and A. Montanari. Hypothesis testing in high-dimensional regression under the gaussian random design model: Asymptotic theory. arXiv Preprint arXiv:1301.4240, (To appear in IEEE Transactions on Information Theory ), 2013b.
25. A. Javanmard and A. Montanari. Nearly optimal sample size in hypothesis testing for high-dimensional regression. In 51st Annual Allerton Conference, pages 1427-1434, Monticello, IL, June 2013c.
26. Y. Koren, R. Bell, and C. Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8):30-37, August 2009.
27. E. L. Lehmann and J. P. Romano. Testing Statistical Hypotheses. Springer, 2005.
28. R. Lockhart, J. Taylor, R. J. Tibshirani, and R. Tibshirani. A significance test for the Lasso. Annals of Statistics, 42(2):413-468, 2014.
29. M. Lustig, D. L. Donoho, J. M. Santos, and J. M. Pauly. Compressed sensing MRI. IEEE Signal Processing Magazine, 25:72-82, 2008.
30. S. G. Mallat and Z. Zhang. Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing, 41(12):3397-3415, 1993.
31. N. Meinshausen and P. Bühlmann. High-dimensional graphs and variable selection with the Lasso. The Annals of Statistics, 34:1436-1462, 2006.
32. N. Meinshausen and P. Bühlmann. Stability selection. J. R. Statist. Soc. B, 72:417-473, 2010.
33. N. Meinshausen, L. Meier, and P. Bühlmann. p-values for high-dimensional regression. Journal of the American Statistical Association, 104(488):1671-1681, 2009.
34. J. Minnier, L. Tian, and T. Cai. A perturbation method for inference on regularized regression estimates. Journal of the American Statistical Association, 106(496), 2011.
35. J. Peng, J. Zhu, A. Bergamaschi, W. Han, D.-Y. Noh, J. R. Pollack, and P. Wang. Regularized multivariate regression for identifying master predictors with application to integrative genomics study of breast cancer. The Annals of Applied Statistics, 4(1):53-77, 2010.
36. S. Reid, R. Tibshirani, and J. Friedman. A study of error variance estimation in Lasso regression. arXiv Preprint arXiv:1311.5274, 2013.
37. Z. Ren, T. Sun, C.-H. Zhang, and H. H. Zhou. Asymptotic normality and optimalities in estimation of large gaussian graphical model. arXiv Preprint arXiv:1309.6024, 2013.
38. M. Rudelson and Z. Shuheng. Reconstruction from anisotropic random measurements. IEEE Transactions on Information Theory, 59(6):3434-3447, 2013.
39. 1-penalization for mixture regression models (with discussion). Test, 19(2):209-256, 2010.
40. T. Sun and C.-H. Zhang. Scaled sparse linear regression. Biometrika, 99(4):879-898, 2012.
41. R. Tibshirani. Regression shrinkage and selection with the Lasso. J. Royal. Statist. Soc B, 58:267-288, 1996.
42. S. van de Geer and P. Bühlmann. On the conditions used to prove oracle results for the Lasso. Electron. J. Statist., 3:1360-1392, 2009.
43. S. van de Geer, P. Bühlmann, Y. Ritov, and R. Dezeure. On asymptotically optimal confidence regions and tests for high-dimensional models. arXiv Preprint arXiv:1303.0518, (To appear in Annals of Statistics ), 2014.
44. A. W. Van der Vaart. Asymptotic Statistics, volume 3. Cambridge university press, 2000.
45. R. Vershynin. Introduction to the non-asymptotic analysis of random matrices. In Y.C. Eldar and G. Kutyniok, editors, Compressed Sensing: Theory and Applications, pages 210-268. Cambridge University Press, 2012.
46. 1-constrained quadratic programming. IEEE Trans. on Inform. Theory, 55:2183-2202, 2009.
47. L. Wasserman. All of Statistics: A Concise Course in Statistical Inference. Springer Verlag, 2004.
48. C-H. Zhang. Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 38(2):894-942, 2010.
49. C.-H. Zhang and S. S. Zhang. Confidence intervals for low-dimensional parameters in highdimensional linear models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 76:217-242, 2014.
50. P. Zhao and B. Yu. On model selection consistency of Lasso. The Journal of Machine Learning Research, 7:2541-2563, 2006.