Correlated variables in regression: Clustering and sparse estimation

Journal of Statistical Planning and Inference

Volumn 143, Issue 11, 2013, Pages 1835-1858

  Bühlmann, Peter ^a   Rütimann, Philipp ^a   van de Geer, Sara ^a   Zhang, Cun Hui ^b  

^a Rutgers University ^*

^b RUTGERS UNIVERSITY   (United States)

Author keywords

Canonical correlation; Group Lasso; Hierarchical clustering; High dimensional inference; Lasso; Oracle inequality; Variable screening; Variable selection

Indexed keywords

EID: 84883299225     PISSN: 03783758     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jspi.2013.05.019     Document Type: Article

References (34)

1. Anderson T. An Introduction to Multivariate Statistical Analysis 1984, Wiley. 2nd ed.
2. Baba K., Shibata R., Sibuya M. Partial correlation and conditional correlation as measures of conditional independence. Australian & New Zealand Journal of Statistics 2004, 46:657-664.
3. Balding D. A tutorial on statistical methods for population association studies. Nature Reviews Genetics 2007, 7:781-791.
4. Bickel P., Ritov Y., Tsybakov A. Simultaneous analysis of Lasso and Dantzig selector. Annals of Statistics 2009, 37:1705-1732.
5. Bondell H., Reich B. Simultaneous regression shrinkage, variable selection and clustering of predictors with OSCAR. Biometrics 2008, 64:115-123.
6. Bühlmann P., van de Geer S. Statistics for High-Dimensional Data: Methods, Theory and Applications 2011, Springer Verlag.
7. Bühlmann, P., Kalisch, M., Meier, L. High-dimensional statistics with a view towards applications in biology. Annual Review in Statistics, in press.
8. Carlson C., Eberle M., Rieder M., Yi Q., Kruglyak L., Nickerson D. Selecting a maximally informative set of single-nucleotide polymorphisms for association analyses using linkage disequilibrium. American Journal of Human Genetics 2004, 74:106-120.
9. Davidson K., Szarek S. Local operator theory, random matrices and Banach spaces. Handbook in Banach Spaces 2001, vol. I:317-366. Elsevier.
10. Dettling M., Bühlmann P. Finding predictive gene groups from microarray data. Journal of Multivariate Analysis 2004, 90:106-131.
11. Friedman J., Hastie T., Tibshirani R. Regularized paths for generalized linear models via coordinate descent. Journal of Statistical Software 2010, 33:1-22.
12. Hastie T., Tibshirani R., Botstein D., Brown P. Supervised harvesting of expression trees. Genome Biology 2001, 2:1-12.
13. Hastie T., Tibshirani R., Eisen M., Alizadeh A., Levy R., Staudt L., Chan W., Botstein D., Brown P. 'Gene shaving' as a method for identifying distinct sets of genes with similar expression patterns. Genome Biology 2000, 1:1-21.
14. Huang J., Ma S., Li H., Zhang C.-H. The sparse Laplacian shrinkage estimator for high-dimensional regression. Annals of Statistics 2011, 39:2021-2046.
15. Kendall M. A Course in Multivariate Analysis 1957, Griffin, London.
16. Meier L., van de Geer S., Bühlmann P. High-dimensional additive modeling. Annals of Statistics 2009, 37:3779-3821.
17. Meinshausen N. Relaxed Lasso. Computational Statistics & Data Analysis 2007, 52:374-393.
18. Meinshausen N., Bühlmann P. High-dimensional graphs and variable selection with the Lasso. Annals of Statistics 2006, 34:1436-1462.
19. Meinshausen N., Yu B. Lasso-type recovery of sparse representations for high-dimensional data. Annals of Statistics 2009, 37:246-270.
20. Segal M., Dahlquist K., Conklin B. Regression approaches for microarray data analysis. Journal of Computational Biology 2003, 10:961-980.
21. She Y. Sparse regression with exact clustering. Electronic Journal of Statistics 2010, 4:1055-1096.
22. Simon, N., Friedman, J., Hastie, T., Tibshirani, R., 2013. A sparse-group Lasso. Journal of Computational and Graphical Statistics, 22, 231-245.
23. Sun T., Zhang C.-H. Scaled sparse linear regression. Biometrika 2012, 99:879-898.
24. Tibshirani R. Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society, Series B 1996, 58:267-288.
25. Tolosi L., Lengauer T. Classification with correlated features: unreliability of feature ranking and solutions. Bioinformatics 2011, 27:1986-1994.
26. van de Geer, S., 2007. The deterministic Lasso. In: JSM Proceedings, 2007, 140. American Statistical Association.
27. van de Geer S., Bühlmann P. On the conditions used to prove oracle results for the Lasso. Electronic Journal of Statistics 2009, 3:1360-1392.
28. van de Geer, S., Lederer, J., 2012. The Lasso, correlated design, and improved oracle inequalities. In: IMS Collections 9 (From Probability to Statistics and Back: High-dimensional models and processes), 303-376.
29. Yuan M., Lin Y. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society, Series B 2006, 69:49-67.
30. Zhang C.-H., Huang J. The sparsity and bias of the Lasso selection in high-dimensional linear regression. Annals of Statistics 2008, 36:1567-1594.
31. Zhao P., Yu B. On model selection consistency of Lasso. Journal of Machine Learning Research 2006, 7:2541-2563.
32. Zou H. The adaptive Lasso and its oracle properties. Journal of the American Statistical Association 2006, 101:1418-1429.
33. Zou H., Hastie T. Regularization and variable selection via the Elastic Net. Journal of the Royal Statistical Society Series B 2005, 67:301-320.
34. Zou H., Li R. One-step sparse estimates in nonconcave penalized likelihood models (with discussion). Annals of Statistics 2008, 36:1509-1566.