Sparse estimation of large covariance matrices via a nested Lasso penalty

Annals of Applied Statistics

Volumn 2, Issue 1, 2008, Pages 245-263

  Levina, Elizaveta ^a   Rothman, Adam ^a   Zhu, Ji ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Cholesky decomposition; Covariance matrix; High dimension low sample size; Large p small n; Lasso; Sparsity

Indexed keywords

EID: 62349114549     PISSN: 19326157     EISSN: 19417330     Source Type: Journal    
DOI: 10.1214/07-AoAs139     Document Type: Article

References (34)

1. Adam, B., Qu, Y., Davis, J., Ward, M., Clements, M., Cazares, L., Semmes, O., Schellhammer, P., Yasui, Y., Feng, Z. and Wright, G. (2002). Serum protein fingerprinting coupled with a pattern-matching algorithm distinguishes prostate cancer from benign prostate hyperplasia and healthy men., Cancer Research 62 3609-3614.
2. Anderson, T. W. (1958)., An Introduction to Multivariate Statistical Analysis. Wiley, New York. Mathematical Reviews (MathSciNet): MR91588
3. Banerjee, O., d'Aspremont, A. and El Ghaoui, L. (2006). Sparse covariance selection via robust maximum likelihood estimation. In, Proceedings of ICML.
4. Bickel, P. J. and Levina, E. (2004). Some theory for Fisher's linear discriminant function, "naive Bayes," and some alternatives when there are many more variables than observations., Bernoulli 10 989-1010. Mathematical Reviews (MathSciNet): MR2108040 Digital Object Identifier: doi:10.3150/bj/1106314847 Project Euclid: euclid.bj/1106314847
5. Bickel, P. J. and Levina, E. (2007). Regularized estimation of large covariance matrices., Ann. Statist. To appear. Mathematical Reviews (MathSciNet): MR2387969 Zentralblatt MATH: 1132.62040 Digital Object Identifier: doi:10.1214/009053607000000758 Project Euclid: euclid.aos/1201877299
6. Dempster, A. (1972). Covariance selection., Biometrics 28 157-175.
7. Dey, D. K. and Srinivasan, C. (1985). Estimation of a covariance matrix under Stein's loss., Ann. Statist. 13 1581-1591. Mathematical Reviews (MathSciNet): MR811511 Zentralblatt MATH: 0582.62042 Digital Object Identifier: doi:10.1214/aos/1176349756 Project Euclid: euclid.aos/1176349756
8. Diggle, P. and Verbyla, A. (1998). Nonparametric estimation of covariance structure in longitudinal data., Biometrics 54 401-415.
9. Djavan, B., Zlotta, A., Kratzik, C., Remzi, M., Seitz, C., Schulman, C. and Marberger, M. (1999). Psa, psa density, psa density of transition zone, free/total psa ratio, and psa velocity for early detection of prostate cancer in men with serum psa 2.5 to 4.0 ng/ml., Urology 54 517-522.
10. Fan, J., Fan, Y. and Lv, J. (2008). High dimensional covariance matrix estimation using a factor model., J. Econometrics. To appear. Mathematical Reviews (MathSciNet): MR2472991 Digital Object Identifier: doi:10.1016/j.jeconom.2008.09.017
11. Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties., J. Amer. Statist. Assoc. 96 1348-1360. Mathematical Reviews (MathSciNet): MR1946581 Zentralblatt MATH: 1073.62547 Digital Object Identifier: doi:10.1198/016214501753382273 JSTOR: links.jstor.org
12. Friedman, J. (1989). Regularized discriminant analysis., J. Amer. Statist. Assoc. 84 165-175. Mathematical Reviews (MathSciNet): MR999675 Digital Object Identifier: doi:10.2307/2289860 JSTOR: links.jstor.org
13. Friedman, J., Hastie, T., Höfling, H. G. and Tibshirani, R. (2007). Pathwise coordinate optimization., Ann. Appl. Statist. 1 302-332. Mathematical Reviews (MathSciNet): MR2415737 Zentralblatt MATH: 05226935 Digital Object Identifier: doi:10.1214/07-AOAS131 Project Euclid: euclid.aoas/1196438020
14. Fu, W. (1998). Penalized regressions: The bridge versus the lasso., J. Comput. Graph. Statist. 7 397-416. Mathematical Reviews (MathSciNet): MR1646710 Digital Object Identifier: doi:10.2307/1390712 JSTOR: links.jstor.org
15. Furrer, R. and Bengtsson, T. (2007). Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants., J. Multivariate Anal. 98 227-255. Mathematical Reviews (MathSciNet): MR2301751 Digital Object Identifier: doi:10.1016/j.jmva.2006.08.003
16. Haff, L. R. (1980). Empirical Bayes estimation of the multivariate normal covariance matrix., Ann. Statist. 8 586-597. Mathematical Reviews (MathSciNet): MR568722 Zentralblatt MATH: 0441.62045 Digital Object Identifier: doi:10.1214/aos/1176345010 Project Euclid: euclid.aos/1176345010
17. Hastie, T., Tibshirani, R. and Friedman, J. (2001)., The Elements of Statistical Learning. Springer, Berlin. Mathematical Reviews (MathSciNet): MR1851606
18. Hoerl, A. E. and Kennard, R. W. (1970). Ridge regression: Biased estimation for nonorthogonal problems., Technometrics 12 55-67.
19. Huang, J., Liu, N., Pourahmadi, M. and Liu, L. (2006). Covariance matrix selection and estimation via penalised normal likelihood., Biometrika 93 85-98. Mathematical Reviews (MathSciNet): MR2277742 Zentralblatt MATH: 1152.62346 Digital Object Identifier: doi:10.1093/biomet/93.1.85
20. Johnstone, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis., Ann. Statist. 29 295-327. Mathematical Reviews (MathSciNet): MR1863961 Zentralblatt MATH: 1016.62078 Digital Object Identifier: doi:10.1214/aos/1009210544 Project Euclid: euclid.aos/1009210544
21. Johnstone, I. M. and Lu, A. Y. (2007). Sparse principal components analysis., J. Amer. Statist. Assoc. To appear.
22. Ledoit, O. and Wolf, M. (2003). A well-conditioned estimator for large-dimensional covariance matrices., J. Multivariate Anal. 88 365-411. Mathematical Reviews (MathSciNet): MR2026339 Zentralblatt MATH: 1032.62050 Digital Object Identifier: doi:10.1016/S0047-259X(03)00096-4
23. Mardia, K. V., Kent, J. T. and Bibby, J. M. (1979)., Multivariate Analysis. Academic Press, New York. Mathematical Reviews (MathSciNet): MR560319
24. Pannek, J. and Partin, A. (1998). The role of psa and percent free psa for staging and prognosis prediction in clinically localized prostate cancer., Semin. Urol. Oncol. 16 100-105.
25. Pourahmadi, M. (1999). Joint mean-covariance models with applications to longitudinal data: Unconstrained parameterisation., Biometrika 86 677-690. Mathematical Reviews (MathSciNet): MR1723786 Zentralblatt MATH: 0949.62066 Digital Object Identifier: doi:10.1093/biomet/86.3.677 JSTOR: links.jstor.org
26. Smith, M. and Kohn, R. (2002). Parsimonious covariance matrix estimation for longitudinal data., J. Amer. Statist. Assoc. 97 1141-1153. Mathematical Reviews (MathSciNet): MR1951266 Zentralblatt MATH: 1041.62044 Digital Object Identifier: doi:10.1198/016214502388618942 JSTOR: links.jstor.org
27. Stamey, T., Johnstone, I., McNeal, J., Lu, A. and Yemoto, C. (2002). Preoperative serum prostate specific antigen levels between 2 and 22 ng/ml correlate poorly with post-radical prostatectomy cancer morphology: Prostate specific antigen cure rates appear constant between 2 and 9 ng/ml., J. Urol. 167 103-111.
28. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso., J. Roy. Statist. Soc. Ser. B 58 267-288. Mathematical Reviews (MathSciNet): MR1379242 JSTOR: links.jstor.org
29. Tibshirani, R., Hastie, T., Narasimhan, B. and Chu, G. (2003). Class prediction by nearest shrunken centroids, with applications to DNA microarrays., Statist. Sci. 18 104-117. Mathematical Reviews (MathSciNet): MR1997067 Digital Object Identifier: doi:10.1214/ss/1056397488 Project Euclid: euclid.ss/1056397488
30. Tibshirani, R., Saunders, M., Rosset, S., Zhu, J. and Knight, K. (2005). Sparsity and smoothness via the fused lasso., J. Roy. Statist. Soc. Ser. B 67 91-108. Mathematical Reviews (MathSciNet): MR2136641 Zentralblatt MATH: 1060.62049 Digital Object Identifier: doi:10.1111/j.1467-9868.2005.00490.x JSTOR: links.jstor.org
31. Wong, F., Carter, C. and Kohn, R. (2003). Efficient estimation of covariance selection models., Biometrika 90 809-830. Mathematical Reviews (MathSciNet): MR2024759 Digital Object Identifier: doi:10.1093/biomet/90.4.809
32. Wu, W. B. and Pourahmadi, M. (2003). Nonparametric estimation of large covariance matrices of longitudinal data., Biometrika 90 831-844. Mathematical Reviews (MathSciNet): MR2024760 Digital Object Identifier: doi:10.1093/biomet/90.4.831
33. Yuan, M. and Lin, Y. (2007). Model selection and estimation in the Gaussian graphical model., Biometrika 94 19-35. Mathematical Reviews (MathSciNet): MR2367824 Zentralblatt MATH: 1142.62408 Digital Object Identifier: doi:10.1093/biomet/asm018
34. Zhao, P., Rocha, G. and Yu, B. (2006). Grouped and hierarchical model selection through composite absolute penalties. Technical Report 703, Dept. Statistics, UC, Berkeley.