Covariance regularization by thresholding

Annals of Statistics

Volumn 36, Issue 6, 2008, Pages 2577-2604

  Bickel, Peter J ^a   Levina, Elizaveta ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Covariance estimation; High dimension low sample size.; Large p small n; Regularization; Sparsity; Thresholding

Indexed keywords

EID: 62349112885     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/08-AOS600     Document Type: Article

References (36)

1. ABRAMOVICH, F., BENJAMINI, Y., DONOHO, D. and JOHNSTONE, I. (2006). Adapting to unknown sparsity by controlling the false discovery rate. Ann. Statist. 34 584 - 653. MR2281879
2. BICKEL, P. J. and LEVINA, E. (2004). Some theory for Fisher's linear discriminant function, "naive Bayes," and some alternatives when there are many more variables than observations. Bernoulli 10 989-1010. MR2108040
3. BICKEL, P. J. and LEVINA, E. (2008). Regularized estimation of large covariance matrices. Ann. Statist. 36 199-227. MR2387969
4. BICKEL, P. J., RITOV, Y. and ZAKAI, A. (2006). Some theory for generalized boosting algorithms. J. Mach. Learn. Res. 7 705-732. MR2274384
5. D'ASPREMONT, A., BANERJEE, O. and EL GHAOUI, L. (2007). First-order methods for sparse covariance selection. SIAM J. Matrix Anal. Appl. 30 56-66. MR2399568
6. D'ASPREMONT, A., EL GHAOUI, L., JORDAN, M. I. and LANCKRIET, G. R. G. (2007). A direct formulation for sparse PCA using semidefinite programming. SIAM Rev. 49 434- 448. MR2353806
7. DEY, D. K. and SRINIVASAN, C. (1985). Estimation of a covariance matrix under Stein's loss. Ann. Statist. 13 1581-1591. MR0811511
8. DONOHO, D. L. and JOHNSTONE, I. M. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika 81 425-455. MR1311089
9. DUDOIT, S. and VAN DER LAAN, M. J. (2005). Asymptotics of cross-validated risk estimation in estimator selection and performance assessment. Statist. Methodol. 2 131-154. MR2161394
10. EL KAROUI, N. (2007a). Operator norm consistent estimation of large-dimensional sparse covariance matrices. Ann. Statist. To appear.
11. EL KAROUI, N. (2007b). Spectrum estimation for large-dimensional covariance matrices using random matrix theory. Ann. Statist. To appear.
12. EL KAROUI, N. (2007C). Tracy-Widom limit for the largest eigenvalue of a large class of complex sample covariance matrices. Ann. Probab. 35 663-714. MR2308592
13. FAN, J., FAN, Y. and Lv, J. (2008). High-dimensional covariance matrix estimation using a factor model. J. Econometrics. To appear.
14. FAN, J., FENG, Y. and Wu, Y. (2007). Network exploration via the adaptive LASSO and SCAD penalties. Unpublished manuscript.
15. FAN, J. and LI, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348-1360. MR 1946581
16. FRIEDMAN, J., HASTIE, T. and TIBSHIRANI, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics 9 432-441.
17. FURRER, R. and BENGTSSON, T. (2007). Estimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants. J. Multivariate Anal. 98 227-255. MR2301751
18. GOLUB, G. H. and VAN LOAN, C. F. (1989). Matrix Computations, 2nd ed. Johns Hopkins Univ. Press, Baltimore, Ml). MR1002570
19. GYORFI, L., KOHLER, M., KRZYZAK, A. and WALK, H. (2002). A Distribution-Free Theory of Nonparametric Regression. Springer, New York. MR 1920390
20. HAFF, L. R. (1980). Empirical Bayes estimation of the multivariate normal covariance matrix. Ann. Statist. 8 586-597. MR0568722
21. HUANG, J., LIu, N., POURAHMADI, M. and LIu, L. (2006). Covariance matrix selection and estimation via penalised normal likelihood. Biometrika 93 85-98. MR2277742
22. JOHNSTONE, I. and SILVERMAN, B. (2005). Empirical Bayes selection of wavelet thresholds. Ann. Statist. 33 1700-1752. MR2166560
23. JOHNSTONE, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295-327. MR1863961
24. JOHNSTONE, I. M. and LU, A. Y. (2004). Sparse principal components analysis. Unpublished manuscript.
25. LAM, C. and FAN, J. (2007). Sparsistency and rates of convergence in large covariance matrices estimation. Manuscript.
26. LEDOIT, O. and WOLF, M. (2003). A well-conditioned estimator for large-dimensional co- variance matrices. J. Multivariate Anal. 88 365-411. MR2026339
27. LEDOUX, M. (2001). The Concentration of Measure Phenomenon. Amer. Math. Soc., Providence, RI. MR 1849347
28. LEVINA, E., ROTHMAN, A. J. and ZHU, J. (2008). Sparse estimation of large covariance matrices via a nested Lasso penalty. Ann. Appl. Statist. 2 245-263.
29. MARČENKO, V. A. and PASTUR, L. A. (1967). Distributions of eigenvalues of some sets of random matrices. Math. USSR-Sb 1 507-536.
30. PAUL, D. (2007). Asymptotics of the leading sample eigenvalues for a spiked covariance model. Statist. Sinica 17 1617-1642. MR2399865
31. ROTHMAN, A. J., BICKEL, P. J., LEVINA, E. and ZHU, J. (2008). Sparse permutation invariant covariance estimation. Electron. J. Stat. 2 494-515.
32. SAULIS, L. and STATULEVIČIUS, V. A. (1991). Limit Theorems for Large Deviations. Kluwer, Dordrecht. MR 1171883
33. VAN DER VAART, A. W. and WELLNER, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. Springer, New York. MR1385671
34. Wu, W. B. and POURAHMADI, M. (2003). Nonparametric estimation of large covariance matrices of longitudinal data. Biometrika 90 831-844. MR2024760
35. YUAN, M. and LIN, Y. (2007). Model selection and estimation in the Gaussian graphical model. Biometrika 94 19-35. MR2367824
36. Zou, H., HASTIE, T. and TIBSHIRANI, R. (2006). Sparse principal components analysis. J. Comput. Graph. Statist. 15 265-286. MR2252527