Minimax bounds for sparse PCA with noisy high-dimensional data

Annals of Statistics

Volumn 41, Issue 3, 2013, Pages 1055-1084

  Birnbaum, Aharon ^a   Johnstone, Iain M ^b   Nadler, Boaz ^c   Paul, Debashis ^d  

^a HEBREW UNIVERSITY OF JERUSALEM   (Israel)

^b STANFORD UNIVERSITY   (United States)

^c WEIZMANN INSTITUTE OF SCIENCE   (Israel)

^d UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

High dimensional data; Minimax risk; Principal component analysis; Sparsity; Spiked covariance model

Indexed keywords

EID: 84879351019     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/12-AOS1014     Document Type: Article

References (37)

1. AMINI, A. A. and WAINWRIGHT, M. J. (2009). High-dimensional analysis of semidefinite relaxations for sparse principal components. Ann. Statist. 37 2877-2921. MR2541450
2. ANDERSON, T. W. (1963). Asymptotic theory for principal component analysis. Ann. Math. Statist. 34 122-148. MR0145620
3. BAIK, J. and SILVERSTEIN, J. W. (2006). Eigenvalues of large sample covariance matrices of spiked population models. J. Multivariate Anal. 97 1382-1408. MR2279680
4. BICKEL, P. J. and LEVINA, E. (2008a). Covariance regularization by thresholding. Ann. Statist. 36 2577-2604. MR2485008
5. BICKEL, P. J. and LEVINA, E. (2008b). Regularized estimation of large covariance matrices. Ann. Statist. 36 199-227. MR2387969
6. BIRGÉ, L. (2001). A new look at an old result: Fano's lemma. Technical report, Univ. Paris 6.
7. CAI, T. and LIU, W. (2011). Adaptive thresholding for sparse covariance matrix estimation. J. Amer. Statist. Assoc. 106 672-684. MR2847949
8. CAI, T. T., MA, Z. and WU, Y. (2012). Sparse PCA: Optimal rates and adaptive estimation. Technical report. Available at arXiv:1211.1309.
9. CAI, T. T., ZHANG, C.-H. and ZHOU, H. H. (2010). Optimal rates of convergence for covariance matrix estimation. Ann. Statist. 38 2118-2144. MR2676885
10. CAI, T. T. and ZHOU, H. H. (2012). Minimax esrimation of large covariance matrices under l1 norm. Statist. Sinica 22 1319-1378.
11. D'ASPREMONT, A., EL GHAOUI, L., JORDAN, M. I. and LANCKRIET, G. R. G. (2007). A direct formulation for sparse PCA using semidefinite programming. SIAM Rev. 49 434-448 (electronic). MR2353806
12. DAVIDSON, K. R. and SZAREK, S. J. (2001). Local operator theory, random matrices and Banach spaces. In Handbook of the Geometry of Banach Spaces, Vol. I (W. B. Johnson and J. Lindenstrauss, eds.) 317-366. North-Holland, Amsterdam. MR1863696
13. EL KAROUI, N. (2008). Operator norm consistent estimation of large-dimensional sparse covariance matrices. Ann. Statist. 36 2717-2756. MR2485011
14. JOHNSTONE, I. M. (2001). Chi-square oracle inequalities. In State of the Art in Probability and Statistics (Leiden, 1999) (M. de Gunst, C. Klaassen and A. van der Waart, eds.). Institute of Mathematical Statistics Lecture Notes - Monograph Series 36 399-418. IMS, Beachwood, OH. MR1836572
15. JOHNSTONE, I. M. and LU, A. Y. (2009). On consistency and sparsity for principal components analysis in high dimensions. J. Amer. Statist. Assoc. 104 682-693. MR2751448
16. JOLLIFFE, I. T. (2002). Principal Component Analysis. Springer, Berlin.
17. KATO, T. (1980). Perturbation Theory of Linear Operators. Springer, New York.
18. KRITCHMAN, S. andNADLER, B. (2008). Determining the number of components in a factor model from limited noisy data. Chemometrics and Intelligent Laboratory Systems 94 19-32.
19. LU, A. Y. (2002). Sparse principal components analysis for functional data. Ph.D. thesis, Stanford Univ., Stanford, CA.
20. MA, Z. (2011). Sparse principal component analysis and iterative thresholding. Technical report, Dept. Statistics, The Wharton School, Univ. Pennsylvania, Philadelphia, PA.
21. MUIRHEAD, R. J. (1982). Aspects of Multivariate Statistical Theory.Wiley, New York. MR0652932
22. NADLER, B. (2008). Finite sample approximation results for principal component analysis: A matrix perturbation approach. Ann. Statist. 36 2791-2817. MR2485013
23. NADLER, B. (2009). Discussion of "On consistency and sparsity for principal component analysis in high dimensions." J. Amer. Statist. Assoc. 104 694-697. MR2751449
24. ONATSKI, A. (2006). Determining the number of factors from empirical distribution of eigenvalues. Technical report, Columbia Univ.
25. PAUL, D. (2005). Nonparametric estimation of principal components. Ph.D. thesis, Stanford Univ. Stanford, CA.
26. PAUL, D. (2007). Asymptotics of sample eigenstructure for a large dimensional spiked covariance model. Statist. Sinica 17 1617-1642. MR2399865
27. PAUL, D. and JOHNSTONE, I. M. (2007). Augmented sparse principal component analysis for high dimensional data. Technical report, Univ. California, Davis. Available at arXiv:1202.1242.
28. ROTHMAN, A. J., LEVINA, E. and ZHU, J. (2009). Generalized thresholding of large covariance matrices. J. Amer. Statist. Assoc. 104 177-186. MR2504372
29. SHEN, H. and HUANG, J. Z. (2008). Sparse principal component analysis via regularized low rank matrix approximation. J. Multivariate Anal. 99 1015-1034. MR2419336
30. SHEN, D., SHEN, H. and MARRON, J. S. (2011). Consistency of sparse PCA in high dimension, low sample size contexts. Technical report. Available at http://arxiv.org/pdf/1104.4289v1.pdf.
31. TIPPING, M. E. and BISHOP, C. M. (1999). Probabilistic principal component analysis. J. R. Stat. Soc. Ser. B Stat. Methodol. 61 611-622. MR1707864
32. VAN TREES, H. L. (2002). Optimum Array Processing. Wiley, New York.
33. VU, V. Q. and LEI, J. (2012). Minimax rates of estimation for sparse PCA in high dimensions. Technical report. Available at http://arxiv.org/pdf/1202. 0786.pdf.
34. WITTEN, D.M., TIBSHIRANI, R. and HASTIE, T. (2009). A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. Biostatistics 10 515-534.
35. YANG, Y. and BARRON, A. (1999). Information-theoretic determination of minimax rates of convergence. Ann. Statist. 27 1564-1599. MR1742500
36. ZONG, C. (1999). Sphere Packings. Springer, New York. MR1707318
37. ZOU, H., HASTIE, T. and TIBSHIRANI, R. (2006). Sparse principal component analysis. J. Comput. Graph. Statist. 15 265-286. MR2252527