Minimax sparse principal subspace estimation in high dimensions

Annals of Statistics

Volumn 41, Issue 6, 2013, Pages 2905-2947

  Vu, Vincent Q ^a   Lei, Jing ^b  

^a OHIO STATE UNIVERSITY   (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Highdimensional statistics; Minimax bounds; Principal components analysis; Random matrices; Sparsity; Subspace estimation

Indexed keywords

EID: 84891937960     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/13-AOS1151     Document Type: Article

References (37)

1. AMINI, A. A. and WAINWRIGHT, M. J. (2009). High-dimensional analysis of semidefinite relaxations for sparse principal components. Ann. Statist. 37 2877-2921. MR2541450
2. BHATIA, R. (1997). Matrix Analysis. Graduate Texts in Mathematics 169. Springer, New York. MR1477662
3. BIRNBAUM, A., JOHNSTONE, I.M., NADLER, B. and PAUL, D. (2013). Minimax bounds for sparse PCA with noisy high-dimensional data. Ann. Statist. 41 1055-1084. MR3113803
4. CHEN, X., ZOU, C. and COOK, R. D. (2010). Coordinate-independent sparse sufficient dimension reduction and variable selection. Ann. Statist. 38 3696-3723. MR2766865
5. CHIKUSE, Y. (2003). Statistics on Special Manifolds. Lecture Notes in Statistics 174. Springer, New York. MR1960435
6. D'ASPREMONT, A., EL GHAOUI, L., JORDAN, M. I. and LANCKRIET, G. R. G. (2007). A direct formulation for sparse PCA using semidefinite programming. SIAM Rev. 49 434-448 (electronic). MR2353806
7. q-error. Probab. Theory Related Fields 99 277303.
8. EDELMAN, A., ARIAS, T. A. and SMITH, S. T. (1999). The geometry of algorithms with orthogonality constraints. SIAM J. Matrix Anal. Appl. 20 303-353. MR1646856
9. GILBERT, E. N. (1952). A comparison of signalling alphabets. Bell System Technical Journal 31 504522.
10. HOTELLING, H. (1933). Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology 24 498520.
11. JOHNSTONE, I. M. and LU, A. Y. (2009). On consistency and sparsity for principal components analysis in high dimensions. J. Amer. Statist. Assoc. 104 682-693. MR2751448
12. JOLLIFFE, I. T., TRENDAFILOV, N. T. and UDDIN, M. (2003). A modified principal component technique based on the LASSO. J. Comput. Graph. Statist. 12 531-547. MR2002634
13. JOURNÉE, M., NESTEROV, Y., RICHTÁRIK, P. and SEPULCHRE, R. (2010). Generalized power method for sparse principal component analysis. J. Mach. Learn. Res. 11 517-553. MR2600619
14. LEDOUX, M. (2001). The Concentration of Measure Phenomenon. Mathematical Surveys and Monographs 89. Amer. Math. Soc., Providence, RI. MR1849347
15. LOUNICI, K. (2013). Sparse principal component analysis with missing observations. In High Dimensional Probability VI (C. Houdré, D. M. Mason, J. Rosínski and J. A. Wellner, eds.). Progress in Probability 66 327-356. Springer, Basel.
16. MA, Z. (2013). Sparse principal component analysis and iterative thresholding. Ann. Statist. 41 772-801. MR3099121
17. MASSART, P. (2007). Concentration Inequalities and Model Selection. Lecture Notes in Math. 1896. Springer, Berlin. MR2319879
18. MENDELSON, S. (2010). Empirical processes with a bounded ψ1 diameter. Geom. Funct. Anal. 20 988-1027. MR2729283
19. NADLER, B. (2008). Finite sample approximation results for principal component analysis: A matrix perturbation approach. Ann. Statist. 36 2791-2817. MR2485013
20. NEGAHBAN, S. N., RAVIKUMAR, P., WAINWRIGHT, M. J. and YU, B. (2012). A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers. Statist. Sci. 27 538-557. MR3025133
21. PAJOR, A. (1998). Metric entropy of the Grassmannmanifold. In Convex Geometric Analysis (Berkeley, CA, 1996). Mathematical Sciences Research Institute Publications 34 181-188. Cambridge Univ. Press, Cambridge. MR1665590
22. PAUL, D. (2007). Asymptotics of sample eigenstructure for a large dimensional spiked covariance model. Statist. Sinica 17 1617-1642. MR2399865
23. PEARSON, K. (1901). On lines and planes of closest fit to systems of points in space. Philosophical Magazine 2 559572.
24. RASKUTTI, G., WAINWRIGHT, M. J. and YU, B. (2011). Minimax rates of estimation for high-dimensional linear regression over ℓq balls. IEEE Trans. Inform. Theory 57 6976-6994. MR2882274
25. SHEN, H. and HUANG, J. Z. (2008). Sparse principal component analysis via regularized low rank matrix approximation. J. Multivariate Anal. 99 1015-1034. MR2419336
26. SHEN, D., SHEN, H. and MARRON, J. S. (2013). Consistency of sparse PCA in high dimension, low sample size contexts. J. Multivariate Anal. 115 317-333. MR3004561
27. STEWART, G. W. and SUN, J. G. (1990). Matrix Perturbation Theory. Academic Press, Boston, MA. MR1061154
28. TIBSHIRANI, R. (1996). Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B Stat. Methodol. 58 267-288. MR1379242
29. VAN DER VAART, A.W. and WELLNER, J. A. (1996). Weak Convergence and Empirical Processes. Springer, New York. MR1385671
30. VARŠAMOV, R. R. (1957). The evaluation of signals in codes with correction of errors. Dokl. Akad. Nauk SSSR (N.S) 117 739-741. MR0095090
31. VU, V. Q. and LEI, J. (2012a). Minimax rates of estimation for sparse PCA in high dimensions. In Proceedings of the 15th International Conference on Artificial Intelligence and Statistics (AISTATS). JMLR Workshop and Conference Proceedings Volume 22.
32. VU, V. Q. and LEI, J. (2012b). Squared-norm empirical process in Banach space. Available at arXiv:1312.1005.
33. WITTEN, D.M., TIBSHIRANI, R. and HASTIE, T. (2009). A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. Biostatistics 10 515534.
34. YU, B. (1997). Assouad, Fano, and Le Cam. In Festschrift for Lucien Le Cam (D. Pollard, E. Torgersen and G. L. Yang, eds.) 423-435. Springer, New York.
35. YUAN, M. and LIN, Y. (2006). Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B Stat. Methodol. 68 49-67. MR2212574
36. ZHAO, P., ROCHA, G. and YU, B. (2009). The composite absolute penalties family for grouped and hierarchical variable selection. Ann. Statist. 37 3468-3497. MR2549566
37. ZOU, H., HASTIE, T. and TIBSHIRANI, R. (2006). Sparse principal component analysis. J. Comput. Graph. Statist. 15 265-286. MR2252527