Sparse principal component analysis and iterative thresholding

Annals of Statistics

Volumn 41, Issue 2, 2013, Pages 772-801

  Ma, Zongming ^a  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

Dimension reduction; High dimensional statistics; Principal component analysis; Principal subspace; Sparsity; Spiked covariance model; Thresholding

Indexed keywords

EID: 84877623616     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/13-AOS1097     Document Type: Article

References (36)

1. AMINI, A. A. and WAINWRIGHT, M. J. (2009). High-dimensional analysis of semidefinite relaxations for sparse principal components. Ann. Statist. 37 2877-2921. MR2541450.
2. ANDERSON, T. W. (1963). Asymptotic theory for principal component analysis. Ann. Math. Statist. 34 122-148. MR0145620.
3. D'ASPREMONT, A., EL GHAOUI, L., JORDAN, M. I. and LANCKRIET, G. R. G. (2007). A direct formulation for sparse PCA using semidefinite programming. SIAM Rev. 49 434- 448 (electronic). MR2353806.
4. DAVIS, C. and KAHAN, W. M. (1970). The rotation of eigenvectors by a perturbation. III. SIAM J. Numer. Anal. 7 1-46. MR0264450.
5. DONOHO, D. L. (1993). Unconditional bases are optimal bases for data compression and for statistical estimation. Appl. Comput. Harmon. Anal. 1 100-115. MR1256530.
6. FAN, J. and LI, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96 1348-1360. MR1946581.
7. GOLUB, G. H. and VAN LOAN, C. F. (1996). Matrix Computations, 3rd ed. Johns Hopkins Univ. Press, Baltimore, MD. MR1417720.
8. HOTELLING, H. (1933). Analysis of a complex of statistical variables into principal components. J. Educ. Psychol. 24 417-441, 498-520.
9. HOYLE, D. C. and RATTRAY, M. (2004). Principal-component-analysis eigenvalue spectra from data with symmetry-breaking structure. Phys. Rev. E (3) 69 026124.
10. JOHNSTONE, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295-327. MR1863961.
11. JOHNSTONE, I. M. and LU, A. Y. (2009). On consistency and sparsity for principal components analysis in high dimensions. J. Amer. Statist. Assoc. 104 682-693. MR2751448.
12. JOLLIFFE, I. T., TRENDAFILOV, N. T. and UDDIN, M. (2003). A modified principal component technique based on the LASSO. J. Comput. Graph. Statist. 12 531-547. MR2002634.
13. JUNG, S. and MARRON, J. S. (2009). PCA consistency in high dimension, low sample size context. Ann. Statist. 37 4104-4130. MR2572454.
14. LU, A. Y. (2002). Sparse principal component analysis for functional data. Ph.D. thesis, Stanford Univ., Stanford, CA. MR2703298.
15. MA, Z. (2013). Supplement to "Sparse principal component analysis and iterative thresholding." DOI:10.1214/13-AOS1097SUPP.
16. MALLAT, S. (2009). A Wavelet Tour of Signal Processing: The Sparse Way. Academic Press, New York.
17. NADLER, B. (2008). Finite sample approximation results for principal component analysis: A matrix perturbation approach. Ann. Statist. 36 2791-2817. MR2485013.
18. NADLER, B. (2009). Discussion of "On consistency and sparsity for principal components analysis in high dimensions, " by I. M. Johnstone and A. Y. Lu. J. Amer. Statist. Assoc. 104 694-697. MR2751449.
19. ONATSKI, A. (2012). Asymptotics of the principal components estimator of large factor models with weakly influential factors. J. Econometrics 168 244-258. MR2923766.
20. PAUL, D. (2005). Nonparametric estimation of principal components. Ph.D. thesis, Stanford Univ. MR2707156.
21. PAUL, D. (2007). Asymptotics of sample eigenstructure for a large dimensional spiked covariance model. Statist. Sinica 17 1617-1642. MR2399865.
22. PAUL, D. and JOHNSTONE, I. M. (2007). Augmented sparse principal component analysis for high dimensional data. Available at arXiv:1202.1242v1.
23. PEARSON, K. (1901). On lines and planes of closest fit to systems of points in space. Philos. Mag. Ser. 6 2 559-572.
24. RAMSAY, J. O. and SILVERMAN, B. W. (2005). Functional Data Analysis, 2nd ed. Springer, New York. MR2168993.
25. REIMANN, P., VAN DEN BROECK, C. and BEX, G. J. (1996). A Gaussian scenario for unsupervised learning. J. Phys. A 29 3521-3535.
26. SHEN, D., SHEN, H. and MARRON, J. S. (2011). Consistency of sparse PCA in high dimension, low sample size contexts.
27. SHEN, H. and HUANG, J. Z. (2008). Sparse principal component analysis via regularized low rank matrix approximation. J. Multivariate Anal. 99 1015-1034. MR2419336.
28. STEWART, G. W. and SUN, J. G. (1990). Matrix Perturbation Theory. Academic Press, Boston, MA. MR1061154.
29. TSAY, R. S. (2005). Analysis of Financial Time Series, 2nd ed. Wiley, Hoboken, NJ. MR2162112.
30. ULFARSSON, M. O. and SOLO, V. (2008). Sparse variable PCA using geodesic steepest descent. IEEE Trans. Signal Process. 56 5823-5832. MR2518261.
31. VARMUZA, K. and FILZMOSER, P. (2009). Introduction to Multivariate Statistical Analysis in Chemometrics. CRC Press, Boca Raton, FL.
32. WAX, M. and KAILATH, T. (1985). Detection of signals by information theoretic criteria. IEEE Trans. Acoust. Speech Signal Process. 33 387-392. MR0788604.
33. WEDIN, P.-Å. (1972). Perturbation bounds in connection with singular value decomposition. Nordisk Tidskr. Informationsbehandling (BIT) 12 99-111. MR0309968.
34. WITTEN, D. M., TIBSHIRANI, R. and HASTIE, T. (2009). A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. Biostatistics 10 515-534.
35. YUAN, X. T. and ZHANG, T. (2011). Truncated power method for sparse eigenvalue problems. Available at arXiv:1112.2679v1.
36. ZOU, H., HASTIE, T. and TIBSHIRANI, R. (2006). Sparse principal component analysis. J. Comput. Graph. Statist. 15 265-286. MR2252527.