On consistency and sparsity for principal components analysis in high dimensions

Journal of the American Statistical Association

Volumn 104, Issue 486, 2009, Pages 682-693

  Johnstone, Iain M ^a   Lu, Arthur Yu ^b  

^a STANFORD UNIVERSITY   (United States)

^b Renaissance Technologies LLC   (United States)

Author keywords

Eigenvector estimation; Reduction of dimension; Regularization; Thresholding; Variable selection

Indexed keywords

EID: 66549088006     PISSN: 01621459     EISSN: None     Source Type: Journal    
DOI: 10.1198/jasa.2009.0121     Document Type: Article

References (37)

1. Amini, A. A., and Wainwright, M. J. (2009), "High-Dimensional Analysis of Semidefinite Relaxations for Sparse Principal Components," The Annals of Statistics.
2. Anderson, T. W. (1963), "Asymptotic Theory for Principal Component Analysis," Annals of Mathematical Statistics, 34, 122-148.
3. Biehl, M., and Mietzner, A. (1994), "Statistical Mechanics of Unsupervised Structure Recognition," Journal of Physics A: Mathematical and General, 27, 1885-1897.
4. Cherkassky, V., and Mulier, F. (1998), Learning from Data, New York: Wiley.
5. d'Aspremont, A., El Ghaoui, L., Jordan, M., and Lanckriet, G. (2007), "A Direct Formulation for Sparse PCA Using Semidefinite Programming," SIAM Review, 49, 434-448.
6. Dembo, A., and Zeitouni, O. (1993), Large Deviations Techniques and Applications, Boston: Jones and Bartlett.
7. Donoho, D. (1993), "Unconditional Bases Are Optimal Bases for Data Compression and Statistical Estimation," Applied and Computational Harmonic Analysis, 1, 100-115.
8. Donoho, D. L., Johnstone, I. M., Kerkyacharian, G., and Picard, D. (1995), "Wavelet Shrinkage: Asymptopia" (with discussion). Journal of the Royal Statistical Society, Ser. B, 57, 301-369.
9. Du, Q., and Fowler, J. E. (2008), "Low-Complexity Principal Component Analysis for Hyperspectral Image Compression," International Journal of High Performance Computing Applications, 22, 438-448.
10. Feng, G. C., Yuen, P. C., and Dai, D. Q. (2000), "Human Face Recognition Using PCA on Wavelet Subband," Journal of Electronic Imaging, 9, 226-233.
11. Fowler, J. E. (2008), "Compressive-Projection Principal Component Analysis for the Compression of Hyperspectral Signatures," in Data Compression Conference, 2008. DCC 2008, eds. J. A. Storer and M. W. Marcellin, Snowbird, UT: IEEE, pp. 83-92.
12. Geman, S. (1980), "A Limit Theorem for the Norm of Random Matrices," Annals of Probability, 8, 252-261.
13. Golub, G. H., and Van Loan, C. F. (1996), Matrix Computations (3rd ed.), Baltimore: Johns Hopkins University Press.
14. Hoyle, D. C., and Rattray, M. (2004), "Principal-Component-Analysis Eigenvalue Spectra from Data with Symmetry Breaking Structure," Physical Review E: Statistical, Nonlinear, and Soft Matter Physics, 69, 026124.
15. Johnstone, I.M. (2001), "Chi Square Oracle Inequalities," in Festschrift forWillem R. van Zwet (vol. 36 of IMS Lecture Notes-Monographs), eds. M. de Gunst, C. Klaassen, and A. van der Waart, Beachwood, OH: Institute of Mathematical Statistics, pp. 399-418.
16. (2003), Function Estimation and Gaussian Sequence Models. Draft of a monograph. Available at www-stat.stanford.edu/;imj.
17. Johnstone, I. M., and Lu, A. Y. (2004), "Sparse Principal Components Analysis," Technical Report, Stanford University, Dept. of Statistics. Available at arxiv.org as e-print 0901.4392.
18. Jolliffe, I. T., Trendafilov, N. T., and Uddin, M. (2003), "A Modified Principal Component Technique Based on the LASSO," Journal of Computational and Graphical Statistics, 12, 531-547.
19. Kaewpijit, S., Le Moigne, J., and El-Ghazawi, T. (2002), "A Wavelet-Based PCA Reduction for Hyperspectral Imagery," in Geoscience and Remote Sensing Symposium, 2002. IGARSS'02. 2002 IEEE International (5th ed.), Washington, DC: IEEE, pp. 2581-2583.
20. Lu, A. Y. (2002), "Sparse Principal Components Analysis for Functional Data," Ph.D. dissertation, Stanford University, Dept. of Statistics.
21. Mallat, S. (1999), A Wavelet Tour of Signal Processing (2nd ed.), New York: Academic Press.
22. Muirhead, R. J. (1982), Aspects ofMultivariate Statistical Theory, NewYork:Wiley.
23. Nadler, B. (2008), "Finite Sample Approximation Results for Principal Component Analysis: A Matrix Perturbation Approach," The Annals of Statistics, 36, 2791-2817.
24. Nadler, B., and Coifman, R. (2005), "The Prediction Error in CLS and PLS: The Importance of Feature Selection Prior to Multivariate Calibration," Journal of Chemometrics, 19, 107-118.
25. Paul, D. (2007), "Asymptotics of Sample Eigenstructure for a Large Dimensional Spiked Covariance Model," Statistica Sinica, 17, 1617-1642.
26. Paul, D., and Johnstone, I. (2004), "Estimation of Principal Components through Coordinate Selection," Technical Report, Stanford University, Dept. of Statistics.
27. Ramsay, J. O., and Silverman, B. W. (1997), Functional Data Analysis, Berlin: Springer.
28. Reimann, P., Van den Broeck, C., and Bex, G. J. (1996), "A Gaussian Scenario for Unsupervised Learning," Journal of Physics A: Mathematical and General, 29, 3521-3535.
29. Rice, J. A., and Silverman, B.W. (1991), "Estimating the Mean and Covariance Structure Nonparametrically When the Data Are Curves," Journal of the Royal Statistical Society, Ser. B, 53, 233-243.
30. Silverman, B. W. (1996), "Smoothed Functional Principal Components Analysis by Choice of Norm," The Annals of Statistics, 24, 1-24.
31. Silverstein, J. W. (1985), "The Smallest Eigenvalue of a Large Dimensional Wishart Matrix," Annals of Probability, 13, 1364-1368.
32. Watkin, T. L. H., and Nadal, J.- P. (1994), "Optimal Unsupervised Learning," Journal of Physics A: Mathematical and General, 27, 1899-1915.
33. Wickerhauser, M. V. (1994a), "Large-Rank Approximate Principal Component Analysis with Wavelets for Signal Feature Discrimination and the Inversion of Complicated Maps," Journal of Chemical Information and Computer Sciences, 34, 1036-1046.
34. Wickerhauser, M. V. (1994b), "Two Fast Approximate Wavelet Algorithms for Image Processing, Classification, and Recognition," Optical Engineering, 33, 2225-2235. (Special issue on Adapted Wavelet Analysis.)
35. Wolf, L., and Bileschi, S. (2005), "Combining Variable Selection with Dimensionality Reduction," in Computer Vision and Pattern Recognition, 2005 (vol. 2), Washington, DC: IEEE Computer Society, pp. 801-806.
36. Wolf, L., and Shashua, A. (2005), "Feature Selection for Unsupervised and Supervised Inference: The Emergence of Sparsity in a Weighted-Based Approach," Journal of Machine Learning Research, 6, 1855-1887.
37. Zou, H., Hastie, T., and Tibshirani, R. (2006), "Sparse Principal Component Analysis," Journal of Computational and Graphical Statistics, 15, 265-286.