Finite sample approximation results for principal component analysis: A matrix perturbation approach

Annals of Statistics

Volumn 36, Issue 6, 2008, Pages 2791-2817

  Nadler, Boaz ^a  

^a WEIZMANN INSTITUTE OF SCIENCE   (Israel)

Author keywords

Matrix perturbation; Phase transition; Principal component analysis; Random matrix theory; Spiked covariance model

Indexed keywords

EID: 62349121558     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/08-AOS618     Document Type: Article

References (37)

1. ANDERSON, T. W. (1963). Asymptotic theory for principal component analysis. Ann. Math. Statist. 34 122-148. MR0145620
2. ANDERSON, T. W. (1984). An Introduction to Multivariate Statistical Analysis, 2nd ed. Wiley, New York. MR0771294
3. BAIK, J., BEN AROUS, G. and PECHE, S. (2005). Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices. Ann. Probab. 33 1643-1697. MR2165575
4. BAIK, J. and SILVERSTEIN, J. W. (2006). Eigenvalues of large sample covariance matrices of spiked population models. J. Multivariate Anal. 97 1382-1408. MR2279680
5. BIEHL, M. and MIETZNER, A. (1994). Statistical- mechanics of unsupervised structure recognition. J. Phys. A 27 1885. MR1280357
6. BUCKHEIT, J. and DONOHO, D. L. (1995). Improved linear discrimination using time frequency dictionaries. Proc. SPIE 2569 540-551.
7. DAVIDSON, K. R. and SZAREK, S. (2001). Local operator theory, random matrices and Banach spaces. In Handbook on the Geometry of Banach Spaces (W. B. Johnson and J. Linden-strauss, eds.) 1 317-366. North-Holland, Amsterdam. MR1863696
8. DAVIS, C. and KAHAN, W. M. (1970). The rotation of eigenvectors by a perturbation. III. SIAMJ. Numer. Anal. 70 1-47. MR0264450
9. DOZIER, R. B. and SILVERSTEIN, J. W. (2007). On the empirical distribution of eigenvalues of large-dimensional information-plus- noise type matrices. J. Multivariate Anal. 98 678-694. MR2322123
10. EATON, M. and TYLER, D. E. (1991). On Wielandt's inequality and its application to the asymptotic distribution of the eigenvalues of a random symmetric matrix. Ann. Statist. 19 260-271. MR1091849
11. EL KAROUI, N. (2007). Tracy-Widom limit for the largest eigenvalue of a large class of complex sample covariance matrices. Ann. Probab. 35 663-714. MR2308592
12. GIRSHICK, M. A. (1939). On the sampling theory of the roots of determinantal equations. Ann. Math. Statist. 10 203-204. MR0000127
13. HORN, R. A. and JOHNSON, C. R. (1990). Matrix Analysis. Cambridge Univ. Press. MR1084815
14. HOYLE, D. C. and RATTRAY, M. (2003). PCA learning for sparse high-dimensional data, Europhys. Lett. 62 117-123.
15. HYVARINEN, A., KARHUNEN, J. and OJA, E. (2001). Independent Component Analysis. Wiley, New York.
16. IPSEN, I. C. F. and NADLER, B. (2008). Refined perturbation bounds for eigenvalues of Hermitian and non-Hermitian matrices. SIAM J. Matrix Anal. Appl. To appear.
17. JACKSON, J. D. (1991). A User's Guide to Principal Components. Wiley.
18. JOHNSTONE, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295-327. MR1863961
19. JOHNSTONE, I. M. and Lu, A. Y. (2008). Sparse principal components analysis. J. Amer. Statist. Assoc. To appear.
20. JOLLIFFE, I. T. (2002). Principal Component Analysis, 2nd ed. Springer, New York. MR2036084
21. KATO, T. (1995). Perturbation Theory for Linear Operators, 2nd ed. Springer, Berlin. MR 1335452
22. KRITCHMAN, S. and NADLER, B. (2008). Determining the number of components in a factor model from limited noisy data. Chemom. Int. Lab. Sys. 94 19-32.
23. LAWLEY, D. N. (1956). Tests of significance for the latent roots of covariance and correlation matrices. Biometrika 43 128-136. MR0078610
24. MARČENKO, V. A. and PASTUR, L. A. (1967). Distribution for some sets of random matrices. Math. USSR-Sb 1 457-483.
25. NADLER, B. and COIFMAN, R. R. (2005). Partial least squares, Beer's law and the net analyte signal: Statistical modeling and analysis. J. Chemometrics 19 45-54.
26. NADLER, B. and COIFMAN, R. R. (2005). The prediction error in CLS and PLS: The importance of feature selection prior to multivariate calibration. J. Chemom. 19 107-118.
27. NAES, T., ISAKSSON, T., FEARN, T. and DAVIS, T. (2002). Multivariate Calibration and Classification. NIR Publications, Chichester, UK.
28. O'LEARY, D. P. and STEWART, G. W. (1990). Computing the eigenvalues and eigenvectors of symmetric arrowhead matrices. J. Comput. Phys. 90 497-505. MR1071882
29. ONATSKI, A. (2007). Asymptotic distribution of the principal components estimator of large factor models when factors are relatively weak. Available at http://www.columbia.edu/~ao2027/inference45.pdf.
30. PARLETT, B. N. (1980). The Symmetric Eigenvalue Problem. Prentice-Hall, Englewood Cliffs, NJ. MR0570116
31. PAUL, D. (2007). Asymptotics of sample eigenstructure for a large dimensional spiked covariance model. Statist. Sinica 17 1617-1642. MR2399865
32. REIMANN, P., VAN DEN BROECK, C. and BEX, G. J. (1996). A Gaussian scenario for unsupervised learning. J. Phys. A 29 3521-3535.
33. SILVERSTEIN, J. W. and BAI, Z. D. (1995). On the empirical distribution of eigenvalues of a class of large-dimensional random matrices. J. Multivariate Anal. 54 175-192. MR1345534
34. SILVERSTEIN, J. W. and CHOI, S. I. (1995). Analysis of the limiting spectral distribution of large-dimensional random matrices. J. Multivariate Anal. 54 295-309. MR1345541
35. STEWART, G. W. (1990). Stochastic perturbation theory. SIAM Rev. 32 579-610. MR1084571
36. STEWART, G. W. (1991). Perturbation theory for the singular value decomposition. In SVD and Signal Processing II: Algorithms, Analysis and Applications (R. J. Vaccaro, ed.) 99-109.
37. WATKIN, T. H. and NADAL, J.-P. (1994). Optimal unsupervised learning. J. Phys. A 27 1899-1915.MR1280358