High-dimensional analysis of semidefinite relaxations for sparse principal components

Annals of Statistics

Volumn 37, Issue 5 B, 2009, Pages 2877-2921

  Amini, Arash A ^a   Wainwright, Martin J ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Convex relaxation; High dimensional statistics; Principal component analysis; Random matrices; Semidefinite programming; Sparsity; Spectral analysis; Spiked covariance ensembles; Wishart ensembles

Indexed keywords

EID: 69049101180     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/08-AOS664     Document Type: Article

References (41)

1. ANDERSON, T. W. (1984). An Introduction to Multivariate Statistical Analysis. Wiley, New York. MR0771294
2. BAIK, J. and SILVERSTEIN, J. W. (2006). Eigenvalues of large sample covariance matrices of spiked populations models. J. Multivariate Anal. 97 1382-1408. MR2279680
3. BICKEL, P. and LEVINA, E. (2008). Regularized estimation of large covariance matrices. Ann. Statist. 36 199-227. MR2387969
4. BIRGÉ, L. (2001). An alternative point of view on Lepski's method. In State of the Art in Probability and Statistics. IMS Lecture Notes 37 113-133.MR1836557
5. BULDYGIN, V. V. and KOZACHENKO, Y. V. (2000). Metric Characterization of Random Variables and Random Processes. Amer. Math. Soc., Providence, RI. MR1743716
6. CANDES, E. and TAO, T. (2006). The Dantzig selector: Statistical estimation when p is much larger than n. Ann. Statist. 35 2313-2351. MR2382644
7. C0VER, T.M. and THOMAS, J. A. (1991). Elements of Information Theory. Wiley, New York. MR1122806
8. D'ASPREM0NT, A., BANNERJEE, 0. and EL GHAOUI, L. (2007). First order methods for sparse covariance selection. SIAM J. Matrix Anal. Appl. 30 56-66. MR2399568
9. D'ASPREM0NT, A., EL GHA0UI, L., J0RDAN, M.I. and LANCKRIET, G. R. G. (2007). A direct formulation for sparse PCA using semidefinite programming. SIAM Rev. 49 434-448. MR2353806
10. DAVIDS0N, K.R. and SZAREK, S. J. (2001). Local operator theory, random matrices, and Banach spaces. In Handbook ofBanach Spaces 1 317-336. Elsevier, Amsterdam, NL. MR1863696
11. DONOHO, D. (2006). Compressed sensing. IEEE Trans. Info. Theory 52 1289-1306. MR2241189
12. EL KAROUI, N. (2008). Operator norm consistent estimation of large-dimensional sparse co-variance matrices. Ann. Statist. 36 2717-2756.
13. GEMAN, S. (1980). A limit theorem for the norm of random matrices. Ann. Probab. 8 252-261. MR0566592
14. GRADSHTEYN, I. S. and RYZHIK, I. M. (2000). Tables of Integrals, Series, and Products. Academic Press, New York, NY. MR1773820
15. GRIMMETT, G. R. and STIRZAKER, D. R. (1992). Probability andRandom Processes. Oxford Science Publications, Clarendon Press, Oxford. MR0667520
16. HAS'MINSKII, R. Z. (1978). A lower bound on the risks of nonparametric estimates of densities in the uniform metric. Theory Probab. Appl. 23 794-798. MR0516279
17. HIRIART-URRUTY, J. and LEMARÉCHAL, C. (1993). Convex Analysis and Minimization Algorithms 1. Springer, New York.
18. H0RN, R.A. and JOHNSON, C. R. (1986). Matrix Analysis. Cambridge Univ. Press, New York, NY. MR1084815
19. JOHNSTONE, I. M. (2001). Chi-square oracle inequalities. In State of the Art in Probability and Statistics IMS Lecture Notes 37 (M. de Gunst, C. Klaassen and A. van der Vaart, eds.) 399-418. Institute of Mathematical Statistics. MR1836572
20. JOHNSTONE, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295-327. MR1863961
21. JOHNSTONE, I.M.andLU, A. (2004). Sparse principal components. Technical report, Stanford Univ.
22. JOLLIFFE, I. T. (2004). Principal Component Analysis. Springer, New York.
23. J0LLIFFE, I. T., TRENDAFIL0V, N. T. and UDDIN, M. (2003). A modified principal component technique based on the LASSO. J. Comput. Graph. Statist. 12 531-547. MR2002634
24. LAURENT, B. and MASSART, P. (1998). Adaptive estimation of a quadratic functional by model selection. Ann. Statist. 28 1303-1338. MR1805785
25. LEDOUX, M. (2001). The Concentration of Measure Phenomenon. Amer. Math. Soc., Providence, RI. MR1849347
26. LED0UX, M. and TALAGRAND, M. (1991). Probability in Banach Spaces: Isoperimetry and Processes. Springer, New York. MR1102015
27. MATOUSEK, J. (2002). Lectures on Discrete Geometry. Springer, New York. MR1899299
28. MEINSHAUSEN, N. and BUHLMANN, P. (2006). High-dimensional graphs and variable selection with the Lasso. Ann. Statist. 34 1436-1462. MR2278363
29. MOGHADDAM, B., WEISS, Y. and AVIDAN, S. (2005). Spectral bounds for sparse PCA: Exact and greedy algorithms. In Neural Information Processing Systems (NIPS). Vancouver, Canada.
30. PAUL, D. (2005). Nonparametric estimation of principal components. Ph.D. thesis, Stanford Univ.
31. PAUL, D. (2007). Asymptotics of sample eigenstructure for a large-dimensional spiked covariance model. Statist. Sinica 17 1617-1642. MR2399865
32. PAUL, D.andJOHNSTONE, I. (2008). Augmented sparse principal component analysis for high-dimensional data. Technical report, UC Davis.
33. ROCKAFELLAR, G. (1970). Convex Analysis. Princeton Univ. Press, Princeton. MR0274683
34. STEWART, G. W. and SUN, J. G. (1990). Matrix Perturbation Theory. Academic Press, New York. MR1061154
35. TROPP, J. (2006). Just relax: Convex programming methods for identifying sparse signals in noise. IEEE Trans. Inform. Theory 52 1030-1051. MR2238069
36. WAINWRIGHT, M. J. (2006). Sharp thresholds for high-dimensional and noisy recovery of sparsity using the Lasso. In IEEE Trans. Inform. Theory. Technical Report 709, Dept. Statistics, UC Berkeley.
37. WAINWRIGHT, M. J. (2007). Information-theoretic bounds for sparsity recovery in the highdimensional and noisy setting. In Int. Symposium on Information Theory. Technical Report 725, Dept. Statistics, UC Berkeley.
38. YANG, Y.andBARRON, A. (1999). Information-theoretic determination of minimax rates of convergence. Ann. Statist. 27 1564-1599. MR1742500
39. YUAN, M.andLIN, Y. (2007). Model selection and estimation in the Gaussian graphical model. Biometrika 94 19-35. MR2367824
40. ZHANG, Z., ZHA, H. and SIMON, H. (2002). Low-rank approximations with sparse factors I: Basic algorithms and error analysis. SIAM J. Matrix Anal. Appl. 23 706-727.
41. ZOU, H., HASTIE, T. and TIBSHIRANI, R. (2006). Sparse principal component analysis. J. Comput. Graph. Statist. 15 262-286. MR2252527