Optimal detection of sparse principal components in high dimension

Annals of Statistics

Volumn 41, Issue 4, 2013, Pages 1780-1815

  Berthet, Quentin ^a   Rigollet, Philippe ^a  

^a Princeton University   (United States)

Author keywords

High dimensional detection; Minimax lower bounds; Planted clique; Semidefinite relaxation; Sparse principal component analysis; Spiked covariance model

Indexed keywords

EID: 84885061765     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/13-AOS1127     Document Type: Article

References (74)

1. ADDARIO-BERRY, L., BROUTIN, N., DEVROYE, L. and LUGOSI, G. (2010). On combinatorial testing problems. Ann. Statist. 38 3063-3092.
2. ALON, N., KRIVELEVICH, M. and SUDAKOV, B. (1998). Finding a large hidden clique in a random graph. Random Structures Algorithms 13 457-466.
3. ALON, U., BARKAI, N., NOTTERMAN, D. A., GISH, K., YBARRA, S., MACK, D. and LEVINE, A. J. (1999). Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proc. Natl. Acad. Sci. USA 96 6745-6750. (Pubitemid 29274954)
4. ALON, N., ANDONI, A., KAUFMAN, T., MATULEF, K., RUBINFELD, R. and XIE, N. (2007). Testing k-wise and almost k-wise independence. In STOC'07-Proceedings of the 39th Annual ACM Symposium on Theory of Computing 496-505. (Pubitemid 47630769)
5. ALON, N., ARORA, S., MANOKARAN, R., MOSHKOVITZ, D. and WEINSTEIN, O. (2011). On the inapproximability of the densest κ-subgraph problem. Unpublished manuscript.
6. AMES, B. P. W. and VAVASIS, S. A. (2011). Nuclear norm minimization for the planted clique and biclique problems. Math. Program. 129 69-89.
7. AMINI, A. A. and WAINWRIGHT, M. J. (2009). High-dimensional analysis of semidefinite relaxations for sparse principal components. Ann. Statist. 37 2877-2921.
8. ARIAS-CASTRO, E., BUBECK, S. and LUGOSI, G. (2012). Detection of correlations. Ann. Statist. 40 412-435.
9. ARIAS-CASTRO, E., CANDÈS, E. J. and DURAND, A. (2011). Detection of an anomalous cluster in a network. Ann. Statist. 39 278-304.
10. ARIAS-CASTRO, E., CANDÈS, E. J. and PLAN, Y. (2011). Global testing under sparse alternatives: ANOVA, multiple comparisons and the higher criticism. Ann. Statist. 39 2533-2556.
11. BACH, F., AHIPASAOGLU, S. D. and D'ASPREMONT, A. (2010). Convex relaxations for subset selection. Preprint. Available at arXiv:1006.3601v1.
12. BAI, Z. D. (1999). Methodologies in spectral analysis of large-dimensional random matrices, a review. Statist. Sinica 9 611-677.
13. BAIK, J., BEN AROUS, G. and PECHE, S. (2005). Phase transition of the largest eigenvalue for nonnull complex sample covariance matrices. Ann. Probab. 33 1643-1697. (Pubitemid 41560766)
14. BAIK, J. and SILVERSTEIN, J. W. (2006). Eigenvalues of large sample covariance matrices of spiked population models. J. Multivariate Anal. 97 1382-1408. (Pubitemid 43705671)
15. BANDEIRA, A. S., DOBRIBAN, E., MIXON, D. G. and SAWIN, W. F. (2012). Certifying the restricted isometry property is hard. Preprint. Available at arXiv:1204.1580.
16. BARAUD, Y. (2002). Non-asymptotic minimax rates of testing in signal detection. Bernoulli 8 577-606. (Pubitemid 44242664)
17. BICKEL, P. J. and LEVINA, E. (2008). Covariance regularization by thresholding. Ann. Statist. 36 2577-2604.
18. BIRNBAUM, A., and JOHNSTONE., I. M., NADLER, B. and PAUL, D. (2013). Minimax bounds for sparse PCA with noisy high-dimensional data. Ann. Statist. 41 1055-1084.
19. BOYD, S. and VANDENBERGHE, L. (2004). Convex Optimization. Cambridge Univ. Press, Cambridge.
20. BRUBAKER, S. C. and VEMPALA, S. S. (2009). Random tensors and planted cliques. In Approximation, Randomization, and Combinatorial Optimization. Lecture Notes in Computer Science 5687 406-419.
21. BUTUCEA, C. and INGSTER, Y. I. (2013). Detection of a sparse submatrix of a high-dimensional noisy matrix. Bernoulli. To appear. Available at arXiv:1109.0898v1.
22. CAI, T. T., MA, Z. and WU, Y. (2012). Sparse PCA: Optimal rates and adaptive estimation. Preprint. Available at arXiv:1211.1309.
23. CAI, T. T., ZHANG, C.-H. and ZHOU, H. H. (2010). Optimal rates of convergence for covariance matrix estimation. Ann. Statist. 38 2118-2144.
24. CHANDRASEKARAN, V. and JORDAN, M. I. (2013). Computational and statistical tradeoffs via convex relaxation. Proc. Natl. Acad. Sci. 110 E1181-E1190.
25. CHEN, X. (2011). Adaptive elastic-net sparse principal component analysis for pathway association testing. Stat. Appl. Genet. Mol. Biol. 10 Art. 48, 23.
26. D'ASPREMONT, A., BACH, F. and EL GHAOUI, L. (2008). Optimal solutions for sparse principal component analysis. J. Mach. Learn. Res. 9 1269-1294.
27. D'ASPREMONT, A., BACH, F. and GHAOUI, L. E. (2012). Approximation bounds for sparse principal component analysis. Preprint. Available at arXiv:1205.0121.
28. D'ASPREMONT, A., EL GHAOUI, L., JORDAN, M. I. and LANCKRIET, G. R. G. (2007). A direct formulation for sparse PCA using semidefinite programming. SIAM Rev. 49 434-448 (electronic). (Pubitemid 47377693)
29. DEKEL, Y., GUREL-GUREVICH, O. and PERES, Y. (2011). Finding hidden cliques in linear time with high probability. In ANALCO11-Workshop on Analytic Algorithmics and Combinatorics 67-75.
30. DONOHO, D. and JIN, J. (2004). Higher criticism for detecting sparse heterogeneous mixtures. Ann. Statist. 32 962-994. (Pubitemid 41250290)
31. EL KAROUI, N. (2008). Operator norm consistent estimation of large-dimensional sparse covariance matrices. Ann. Statist. 36 2717-2756.
32. FEIGE, U. and KRAUTHGAMER, R. (2000). Finding and certifying a large hidden clique in a semirandom graph. Random Structures Algorithms 16 195-208.
33. FEIGE, U. and KRAUTHGAMER, R. (2003). The probable value of the Lovász-Schrijver relaxations for maximum independent set. SIAM J. Comput. 32 345-370 (electronic).
34. FEIGE, U. and RON, D. (2010). Finding hidden cliques in linear time. In 21st International Meeting on Probabilistic, Combinatorial, and Asymptotic Methods in the Analysis of Algorithms (AofA'10) 189-203.
35. FELDMAN, V., GRIGORESCU, E., REYZIN, L., VEMPALA, S. and XIAO, Y. (2013). Statistical algorithms and a lower bound for planted clique. In Proceedings of the 45th Annual ACM Symposium on Theory of Computing, STOC'13 655-664.
36. FÉRAL, D. and PECHE, S. (2009). The largest eigenvalues of sample covariance matrices for a spiked population: Diagonal case. J. Math. Phys. 50 073302, 33.
37. FRIEZE, A. and KANNAN, R. (2008). A new approach to the planted clique problem. In FSTTCS 2008: IARCS Annual Conference on Foundations of Software Technology and Theoretical Computer Science. LIPIcs. Leibniz Int. Proc. Inform. 2 187-198.
38. GEMAN, S. (1980). A limit theorem for the norm of random matrices. Ann. Probab. 8 252-261.
39. GOEMANS, M. X. and WILLIAMSON, D. P. (1995). Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. J. Assoc. Comput. Mach. 42 1115-1145.
40. HAZAN, E. and KRAUTHGAMER, R. (2011). How hard is it to approximate the best Nash equilibrium? SIAM J. Comput. 40 79-91.
41. INGSTER, Y. I. (1982). The asymptotic efficiency of tests for a simple hypothesis against a composite alternative. Teor. Veroyatn. Primen. 27 587-592.
42. INGSTER, Y. I., TSYBAKOV, A. B. and VERZELEN, N. (2010). Detection boundary in sparse regression. Electron. J. Stat. 4 1476-1526.
43. JENATTON, R., OBOZINSKI, G. and BACH, F. (2010). Structured sparse principal component analysis. In Proceedings of the International Conference on Artificial Intelligence and Statistics (AISTAT S).
44. JERRUM, M. (1992). Large cliques elude the Metropolis process. Random Structures Algorithms 3 347-359.
45. JOHNSTONE, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. Ann. Statist. 29 295-327.
46. JOHNSTONE, I. M. and LU, A. Y. (2009). On consistency and sparsity for principal components analysis in high dimensions. J. Amer. Statist. Assoc. 104 682-693.
47. JOURNEE, M., NESTEROV, Y., RICHTÁRIK, P. and SEPULCHRE, R. (2010). Generalized power method for sparse principal component analysis. J. Mach. Learn. Res. 11 517-553.
48. JUELS, A. and PEINADO, M. (2000). Hiding cliques for cryptographic security. Des. Codes Cryptogr. 20 269-280.
49. KARP, R. M. (1972). Reducibility among combinatorial problems. In Complexity of Computer Computations (Proc. Sympos., IBM Thomas J. Watson Res. Center, Yorktown Heights, N.Y., 1972) 85-103. Plenum, New York.
50. KOIRAN, P. and ZOUZIAS, A. (2012). Hidden cliques and the certification of the restricted isometry property. Preprint. Available at arXiv:1211.0665.
51. KRIVELEVICH, M. and VU, V. H. (2002). Approximating the independence number and the chromatic number in expected polynomial time. J. Comb. Optim. 6 143-155. (Pubitemid 34176300)
52. KUČERA, L. (1995). Expected complexity of graph partitioning problems. Discrete Appl. Math. 57 193-212.
53. LAURENT, B. and MASSART, P. (2000). Adaptive estimation of a quadratic functional by model selection. Ann. Statist. 28 1302-1338. (Pubitemid 33244915)
54. LOH, P.-L. and WAINWRIGHT, M. J. (2012). High-dimensional regression with noisy and missing data: Provable guarantees with nonconvexity. Ann. Statist. 40 1637-1664.
55. LU, Z. and ZHANG, Y. (2012). An augmented Lagrangian approach for sparse principal component analysis. Math. Program. 135 149-193.
56. MA, S. (2011). Alternating direction method of multipliers for sparse principal component analysis. Preprint. Available at arXiv:1111.6703v1.
57. MA, Z. (2013). Sparse principal component analysis and iterative thresholding. Ann. Statist. 41 772-801.
58. NADLER, B. (2008). Finite sample approximation results for principal component analysis: A matrix perturbation approach. Ann. Statist. 36 2791-2817.
59. NESTEROV, Y. (2003). Introductory Lectures on Convex Optimization. Springer, New York.
60. NESTEROV, Y. and NEMIROVSKII, A. (1987). Interior-Point Polynomial Algorithms in Convex Programming. Society for Industrial Mathematics 13.
61. ONATSKI, A., MOREIRA, M. J. and HALLIN, M. (2013). Asymptotic power of sphericity tests for high-dimensional data. Ann. Statist. 41 1204-1231.
62. PAUL, D. (2007). Asymptotics of sample eigenstructure for a large dimensional spiked covariance model. Statist. Sinica 17 1617-1642.
63. PAUL, D. and JOHNSTONE, I. M. (2012). Augmented sparse principal component analysis for high dimensional data. Preprint. Available at arXiv:1202.1242v1.
64. ROSSMAN, B. (2010). Average-case complexity of detecting cliques. Ph.D. thesis, Massachusetts Institute of Technology.
65. SHEN, D., SHEN, H. and MARRON, J. S. (2013). Consistency of sparse PCA in high dimension, low sample size contexts. J. Multivariate Anal. 115 317-333.
66. SPENCER, J. (1994). Ten Lectures on the Probabilistic Method, 2nd ed. CBMS-NSF Regional Conference Series in Applied Mathematics 64.
67. SUN, X. and NOBEL, A. B. (2008). On the size and recovery of submatrices of ones in a random binary matrix. J. Mach. Learn. Res. 9 2431-2453.
68. SUN, X. and NOBEL, A. B. (2013). On the maximal size of large-average and ANOVA-fit submatrices in a Gaussian random matrix. Bernoulli 19 275-294.
69. TSYBAKOV, A. B. (2009). Introduction to Nonparametric Estimation. Springer, New York. Revised and extended from the 2004 French original, translated by Vladimir Zaiats.
70. VERSHYNIN, R. (2012). Introduction to the non-asymptotic analysis of random matrices. In Compressed Sensing 210-268.
71. VERZELEN, N. (2012). Minimax risks for sparse regressions: Ultra-high dimensional phenomenons. Electron. J. Stat. 6 38-90.
72. VU, V. and LEI, J. (2012). Minimax rates of estimation for sparse pca in high dimensions. In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics April 21-23, 2012, La Palma, Canary Islands, Vol. 22 of JMLR W & CP 1278-1286.
73. WRIGHT, J., GANESH, A., YANG, A., ZHOU, Z. and MA, Y. (2011). Sparsity and robustness in face recognition. Preprint. Available at arXiv:1111.1014.
74. YIN, Y. Q., BAI, Z. D. and KRISHNAIAH, P. R. (1988). On the limit of the largest eigenvalue of the large-dimensional sample covariance matrix. Probab. Theory Related Fields 78 509-521.