Complexity theoretic lower bounds for sparse principal component detection

Journal of Machine Learning Research

Volumn 30, Issue , 2013, Pages 1046-1066

  Berthet, Quentin ^a   Rigollet, Philippe ^a  

^a PRINCETON UNIVERSITY   (United States)

Author keywords

Planted clique; Polynomial time reduction; Sparse principal component analysis

Indexed keywords

COMPUTATIONAL EFFICIENCY; MATHEMATICAL PROGRAMMING; POLYNOMIAL APPROXIMATION;

COMPUTATIONALLY EFFICIENT; PLANTED CLIQUES; POLYNOMIAL-TIME REDUCTION; PRINCIPAL COMPONENTS; SEMI-DEFINITE PROGRAMMING; SIGNAL STRENGTHS; SPARSE PRINCIPAL COMPONENT ANALYSIS; STATISTICAL PERFORMANCE;

PRINCIPAL COMPONENT ANALYSIS;

EID: 84898046942     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (48)

1. Louigi Addario-Berry, Nicolas Broutin, Luc Devroye, and Gábor Lugosi. On combinatorial testing problems. Annals of Statistics, 38(5): 3063-3092, 08 2010.
2. Noga Alon, Michael Krivelevich, and Benny Sudakov. Finding a large hidden clique in a random graph. In Proceedings of the ninth annual ACM-SIAM symposium on Discrete algorithms, SODA '98, pages 594-598, Philadelphia, PA, USA, 1998. Society for Industrial and Applied Mathematics.
3. Noga Alon, Alexandr Andoni, Tali Kaufman, Kevin Matulef, Ronitt Rubinfeld, and Ning Xie. Testing k-wise and almost k-wise independence. In STOC'07 - Proceedings of the 39th Annual ACM Symposium on Theory of Computing, pages 496-505. ACM, New York, 2007.
4. Noga Alon, Sanjeev Arora, Rajsekar Manokaran, Dana Moshkovitz, and Omri Weinstein. On the inapproximability of the densest κ-subgraph problem. Unpublished, April 2011.
5. Brendan P.W. Ames and Stephen A. Vavasis. Nuclear norm minimization for the planted clique and biclique problems. Mathematical Programming, 129: 69-89, 2011.
6. Arash A. Amini and Martin J. Wainwright. High-dimensional analysis of semidefinite relaxations for sparse principal components. Annals of Statistics, 37(5B): 2877-2921, 03 2009.
7. E. Arias-Castro, S. Bubeck, and G. Lugosi. Detecting positive correlations in a multivariate sample. Arxiv Preprint, 2013. arXiv: 1202.5536.
8. Ery Arias-Castro and Nicolas Verzelen. Community detection in random networks. Arxiv Preprint, 02 2013. URL http://arXiv.org/abs/1302.7099.
9. Ery Arias-Castro, Emmanuel J. Candès, and Arnaud Durand. Detection of an anomalous cluster in a network. Annals of Statistics, 39, 01 2011a.
10. Ery Arias-Castro, Emmanuel J. Candès, and Yaniv Plan. Global testing under sparse alternatives: ANOVA, multiple comparisons and the higher criticism. Ann. Statist., 39(5): 2533-2556, 2011b. ISSN 0090-5364. doi: 10.1214/11-AOS910. URL http://dx.doi.org/10.1214/11-AOS910.
11. Ery Arias-Castro, Sébastien Bubeck, and Gábor Lugosi. Detection of correlations. Ann. Statist., 40(1): 412-435, 2012.
12. Francis Bach, Selin Damla Ahipasaoglu, and Alexandre d'Aspremont. Convex relaxations for subset selection. Arxiv Preprint, 06 2010. URL http://arXiv.org/abs/1006.3601v1.
13. Sivaraman Balakrishnan, Mladen Kolar, Alessandro Rinaldo, Aarti Singh, and Larry Wasserman. Statistical and computational tradeoffs in biclustering. NIPS 2011 Workshop on Computational Trade-offs in Statistical Learning, 2011.
14. Quentin Berthet and Philippe Rigollet. Optimal detection of sparse principal components in high dimension. ArXiv: 1202.5070, 02 2012. URL http://arXiv.org/abs/1202.5070.
15. Ravi B. Boppana. Eigenvalues and graph bisection: An average-case analysis. In Foundations of Computer Science, 1987., 28th Annual Symposium on, pages 280-285, oct. 1987.
16. Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge University Press, Cambridge, 2004.
17. Cristina Butucea and Yuri I. Ingster. Detection of a sparse submatrix of a high-dimensional noisy matrix. Bernoulli (to appear), 2013.
18. T. Tony Cai, Zongming Ma, and Yihong Wu. Sparse PCA: Optimal rates and adaptive estimation. Arxiv Preprint, 11 2012. URL http://arXiv.org/abs/1211. 1309.
19. Venkat Chandrasekaran and Michael I. Jordan. Computational and statistical tradeoffs via convex relaxation. Proceedings of the National Academy of Sciences, 2013.
20. Alexandre d'Aspremont, Laurent El Ghaoui, Michael I. Jordan, and Gert R. G. Lanckriet. A direct formulation for sparse PCA using semidefinite programming. SIAM Review, 49 (3): 434-448, July 2007.
21. Alexandre d'Aspremont, Francis Bach, and Laurent El Ghaoui. Approximation bounds for sparse principal component analysis. ArXiv: 1205.0121, May 2012. URL http://arXiv.org/abs/1205.0121.
22. Yael Dekel, Ori Gurel-Gurevich, and Yuval Peres. Finding hidden cliques in linear time with high probability. Arxiv Preprint, 2010. URL http://arXiv.org/abs/1010.2997v1.
23. Persi Diaconis and David Freedman. Finite exchangeable sequences. Ann. Probab., 8(4): 745-764, 1980.
24. Richard Dudley. Uniform Central Limit Theorems. Cambridge University Press, 1999.
25. Uriel Feige and Robert Krauthgamer. Finding and certifying a large hidden clique in a semirandom graph. Random Structures Algorithms, 16(2): 195-208, 2000.
26. Uriel Feige and Robert Krauthgamer. The probable value of the Lovász-Schrijver relaxations for maximum independent set. SIAM J. Comput., 32(2): 345-370 (electronic), 2003.
27. Uriel Feige and Dorit Ron. Finding hidden cliques in linear time. In 21st International Meeting on Probabilistic, Combinatorial, and Asymptotic Methods in the Analysis of Algorithms (AofA'10), Discrete Math. Theor. Comput. Sci. Proc., AM, pages 189-203. Assoc. Discrete Math. Theor. Comput. Sci., Nancy, 2010.
28. Vitaly Feldman, Elena Grigorescu, Lev Reyzin, Santosh Vempala, and Ying Xiao. Statistical algorithms and a lower bound for planted clique. In Proceedings of the Fourty-Fifth Annual ACM Symposium on Theory of Computing, STOC 2013, 2013.
29. 1-ε. In 37th Annual Symposium on Foundations of Computer Science (Burlington, VT, 1996), pages 627-636. IEEE Comput. Soc. Press, Los Alamitos, CA, 1996.
30. 1-ε. Acta Math., 182(1): 105-142, 1999.
31. Elad Hazan and Robert Krauthgamer. How hard is it to approximate the best nash equilibrium? SIAM J. Comput., 40(1): 79-91, 2011.
32. Mark Jerrum. Large cliques elude the Metropolis process. Random Structures Algorithms, 3(4): 347-359, 1992.
33. Iain M. Johnstone and Arthur Yu Lu. On consistency and sparsity for principal components analysis in high dimensions. J. Amer. Statist. Assoc., 104(486): 682-693, 2009.
34. Ari Juels and Marcus Peinado. Hiding cliques for cryptographic security. Des. Codes Cryptogr., 20(3): 269-280, 2000.
35. M. Kolar, S. Balakrishnan, A. Rinaldo, and A. Singh. Minimax localization of structural information in large noisy matrices. Advances in Neural Information Processing Systems, 2011.
36. Michael Krivelevich and Van H. Vu. Approximating the independence number and the chromatic number in expected polynomial time. J. Comb. Optim., 6(2): 143-155, 2002.
37. Luděk Kučera. Expected complexity of graph partitioning problems. Discrete Appl. Math., 57(2-3): 193-212, 1995. Combinatorial optimization 1992 (CO92) (Oxford).
38. Zongming Ma. Sparse principal component analysis and iterative thresholding. Ann. Statist. (to appear), 2013.
39. Pascal Massart. Concentration inequalities and model selection, Volume 1896 of Lecture Notes in Mathematics. Springer, Berlin, 2007. Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour, July 6-23, 2003, With a foreword by Jean Picard.
40. Frank McSherry. Spectral partitioning of random graphs. In 42nd IEEE Symposium on Foundations of Computer Science (Las Vegas, NV, 2001), pages 529-537. IEEE Computer Soc., Los Alamitos, CA, 2001.
41. Benjamin Rossman. Average-Case Complexity of Detecting Cliques. ProQuest LLC, Ann Arbor, MI, 2010. Thesis (Ph.D.)-Massachusetts Institute of Technology.
42. Shai Shalev-Shwartz, Ohad Shamir, and Eran Tomer. Using more data to speed-up training time. In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics April 21-23, 2012 La Palma, Canary Islands., Volume 22 of JMLR W&CP, pages 1019-1027, 2012.
43. Joel Spencer. Ten lectures on the probabilistic method, Volume 64 of CBMS-NSF Regional Conference Series in Applied Mathematics. Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, second edition, 1994.
44. Xing Sun and Andrew B. Nobel. On the maximal size of large-average and ANOVA-fit submatrices in a Gaussian random matrix. Bernoulli, 19(1): 275-294, 2013.
45. Alexandre B. Tsybakov. Introduction to nonparametric estimation. Springer Series in Statistics. Springer, New York, 2009. Revised and extended from the 2004 French original, Translated by Vladimir Zaiats.
46. Roman Vershynin. Introduction to the non-asymptotic analysis of random matrices. Arxiv Preprint, 11 2010. URL http://arXiv.org/abs/1011.3027v7.
47. Vincent Vu and Jing Lei. Minimax rates of estimation for sparse pca in high dimensions. In Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics April 21-23, 2012 La Palma, Canary Islands., Volume 22 of JMLR W&CP, pages 1278-1286, 2012.
48. David Zuckerman. Linear degree extractors and the inapproximability of max clique and chromatic number. In Proceedings of the thirty-eighth annual ACM symposium on Theory of computing, STOC '06, pages 681-690, New York, NY, USA, 2006. ACM.