Truncated power method for sparse eigenvalue problems

Journal of Machine Learning Research

Volumn 14, Issue 1, 2013, Pages 899-925

  Yuan, Xiao Tong ^a   Zhang, Tong ^a  

^a RUTGERS UNIVERSITY   (United States)

Author keywords

Densest k subgraph; Power method; Sparse eigenvalue; Sparse principal component analysis

Indexed keywords

DENSEST K-SUBGRAPH; EFFECTIVE SOLUTION; EIGEN-VALUE; EIGENVALUE PROBLEM; EMPIRICAL PERFORMANCE; NONCONVEX OPTIMIZATION PROBLEM; POWER METHOD; SPARSE PRINCIPAL COMPONENT ANALYSIS;

GRAPH THEORY; PRINCIPAL COMPONENT ANALYSIS; SWITCHING SYSTEMS; VIRTUAL REALITY;

EIGENVALUES AND EIGENFUNCTIONS;

EID: 84877607371     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (37)

1. A. Alizadeh, M. Eisen, R. Davis, C. Ma, I. Lossos, and A. Rosenwald. Distinct types of diffuse large b-cell lymphoma identified by gene expression profiling. Nature, 403:503-511, 2000.
2. A. Alon, N. Barkai, D. A. Notterman, K. Gish, S. Ybarra, D. Mack, and A. J. Levine. Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Cell Biology, 96:6745-6750, 1999.
3. A. A. Amini and M. J. Wainwright. High-dimensional analysis of semidefinite relaxiation for sparse principal components. Annals of Statistics, 37:2877-2921, 2009.
4. Y. Asahiro, R. Hassin, and K. Iwama. Complexity of finidng dense subgraphs. Discrete Applied Mathematics, 211(1-3):15-26, 2002.
5. A. Billionnet and F. Roupin. A deterministic algorithm for the densest k-subgraph problem using linear programming. Technical report, Technical Report, No. 486, CEDRIC, CNAM-IIE, Paris, 2004.
6. T. Cai, Z. Ma, and Y. Wu. Sparse pca: Optimal rates and adaptive estimation. 2012. URL arxiv. org/pdf/1211.1309v1.pdf.
7. E. J. Candes and T. Tao. Decoding by linear programming. IEEE Transactions on Information Theory, 51:4203-4215, 2005.
8. D. G. Corneil and Y. Perl. Clustering and domination in perfect graphs. Discrete Applied Mathematics, 9:27-39, 1984.
9. A. d'Aspremont, L. El Ghaoui, M. I. Jordan, and G. R. G. Lanckriet. A direct formulation for sparse pca using semidefinite programming. SIAM Review, 49:434-448, 2007.
10. A. d'Aspremont, F. Bach, and L. El Ghaoui. Optimal solutions for sparse principal component analysis. Journal of Machine Learning Research, 9:1269-1294, 2008.
11. U. Feige, G. Kortsarz, and D. Peleg. The dense k-subgraph problem. Algorithmica, 29(3):410-421, 2001.
12. X. Geng, T. Liu, T. Qin, and H. Li. Feature selection for ranking. In Proceedings of the 30th Annual International ACM SIGIR Conference (SIGIR'07), 2007.
13. D. Gibson, R. Kumar, and A. Tomkins. Discovering large dense subgraphs in massive graphs. In Proceedings of the 31st International Conference on Very Large Data Bases (VLDB'05), pages 721-732, 2005.
14. G. H. Golub and C.F. Van Loan. Matrix Computations. Johns Hopkins University Press, Baltimore, MD, third edition, 1996.
15. J. Jeffers. Two case studies in the application of principal components. Applied Statistics, 16(3): 225-236, 1967.
16. P. Jiang, J. Peng, M. Heath, and R. Yang. Finding densest k-subgraph via 1-mean clustering and low-dimension approximation. Technical report, 2010.
17. I. M. Johnstone. On the distribution of the largest eigenvalue in principal components analysis. Annals of Statistics, 29:295-327, 2001.
18. I. T. Jolliffe, N. T. Trendafilov, and M. Uddin. A modified principal component technique based on the lasso. Journal of Computational and Graphical Statistics, 12(3):531-547, 2003.
19. M. Journée, Y. Nesterov, P. Richtárik, and Rodolphe Sepulchre. Generalized power method for sparse principal component analysis. Journal of Machine Learning Research, 11:517-553, 2010.
20. S. Khot. Ruling out ptas for graph min-bisection, dense k-subgraph, and bipartite clique. SIAM Journal on Computing, 36(4):1025-1071, 2006.
21. S. Khuller and B. Saha. On finding dense subgraphs. In Proceedings of the 36th International Colloquium on Automata, Languages and Programming (ICALP'09), pages 597-608, 2009.
22. R. Kumar, P. Raghavan, S. Rajagopalan, and A. Tomkins. Trawling the web for emerging cybercommunities. In Proceedings of the 8th World Wide Web Conference (WWW'99), pages 403-410, 1999.
23. M. Liazi, I. Milis, and V. Zissimopoulos. A constant approximation algorithm for the densest ksubgraph problem on chordal graphs. Information Processing Letters, 108(1):29-32, 2008.
24. Z. Ma. Sparse principal component analysis and iterative thresholding. Annals of Statistics, to appear, 2013.
25. L. Mackey. Deflation methods for sparse pca. In Proceedings of the 22nd Annual Conference on Neural Information Processing Systems (NIPS'08), 2008.
26. J. Mairal, F. Bach, J. Ponce, and G. Sapiro. Online learning for matrix factorization and sparse coding. Journal of Machine Learning Research, 11:10-60, 2010.
27. B. Moghaddam, Y. Weiss, and S. Avidan. Generalized spectral bounds for sparse lda. In Proceedings of the 23rd International Conference on Machine Learning (ICML'06), pages 641-648, 2006.
28. D. Paul and I.M. Johnstone. Augmented sparse principal component analysis for high dimensional data. 2012. URL arxiv.org/pdf/1202.1242v1.pdf.
29. S. S. Ravi, D. J. Rosenkrantz, and G. K. Tayi. Heuristic and special case algorithms for dispersion problems. Operations Research, 42:299-310, 1994.
30. D. Shen, H. Shen, and J.S. Marron. Consistency of sparse pca in high dimension, low sample size contexts. Journal of Multivariate Analysis, 115:317-333, 2013.
31. H. Shen and J. Z. Huang. Sparse principal component analysis via regularized low rank matrix approximation. Journal of Multivariate Analysis, 99(6):1015-1034, 2008.
32. A. Srivastav and K. Wolf. Finding dense subgraphs with semidefinite programming. In Proceedings of International Workshop on Approximation Algorithms for Combinatorial Optimization (APPROX'98), pages 181-191, 1998.
33. R. Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B, 58(1):267-288, 1996.
34. D. M. Witten, R. Tibshirani, and T. Hastie. A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. Biostatistics, 10(3):515-534, 2009.
35. R. Yang. New approximation methods for solving binary quadratic programming problem. Technical report, Master Thesis, Department of Industrial and Enterprise Systems Engineering, University of Illnois at Urbana-Champaign, 2010.
36. Y. Y. Ye and J. W. Zhang. Approximation of dense-n/2-subgraph and the complement of minbisection. Journal of Global Optimization, 25:55-73, 2003.
37. H. Zou, T. Hastie, and R. Tibshirani. Sparse principal component analysis. Journal of Computational and Graphical Statistics, 15(2):265-286, 2006.