Nuclear norm minimization via active subspace selection

31st International Conference on Machine Learning, ICML 2014

Volumn 1, Issue , 2014, Pages 871-882

  Hsieh, Cho Jui ^a   Olsen, Peder A ^b  

^a University of Texas at Austin   (United States)

^b IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

LEARNING SYSTEMS; RELAXATION PROCESSES;

ALTERNATING LEAST SQUARES; CONVEX RELAXATION; MATRIX COMPLETION PROBLEMS; MATRIX PROBLEM; NUCLEAR NORM MINIMIZATIONS; SECOND-ORDER METHODS; SUB-SPACES; SUBSPACE SELECTION;

PROBLEM SOLVING;

EID: 84919790414     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (38)

1. Amit, Y., Fink, M., Srebro, N., and Ullman, S. Uncovering shared structures in multiclass classification. In ICML, pp. 17-24, 2007.
2. Arbenz, Peter. Lecture Notes on Solving Large Scale Eigenvalue Problems. 2010.
3. Argyriou, A., Evgeniou, T., and Pontil, M. Convex multi-task feature learning. Machine Learning, 73:243-272, 2008.
4. Avron, H., Kale, S., Kasiviswanathan, S., and Sindhwani, V. Efficient and practical stochastic subgradient descent for nuclear norm regularization. In ICML, 2012.
5. Bartels, RH and Stewart, GW. Algorithm 432: Solution of the matrix equation AX+ XB= C [F4]. Communications of the ACM, 15(9):820-826, 1972. ISSN 0001-0782.
6. Bertsekas, Dimitri P. Nonlinear Programming. Athena Scientific, Belmont, MA 02178-9998, second edition, 1999.
7. Cabrai, R. S., Torre, F., Costeira, J. P., and Bernardino, A. Matrix completion for multi-label image classification. In NIPS, pp. 190-198, 2011.
8. Cai, J. F., Candes, E. J., and Shen, Z. A singular value thresholding algorithm for matrix completion. SIAM J. Optimization, 20 (4):1956-1982, 2010.
9. Candès, E. J. and Recht, B. Exact matrix completion via convex optimization. Foundations of Computational mathematics, 9 (6):717-772, 2009.
10. Candés, E. J. and Tao, T. The power of convex relaxation: Nearoptimal matrix completion. IEEE Trans. Inform. Theory, 56 (5):2053-2080, 2009.
11. Daubechies, I., Défrise, M., and Mol, C. De. An iterative thresh olding algorithm for linear inverse problems with a sparsity constraint. Communications on pure and applied mathemat ics, 57(11):1413-1457, 2004.
12. Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. ImageNet: A large-scale hierarchical image database. In CVPR, pp. 248-255, 2009.
13. Dror, G., Koenigstein, N., Koren, Y., and Weimer, M. The Yahoo. music dataset and KDD-Cup'll. In JMLR Workshop and Conference Proceedings: Proceedings of KDD Cup 2011 Competition, volume 18, pp. 3-18, 2012.
14. Dudik, M., Harchaoui, Z., and Malick, J. Lifted coordinate de scent for learning with trace-norm regularization. In AISTATS, 2012.
15. Friedland, S. and Torokhti, A. Generalized rank-constrained ma trix approximation. SIAM Journal on Matrix Analysis and Applications, 29(2):656-659, 2006.
16. Giraud, C. Low rank multivariate regression. Electronic Journal of Statistics, 5:775-799, 2011.
17. Halko, N., Martinsson, P. G., and Tropp, J. A. Finding structure with randomness: Probabilistic algorithms for constructing ap proximate matrix decompositions. SIAM Rev, 53(2):217-288, 2011.
18. Hazan, E. Sparse approximate solutions to semidefinite programs. LATIN, pp. 306-316, 2008.
19. Higham, N. J. Newtons method for the matrix square root. Mathematics of Computation, 46(174):537-549, 1986.
20. Hsieh, C.-J., Sustik, M. A., Dhillon, I. S., and Ravikumar, P. Sparse inverse covariance matrix estimation using quadratic approximation. In NIPS, 2011.
21. Hsieh, C.-J., Sustik, M. A., Dhillon, I. S., Ravikumar, P., and Poldrack, R. A. Big & Quic: Sparse inverse covariance estimation for a million variables. In NIPS, 2013.
22. Jaggi, M. and Sulovsky, M. A simple algorithm for nuclear norm regularized problems. In ICML, pp. 471-478, 2010.
23. Ji, S. and Ye, J. An accelerated gradient method for trace norm minimization. In ICML, pp. 329-336, 2009.
24. Koren, Y., Bell, R. M, and Volinsky, C. Matrix factorization techniques for recommender systems. IEEE Computer, 42:30-37, 2009.
25. Li, R.-C. Relative Perturbation Theory: II. Eigenspace and Sin gular subspace Variations. SIAM Journal on Matrix Analysis and Applications, 20(2)471-492, 1998.
26. Mazumder, R., Hastie, T., and Tibshirani, R. Spectral regulariza tion algorithms for learning large incomplete matrices. JMLR, 11:2287-2322, 2010.
27. Mishra, B., Meyer, G., Bach, F., and Sepulchre, R. Low-rank optimization with trace norm penalty, arxiv: 1112.2318, 2013.
28. Nocedal, J. and Wright, S. J. Numerical optimization, volume 2. Springer New York, 1999.
29. Olsen, P., Oztoprak, F., Nocedal, J., and Rennie, S. Newton-like methods for sparse inverse covariance estimation. In NIPS, 2012.
30. Recht, B., Fazel, M., and Parrilo, P. A. Guaranteed minimum rank solutions to linear matrix equations via nuclear norm min imization. SIAM Review, 52(3)471-501, 2010.
31. Shalev-Shwartz, S., Gonen, A., and Shamir, O. Large-scale con vex minimization with a low-rank constraint. In ICML, pp. 329-336, 2011.
32. Toh, K.-C. and Yun, S. An accelerated proximal gradient algo rithm for nuclear norm regularized least squares problems. J. Optimization, 6:615-640, 2010.
33. Watson, G. A. Characterization of the subdifferential of some matrix norms. Linear Algebra and its Applications, 170:33-45, 1992.
34. Wen, Z., Yin, W., and Zhang. Y. Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm. Mathemetical Programming Computa tion, 4(4):333-361, 2012.
35. Yu, Y. and Schuurmans, D. Rank/norm regularization with closed-form solutions: application to subspace clustering. In UAI, pp. 778-785, 2011.
36. Yuan, G.-X., Ho, C.-H., and Lin, C.-J. An improved GLMNET for LI-regularized logistic regression. JMLR, 13:1999-2030, 2012.
37. Yuan, M., Ekici, A., Lu, Z., and Monteiro, R. Dimension reduc tion and coefficient estimation in multivariate linear regression. J. R. Statist. Soc, pp. 329-346, 2007.
38. Zhang, X., Yu, Y., and Schuurmans, D. Accelerated training for matrix-norm regularization: A boosting approach. In NIPS, 2012.