Optimal exact least squares rank minimization

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Volumn , Issue , 2012, Pages 480-488

  Xiang, Shuo ^a   Zhu, Yunzhang ^b   Shen, Xiaotong ^b   Ye, Jieping ^a  

^a ARIZONA STATE UNIVERSITY   (United States)

^b UNIVERSITY OF MINNESOTA   (United States)

Author keywords

global optimality; nonconvex; rank minimization

Indexed keywords

ESTIMATION ERRORS; GLOBAL OPTIMALITY; GLOBAL SOLUTIONS; LEAST SQUARE; LEAST-SQUARES FORMULATION; LOSS FUNCTIONS; MULTI VARIATE ANALYSIS; NOISY DATA; NONCONVEX; NP-HARD; RANK CONSTRAINTS; SOLUTION PATH;

ALGORITHMS; DATA MINING; ESTIMATION; IMAGE RECONSTRUCTION; PARAMETER ESTIMATION;

OPTIMIZATION;

EID: 84866045291     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2339530.2339609     Document Type: Conference Paper

References (35)

1. J. Abernethy, F. Bach, T. Evgeniou, and J. Vert. Low-rank matrix factorization with attributes. Arxiv preprint cs/0611124, 2006.
2. Y. Amit, M. Fink, N. Srebro, and S. Ullman. Uncovering shared structures in multiclass classification. In Proceedings of the 24th Annual International Conference on Machine Learning, pages 17-24. ACM, 2007.
3. T. Anderson. Estimating linear restrictions on regression coefficients for multivariate normal distributions. The Annals of Mathematical Statistics, 22(3):327-351, 1951.
4. T. Anderson. Asymptotic distribution of the reduced rank regression estimator under general conditions. The Annals of Statistics, 27(4):1141-1154, 1999.
5. T. André, R. Nowak, and B. Van Veen. Low-rank estimation of higher order statistics. Signal Processing, IEEE Transactions on, 45(3):673-685, 1997.
6. A. Argyriou, T. Evgeniou, and M. Pontil. Multi-task feature learning. Advances in Neural Information Processing Systems, 19:41, 2007.
7. F. Bunea, Y. She, and M. Wegkamp. Optimal selection of reduced rank estimators of high-dimensional matrices. The Annals of Statistics, 39(2):1282-1309, 2011.
8. J. Cai, E. Candès, and Z. Shen. A singular value thresholding algorithm for matrix completion. SIAM Journal of Optimization, 20(4):1956-1982, 2010.
9. E. Candes and Y. Plan. Matrix completion with noise. Arxiv preprint arXiv:0903.3131, 2009.
10. E. Candès and B. Recht. Exact matrix completion via convex optimization. Foundations of Computational Mathematics, 9(6):717-772, 2009.
11. J. Chen, J. Liu, and J. Ye. Learning incoherent sparse and low-rank patterns from multiple tasks. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 1179-1188. ACM, 2010.
12. C. Eckart and G. Young. The approximation of one matrix by another of lower rank. Psychometrika, 1(3):211-218, 1936.
13. M. Fazel, H. Hindi, and S. Boyd. A rank minimization heuristic with application to minimum order system approximation. In American Control Conference, 2001. Proceedings of the 2001, volume 6, pages 4734-4739. IEEE, 2001.
14. S. Friedland and A. Torokhti. Generalized rank-constrained matrix approximations. SIAM Journal on Matrix Analysis and Applications, 29(2):656-659, 2007.
15. A. Izenman. Reduced-rank regression for the multivariate linear model. Journal of multivariate analysis, 5(2):248-264, 1975.
16. M. Jaggi and M. Sulovskỳ. A Simple Algorithm for Nuclear Norm Regularized Problems. In Proceedings of the 27th Annual International Conference on Machine Learning, 2010.
17. P. Jain, R. Meka, and I. Dhillon. Guaranteed rank minimization via singular value projection. Advances in Neural Information Processing Systems, 23:937-945, 2010.
18. H. Ji, C. Liu, Z. Shen, and Y. Xu. Robust video denoising using low rank matrix completion. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pages 1791-1798. IEEE, 2010.
19. S. Ji and J. Ye. An accelerated gradient method for trace norm minimization. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 457-464, 2009.
20. B. Kulis, A. Surendran, and J. Platt. Fast low-rank semidefinite programming for embedding and clustering. In Eleventh International Conference on Artifical Intelligence and Statistics, AISTATS 2007, 2007.
21. G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, and Y. Ma. Robust recovery of subspace structures by low-rank representation. Arxiv preprint arXiv:1010.2955, 2010.
22. X. Luo. High dimensional low rank and sparse covariance matrix estimation via convex minimization. Arxiv preprint arXiv:1111.1133, 2011.
23. B. K. Natarajan. Sparse approximation solutions to linear systems. SIAM J. Comput., 24(2):227-234, 1995.
24. S. Negahban and M. J. Wainwright. Estimation of (near) low-rank matrices with noise and high-dimensional scaling. Annals of Statistics, 39(2):1069-1097, 2011.
25. T. Pong, P. Tseng, S. Ji, and J. Ye. Trace norm regularization: Reformulations, algorithms, and multi-task learning. SIAM Journal on Optimization, 20:3465-3489, 2010.
26. C. Rao. Matrix approximations and reduction of dimensionality in multivariate statistical analysis. Multivariate analysis, 5:3-22, 1980.
27. B. Recht, M. Fazel, and P. Parrilo. Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization. SIAM Review, 52(471), 2010.
28. G. Reinsel and R. Velu. Multivariate reduced-rank regression: theory and applications. Springer New York, 1998.
29. B. Savas and I. Dhillon. Clustered low rank approximation of graphs in information science applications. In Proceedings of the SIAM Data Mining Conference., 2011.
30. S. Shalev-Shwartz, A. Gonen, and O. Shamir. Large-Scale Convex Minimization with a Low-Rank Constraint. In Proceedings of the 28th Annual International Conference on Machine Learning, 2011.
31. X. Shen and H. Huang. Optimal model assessment, selection, and combination. Journal of the American Statistical Association, 101(474):554-568, 2006.
32. D. Sondermann. Best approximate solutions to matrix equations under rank restrictions. Statistical Papers, 27(1):57-66, 1986.
33. N. Srebro, J. Rennie, and T. Jaakkola. Maximum-margin matrix factorization. Advances in Neural Information Processing Systems, 17:1329-1336, 2005.
34. K. Toh and S. Yun. An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems. Pacific J. Optim, 6:615-640, 2010.
35. J. Wright, A. Ganesh, S. Rao, and Y. Ma. Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization. Advances in Neural Information Processing Systems, 2009.