Fast and accurate matrix completion via truncated nuclear norm regularization

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 35, Issue 9, 2013, Pages 2117-2130

  Hu, Yao ^a   Zhang, Debing ^a   Ye, Jieping ^b   Li, Xuelong ^c   He, Xiaofei ^a  

^a ZHEJIANG UNIVERSITY   (China)

^b Arizona State University   (United States)

^c XI AN INSTITUTE OF OPTICS AND PRECISION MECHANICS   (China)

Author keywords

accelerated proximal gradient method; alternating direction method of multipliers; Matrix completion; nuclear norm minimization

Indexed keywords

ACCELERATED PROXIMAL GRADIENT METHODS; ALTERNATING DIRECTION METHOD OF MULTIPLIERS; FASTER CONVERGENCE; LOW-RANK MATRIX APPROXIMATIONS; MATRIX COMPLETION; NUCLEAR NORM MINIMIZATIONS; NUCLEAR NORM REGULARIZATIONS; OPTIMIZATION PROBLEMS;

ALGORITHMS; GRADIENT METHODS; OPTIMIZATION; RELAXATION PROCESSES;

PROBLEM SOLVING;

EID: 84880896128     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2012.271     Document Type: Article

References (48)

1. E.J. Candés and B. Recht, "Exact Matrix Completion via Convex Optimization," Foundations on Computational Math., vol. 9, pp. 717-772, 2009.
2. E.J. Candés and T. Tao, "The Power of Convex Relaxation: Near-Optimal Matrix Completion," IEEE Trans. Information Theory, vol. 56, no. 5, pp. 2053-2080, May 2009.
3. J. Liu, P. Musialski, P. Wonka, and J. Ye, "Tensor Completion for Estimating Missing Values in Visual Data," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 208-220, Jan. 2013.
4. A. Eriksson and A. van den Hengel, "Efficient Computation of Robust Low-Rank Matrix Approximations in the Presence of Missing Data Using the l1 Norm," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
5. T. Okatani, T. Yoshida, and K. Deguchi, "Efficient Algorithm for Low-Rank Matrix Factorization with Missing Components and Performance Comparison of Latest Algorithms," Proc. IEEE Int'l Conf. Computer Vision, 2011.
6. H.-Y. Shum, K. Ikeuchi, and R. Reddy, "Principal Component Analysis with Missing Data and Its Application to Polyhedral Object Modeling," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 9, pp. 854-867, Sept. 1995.
7. S.J. Ting Kei Tong, P. Tseng, and J. Ye, "Trace Norm Regularization: Reformulation, Algorithms, and Multi-Task Learning," SIAM J. Optimization, vol. 20, no. 6, pp. 3465-3489, 2010.
8. N. Komodakis and G. Tziritas, "Image Completion Using Global Optimization," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
9. C. Rasmussen and T. Korah, "Spatiotemporal Inpainting for Recovering Texture Maps of Partially Occluded Building Facades," Proc. IEEE Int'l Conf. Image Processing, 2005.
10. H. Ji, C. Liu, Z. Shen, and Y. Xu, "Robust Video Denoising Using Low Rank Matrix Completion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
11. Y. Koren, "Factorization Meets the Neighborhood: A Multifaceted Collaborative Filtering Model," Proc. 14th ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, 2008.
12. H. Steck, "Training and Testing of Recommender Systems on Data Missing Not at Random," Proc. 16th ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, 2010.
13. B. Recht, M. Fazel, and P.A. Parrilo, "Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization," SIAM Rev., vol. 52, no. 3, pp. 471-501, 2010.
14. S. Ma, D. Goldfarb, and L. Chen, "Fixed Point and Bregman Iterative Methods for Matrix Rank Minimization," Math. Programming, vol. 128, nos. 1-2, pp. 321-353, 2011.
15. B. Recht, "A Simpler Approach to Matrix Completion," J. Machine Learning Research, vol. 12, pp. 413-3430, 2011.
16. E.J. Candés and Y. Plan, "Matrix Completion with Noise," Proc. IEEE, vol. 98, no. 6, pp. 925-936, 2009.
17. J.F. Cai, E.J. Candés, and Z. Shen, "A Singular Value Thresholding Algorithm for Matrix Completion," SIAM J. Optimization, vol. 20, pp. 1956-1982, 2010.
18. K.-C. Toh and S. Yun, "An Accelerated Proximal Gradient Algorithm for Nuclear Norm Regularized Least Squares Problems," Pacific J. Optimization, pp. 615-640, 2010.
19. J. Wright, A. Ganesh, S. Rao, Y. Peng, and Y. Ma, "Robust Principal Component Analysis: Exact Recovery of Corrupted Low-Rank Matrices via Convex Optimization," Proc. Advances in Neural Information Processing Systems, 2009.
20. D. Zhang, Y. Hu, J. Ye, X. Li, and X. He, "Matrix Completion by Truncated Nuclear Norm Regularization," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2012.
21. R.S. Cabral, F. De la Torre, J.P. Costeira, and A. Bernardino, "Matrix Completion for Multi-Label Image Classification," Proc. Advances in Neural Information Processing Systems, 2011.
22. R. Foygel, R.R. Salakhutdinov, O. Shamir, and N. Srebro, "Learning with the Weighted Trace-Norm under Arbitrary Sampling Distributions," Proc. Advances in Neural Information Processing Systems, 2011.
23. A. Goldberg, X.J. Zhu, B. Recht, J. Xu, and R. Nowak, "Transduction with Matrix Completion: Three Birds with One Stone," Proc. Advances in Neural Information Processing Systems, 2010.
24. Y. Peng, A. Ganesh, J. Wright, W. Xu, and Y. Ma, "RASL: Robust Alignment by Sparse and Low-Rank Decomposition for Linearly Correlated Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 34, no. 11, pp. 2233-2246, Nov. 2012.
25. R. Salakhutdinov and N. Srebro, "Collaborative Filtering in a Non-Uniform World: Learning with the Weighted Trace Norm," Proc. Advances in Neural Information Processing Systems, 2010.
26. M. Fazel, H. Hindi, and S.P. Boyd, "A Rank Minimization Heuristic with Application to Minimum Order System Approximation," Proc. Am. Control Conf., pp. 4734-4739. 2001 (Pubitemid 32837075)
27. W. Dai, O. Milenkovic, and E. Kerman, "Subspace Evolution and Transfer (Set) for Low-Rank Matrix Completion," IEEE Trans. Signal Processing, vol. 59, no. 7, pp. 3120-3132, July 2011.
28. M. Fazel, "Matrix Rank Minimization with Applications," PhD thesis, Stanford Univ., 2002.
29. R.H. Tutuncu, K.C. Toh, and M.J. Todd, Sdpt3-a Matlab Software Package for Semidefinite Quadratic Linear Programming, Version 3.0, 2001.
30. J.F. Sturm, Using Sedumi 1.02, a Matlab Toolbox for Optimization over Symmetric Cones, 1998.
31. S. Ji and J. Ye, "An Accelerated Gradient Method for Trace Norm Minimization," Proc. 26th Ann. Int'l Conf. Machine Learning, 2009.
32. N. Srebro, J.D.M. Rennie, and T.S. Jaakkola, "Maximum-Margin Matrix Factorization," Proc. Advances in Neural Information Processing Systems, 2005.
33. J.D.M. Rennie and N. Srebro, "Fast Maximum Margin Matrix Factorization for Collaborative Prediction," Proc. 22nd Int'l Conf. Machine Learning, 2005.
34. G. Taká cs, I. Pilászy, B. Németh, and D. Tikk, "Scalable Collaborative Filtering Approaches for Large Recommender Systems," J. Machine Learning Research, 2009.
35. S. Burer and R.D.C. Monteiro, "Local Minima and Convergence in Low-Rank Semidefinite Programming," Math. Programming, vol. 103, no. 3, pp. 427-444, 2005. (Pubitemid 40925236)
36. R.H. Keshavan, A. Montanari, and S. Oh, "Matrix Completion from Noisy Entries," J. Machine Learning Research, vol. 11, pp. 2057-2078, 2010.
37. R. Meka, P. Jain, and I.S. Dhillon, "Guaranteed Rank Minimization via Singular Value Projection," Proc. Advances in Neural Information Processing Systems, 2010.
38. J. Yang and X. Yuan, "Linearized Augmented Lagrangian and Alternating Direction Methods for Nuclear Norm Minimization," submitted, 2011.
39. Z. Lin, R. Liu, and Z. Su, "Linearized Alternating Direction Method with Adaptive Penalty for Low-Rank Representation," Proc. Advances in Neural Information Processing Systems, 2011.
40. M. Li, J.T. Kwok, and B.-L. Lu, "Making Large-Scale Nyström Approximation Possible," Proc. 27th Ann. Int'l Conf. Machine Learning, 2010.
41. S. Boyd, N. Parikh, E. Chu, and J. Eckstein, "Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers," Information Systems J., vol. 3, no. 1, pp. 1-122, 2010.
42. Y. Nesterov, "Gradient Methods for Minimizing Composite Objective Function," technical report, 2007.
43. A. Beck and M. Teboulle, "A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems," SIAM J. Imaging Sciences, vol. 2, no. 1, pp. 183-202, 2009.
44. Y. Nesterov, "A Method of Solving a Convex Programming Problem with Convergence Rate o(1/sqr(k))," Soviet Math. Doklady, vol. 27, pp. 372-376, 1983.
45. B. He, M. Tao, and X. Yuan, "Alternating Direction Method with Gaussian Back Substitution for Separable Convex Programming," SIAM J. Optimization, vol. 22, no. 2, pp. 313-340, 2012.
46. M. Tao and X. Yuan, "Recovering Low-Rank and Sparse Components of Matrices from Incomplete and Noisy Observations," SIAM J. Optimizaiton, vol. 21, no. 1, pp. 57-81, 2011.
47. B.S. He, H. Yang, and S.L. Wang, "Alternating Direction Method with Self-Adaptive Penalty Parameters for Monotone Variational Inequalities," J. Optimization Theory and Applications, vol. 106, no. 2, pp. 337-356, 2000.
48. R.H. Keshavan, A. Montanari, and S. Oh, "Matrix Completion from a Few Entries," IEEE Trans. Information Theory, vol. 56, pp. 2980-2998, 2010.