Guaranteed Matrix Completion via Nonconvex Factorization

Proceedings - Annual IEEE Symposium on Foundations of Computer Science, FOCS

Volumn 2015-December, Issue , 2015, Pages 270-289

  Sun, Ruoyu ^a   Luo, Zhi Quan ^b  

^a UNIVERSITY OF MINNESOTA   (United States)

^b THE CHINESE UNIVERSITY OF HONG KONG   (China)

Author keywords

matrix completion; matrix factorization; nonconvex optimization; perturbation analysis

Indexed keywords

ALGORITHMS; FACTORIZATION; ITERATIVE METHODS; MATRIX ALGEBRA; RECOVERY; STOCHASTIC SYSTEMS;

ALTERNATING MINIMIZATION; MATRIX COMPLETION; MATRIX FACTORIZATIONS; NONCONVEX OPTIMIZATION; PERTURBATION ANALYSIS; STANDARD OPTIMIZATION; STOCHASTIC GRADIENT DESCENT; THEORETICAL GUARANTEES;

OPTIMIZATION;

EID: 84960383894     PISSN: 02725428     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/FOCS.2015.25     Document Type: Conference Paper

References (59)

1. Y. Koren, R. Bell, and C. Volinsky, "Matrix factorization techniques for recommender systems, " Computer, vol. 42, no. 8, pp. 30-37, 2009.
2. P. Chen and D. Suter, "Recovering the missing components in a large noisy low-rank matrix: Application to SFM, " IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 8, pp. 1051-1063, 2004.
3. Z. Liu and L. Vandenberghe, "Interior-point method for nuclear norm approximation with application to system identification, " SIAM Journal on Matrix Analysis and Applications, vol. 31, no. 3, pp. 1235-1256, 2009.
4. E. J. Candès and B. Recht, "Exact matrix completion via convex optimization, " Foundations of Computational mathematics, vol. 9, no. 6, pp. 717-772, 2009.
5. E. J. Candès and T. Tao, "The power of convex relaxation: Near-optimal matrix completion, " IEEE Transactions on Information Theory, vol. 56, no. 5, pp. 2053-2080, 2010.
6. D. Gross, "Recovering low-rank matrices from few coefficients in any basis, " IEEE Transactions on Information Theory, vol. 57, no. 3, pp. 1548-1566, 2011.
7. B. Recht, "A simpler approach to matrix completion, " The Journal of Machine Learning Research, vol. 12, pp. 3413-3430, 2011.
8. J.-F. Cai, E. J. Candès, and Z. Shen, "A singular value thresholding algorithm for matrix completion, " SIAM Journal on Optimization, vol. 20, no. 4, pp. 1956-1982, 2010.
9. S. Ma, D. Goldfarb, and L. Chen, "Fixed point and bregman iterative methods for matrix rank minimization, " Mathematical Programming, vol. 128, no. 1-2, pp. 321-353, 2011.
10. K.-C. Toh and S. Yun, "An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems, " Pacific Journal of Optimization, vol. 6, no. 615-640, p. 15, 2010.
11. A. Agarwal, S. Negahban, and M. J. Wainwright, "Fast global convergence of gradient methods for high-dimensional statistical recovery, " The Annals of Statistics, vol. 40, no. 5, pp. 2452-2482, 2012.
12. K. Hou, Z. Zhou, A. M.-C. So, and Z.-Q. Luo, "On the linear convergence of the proximal gradient method for trace norm regularization, " in Advances in Neural Information Processing Systems (NIPS), 2013, pp. 710-718.
13. A. P. Singh and G. J. Gordon, "A unified view of matrix factorization models, " in Machine Learning and Knowledge Discovery in Databases. Springer, 2008, pp. 358-373.
14. G. Takács, I. Pilászy, B. Németh, and D. Tikk, "Major components of the gravity recommendation system, " ACM SIGKDD Explorations Newsletter, vol. 9, no. 2, pp. 80-83, 2007.
15. H. Keshavan, "Efficient algorithms for collaborative filtering, " Ph. D. dissertation, Stanford University, 2012.
16. P. Jain, P. Netrapalli, and S. Sanghavi, "Low-rank matrix completion using alternating minimization, " in Proceedings of the forty-fifth annual ACM symposium on Theory of computing (STOC). ACM, 2013, pp. 665-674.
17. M. Hardt, "Understanding alternating minimization for matrix completion, " in 2014 IEEE 55th Annual Symposium on Foundations of Computer Science (FOCS). IEEE, 2014, pp. 651-660.
18. M. Hardt and M. Wootters, "Fast matrix completion without the condition number, " in Proceedings of The 27th Conference on Learning Theory (COLT), 2014, pp. 638-678.
19. Y. Zhou, D. Wilkinson, R. Schreiber, and R. Pan, "Large-scale parallel collaborative filtering for the netflix prize, " in Algorithmic Aspects in Information and Management. Springer, 2008, pp. 337-348.
20. Z. Wen, W. Yin, and Y. Zhang, "Solving a low-rank factorization model for matrix completion by a nonlinear successive overrelaxation algorithm, " Mathematical Programming Computation, vol. 4, no. 4, pp. 333-361, 2012.
21. S. Funk, "Netflix update: Try this at home, " http://sifter. org/ simon/journal/20061211. html.
22. A. Paterek, "Improving regularized singular value decomposition for collaborative filtering, " in Proceedings of KDD cup and workshop, vol. 2007, 2007, pp. 5-8.
23. R. Gemulla, E. Nijkamp, P. J. Haas, and Y. Sismanis, "Large-scale matrix factorization with distributed stochastic gradient descent, " in Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2011, pp. 69-77.
24. B. Recht and C. Ré, "Parallel stochastic gradient algorithms for large-scale matrix completion, " Mathematical Programming Computation, vol. 5, no. 2, pp. 201-226, 2013.
25. Y. Zhuang, W.-S. Chin, Y.-C. Juan, and C.-J. Lin, "A fast parallel sgd for matrix factorization in shared memory systems, " in Proceedings of the 7th ACM Conference on Recommender Systems. ACM, 2013, pp. 249-256.
26. I. Pilászy, D. Zibriczky, and D. Tikk, "Fast als-based matrix factorization for explicit and implicit feedback datasets, " in Proceedings of the fourth ACM conference on Recommender systems. ACM, 2010, pp. 71-78.
27. H.-F. Yu, C.-J. Hsieh, S. Si, and I. S. Dhillon, "Scalable coordinate descent approaches to parallel matrix factorization for recommender systems. " in ICDM, 2012, pp. 765-774.
28. T. Hastie, R. Mazumder, J. Lee, and R. Zadeh, "Matrix completion and low-rank svd via fast alternating least squares, " arXiv preprint arXiv:1410. 2596, 2014.
29. R. Sun, "Matrix completion via nonconvex factorization: Algorithms and theory, " Ph. D. dissertation, University of Minnesota, 2015.
30. R. H. Keshavan, A. Montanari, and S. Oh, "Matrix completion from a few entries, " IEEE Transactions on Information Theory, vol. 56, no. 6, pp. 2980-2998, 2010.
31. E. Candès, X. Li, and M. Soltanolkotabi, "Phase retrieval via wirtinger flow: Theory and algorithms, " arXiv preprint arXiv:1407. 1065, 2014.
32. D. Gross, Y.-K. Liu, S. T. Flammia, S. Becker, and J. Eisert, "Quantum state tomography via compressed sensing, " arXiv preprint, http://arxiv. org/abs/0909. 3304v1, 2009.
33. P. Jain and P. Netrapalli, "Fast exact matrix completion with finite samples, " arXiv preprint arXiv:1411. 1087, 2014.
34. C. De Sa, K. Olukotun, and C. Ré, "Global convergence of stochastic gradient descent for some nonconvex matrix problems, " arXiv preprint arXiv:1411. 1134, 2014.
35. P. Netrapalli, P. Jain, and S. Sanghavi, "Phase retrieval using alternating minimization, " in Advances in Neural Information Processing Systems (NIPS), 2013, pp. 2796-2804.
36. C.-H. Zhang and T. Zhang, "A general theory of concave regularization for high-dimensional sparse estimation problems, " Statistical Science, vol. 27, no. 4, pp. 576-593, 2012.
37. P.-L. Loh and M. Wainwright, "Regularized m-estimators with nonconvexity: Statistical and algorithmic theory for local optima, " in Advances in Neural Information Processing Systems, 2013, pp. 476-484.
38. J. Fan, L. Xue, and H. Zou, "Strong oracle optimality of folded concave penalized estimation, " The Annals of Statistics, vol. 42, no. 3, pp. 819-849, 2014.
39. X.-T. Yuan and T. Zhang, "Truncated power method for sparse eigenvalue problems, " The Journal of Machine Learning Research, vol. 14, no. 1, pp. 899-925, 2013.
40. Z. Wang, H. Lu, and H. Liu, "Nonconvex statistical optimization: Minimax-optimal sparse pca in polynomial time, " arXiv preprint arXiv:1408. 5352, 2014.
41. P. Netrapalli, U. Niranjan, S. Sanghavi, A. Anandkumar, and P. Jain, "Non-convex robust pca, " in Advances in Neural Information Processing Systems, 2014, pp. 1107-1115.
42. S. Balakrishnan, M. Wainwright, and B. Yu, "Statistical guarantees for the EM algorithm: From population to sample-based analysis, " arXiv preprint arXiv:1408. 2156, 2014.
43. Z. Wang, Q. Gu, Y. Ning, and H. Liu, "High dimensional expectation-maximization algorithm: Statistical optimization and asymptotic normality, " arXiv preprint arXiv:1412. 8729, 2014.
44. U. Feige and E. Ofek, "Spectral techniques applied to sparse random graphs, " Random Structures & Algorithms, vol. 27, no. 2, pp. 251-275, 2005.
45. P.-A?. Wedin, "Perturbation bounds in connection with singular value decomposition, " BIT Numerical Mathematics, vol. 12, no. 1, pp. 99-111, 1972.
46. Y. Chen, S. Bhojanapalli, S. Sanghavi, and R. Ward, "Coherent matrix completion, " in Proceedings of The 31st International Conference on Machine Learning (ICML), 2014, pp. 674-682.
47. S. Bhojanapalli and P. Jain, "Universal matrix completion, " arXiv preprint arXiv:1402. 2324, 2014.
48. W. I. Zangwill, "Non-linear programming via penalty functions, " Management science, vol. 13, no. 5, pp. 344-358, 1967.
49. D. P. Bertsekas, "Nonlinear programming, " 1999.
50. P. Tseng, "Convergence of a block coordinate descent method for nondifferentiable minimization, " Journal of optimization theory and applications, vol. 109, no. 3, pp. 475-494, 2001.
51. Y. Nesterov, "Efficiency of coordinate descent methods on huge-scale optimization problems, " SIAM Journal on Optimization, vol. 22, no. 2, pp. 341-362, 2012.
52. M. Razaviyayn, M. Hong, and Z.-Q. Luo, "A unified convergence analysis of block successive minimization methods for nonsmooth optimization, " SIAM Journal on Optimization, vol. 23, no. 2, pp. 1126-1153, 2013.
53. H. Baligh, M. Hong, W.-C. Liao, Z.-Q. Luo, M. Razaviyayn, M. Sanjabi, and R. Sun, "Cross-layer provision of future cellular networks: A WMMSE-based approach, " IEEE Signal Processing Magazine, vol. 31, no. 6, pp. 56-68, 2014.
54. M. Hong, R. Sun, H. Baligh, and Z.-Q. Luo, "Joint base station clustering and beamformer design for partial coordinated transmission in heterogeneous networks, " IEEE Journal on Selected Areas in Communications (JSAC), vol. 31, no. 2, pp. 226-240, February 2013.
55. L. Grippo and M. Sciandrone, "On the convergence of the block nonlinear gauss-seidel method under convex constraints, " Operations Research Letters, vol. 26, no. 3, pp. 127-136, 2000.
56. R. Sun, Z.-Q. Luo, and Y. Ye, "On the expected convergence of randomly permuted ADMM, " arXiv preprint arXiv:1503. 06387, 2015.
57. Z.-Q. Luo and P. Tseng, "Analysis of an approximate gradient projection method with applications to the backpropagation algorithm, " Optimization Methods and Software, vol. 4, no. 2, pp. 85-101, 1994.
58. D. P. Bertsekas and J. N. Tsitsiklis, "Gradient convergence in gradient methods with errors, " SIAM Journal on Optimization, vol. 10, no. 3, pp. 627-642, 2000.
59. G. W. Stewart, "Perturbation theory for the singular value decomposition, " 1998.