Low-rank matrix completion by Riemannian optimization

SIAM Journal on Optimization

Volumn 23, Issue 2, 2013, Pages 1214-1236

  Vandereycken, Bart ^a  

^a EPFL   (Switzerland)

Author keywords

Differential geometry; Low rank matrices; Matrix completion; Newton; Nonlinear conjugate gradients; Optimization on manifolds; Riemannian manifolds

Indexed keywords

DIFFERENTIAL GEOMETRY; LOW-RANK MATRICES; MATRIX COMPLETION; NONLINEAR CONJUGATE GRADIENT; OPTIMIZATION ON MANIFOLDS; RIEMANNIAN MANIFOLD;

ALGORITHMS; CONJUGATE GRADIENT METHOD; GEOMETRY; MATRIX ALGEBRA; NEWTON-RAPHSON METHOD;

OPTIMIZATION;

EID: 84880713057     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/110845768     Document Type: Article

References (53)

1. P.-A. Absil, C. G. Baker, and K. A. Gallivan, Trust-region methods on Riemannian manifolds, Found. Comput. Math., 7 (2007), pp. 303-330.
2. P.-A. Absil, R. Mahony, and R. Sepulchre, Optimization Algorithms on Matrix Manifolds, Princeton University Press, Princeton, NJ, 2008.
3. P.-A. Absil and J. Malick, Projection-like retractions on matrix manifolds, SIAM J. Optim., 22 (2012), pp. 135-158.
4. R. L. Adler, J.-P. Dedieu, J. Y. Margulies, M. Martens, and M. Shub, Newton's method on Riemannian manifolds and a geometric model for the human spine, IMA J. Numer. Anal., 22 (2002), pp. 359-390.
5. L. Balzano, R. Nowak, and B. Recht, Online identification and tracking of subspaces from highly incomplete information, in Proceedings of the Allerton Conference on Communication, Control, and Computing, Momticello, IL, 2010.
6. J. Bennett and S. Lanning, The Netflix Prize, in Proceedings of the KDD-Cup andWorkshop at the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Jose, CA, 2007.
7. D. P. Bertsekas, Nonlinear Programming, Athena Scientific, Nashua, NH, 1999.
8. N. Boumal and P.-A. Absil, RTRMC: A Riemannian trust-region method for low-rank matrix completion, in Proceedings of the Neural Information Processing Systems Conference (NIPS), Granada, Spain, 2011.
9. N. Boumal and P.-A. Absil, Low-Rank Matrix Completion via Trust-Regions on the Grassmann Manifold, Technical report 2012.07, Université catholique de Louvain, INMA, Louvain-La-Neuve, Belgium, 2012.
10. W. Bruns and U. Vetter, Determinantal rings, Lecture Notes in Math. 1327, Springer-Verlag, Berlin, 1988.
11. S. Burer and R. D. C. Monteiro, A nonlinear programming algorithm for solving semidefinite programs via low-rank factorization, Math. Program., 95 (2003), pp. 329-357.
12. S. Burer and R. D. C. Monteiro, Local minima and convergence in low-rank semidefinite programming, Math. Program., 103 (2005), pp. 427-444.
13. J.-F. Cai, E. J. Candès, and Z. Shen, A singular value thresholding algorithm for matrix completion, SIAM J. Optim., 20 (2010), pp. 1956-1982.
14. E. J. Cand'es and Y. Plan, Matrix completion with noise, Proceedings of the IEEE, 98 (2010), pp. 925-936.
15. E. J. Candès and T. Tao, The power of convex relaxation: Near-optimal matrix completion, IEEE Trans. Inform. Theory, 56 (2009), pp. 2053-2080.
16. E. Candès and B. Recht, Exact matrix completion via convex optimization, Found. Comput. Math., 9 (2009), pp. 717-772.
17. T. P. Cason, P.-A. Absil, and P. Van Dooren, Iterative methods for low rank approximation of graph similarity matrices, Linear Algebra Appl., 438 (2013), pp. 1863-1882.
18. W. Dai, O. Milenkovic, and E. Kerman, Subspace evolution and transfer (SET) for low-rank matrix completion, IEEE Trans. Signal Process., 59 (2011), pp. 3120-3132.
19. W. Dai and O. Milenkovic, SET: An algorithm for consistent matrix completion, in International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Dallas, TX, 2010, pp. 3646-3649.
20. A. Edelman, T. A. Arias, and S. T. Smith, The geometry of algorithms with orthogonality constraints, SIAM J. Matrix Anal. Appl., 20 (1999), pp. 303-353.
21. D. Goldfarb and S. Ma, Convergence of fixed point continuation algorithms for matrix rank minimization, Found. Comput. Math., 11 (2011), pp. 183-210.
22. G. H. Golub and C. F. Van Loan, Matrix Computations, 3rd ed., Johns Hopkins Stud. Math. Sci., Baltimore, MD, 1996.
23. G. H. Golub and V. Pereyra, The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate, SIAM J. Numer. Anal., 10 (1973), pp. 413-432.
24. H. Harbrecht, M. Peters, and R. Schneider, On the low-rank approximation by the pivoted Cholesky decomposition, Appl. Numer. Math., 62 (2012), pp. 428-440.
25. U. Helmke and J. B. Moore, Optimization and Dynamical Systems, Springer-Verlag, London, 1994.
26. M. Journée, F. Bach, P.-A. Absil, and R. Sepulchre, Low-rank optimization on the cone of positive semidefinite matrices, SIAM J. Optim., 20 (2010), pp. 2327-2351.
27. R. H. Keshavan, A. Montanari, and S. Oh, Matrix completion from a few entries, IEEE Trans. Inform. Theory, 56 (2010), pp. 2980-2998.
28. R. Keshavan, A. Montanari, and S. Oh, Matrix completion from noisy entries, J. Mach. Learn. Res., 11 (2010), pp. 2057-2078.
29. O. Koch and C. Lubich, Dynamical low-rank approximation, SIAM J. Matrix Anal. Appl., 29 (2007), pp. 434-454.
30. D. Kressner and C. Tobler, Krylov subspace methods for linear systems with tensor product structure, SIAM J. Matrix Anal. Appl., 31 (2010), pp. 1688-1714.
31. R. M. Larsen, PROPACK - Software for Large and Sparse SVD Calculations. http://soi. stanford.edu/∼rmunk/PROPACK (2004).
32. J. M. Lee, Introduction to smooth manifolds, Grad. Texts in Math. 218, Springer-Verlag, New York, 2003.
33. K. Lee and Y. Bresler, ADMiRA: Atomic decomposition for minimum rank approximation, IEEE Trans. Inform. Theory, 56 (2010), pp. 4402-4416.
34. A. S. Lewis and J. Malick, Alternating projections on manifolds, Math. Oper. Res., 33 (2008), pp. 216-234.
35. Z. Lin, M. Chen, L. Wu, and Yi Ma, The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices, Technical report UILU-ENG-09-2215, University of Illinois, Urbana, IL, 2009.
36. Y.-J. Liu, D. Sun, and K.-C. Toh, An implementable proximal point algorithmic framework for nuclear norm minimization, Math. Program., 133 (2012), pp. 399-436.
37. S. Ma, D. Goldfarb, and L. Chen, Fixed point and Bregman iterative methods for matrix rank minimization, Math. Program., 128 (2011), pp. 321-353.
38. R. Meka, P. Jain, and I. S. Dhillon, Guaranteed rank minimization via singular value projection, in Proceedings of the Neural Information Processing Systems Conference (NIPS), 2010.
39. G. Meyer, S. Bonnabel, and R. Sepulchre, Linear regression under fixed-rank constraints: A Riemannian approach, in Proceedings of the 28th International Conference on Machine Learning (ICML2011), Bellevue, WA, 2011.
40. G. Meyer, S. Bonnabel, and R. Sepulchre, Regression on fixed-rank positive semidefinite matrices: A Riemannian approach, J. Mach. Learn. Res., 12 (2011), pp. 593-625.
41. G. Meyer, Geometric Optimization Algorithms for Linear Regression on Fixed-Rank Matrices, Ph.D. thesis, University of Liège, Liège, Belgium, 2011.
42. M. Michenkov'a, Numerical algorithms for low-rank matrix completion problems, http://www. math.ethz.ch/∼kressner/students/michenkova.pdf (2011).
43. B. Mishra, G. Meyer, F. Bach, and R. Sepulchre, Low-Rank Optimization with Trace Norm Penalty, preprint, 2011, http://arxiv.org/abs/1112.2318.
44. B. Mishra, G. Meyer, S. Bonnabel, and R. Sepulchre, Fixed-Rank Matrix Factorizations and Riemannian Low-Rank Optimization, preprint, 2012, http:arxiv.org/abs/1209.0430.
45. R. Orsi, U. Helmke, and J. B. Moore, A Newton-like method for solving rank constrained linear matrix inequalities, Automatica, 42 (2006), pp. 1875-1882.
46. C. Qi, K. A. Gallivan, and P.-A. Absil, Riemannian BFGS algorithm with applications, in Recent Advances in Optimization and its Applications in Engineering, Springer, to appear.
47. U. Shalit, D. Weinshall, and G. Chechik, Online learning in the manifold of low-rank matrices, in Advances in Neural Inf. Process. Syst. 23, J. Lafferty, C. K. I. Williams, J. Shawe-Taylor, R. S. Zemel, and A. Culotta, eds., 2010, pp. 2128-2136.
48. U. Shalit, D. Weinshall, and G. Chechik, Online learning in the embedded manifold of low-rank matrices, J. Mach. Learn. Res., 13 (2012), pp. 429-458.
49. S. T. Smith, Optimization techniques on Riemannian manifold, in Hamiltonian and Gradient Flows, Algorithms and Control, vol. 3, AMS, Providence, RI, 1994, pp. 113-136.
50. K. Toh and S. Yun, An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems, Pacific J. Optimization, 6 (2010), pp. 615-640.
51. B. Vandereycken and S. Vandewalle, A Riemannian optimization approach for computing low-rank solutions of Lyapunov equations, SIAM J. Matrix Anal. Appl., 31 (2010), pp. 2553-2579.
52. B. Vandereycken, Low-rank matrix completion by Riemannian optimization (extended version), preprint, http://arxiv.org/abs/1209.3834.
53. Z.Wen, W. Yin, and Yin Zhang, Solving a low-rank factorization model for matrix completion by a non-linear successive over-relaxation algorithm, Math. Program. Comput., 4 (2012), pp. 333-361.