A singular value thresholding algorithm for matrix completion

SIAM Journal on Optimization

Volumn 20, Issue 4, 2010, Pages 1956-1982

  Cai, Jian Feng ^a   Candès, Emmanuel J ^b   Shen, Zuowei ^c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^c NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Lagrange dual function; Linearized Bregman iteration; Matrix completion; Nuclear norm minimization; Singular value thresholding; Uzawa's algorithm

Indexed keywords

BREGMAN ITERATION; LAGRANGE DUAL; SINGULAR VALUES; THRESHOLDING; UZAWA'S ALGORITHMS;

ALGORITHMS; LAGRANGE MULTIPLIERS; LINEARIZATION; OPTIMIZATION; RELAXATION PROCESSES;

COMPUTATIONAL EFFICIENCY;

EID: 77951291046     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/080738970     Document Type: Article

References (68)

1. ACM SIGKDD and Netflix, Proceedings of kdd cup and workshop, http://www.cs.uic.edu/liub/KDD-cup-2007/proceedings.html.
2. J. Abernethy, F. Bach, T. Evgeniou, and J. P. Vert, Low-rank matrix factorization with attributes, Arxiv preprint cs/0611124, 2006.
3. Y. Amit, M. Fink, N. Srebro, and S. Ullman, Uncovering shared structures in multiclass classification, in Proceedings of the 24th International Conference on Machine Learning, ACM, Providence, RI, 2007, pp. 17-24.
4. A. Argyriou, T. Evgeniou, and M. Pontil, Multi-task feature learning, Adv. Neural Inform. Process. Syst., 19 (2007), pp. 41-48.
5. J. Bect, L. Blanc-Feraud, G. Aubert, and A. Chambolle, A-unified variational framework for image restoration, in Proc. ECCV, Vol.3024, Springer, New York, 2004, pp. 1-13.
6. D. Bertsekas, On the Goldstein-Levitin-Polyak gradient projection method, IEEE Trans. Automat. Control, 21 (1976), pp. 174-184.
7. S. P. Boyd and L. Vandenberghe, Convex Optimization, Cambridge University Press, London, 2004.
8. J. Burkardt, Cities-City distance datasets, http://people.sc.fsu.edu/- burkardt/datasets/cities/cities.html.
9. J.-F. Cai, R. H. Chan, L. Shen, and Z. Shen, Restoration of chopped and nodded images by framelets, SIAM J. Sci. Comput., 30 (2008), pp. 1205-1227.
10. J.-F. Cai, R. H. Chan, and Z. Shen, A framelet-based image inpainting algorithm, Appl. Comput. Harmon. Anal., 24 (2008), pp. 131-149.
11. J.-F. Cai, S. Osher, and Z. Shen, Convergence of the linearized Bregman iteration for-1-norm minimization, Math. Comp., 78 (2009), pp. 2127-2136.
12. J.-F. Cai, S. Osher, and Z. Shen, Linearized Bregman iterations for compressed sensing, Math. Comp., 78 (2009), pp. 1515-1536.
13. J.-F. Cai, S. Osher, and Z. Shen, Linearized Bregman iterations for frame-based image deblurring, SIAM J. Imaging Sci., 2 (2009), pp. 226-252.
14. E. J. Candès and F. Guo, New multiscale transforms, minimum total variation synthesis: Applications to edge-preserving image reconstruction, Signal Process., 82 (2002), pp. 1519-1543.
15. E. J. Candès and Y. Plan, Matrix completion with noise, Proc. IEEE, to appear.
16. E. J. Candès and B. Recht, Exact matrix completion via convex optimization, Found. Comput. Math., 9 (2009), pp. 717-772.
17. E. Candès and J. Romberg, Sparsity and incoherence in compressive sampling, Inverse Problems,23 (2007), pp. 969-985.
18. E. Candès and T. Tao, The Dantzig selector: Statistical estimation when p is much larger than n, Ann. Statist., 35 (2007), pp. 2313-2351.
19. E. J. Candès, J. Romberg, and T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inform. Theory, 52 (2006), pp. 489-509.
20. E. J. Candès and T. Tao, Decoding by linear programming, IEEE Trans. Inform. Theory, 51 (2005), pp. 4203-4215.
21. E. J. Candès and T. Tao, Near-optimal signal recovery from random projections: Universal encoding strategies?, IEEE Trans. Inform. Theory, 52 (2006), pp. 5406-5425.
22. A. Chai and Z. Shen, Deconvolution: A wavelet frame approach, Numer. Math., 106 (2007), pp. 529-587.
23. R. H. Chan, T. F. Chan, L. Shen, and Z. Shen, Wavelet algorithms for high-resolution image reconstruction, SIAM J. Sci. Comput., 24 (2003), pp. 1408-1432.
24. P. Chen and D. Suter, Recovering the missing components in a large noisy low-rank matrix: Application to SFM, IEEE Trans. Pattern Anal. Machine Intelligence, 26 (2004), pp. 1051-1063.
25. Y. C. Cheng, On the gradient-projection method for solving the nonsymmetric linear complementarity problem, J. Optim. Theory Appl., 43 (1984), pp. 527-541.
26. P. L. Combettes and V. R. Wajs, Signal recovery by proximal forward-backward splitting, Multiscale Model. Simul., 4 (2005), pp. 1168-1200.
27. J. Darbon and S. Osher, Fast discrete optimization for sparse approximations and deconvolutions, Department of Mathematics, UCLA, preprint, 2007.
28. I. Daubechies, M. Defrise, and C. De Mol, An iterative thresholding algorithm for linear inverse problems with a sparsity constraint, Comm. Pure Appl. Math., 57 (2004), pp. 1413-1457.
29. I. Daubechies, G. Teschke, and L. Vese, Iteratively solving linear inverse problems under general convex constraints, Inverse Probl. Imaging, 1 (2007), pp. 29-46.
30. D. L. Donoho, Compressed sensing, IEEE Trans. Inform. Theory, 52 (2006), pp. 1289-1306.
31. B.C. Eaves, On the basic theorem of complementarity, Math. Program., 1 (1971), pp. 68-75.
32. M. Elad, J.-L. Starck, P. Querre, and D. L. Donoho, Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA), Appl. Comput. Harmon. Anal., 19 (2005), pp. 340-358.
33. M. J. Fadili, J.-L. Starck, and F. Murtagh, Inpainting and zooming using sparse representations, Comput. J., 52 (2009), pp. 64-79.
34. M. Fazel, Matrix rank minimization with applications, Ph.D. thesis, Stanford University, Stanford,CA, 2002.
35. M. Fazel, H. Hindi, and S. P. Boyd, Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices, Proc. Amer. Control Conf., 3 (2003), pp. 2156-2162.
36. M. A. T. Figueiredo and R. D. Nowak, An EM algorithm for wavelet-based image restoration, IEEE Trans. Image Process., 12 (2003), pp. 906-916.
37. M. Fukushima, Z.-Q. Luo, and P. Tseng, Smoothing functions for second-order-cone complementarity problems, SIAM J. Optim., 12 (2001/2002), pp. 436-460.
38. A. A. Goldstein, Convex programming in Hilbert space, Bull. Amer. Math. Soc., 70 (1964), pp. 709-710.
39. T. Goldstein and S. Osher, The split Bregman method for L1-regularized problems, SIAM J. Imaging Sci., 2 (2009), pp. 323-343.
40. E. T. Hale, W. Yin, and Y. Zhang, Fixed-point continuation for l1-minimization: Methodology and convergence, SIAM J. Optim., 19 (2008), pp. 1107-1130.
41. N. J. Higham, Functions of matrices, Theory and computation, SIAM, Philadelphia, 2008.
42. J.-B. Hiriart-Urruty and C. Lemaréchal, Convex analysis and minimization algorithms. I, Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], Vol.305, Springer-Verlag, Berlin, 1993.
43. A. N. Iusem, On the convergence properties of the projected gradient method for convex optimization, Comput. Appl. Math., 22 (2003), pp. 37-52.
44. R. M. Larsen, PROPACK-Software for large and sparse SVD calculations, http://sun.stanford.edu/-rmunk/PROPACK/.
45. E. S. Levitin and B. T. Poliak, Constrained minimization methods (Extremum problems from functional-analytic point of view, discussing methods of solving and convergence conditions), USSR Comput. Math. Math. Phys., 6 (1966), pp. 1-50.
46. A. S. Lewis, The mathematics of eigenvalue optimization, Math. Program., 97 (2003), pp. 155-176.
47. E. Liberty, F. Woolfe, P. G. Martinsson, V. Rokhlin, and M. Tygert, Randomized algorithms for the low-rank approximation of matrices, Proc. Nat. Acad. Sci. USA, 104 (2007), pp. 20167-20172.
48. S. Lintner and F. Malgouyres, Solving a variational image restoration model which involves L8 constraints, Inverse Prob., 20 (2004), pp. 815-831.
49. Y.-J. Liu, D. F. Sun, and K. C. Toh, An Implementable Proximal Point Algorithmic Framework for Nuclear Norm Minimization, Department of Mathematics, National University of Singapore, preprint, 2009
50. Z. Liu and L. Vandenberghe, Interior-point method for nuclear norm approximation with application to system identification, SIAM J. Matrix Anal. Appl., 31 (2009), pp. 1235-1256.
51. S. Ma, D. Goldfarb, and L. Chen, Fixed point and Bregman iterative methods for matrix rank minimization, Math. Program., to appear.
52. P. Marcotte and J. H. Wu, On the convergence of projection methods: Application to the decomposition of affine variational inequalities, J. Optim. Theory Appl., 85 (1995), pp. 347-362.
53. M. Mesbahi and G. P. Papavassilopoulos, On the rank minimization problem over a positive semidefinitelinear matrix inequality, IEEE Trans. Automat. Control, 42 (1997), pp. 239-243.
54. S. Osher, M. Burger, D. Goldfarb, J. Xu, and W. Yin, An iterative regularization method for total variation-based image restoration, Multiscale Model. Simul., 4 (2005), pp. 460-489.
55. S. Osher, Y. Mao, B. Dong, and W. Yin, Fast linearized Bregman iteration for compressive sensing and sparse denoising, Commun. Math. Sci., 8 (2010), pp. 93-111.
56. B. Recht, M. Fazel, and P. A. Parrilo, Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, SIAM Rev., to appear.
57. R. T. Rockafellar, Convex analysis, Princeton Mathematical Series 28, Princeton University Press, Princeton, NJ, 1970.
58. J. L. Starck, D. L. Donoho, and E. J. Candès, Astronomical image representation by the curvelet transform, Astronom. Astrophys., 398 (2003), pp. 785-800.
59. K. C. Toh, M. J. Todd, and R. H. Tütüncü, SDPT3-A Matlab software package for semidefinite-quadratic-linear programming, version 3.0, 2001, .
60. K.-C. Toh and S. Yun, An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems, Pacific J. Optim., to appear.
61. C. Tomasi and T. Kanade, Shape and motion from image streams under orthography: A factorization method, Int. J. Comput. Vision, 9 (1992), pp. 137-154.
62. P. Tseng, Applications of a splitting algorithm to decomposition in convex programming and variational inequalities., SIAM J. Control Optim., 29 (1991), pp. 119-138.
63. P. Tseng, A modified forward-backward splitting method for maximal monotone mappings, SIAM J. Control Optim., 38 (2000), pp. 431-446.
64. G.A. Watson, Characterization of the subdifferential of some matrix norms, Linear Algebra Appl., 170 (1992), pp. 33-45.
65. F. Woolfe, E. Liberty, V. Rokhlin, and M. Tygert, A fast randomized algorithm for the approximation of matrices, Appl. Comput. Harmonic Anal., 25 (2008), pp. 335-366.
66. S. J. Wright, R. Nowak, and M. Figueiredo, Sparse reconstruction by separable approximation, IEEE Trans. Signal Process., 57 (2009), pp. 2479-2493.
67. W. Yin, S. Osher, D. Goldfarb, and J. Darbon, Bregman iterative algorithms for-1-minimization with applications to compressed sensing, SIAM J. Imaging Sci., 1 (2008), pp. 143-168.
68. D. L. Zhu and P. Marcotte, Co-coercivity and its role in the convergence of iterative schemes for solving variational inequalities, SIAM J. Optim., 6 (1996), pp. 714-726.