Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization

Mathematics of Computation

Volumn 82, Issue 281, 2013, Pages 301-329

  Yang, Junfeng ^a   Yuan, Xiaoming ^b  

^a NANJING UNIVERSITY   (China)

^b HONG KONG BAPTIST UNIVERSITY   (Hong Kong)

Author keywords

Alternating direction method; Augmented Lagrangian method; Convex programming; Linearized; Low rank; Nuclear norm

Indexed keywords

EID: 84871450765     PISSN: 00255718     EISSN: None     Source Type: Journal    
DOI: 10.1090/S0025-5718-2012-02598-1     Document Type: Article

References (58)

1. J. Abernethy, F. Bach, T. Evgeniou and J.-P. Vert, A new approach to collaborative filtering: Operator estimation with spectral regularization, Journal of Machine Learning Research, 10(2009), pp. 803-826.
2. A. Argyriou, T. Evgeniou and M. Pontil, Convex multi-task feature learning, Machine Learning, 73(3)(2008), pp. 243-272.
3. J. M. Borwein, and A. S. Lewis, Convex analysis and nonlinear optimization, Springer-Verlag, 2003.
4. J. F. Cai, E. J. Candés and Z. W. Shen, A singular value thresholding algorithm for matrix completion, SIAM Journal on Optimization, 20(4) (2010), pp. 1956-1982.
5. J. F. Cai, S. Osher and Z. Shen, Linearized Bregman iterations for compressed sensing, Mathematics of Computation, 78(267)(2009), pp. 1515-1536.
6. 1-Norm Minimization, Mathematics of Computation, 78(268) (2009), pp. 2127-2136
7. J. F. Cai, S. Osher and Z. Shen, Split Bregman Methods and Frame Based Image Restoration, Multiscale Model. Simul., 8(2)(2009), pp. 337-369.
8. E. J. Candès and B. Recht, Exact matrix completion via convex optimization, Foundations of Computational Mathematics, 9(2009), pp. 717-772.
9. E. J. Cand'es and T. Tao, The power of convex relaxation: near-optimial matrix completion, IEEE Transactions on Information Theory, 56(5) (2009), pp. 2053-2080.
10. C. H. Chen, B. S. He and X. M. Yuan, Matrix completion via alternating direction methods, IMA Journal of Numerical Analysis, to appear.
11. P. L. Combettes and V. R. Wajs, Signal recovery by proximal forward-backward splitting, Multiscale Model. Simul., 4 (2005), pp. 1168-1200.
12. J. Eckstein and M. Fukushima, Some reformulation and applications of the alternating directions method of multipliers, In: Hager, W. W. et al. eds., Large Scale Optimization: State of the Art, Kluwer Academic Publishers, pp. 115-134, 1994.
13. E. Esser, Applications of Lagrangian-based alternating direction methods and connections to split Bregman, preprint, available at http://www.math.ucla.edu/applied/cam/, 2009.
14. M. Fazel, H. Hindi and S. Boyd, A rank minimization heuristic with application to minimum order system approximation, Proceedings American Control Conference, 6(2001), pp. 4734-4739.
15. M. Fukushima, Application of the alternating direction method of multipliers to separable convex programming problems, Computational Optimization and Applications, 1(1992), pp. 93-111.
16. D. Gabay, Application of the method of multipliers to varuational inequalities, In: Fortin, M., Glowinski, R., eds., Augmented Lagrangian methods: Application to the numerical solution of Boundary-Value Problem, North-Holland, Amsterdam, The Netherlands, pp. 299-331, 1983.
17. D. Gabay and B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite element approximations, Computational Mathematics with Applications, 2(1976), pp. 17-40.
18. R. Glowinski and P. Le Tallec, Augmented Lagrangian and Operator Splitting Methods in Nonlinear Mechanics, SIAM Studies in Applied Mathematics, Philadelphia, PA, 1989.
19. T. Goldstein and S. Osher, The split Bregman method for L1-regularized problems, SIAM Journal on Imaging Science, 2(2) (2009), pp. 323-343.
20. E. Hale, W. Yin and Y. Zhang, Fixed-point continuation for ℓ1-minimization: methodology and convergence, SIAM Journal on Optimization, 19(3) (2008), pp.1107-1130.
21. B. S. He, L. Z. Liao, D. Han and H. Yang, A new inexact alternating directions method for monontone variational inequalities, Mathematical Programming, 92(2002), pp. 103-118.
22. B. S. He, M. H. Xu and X. M. Yuan, Solving large-scale least squares semidefinite programming by alternating direction methods, SIAM Journal on Matrix Analysis and Applications, 32(1) (2011), pp. 136-152.
23. B. S. He and H. Yang, Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities, Operations Research Letters, 23 (1998), pp. 151-161.
24. B. S. He, H. Yang and S. L. Wang, Alternating direction method with self-adaptive penalty parameters for monotone variational inequalities, Journal of Optimization theory and applications, 106 (2000), pp. 337-356.
25. M. R. Hestenes, Multiplier and gradient methods, Journal of Optimization Theory and Applications, 4 (1969), pp. 303-320.
26. S. Ji and J. Ye, An accelerated gradient method for trace norm minimization, The Twenty-Sixth International Conference on Machine Learning, 2009.
27. S. Kontogiorgis and R. R. Meyer, A variable-penalty alternating directions method for convex optimization, Mathematical Programming, 83(1998), pp. 29-53.
28. R. M. Larsen, PROPACK-Software for large and sparse SVD calculations, Available at: http://sun.stanfor.edu/srmunk/PROPACK/, 2005.
29. Z.-C. Lin, M.-M. Chen, L.-Q. Wu and Y. Ma, The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices, manuscript, 2009.
30. J. Liu, S. Ji and J. Ye, SLEP: A Sparse Learning Package, Version 2.0, Available at: http://www.public.asu.edu/~jye02/Software/SLEP, 2010.
31. Y. J. Liu, D. F. Sun and K. C. Toh, An implementable proximal point algorithmic framework for nuclear norm minimization, Mathematical Programming, to appear.
32. S. Q. Ma, D. Goldfarb and L. Chen, Fixed point and Bregman iterative methods for matrix rank minimization, Math. Program., 128 (2011), 321-353.
33. 2), Doklady AN SSSR, 269, pp. 543-547, 1983.
34. Y. E. Nesterov, Smooth minimization of nonsmooth functions, Mathematical Programming, 103 (2005), pp. 127-152.
35. M. K. Ng, P. A. Weiss and X. M. Yuan, Solving constrained total-variation problems via alternating direction methods, SIAM Journal on Scientifc Computing, 32(5) (2010), pp. 2710- 2736.
36. G. Obozinski, B. Taskar and M. I. Jordan, Joint covariate selection and joint subspace selection for multiple classification problems, Statistics and Computing, 2009.
37. S. Osher, M. Burger, D. Goldfarb, J. Xu and W. Yin, An iterative regularization method for total variation-based image restoration, Multiscale Model. Simul. 4(2) (2005), pp. 460-489.
38. T. K. Pong, P. Tseng, S. Ji. and J. Ye, Trace norm regularization: reformulations, algorithms, and multi-task learning, SIAM Journal on Optimization, 20 (2010), pp. 3465-3489.
39. M. J. D. Powell, A method for nonlinear constraints in minimization problems, in Optimization, R. Fletcher, ed., Academic Press, New York, NY, pp. 283-298, 1969.
40. B. Recht, M. Fazel and P. A. Parrilo, Guaranteed minimum rank solutions to linear matrix equations via nuclear norm minimization, SIAM Review, 52(2010), pp. 471-501.
41. R. T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, NJ, 1970.
42. R. T. Rockafellar, Augmented Lagrangians and applications of the proximal point algorithm in convex programming, Mathematics of Operations Research, 1 (1976), 97-116.
43. S. Setzer, G. Steidl and T. Teuber, Deblurring Poissonian images by split Bregman techniques, Journal of Visual Communication and Image Representation, 21 (2010), pp. 193-199.
44. N. Srebro, J. D. M. Rennie and T. S. Jaakkola, Maximum-margin matrix factorization, Advances in Neural Information Processing System, (2005), pp. 1329-1336.
45. J. F. Sturm, Using SeDuMi 1.02, a Matlab toolbox for optimization over symmetric cones, Optimization Methods and Software 11 & 12 (1999), pp. 625-653.
46. W. Y. Sun and Y. X. Yuan, Optimization theory and methods, Nonlinear Programming Series: Springer Optimization and Its Applications, 2006.
47. J. Sun and S. Zhang, A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs, European Journal of Operational Research, 207 (2010), pp. 1210-1220.
48. M. Tao and X. M. Yuan, Recovering low-rank and sparse components of matrices from incomplete and noisy observations, SIAM Journal on Optimization, 21(1) (2011), pp. 57-81.
49. K. C. Toh and S. Yun, An accelerated proximal gradient algorithm for nuclear norm regularized least suqares problems, Pacific Journal of Optimization, 6(2010), pp. 615-640.
50. P. Tseng, Alternating projection-proximal methods for convex programming and variational inequalities, SIAM Journal on Optimization, 7(1997), pp. 951-965.
51. R. H. Tütüncü, K. C. Toh and M. J. Todd, Solving semidefinite-quadrtic-linear programs using SDPT3, Mathematical Programming, 95(2003), pp. 189-217.
52. G. A. Watson, Characterization of the subdifferential of some matrix norms, Linear Algebra and its Applications, 170 (1992), pp. 33-45.
53. Z. W. Wen, D. Goldfarb and W. Yin, Alternating direction augmented Lagrangina methods for semidefinite programming, Mathematical Programming Computation, 2 (2010), pp. 203-230.
54. Z. W. Wen, W. Yin and Y. Zhang, Solving a low-rank factorization model for matrix completion by a non-linear successive over-relaxation algorithm, TR10-07, CAAM, Rice University, 2010.
55. 1-problems in compressive sensing, SIAM Journal on Scientific Computing, 33(1) (2011), pp. 250-278.
56. W. Yin, Analysis and generalizations of the linearized Bregman method, SIAM Journal on Imaging Sciences, 3(4) (2010), pp. 856-877.
57. 1-minimization with applications to compressed sensing. SIAM Journal on Imaging Science, 1 (2008), pp. 143-168.
58. X. Zhang, M. Burger, X. Bresson and S. Osher, Bregmanized Nonlocal Regularization for Deconvolution and Sparse Reconstruction, SIAM Journal on Imaging Science, 3(2010), pp. 253-276.