Augmented Lagrangian alternating direction method for matrix separation based on low-rank factorization

Optimization Methods and Software

Volumn 29, Issue 2, 2014, Pages 239-263

  Shen, Y ^a   Wen, Z ^b   Zhang, Y ^c  

^a NANJING UNIVERSITY   (China)

^b SHANGHAI JIAO TONG UNIVERSITY   (China)

^c Rice University   (United States)

Author keywords

Alternating direction method; Augmented Lagrangian function; Matrix separation

Indexed keywords

ALTERNATING DIRECTION METHODS; AUGMENTED LAGRANGIAN FUNCTIONS; AUGMENTED LAGRANGIANS; DATA PREPROCESSING; FACTORIZATION MODEL; MATRIX SEPARATION; MINIMIZATION ALGORITHMS; SUITABLE CONDITIONS;

LAGRANGE MULTIPLIERS; NUMERICAL METHODS; SEPARATION;

ITERATIVE METHODS;

EID: 84888318611     PISSN: 10556788     EISSN: 10294937     Source Type: Journal    
DOI: 10.1080/10556788.2012.700713     Document Type: Article

References (50)

1. E. Anderson, Z. Bai, C. Bischof, S. Blackford, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, and D. Sorensen, LAPACK Users' Guide, SIAM, Philadelphia, 1999.
2. D.P. Bertsekas and J.N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods, Prentice-Hall, Inc., Upper Saddle River, NJ, 1989.
3. J. Cai, E.J. Candès, and Z. Shen, A singular value thresholding algorithm for matrix completion, SIAM J. Optim. 20(4) (2010), pp. 1956-1982.
4. E.J. Candès and B. Recht, Exact matrix completion via convex optimization, Found. Comput. Math. 9(6) (2009), pp. 717-772.
5. E.J. Candès and J. Romberg, Quantitative robust uncertainty principles and optimally sparse decompositions, Found. Comput. Math. 6(2) (2006), pp. 227-254.
6. E.J. Candès and T. Tao, Near optimal signal recovery from random projections: Universal encoding strategies, IEEE Trans. Inform. Theory 52(1) (2006), pp. 5406-5425.
7. E.J. Candès and T. Tao, The power of convex relaxation: Near-optimal matrix completion, IEEE Trans. Inform. Theory 56(5) (2010), pp. 2053-2080.
8. E.J. Candès, X. Li, Y. Ma, and J. Wright, Robust principal component analysis? Arxiv preprint. Available at arXiv:0912.3599, December 2009.
9. E.J. Candès, J. Romberg, and T. Tao, Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information, IEEE Trans. Inform. Theory 52 (2006), pp. 489-509.
10. V. Chandrasekaran, S. Sanghavi, P.A. Parrilo, and A.S. Willsky, Rank-sparsity incoherence for matrix decomposition, Arxiv preprint. Available at arXiv:0906.2220, 2009.
11. V. Chandrasekaran, S. Sanghavi, P.A. Parrilo, and A.S. Willsky, Sparse and low-rank matrix decompositions, IFAC Symposium on System Identification, 2009. Available at http://ssg.mit.edu/~venkatc/cspw-slr-sysid09.pdf.
12. G. Chen and M. Teboulle, A proximal-based decomposition method for convex minimization problems, Math. Program. 64(1, Ser. A) (1994), pp. 81-101.
13. P.L. Combettes and J.-C. Pesquet, Proximal thresholding algorithm for minimization over orthonormal bases, SIAM J. Optim. 18(4) (2007), pp. 1351-1376.
14. D. Donoho, Compressed sensing, IEEE Trans. Inform. Theory 52 (2006), pp. 1289-1306.
15. J. Eckstein and D.P. Bertsekas, An Alternating Direction Method for Linear Programming, Laboratory for Information and Decision Systems, Massachusetts Institute of Technology, Cambridge, MA, 1967.
16. J. Eckstein and D.P. Bertsekas, On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators, Math. Program. 55(3, Ser. A) (1992), pp. 293-318.
17. M. Elad, Why simple shrinkage is still relevant for redundant representations? IEEE Trans. Inform. Theory 52(12) (2006), pp. 5559-5569.
18. M. Fazel and J. Goodman, Approximations for partially coherent optical imaging systems, technical report, Stanford University, 1998. Available at http://faculty.washington.edu/mfazel/acc03-final.pdf.
19. M. Fazel, H. Hindi, and S. Boyd, Log-det heuristic for matrix rank minimization with applications to hankel and euclidean distance matrices, Proceedings of the American Control Conference, 2003. Available at http://faculty.washington.edu/mfazel/acc03-final.pdf.
20. M. Figueiredo and R. Nowak, An EM algorithm for wavelet-based image restoration, IEEE Trans. Image Process. 12 (2003), pp. 906-916.
21. M. Fortin and R. Glowinski, Augmented Lagrangian methods: Applications to the numerical solution of boundary value problems Vol. 15, in Studies in Mathematics and its Applications, North-Holland Publishing Co., Amsterdam, 1983. Translated from the French by B. Hunt and D. C. Spicer.
22. A. Ganeshy, J. Wright, X. Li, E.J. Candes, and Y. Ma, Dense error correction for low-rank matrices via principal component pursuit, 2010 IEEE International Symposium on Information Theory Proceedings (ISIT), 2010, pp. 1513-1517.
23. R. Glowinski, Numerical Methods for Nonlinear Variational Problems, Springer, Berlin, 1984.
24. R. Glowinski and P. Le Tallec, Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics Vol. 9, SIAM, Philadelphia, PA, 1989.
25. J.P. Haldar and D. Hernando, Rank-constrained solutions to linear matrix equations using powerfactorization, IEEE Signal Process. Lett. 16 (2009), pp. 584-587.
26. E.T. Hale, W. Yin, and Y. Zhang, Fixed-point continuation for l1-minimization: Methodology and convergence, SIAM J. Optim. 19(3) (2008), pp. 1107-1130.
27. B. He, L. Liao, D. Han, and H. Yang, A new inexact alternating directions method for monotone variational inequalities, Math. Program. 92(1, Ser. A) (2002), pp. 103-118.
28. B. He, H. Yang, and S. Wang, Alternating direction method with self-adaptive penalty parameters for monotone variational inequalities, J. Optim. Theory Appl. 106(2) (2000), pp. 337-356.
29. H. Hindi M. Fazel, and S. Boyd, A rank minimization heuristic with application to minimum order system approximation, Proceedings American Control Conference 6, 2001, pp. 4734-4739.
30. K.C. Kiwiel, C.H. Rosa, and A. Ruszczyński, Proximal decomposition via alternating linearization, SIAM J. Optim. 9(3) (1999), pp. 668-689.
31. S. Kontogiorgis and R.R. Meyer, A variable-penalty alternating directions method for convex optimization, Math. Program. 83(1, Ser. A) (1998), pp. 29-53.
32. R.M. Larsen, Propack: Software for large and sparse svd calculations. Available at http://soi.stanford.edu/~rmunk/PROPACK.
33. L. Li, W. Huang, I. Gu, and Q. Tian, Statistical modeling of complex backgrounds for foreground object detection, IEEE Trans. Image Process. 13(11) (2004), pp. 1459-1472.
34. Z. Lin, M. Chen, L. Wu, and Y. Ma, The augmented lagrange multiplier method for exact recovery of a corrupted low-rank matrices, Math. Program. (2009) (submitted).
35. Z. Lin, A. Ganesh, J. Wright, L. Wu, M. Chen, and Y. Ma, Fast convex optimization algorithms for exact recovery of a corrupted low-rank matrix, 2009. Available at http://yima.csl.illinois.edu/psfile/rpca-algorithms.pdf.
36. S. Ma, D. Goldfarb, and L. Chen, Fixed point and Bregman iterative methods for matrix rank minimization, technical report, Department of IEOR, Columbia University, 2008.
37. Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, Applied Optimization, Kluwer Academic Publishers, Boston, 2004.
38. M. Tao and X. Yuan, Recovering low-rank and sparse components of matrices from incomplete and noisy observations, SIAM J. Optim. 21(1) (2011), pp. 57-81.
39. P. Tseng, Alternating projection-proximal methods for convex programming and variational inequalities, SIAM J. Optim. 7(4) (1997), pp. 951-965.
40. P. Tseng, On accelerated proximal gradient methods for convex-concave optimization, SIAM J. Optim. (2008) (submitted).
41. Y. Wang, J. Yang, W. Yin, and Y. Zhang, A new alternating minimization algorithm for total variation image reconstruction, SIAM J. Imaging Sci. 1(3) (2008), pp. 248-272.
42. Z. Wen, D. Goldfarb, and W. Yin, Alternating direction augmented Lagrangian methods for semidefinite programming, technical report, Department of IEOR, Columbia University, 2009.
43. Z. Wen, WYin, and Y. Zhang, Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm, technical report, 2010. Available at http://www.optimization-online.org/DB-FILE/2010/03/ 2581.pdf.
44. J. Wright, A. Ganesh, S. Rao, Y. Peng, and Y. Ma, Robust principal component analysis: Exact recovery of corrupted low-rank matrices by convex optimization, Proceedings of Neural Information Processing Systems (NIPS), December 2009. Available at https://netfiles.uiuc.edu/abalasu2/www/Files/ Wright09-NIPS.pdf.
45. J. Wright, A. Ganesh, S. Rao, Y. Peng, and Y. Ma, Robust principal component analysis: Exact recovery of corrupted low-rank matrices via convex optimization, J. ACM. (2009) (submitted).
46. J. Yang, Y. Zhang, and W. Yin, An efficient tvl1 algorithm for deblurring multichannel images corrupted by impulsive noise, SIAM J. Sci. Comput. 31(4) (2009), pp. 2842-2865.
47. X. Yuan and J. Yang, Sparse and low-rank matrix decomposition via alternating direction methods, technical report, Department of Mathematics, Hong Kong Baptist University, 2009.
48. Y. Zhang, LMaFit:Low-rank matrix fitting, 2009. Available at http://www.caam.rice.edu/~optimization/L1/LMaFit/.
49. Y. Zhang, User's guide for yall1: Your algorithms for L1 optimization, technical report, Rice University, 2009. Available at http://www.caam.rice.edu/ ~zhang/reports/tr0917.pdf.
50. Y. Zhang, An alternating direction algorithm for nonnegative matrix factorization, technical report, Rice University, 2010. Available at http://www.caam.rice.edu/~yzhang/reports/tr1003.pdf.