Recovering low-rank and sparse components of matrices from incomplete and noisy observations

SIAM Journal on Optimization

Volumn 21, Issue 1, 2011, Pages 57-81

  Tao, Min ^a,b   Yuan, Xiaoming ^c  

^a NANJING UNIVERSITY   (China)

^b NANJING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS   (China)

^c HONG KONG BAPTIST UNIVERSITY   (Hong Kong)

Author keywords

Alternating direction method; Augmented Lagrangian method; Low rank; Matrix recovery; Principal component analysis; Sparse

Indexed keywords

ALTERNATING DIRECTION METHODS; AUGMENTED LAGRANGIAN METHOD; LOW-RANK; MATRIX RECOVERY; SPARSE;

ALGORITHMS; GAUSSIAN NOISE (ELECTRONIC); LAGRANGE MULTIPLIERS; RECOVERY; RELAXATION PROCESSES;

PRINCIPAL COMPONENT ANALYSIS;

EID: 79957510064     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/100781894     Document Type: Article

References (61)

1. H. H. BAUSCHKE AND P. L. COMBETTES, A weak-to-strong convergence principle for Fejrmonotone methods in Hilbert spaces, Math. Oper. Res., 26 (2001), pp. 248-264. (Pubitemid 34050614)
2. D. P. BERTSEKAS AND J. N. TSITSIKLIS, Parallel and distributed computation: Numerical methods, Prentice-Hall, Englewood Cliffs, NJ, 1989.
3. A. M. BRUCKSTEIN, D. L DONOHO, AND M. ELAD, From sparse solutions of systems of equations to sparse modeling of signals and images, SIAM Rev., 51 (2009), pp. 34-81.
4. J. F. CAI, E. J. CANDÉS, AND Z. W. SHEN, A singular value thresholding algorithm for matrix completion, SIAM J. Optim., 20 (2010), pp. 1956-1982.
5. E. J. CANDÉS, X. D. LI, Y. MA, AND J. WRIGHT, Robust Principal Component Analysis?, manuscript, 2009.
6. E. J. CANDÉS AND Y. PLAN, Matrix completion with noise, in Proceedings of the IEEE, 2009.
7. E. J. CANDÉS AND B. RECHT, Exact matrix completion via convex optimization, Found. Comput. Math., 9 (2009), pp. 717-772.
8. E. J. CANDÉS AND T. TAO, The power of convex relaxation: Near-optimal matrix completion, IEEE Trans. Inform. Theory, 56 (2009), pp. 2053-2080.
9. V. CHANDRASEKARAN, S. SANGHAVI, P. A. PARRILO, AND A. S. WILLSKYC, Rank-Sparsity Incoherence for Matrix Decomposition, manuscript, 2009, http://arxiv.org/abs/0906.2220.
10. C. H. CHEN, B. S. HE, AND X. M. YUAN, Matrix completion via alternating direction method, IMA J. Numer. Anal., to appear.
11. G. CHEN AND M. TEBOULLE, A proximal-based decomposition method for convex minimization problems, Math. Program., 64 (1994), pp. 81-101.
12. S. CHEN, D. DONOHO, AND M. SAUNDERS, Atomic decomposition by basis pursuit, SIAM J. Sci. Comput., 20 (1998), pp. 33-61. (Pubitemid 128689501)
13. J. DOUGLAS AND H. H. RACHFORD, On the numerical solution of the heat conduction problem in 2 and 3 space variables, Trans. Amer. Math. Soc., 82 (1956), pp. 421-439.
14. D. DONOHO, High-dimensional data analysis: The curses and blessings of dimensionality, AMS Math Challenges Lecture, 2000.
15. J. ECKSTEIN, Some saddle-function splitting methods for convex programming, Optim. Methods Softw., 4 (1994), pp. 75-83.
16. J. ECKSTEIN AND M. FUKUSHIMA, Some reformulation and applications of the alternating directions method of multipliers, in Large Scale Optimization: State of the Art, W. W. Hager, D. W. Hearn, and P. M. Pordulos, eds., Kluwer Academic Publishers, Norwell, MA, 1994, pp. 115-134.
17. E. ESSER, Applications of Lagrangian-Based Alternating Direction Methods and Connections to Split Bregman, manuscript, 2009, http://www.math.ucla.edu/ applied/cam/.
18. F. FACCHINEI AND J.-S. PANG, Finite-Dimensional Variational Inequalities and Complementarity problems, Springer Ser. Oper. Res. 1, Springer-Verlag, New York, 2003.
19. M. FAZEL, Matrix Rank Minimization with Applications, Ph. D. thesis, Stanford University, Stanford, CA, 2002.
20. M. FAZEL AND J. GOODMAN, Approximations for Partially Coherent Optical Imaging Systems, Technical report, Stanford University, Stanford, CA, 1998.
21. M. FAZEL, H. HINDI, AND S. BOYD, A rank minimization heuristic with application to minimum order system approximation, in Proceedings of the American Control Conference, 2001, pp. 4734-4739. (Pubitemid 32837075)
22. M. FAZEL, H. HINDI AND S. BOYD, Log-det heuristic for matrix rank minimization with applications to Hankel and Euclidean distance matrices, in Proceedings of the American Control Conference, 2003.
23. M. FUKUSHIMA, Application of the alternating direction method of multipliers to separable convex programming problems, Comput. Optim. Appl., 1 (1992), pp. 93-111.
24. D. GABAY, Application of the method of multipliers to varuational inequalities, in Augmented Lagrangian Methods: Application to the Numerical Solution of Boundary-Value Problem, M. Fortin and R. Glowinski, eds., Elsevier-North Holland, Amsterdam, The Netherlands, 1983, pp. 299-331.
25. D. GABAY AND B. MERCIER, A dual algorithm for the solution of nonlinear variational problems via finite-element approximations, Comput. Math. Appl., 2 (1976), pp. 17-40.
26. R. GLOWINSKI, Numerical Methods for Nonlinear Variational Problems, Springer-Verlag, Berlin, Heidelberg, 1984.
27. R. GLOWINSKI AND P. LE TALLEC, Augmented Lagrangian and Operator Splitting Methods in Nonlinear Mechanics, SIAM Stud. Appl. Math. 9, SIAM, Philadelphia, PA, 1989.
28. B. S. HE, L. Z. LIAO, D. HAN, AND H. YANG, A new inexact alternating directions method for monontone variational inequalities, Math. Program., 92 (2002), pp. 103-118. (Pubitemid 44769742)
29. B. S. HE, M. H. XU, AND X. M. YUAN, Solving large-scale least squares covariance matrix problems by alternating direction methods, SIAM J. Matrix Anal. Appl., to appear.
30. B. S. HE AND H. YANG, Some convergence properties of a method of multipliers for linearly constrained monotone variational inequalities, Oper. Res. Lett., 23 (1998), pp. 151-161. (Pubitemid 128676387)
31. M. HESTENES, Multiplier and gradient methods, J. Optim. Theory Appl., 4 (1969), pp. 303-320.
32. N. J. HIGHAM, Computing a nearest symmetric positive semidefinite matrix, Linear Algebra Appl., 103 (1988), pp. 103-118.
33. I. T. JOLLIFFE, Princ i pal C om pon ent Analy sis, Springer-Verlag, New York, 1986.
34. R. H. KESHAVAN, A. MONTANARI, AND S. OH, Matrix completion from a few entries, IEEE Trans. Inform. Theory, to appear.
35. R. H. KESHAVAN, A. MONTANARI, AND S. OH, Matrix completion from noisy entries, J. Math. Learn. Res., to appear.
36. S. KONTOGIORGIS AND R. R. MEYER, A variable-penalty alternating directions method for convex optimization, Math. Program., 83 (1998), pp. 29-53. (Pubitemid 128431379)
37. R. M. LARSEN, Lanczos Bidiagonalization with Partial Reorthogonalization, Technical report, Department of Computer Science, Aarhus University, Aarhus, Denmark, 1998.
38. S. L. LAURITZEN, Graphical Models, Oxford University Press, New York, 1996.
39. Z. C. LIN, M. M. CHEN AND L. Q. WU, AND Y. MA, The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices, manuscript, 2009.
40. Z. C. LIN, A. GANESH, J. WRIGHT, L. WU, M. GHEN, AND Y. MA, Fast Convex Optimization Algorithms for Exact Recovery of a Corrupted Low-Rank Matrix, preprint, 2009, http://perception. csl.illinois.edu/jnwright/Lin2009-Algorithms. pdf.
41. Y. J. LIU, D. F. SUN, AND K. C. TOH, An Implementable Proximal Point Algorithmic Framework for Nuclear Norm Minimization, manuscript, 2009.
42. Z. LIU AND L. VANDENBERGHE, Interior-point method for nuclear norm approximation with application to system identification, SIAM J. Matrix Anal. Appl., 31 (2009), pp. 1235-1256.
43. S. Q. MA, D. GOLDFARB, AND L. F. CHEN, Fixed point and Bregman iterative methods for matrix rank minimization, Math. Program., to appear.
44. M. NG, P. A. WEISS, AND X. M. YUAN, Solving constrained total-variation image restoration and reconstruction problems via alternating direction methods, SIAM J. Sci. Comput., 32 (2010), pp. 2710-2736.
45. Y. C. PATI AND T. KAILATH, Phase-shifting masks for microlithography: Automated design and mask requirements, J. Opt. Soc. Amer. A, 11 (1994), pp. 2438-2452.
46. M. J. D. POWELL, A method for nonlinear constraints in minimization problems, in Optimization, R. Fletcher, ed., Academic Press, New York, 1969, pp. 283-298.
47. B. RECHT, M. FAZEL, AND P. A. PARRILO, Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization, SIAM Rev., 52 (2010), pp. 471-501.
48. E. SONTAG, Mathematical Control Theory, Springer-Verlag, New York, 1998.
49. J. F. STURM, Using SeDuMi 1.02, a Matlab toolbox for optimization over symmetric cones, Optim. Methods Softw., 11-12 (1999), pp. 625-653.
50. J. SUN AND S. ZHANG, A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs, European J. Oper. Res., to appear.
51. M. TAO AND J. F. YANG, Alternating Direction Algorithms for Total Variation Deconvolution in Image Reconstruction, http://www.optimization-online. org/DB-HTML/2009/11/2463.html (August 2009).
52. P. TSENG, Alternating projection-proximal methods for convex programming and variational inequalities, SIAM J. Optim., 7 (1997), pp. 951-965.
53. R. H. TÜTÜNCÜ, K. C. TOH, AND M. J. TODD, Solving semidefinite-quadrtic-linear programs using SDPT3, Math. Program., 95 (2003), pp. 189-217. (Pubitemid 44757853)
54. K. C. TOH AND S. YUN, An accelerated proximal gradient algorithm for nuclear norm regularized least squares problems, Pac. J. Optim., 6 (2010), pp. 615-640.
55. L. G. VALIANT, Graph-theoretic arguments in low-level complexity, in Proceedings of the 6th Symposium on Mathematical Foundations of Computer Science, 1977, pp. 162-176.
56. Z. W. WEN, D. GOLDFARB, AND W. YIN, Alternating Direction Augmented Lagrangian Methods for Semidefinite Programming, manuscript, 2009.
57. J. F. YANG, W. T. YIN, Y. ZHANG, AND Y. L. WANG, A fast algorithm for edge-preserving variational multichannel image restoration, SIAM J. Imaging Sci., 2 (2009), pp. 569-592.
58. J. YANG AND Y. ZHANG, Alternating Direction Algorithms for l1-problems in Compressive Sensing, Technical report 09-37, Department of Computational and Applied Mathematics, Rice University, Houston, TX, 2008.
59. X. M. YUAN, Alternating Direction Methods for Sparse Covariance Selection, manuscript, 2009.
60. X. M. YUAN AND J. YANG, Sparse and Low-Rank Matrix Decomposition Via Alternating Direction Methods, manuscript, 2009.
61. Z. ZHOU, X. LI, J. WRIGHT, E. J. CANDÉS, AND Y. MA, Stable principal component pursuit, in Proceedings of the International Symposium on Information Theory, 2010.