On the O(1/N) convergence rate of the Douglas-Rachford alternating direction method

SIAM Journal on Numerical Analysis

Volumn 50, Issue 2, 2012, Pages 700-709

  He, Bingsheng ^a   Yuan, Xiaoming ^b  

^a NANJING UNIVERSITY   (China)

^b HONG KONG BAPTIST UNIVERSITY   (Hong Kong)

Author keywords

Alternating direction method; Convergence rate; Convex programming; Split inexact Uzawa method; Variational inequalities

Indexed keywords

ALTERNATING DIRECTION METHODS; CONVERGENCE RATES; DOUGLAS; ITERATION NUMBERS; SIMPLE APPROACH; SPLIT INEXACT UZAWA METHOD; VARIATIONAL INEQUALITIES;

CONVEX PROGRAMMING; IMAGE PROCESSING;

VARIATIONAL TECHNIQUES;

EID: 84861398963     PISSN: 00361429     EISSN: None     Source Type: Journal    
DOI: 10.1137/110836936     Document Type: Review

References (35)

1. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, Distributed optimization and statistical learning via the alternating direction method of multipliers, Found. Trends Mach. Learning, 3 (2010), pp. 1-122.
2. R. H. Chan, J. Yang, and X. Yuan, Alternating direction method for image inpainting in wavelet domains, SIAM J. Imaging Sci., 4 (2011), pp. 807-826.
3. T. F. Chan and R. Glowinski, Finite Element Approximation and Iterative Solution of a Class of Mildly Nonlinear Elliptic Equations, technical report, Computer Science Department, Stanford University, Stanford, CA, 1978.
4. J. Douglas and H. H. Rachford, On the numerical solution of the heat conduction problem in 2 and 3 space variables, Trans. Amer. Math. Soc., 82 (1956), pp. 421-439.
5. J. Eckstein and D. P. Bertsekas, On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators, Math. Program., 55 (1992), pp. 293-318.
6. E. Esser, Applications of Lagrangian-Based Alternating Direction Methods and Connections to Split Bregman, UCLA CAM Report 09-31, Department of Mathematics, UCLA, Los Angeles, CA, 2009.
7. F. Facchinei and J. S. Pang, Finite-Dimensional Variational Inequalities and Complementarity Problems, Vol. I. Springer Ser. Oper. Res., Springer-Verlag, New York, 2003.
8. M. Fukushima, Application of the alternating directions method of multipliers to separable convex programming problems, Comput. Optim. Appl., 2 (1992), pp. 93-111.
9. M. Fortin and R. Glowinski, Augmented Lagrangian Methods: Applications to the Numerical Solutions of Boundary Value Problems, Stud. Math. Appl. 15, North-Holland, Amsterdam, 1983.
10. D. Gabay and B. Mercier, A dual algorithm for the solution of nonlinear variational problems via finite-element approximations, Comput. Math. Appl., 2 (1976), pp. 17-40.
11. R. Glowinski, Numerical Methods for Nonlinear Variational Problems, Springer-Verlag, New York, Berlin, Heidelberg, Tokyo, 1984.
12. R. Glowinski, Finite element methods for incompressible viscous flow, in Handbook of Numerical Analysis, Vol. IX, P. G. Ciarlet and J. L. Lions, eds., North-Holland, Amsterdam, 2003, pp. 3-1176.
13. R. Glowinski, T. Kärkkäinen, and K. Majava, On the convergence of operator-splitting methods, in Numerical Methods for Scientific Computing, Variational Problems and Applications, Y. Kuznetsov, P. Neittanmaki, and O. Pironneau, eds., Barcelona, 2003.
14. R. Glowinski and A. Marrocco, Approximation par éléments finis d'ordre un et résolution par pénalisation-dualité d'une classe de problèmes non linéaires, RAIRO, R2 (1975), pp. 41-76.
15. R. Glowinski and P. Le Tallec, Augmented Lagrangian and Operator Splitting Methods in Nonlinear Mechanics, SIAM Stud. Appl. Math. 9, SIAM, Philadelphia, 1989.
16. B. S. He, L. Z. Liao, D. R. Han, and H. Yang, A new inexact alternating directions method for monotone variational inequalities, Math. Program., 92 (2002), pp. 103-118. (Pubitemid 44769742)
17. B. He, M. Xu, and X. Yuan, Solving large-scale least squares semidefinite programming by alternating direction methods, SIAM J. Matrix Anal. Appl., 32 (2011), pp. 136-152.
18. M. R. Hestenes, Multiplier and gradient methods, J. Optim. Theory Appl., 4 (1969), pp. 303-320.
19. R. B. Kellogg, Nonlinear alternating direction algorithm, Math. Comp., 23 (1969), pp. 23-38.
20. S. Kontogiorgis and R. R. Meyer, A variable-penalty alternating directions method for convex optimization, Math. Program., 83 (1998), pp. 29-53. (Pubitemid 128431379)
21. P. L. Lions and B. Mercier, Splitting algorithms for the sum of two nonlinear operators, SIAM J. Numer. Anal., 16 (1979), pp. 964-979.
22. G. I. Marchuk, Splitting and alternating direction methods, in Handbook of Numerical Analysis, Vol. I, P. G. Ciarlet and J. L. Lions, eds., North-Holland, Amsterdam, 1990, pp. 197-462.
23. R. D. C. Monteiro and B. F. Svaiter, Iteration-Complexity of Block-Decomposition Algorithms and the Alternating Direction Method of Multipliers, manuscript, School of Industrial and Systems Engineering, The Georgia Institute of Technology, Atlanta, GA, 2010.
24. A. Nemirovski, Prox-method with rate of convergence O(1/t) for variational inequalities with Lipschitz continuous monotone operators and smooth convex-concave saddle point problems, SIAM J. Optim., 15 (2004), pp. 229-251. (Pubitemid 40360669)
25. Y. E. Nesterov, A method for unconstrained convex minimization problem with the rate of convergence O(1/k2), Dokl. Akad. Nauk SSSR, 269 (1983), pp. 543-547.
26. M. K. Ng, P. Weiss, and X. Yuan, Solving constrained total-variation image restoration and reconstruction problems via alternating direction methods, SIAM J. Sci. Comput., 32 (2010), pp. 2710-2736.
27. G. B. Passty, Ergodic convergence to a zero of the sum of monotone operators in Hilbert space, J. Math. Anal. Appl., 72 (1979), pp. 383-390.
28. M. J. D. Powell, A method for nonlinear constraints in minimization problems, in Optimization, R. Fletcher, ed., Academic Press, New York, 1969, pp. 283-298.
29. J. Sun and S. Zhang, A modified alternating direction method for convex quadratically constrained quadratic semidefinite programs, European J. Oper. Res., 207 (2010), pp. 1210-1220.
30. M. Tao and X. Yuan, Recovering low-rank and sparse components of matrices from incomplete and noisy observations, SIAM J. Optim., 21 (2011), pp. 57-81.
31. P. Tseng, Applications of a splitting algorithm to decomposition in convex programming and variational inequalities, SIAM J. Control Optim., 29 (1991), pp. 119-138.
32. P. Tseng, On Accelerated Proximal Gradient Methods for Convex-Concave Optimization, manuscript, University of Washington, WA, 2008.
33. S. Zhang, J. Ang, and J. Sun, An alternating direction method for solving convex nonlinear semidefinite programming problem, Optim., to appear.
34. X. Zhang, M. Burger, X. Bresson, and S. Osher, Bregmanized nonlocal regularization for deconvolution and sparse reconstruction, SIAM J. Imaging Sci., 3 (2010), pp. 253-276.
35. X. Q. Zhang, M. Burger, and S. Osher, A unified primal-dual algorithm framework based on Bregman iteration, J. Sci. Comput., 46 (2010), pp. 20-46.