On the Sublinear Convergence Rate of Multi-block ADMM

Journal of the Operations Research Society of China

Volumn 3, Issue 3, 2015, Pages 251-274

  Lin, Tian Yi ^a   Ma, Shi Qian ^a   Zhang, Shu Zhong ^b  

^a CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

^b University of Minnesota   (United States)

Author keywords

Alternating direction method of multipliers; Convex optimization; Sublinear convergence rate

Indexed keywords

FUNCTIONS;

ALTERNATING DIRECTION METHOD OF MULTIPLIER (ADMM); ALTERNATING DIRECTION METHOD OF MULTIPLIERS; CONSTRAINT VIOLATION; CONVERGENCE RATES; LINEAR CONSTRAINTS; NUMBER OF ITERATIONS; STRONG CONVEXITIES; STRUCTURED CONVEX OPTIMIZATIONS;

CONVEX OPTIMIZATION;

EID: 84941340210     PISSN: 2194668X     EISSN: 21946698     Source Type: Journal    
DOI: 10.1007/s40305-015-0092-0     Document Type: Article

References (37)

1. Gabay, D., Mercier, B.: A dual algorithm for the solution of nonlinear variational problems via finite-element approximations. Comp. Math. Appl. 2, 17–40 (1976)
2. Glowinski, R., Marrocco, A.: Sur l’approximation par éléments finis et la résolution par pénalisation-dualité d’une classe de problèmes de Dirichlet non linéaires. Revue Française d’Automatique, Informatique, Recherche Operationnelle, Serie Rouge (AnalyseNumérique), R-2,pp. 41–76 (1975)
3. Douglas, J., Rachford, H.H.: On the numerical solution of the heat conduction problem in 2 and 3 space variables. Trans. Am. Math. Soc. 82, 421–439 (1956)
4. Peaceman, D.H., Rachford, H.H.: The numerical solution of parabolic elliptic differential equations. SIAM J. Appl. Math. 3, 28–41 (1955)
5. Eckstein, J.: Splitting methods for monotone operators with applications to parallel optimization. PhD thesis, Massachusetts Institute of Technology (1989)
6. Fortin, M., Glowinski, R.: Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems. North-Holland Pub, Co, Amsterdam (1983)
7. Glowinski, R., Le Tallec, P.: Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics. SIAM, Philadelphia (1989)
8. Lions, P.L., Mercier, B.: Splitting algorithms for the sum of two nonlinear operators. SIAM J. Numer. Anal. 16, 964–979 (1979)
9. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)
10. Eckstein, J.: Augmented Lagrangian and alternating direction methods for convex optimization: A tutorial and some illustrative computational results (2012). Preprint http://www.optimization-online.org/DB_HTML/2012/12/3704.html
11. Eckstein, J., Bertsekas, D.P.: On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators. Math. Program. 55, 293–318 (1992)
12. Gabay, D.: Applications of the method of multipliers to variational inequalities. In: Fortin, M., Glowinski, R. (eds.) Augmented Lagrangian Methods: Applications to the Solution of Boundary Value Problems. North-Holland, Amsterdam (1983)
13. Boley, D.: Local linear convergence of the alternating direction method of multipliers on quadratic or linear programs. SIAM J. Optim. 23(4), 2183–2207 (2013)
14. Davis, D., Yin, W.: Faster convergence rates of relaxed Peaceman–Rachford and ADMM under regularity assumptions. Technical report, UCLA CAM Report 14–58 (2014)
15. Deng, W., Yin, W.: On the global and linear convergence of the generalized alternating direction method of multipliers. J. Sci. Comput. (2015). doi:10.1007/s10915-015-0048-x
16. He, B., Yuan, X.: On the $${O}(1/n)$$O(1/n) convergence rate of Douglas–Rachford alternating direction method. SIAM J. Numer. Anal. 50, 700–709 (2012)
17. He, B., Yuan, X.: On nonergodic convergence rate of Douglas–Rachford alternating direction method of multipliers. Numerische Mathematik 130(3), 567–577 (2015)
18. Monteiro, R.D.C., Svaiter, B.F.: Iteration-complexity of block-decomposition algorithms and the alternating direction method of multipliers. SIAM J. Optim. 23, 475–507 (2013)
19. Chen, C., He, B., Ye, Y., Yuan, X.: The direct extension of ADMM for multi-block convex minimization problems is not necessarily convergent. Math. Program. (2014). doi:10.1007/s10107-014-0826-5
20. Peng, Y., Ganesh, A., Wright, J., Xu, W., Ma, Y.: RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2233–2246 (2012)
21. Tao, M., Yuan, X.: Recovering low-rank and sparse components of matrices from incomplete and noisy observations. SIAM J. Optim. 21, 57–81 (2011)
22. Sun, D., Toh, K.-C., Yang, L.: A convergent 3-block semiproximal alternating direction method of multipliers for conic programming with 4-type constraints. SIAM J. Optim. 25, 882–915 (2015)
23. Wang, X., Hong, M., Ma, S., Luo, Z.-Q.: Solving multiple-block separable convex minimization problems using two-block alternating direction method of multipliers (2013). Preprint arXiv:1308.5294
24. Han, D., Yuan, X.: A note on the alternating direction method of multipliers. J. Optim. Theory Appl. 155(1), 227–238 (2012)
25. Chen, C., Shen, Y., You, Y.: On the convergence analysis of the alternating direction method of multipliers with three blocks. Abstract and Applied Analysis, Article ID 183961 (2013). doi:10.1155/2013/183961
26. Cai, X., Han, D., Yuan, X.: The direct extension of ADMM for three-block separable convex minimization models is convergent when one function is strongly convex (2014). Preprint http://www.optimization-online.org/DB_HTML/2014/11/4644.html
27. Li, M., Sun, D., Toh, K.C.: A convergent 3-block semi-proximal ADMM for convex minimization problems with one strongly convex block. Asia-Pacific J. Oper. Res. 32(3), 1550024 (2015)
28. Davis, D., Yin, W.: A three-operator splitting scheme and its optimization applications. Technical report, UCLA CAM Report 15–13 (2015)
29. Lin, T., Ma, S., Zhang, S.: Iteration complexity analysis of multi-block ADMM for a family of convex minimization without strong convexity (2015). Preprint arXiv:1504.03087
30. Lin, T., Ma, S., Zhang, S.: Global convergence of unmodified 3-block ADMM for a class of convex minimization problems (2015). Preprint arXiv:1505.04252
31. Hong, M., Luo, Z.: On the linear convergence of the alternating direction method of multipliers (2012). Preprint arXiv:1208.3922
32. Deng, W., Lai, M., Peng, Z., Yin, W.: Parallel multi-block ADMM with $$o(1/k)$$o(1/k) convergence (2013). Preprint arXiv:1312.3040
33. He, B., Hou, L., Yuan, X.: On full Jacobian decomposition of the augmented Lagrangian method for separable convex programming (2013). Preprint http://www.optimization-online.org/DB_HTML/2013/05/3894.html
34. He, B., Tao, M., Yuan, X.: Alternating direction method with Gaussian back substitution for separable convex programming. SIAM J. Optim. 22, 313–340 (2012)
35. B. He, M. Tao, and X. Yuan. Convergence rate and iteration complexity on the alternating direction method of multipliers with a substitution procedure for separable convex programming (2013). Preprint http://www.optimization-online.org/DB_FILE/2012/09/3611.pdf
36. Hong, M., Chang, T.-H., Wang, X., Razaviyayn, M., Ma, S., Luo, Z.-Q.: A block successive upper bound minimization method of multipliers for linearly constrained convex optimization (2014). Preprint arXiv:1401.7079
37. Lin, T., Ma, S., Zhang, S.: On the global linear convergence of the ADMM with multi-block variables. SIAM J. Optim., to appear (2015)