On the convergence of block coordinate descent type methods

SIAM Journal on Optimization

Volumn 23, Issue 4, 2013, Pages 2037-2060

  Beck, Amir ^a   Tetruashvili, Luba ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Alternating minimization; Block descent methods; Convex optimization; Rate of convergence

Indexed keywords

ALTERNATING MINIMIZATION; BLOCK COORDINATE DESCENTS; CONVEX PROGRAMMING PROBLEMS; DESCENT METHOD; GRADIENT PROJECTION METHODS; GRADIENT PROJECTIONS; LINEAR RATE OF CONVERGENCES; RATE OF CONVERGENCE;

APPROXIMATION THEORY; CONVEX OPTIMIZATION;

ITERATIVE METHODS;

EID: 84892868336     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/120887679     Document Type: Article

References (19)

1. A. Auslender, Optimisation, Masson, Paris, 1976.
2. A. Beck and M. Teboulle, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2 (2009), pp. 183-202.
3. A. Beck and M. Teboulle, Gradient-based algorithms with applications to signal recovery problems, in Convex Optimization in Signal Processing and Communications, D. Palomar and Y. Eldar, eds., Cambridge University Press, Cambridge, UK, 2009, pp. 139-162.
4. D. P. Bertsekas, Nonlinear Programming, 2nd ed., Athena Scientific, Belmont, MA, 1999.
5. D. P. Bertsekas and J. N. Tsitsiklis, Parallel and Distributed Computation, Prentice-Hall, Englewood Cliffs, NJ, 1989.
6. I. S. Dhillon, P. D. Ravikumar, and A. Tewari, Nearest Neighbor Based Greedy Coordinate Descent, in Proceedings of NIPS, 2011, pp. 2160-2168.
7. L. Grippo and M. Sciandrone, Globally convergent block-coordinate techniques for unconstrained optimization, Optim. Methods Soft., 10 (1999), pp. 587-637.
8. T. Luo and P. Tseng, On the convergence of the coordinate descent method for convex differentiable minimization, J. Optim. Theory Appl., 72 (1992), pp. 7-35.
9. T. Luo and P. Tseng, Error bounds and convergence analysis of feasible descent methods: A general approach, Ann. Oper. Res., 46 (1993), pp. 157-178.
10. Y. Nesterov, Introductory Lectures on Convex Optimization, Kluwer, Boston, 2004.
11. 2), Dokl. Akad. Nauk SSSR, 269 (1983), pp. 543-547.
12. Y. E. Nesterov, Gradient Methods for Minimizing Composite Objective Function, http://www.ecore.be/DPs/dp1191313936.pdf (2007).
13. Y. E. Nesterov, Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems, CORE Discussion paper 2010/2, 2010.
14. J. M. Ortega and W. C. Rheinboldt, Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, 1970.
15. B. T. Polyak, Introduction to Optimization, Transl. Ser. Math. Engrg., Optimization Software, Inc., New York, 1987.
16. P. Richtarik and M. Takac, Efficient Serial and Parallel Coordinate Descent Methods for Huge-Scale Truss Topology Design, Oper. Res. Proc. 2011, Springer, 2012, pp. 27-32.
17. P. Richtarik and M. Takac, Iteration Complexity of Randomized Block-Coordinate Descent Methods for Minimizing a Composite Function, Math. Prog. Ser. A, (2012), pulished online.
18. A. Saha and A. Tewari, On the non-asymptotic convergence of cyclic coordinate descent methods, SIAM J. Optim., 23 (2013), pp. 576-601.
19. 1-regularized loss minimization, J. Mach. Learn. Res., 12 (2011), pp. 1865-1892.