Parallel random coordinate descent method for composite minimization: Convergence analysis and error bounds

SIAM Journal on Optimization

Volumn 26, Issue 1, 2016, Pages 197-226

  Necoara, Ion ^a   Clipici, Dragos ^a  

^a UNIVERSITY POLITEHNICA OF BUCHAREST   (Romania)

Author keywords

Composite minimization; Generalized error bound condition; Parallel random coordinate descent algorithm; Partially separable functions; Rate of convergence

Indexed keywords

ERROR ANALYSIS; ERRORS; NONLINEAR EQUATIONS;

CONVERGENCE ANALYSIS; COORDINATE DESCENT; COORDINATE DESCENT METHODS; ERROR BOUND; LINEAR CONVERGENCE RATE; LIPSCHITZ CONTINUITY; PARTIALLY SEPARABLE FUNCTIONS; RATE OF CONVERGENCE;

FUNCTIONS;

EID: 84962393728     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/130950288     Document Type: Article

References (29)

1. A. Beck and L. Tetruashvili, On the convergence of block coordinate descent type methods, SIAM J. Optim., 23 (2013), pp. 2037-2060.
2. C. M. Bishop, Pattern Recognition and Machine Learning, Springer-Verlag, New York, 2006.
3. J. K. Bradley, A. Kyrola, D. Bickson, and C. Guestrin, Parallel coordinate descent for l1-regularized loss minimization, in Proceedings of the 28th International Conference on Machine Learning, Washington, D.C., 2011.
4. X. Chen, M. K. Ng, and C. Zhang, Non-Lipschitzp-regularization and box constrained model for image restoration, IEEE Trans. Image Process., 21 (2012), pp. 4709-4721.
5. M. Hong, X. Wang, M. Razaviyayn, and Z-Q. Luo, Iteration Complexity Analysis of Block Coordinate Descent Methods, Tech. report, University of Minnesota, Minneapolis, MN, 2013; available online from http://arxiv.org/abs/1310.6957.
6. A. Lewis and D. Leventhal, Randomized methods for linear constraints: Convergence rates and conditioning, Math. Oper. Res., 35 (2010), pp. 641-654.
7. Z. Lu and L. Xiao, On the complexity analysis of randomized block-coordinate descent methods, Math. Program., 152 (2015), pp. 615-642.
8. Z. Q. Luo and P. Tseng, Error bounds and convergence analysis of feasible descent methods: A general approach, Ann. Oper. Res., 46 (1993), pp. 157-178.
9. I. Necoara, Random coordinate descent algorithms for multi-agent convex optimization over networks, IEEE Trans. Automat. Control, 58 (2013), pp. 2001-2012.
10. I. Necoara and D. Clipici, Efficient parallel coordinate descent algorithm for convex optimization problems with separable constraints: Application to distributed MPC, J. Process Contr., 23 (2013), pp. 243-253.
11. I. Necoara and D. Clipici, Parallel Coordinate Descent Methods for Composite Minimization: Convergence Analysis and Error Bounds, Tech. report, University Politehnica Bucharest, Bucharest, Romania, 2013; available online from http://arxiv.org/abs/1312.5302.
12. I. Necoara and R. Findeisen, Parallel and distributed random coordinate descent method for convex error bound minimization, in Proceedings of the American Control Conference, 2015, pp. 527-532.
13. I. Necoara and V. Nedelcu, Rate analysis of inexact dual first order methods: Application to dual decomposition, IEEE Trans. Automat. Control, 59 (2014), pp. 1232-1243.
14. I. Necoara, Yu. Nesterov, and F. Glineur, Random Block Coordinate Descent Methods for Linearly Constrained Optimization over Networks, Tech. report, University Politehnica Bucharest, Bucharest, Romania, 2011; available online from http://arxiv.org/abs/1504. 06340.
15. I. Necoara and A. Patrascu, A random coordinate descent algorithm for optimization problems with composite objective function and linear coupled constraints, Comput. Optim. Appl., 57 (2014), pp. 307-337.
16. I. Necoara and J. A. K. Suykens, An interior-point Lagrangian decomposition method for separable convex optimization, J. Optim. Theory Appl., 143 (2009), pp. 567-588.
17. Yu. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, Kluwer, Boston, MA, 2004.
18. Yu. Nesterov, Efficiency of coordinate descent methods on huge-scale optimization problems, SIAM J. Optim., 22 (2012), pp. 341-362.
19. Yu. Nesterov, Gradient methods for minimizing composite objective functions, Mathematical Programming, 140 (2013), pp. 125-161.
20. Z. Peng, M. Yan, and W. Yin, Parallel and Distributed Sparse Optimization, Tech. report, Rice University, Houston, TX, 2013.
21. S. S. Ram, A. Nedić, and V. V. Veeravalli, Incremental stochastic subgradient algorithms for convex optimization, SIAM J. Optim., 20 (2009), pp. 691-717.
22. P. Richtárik and M. Takáč, Distributed Coordinate Descent Method for Learning with Big Data, Tech. report, University of Edinburgh, Edinburgh, Scotland, 2013.
23. P. Richtárik and M. Takáč, Parallel coordinate descent methods for big data optimization, Math. Program., DOI:10.1007/s10107-0150901-6, 2015.
24. P. Richtárik and M. Takáč, Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function, Math. Program., 144 (2014), pp. 1-38.
25. S. M. Robinson, Bounds for error in the solution set of a perturbed linear program, Linear Algebra Appl., 6 (1973), pp. 69-81.
26. R. T. Rockafellar and R. J. Wets, Variational Analysis, Springer-Verlag, Berlin, 1998.
27. S. Ryali, K. Supekar, D. A. Abrams, and V. Menone, Sparse logistic regression for wholebrain classification of FMRI data, NeuroImage, 51 (2010), pp. 752-764.
28. P. Tseng and S. Yun, A coordinate gradient descent method for nonsmooth separable minimization, Math. Program., 117 (2009), pp. 387-423.
29. P. W. Wang and C. J. Lin, Iteration complexity of feasible descent methods for convex optimization, J. Mach. Learn. Res., 14 (2014), pp. 1523-1548.