Accelerated, Parallel, and PROXimal coordinate descent

SIAM Journal on Optimization

Volumn 25, Issue 4, 2015, Pages 1997-2023

  Fercoq, Olivier ^a   Richtárik, Peter ^b  

^a INSTITUT TELECOM   (France)

^b UNIVERSITY OF EDINBURGH   (United Kingdom)

Author keywords

Acceleration; Big data; Complexity; Convex optimization; Parallel methods; Partial separability; Proximal methods; Randomized coordinate descent

Indexed keywords

ACCELERATION; BIG DATA; CONVEX OPTIMIZATION; FUNCTIONS; OPTIMIZATION; RATE CONSTANTS;

COMPLEXITY; COORDINATE DESCENT; PARALLEL METHOD; PARTIAL SEPARABILITY; PROXIMAL METHODS;

ITERATIVE METHODS;

EID: 84953234319     PISSN: 10526234     EISSN: None     Source Type: Journal    
DOI: 10.1137/130949993     Document Type: Article

References (30)

1. A. BECK AND M. TEBOULLE, A fast iterative shrinkage-thresholding algorithm for linear inverse problems, SIAM J. Imaging Sci., 2(2009), pp. 183-202.
2. 1-regularized loss minimization, in the 28th International Conference on Machine Learning, 2011, pp. 321-328.
3. L. EL GHAOUI, V. VIALLON, AND T. RABBANI, Safe Feature Elimination for the Lasso, Technical report, University of California at Berkeley, Berkeley, CA, 2011.
4. O. FERCOQ, Z. QU, P. RICHTÁRIK, AND M. TAKÁČ, Fast distributed coordinate descent for minimizing non-strongly convex losses, in IEEE International Workshop on Machine Learning for Signal Processing, 2014, pp 1-6.
5. O. FERCOQ AND P. RICHTÁRIK, Smooth minimization of nonsmooth functions by parallel coordinate descent, arXiv:1309.5885, 2013.
6. Y. T. LEE AND A. SIDFORD, Efficient accelerated coordinate descent methods and faster algorithms for solving linear systems, arXiv:1305.1922, 2013.
7. D. LEVENTHAL AND A. S. LEWIS, Randomized methods for linear constraints: Convergence rates and conditioning, Math. Oper. Res., 35(2010), pp. 641-654.
8. Q. LIN, Z. LU, AND L. XIAO, An accelerated proximal coordinate gradient method and its application to regularized empirical risk minimization, arXiv:1407.1296, 2014.
9. J. LIU, S. J. WRIGHT, C. RÉ, V. BITTORF, AND S. SRIDHAR, An asynchronous parallel stochastic coordinate descent algorithm, arXiv:1311.1873, 2013.
10. I. NECOARA AND D. CLIPICI, Distributed Coordinate Descent Methods for Composite Minimization, Tech. report, Politehnica University of Bucharest, Bucharest, Romania, 2013.
11. I. NECOARA AND D. CLIPICI, Efficient parallel coordinate descent algorithm for convex optimization problems with separable constraints: application to distributed MPC, J. Process Control, 23(2013), pp. 243-253.
12. I. NECOARA, Y. NESTEROV, AND F. GLINEUR, Efficiency of randomized coordinate descent methods on optimization problems with linearly coupled constraints, Tech. report, Politehnica University of Bucharest, Bucharest, Romania, 2012.
13. Y. NESTEROV, A method of solving a convex programming problem with convergence rate O (1/k2), Soviet Math. Dokl, 27(1983), pp. 372-376.
14. Y. NESTEROV, Smooth minimization of nonsmooth functions, Math. Program., 103(2005), pp. 127-152.
15. Y. NESTEROV, Efficiency of coordinate descent methods on huge-scale optimization problems, SIAM J. Optim., 22(2012), pp. 341-362.
16. J. C. PLATT, Fast training of support vector machines using sequential minimal optimization, in Advances in Kernel Methods - Support Vector Learning, B. Scholkopf, C. Burges, and A. Smola, eds., MIT Press, Cambridge, MA, 1999, pp. 185-208.
17. Z. QU, O. FERCOQ, AND P. RICHTÁRIK, Accelerated Coordinate Descent Method for Strongly Convex Functions, Tech. report, 05/2014, University of Edinburgh, Edinburgh, UK, 2014.
18. P. RICHTÁRIK AND M. TAKÁČ, Distributed coordinate descent method for learning with big data, arXiv:1310.2059, 2013.
19. P. RICHTÁRIK AND M. TAKÁČ, On optimal probabilities in stochastic coordinate descent methods, Optim. Lett., 2015, DOI: 10.1007/s11590-015-0916-1.
20. P. RICHTÁRIK AND M. TAKÁČ, Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function, Math. Program., 144(2014), pp. 1-38.
21. P. RICHTÁRIK AND M. TAKÁČ, Parallel coordinate descent methods for big data optimization problems, Math. Program., Ser. A, (2015), DOI: 10.1007/s10107-015-0901-6.
22. 1-regularized loss minimization, J. Mach. Learn. Res., 12(2011), pp. 1865-1892.
23. S. SHALEV-SHWARTZ AND T. ZHANG, Proximal stochastic dual coordinate ascent, arXiv:1211. 2717, 2012.
24. S. SHALEV-SHWARTZ AND T. ZHANG, Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization, arXiv: 1309.2375, 2013.
25. J. STAMPER, A. NICULESCU-MIZIL, S. RITTER, G. J. GORDON, AND K. R. KOEDINGER, Bridge to algebra 2008-2009. Challenge data set from KDD cup 2010 educational data mining challenge, 2010; available online at http://pslcdatashop. web.cmu.edu/KDDCup/downloads. jsp.
26. M. TAKÁČ, A. BIJRAL, P. RICHTÁRIK, AND N. SREBRO, Mini-batch primal and dual methods for SVMs, in Proceedings of the 30th International Conference on Machine Learning, Vol. 28, 2013, pp. 1022-1030.
27. R. TAPPENDEN, P. RICHTÁRIK, AND J. GONDZIO, Inexact block coordinate descent method: Complexity and preconditioning, arXiv: 1304.5530, 2013.
28. P. TSENG, On accelerated proximal gradient methods for convex-concave optimization, SIAM J. Optim., 2008, submitted.
29. T. T. WU AND K. LANGE, Coordinate descent algorithms for lasso penalized regression, Ann. Appl. Stat., (2008), pp. 224-244.
30. L. XIAO AND Z. LU, On the complexity analysis of randomized block-coordinate descent methods, arXiv: 1305.4723, 2013.