An accelerated proximal coordinate gradient method

Advances in Neural Information Processing Systems

Volumn 4, Issue January, 2014, Pages 3059-3067

  Lin, Qihang ^a   Lu, Zhaosong ^b   Xiao, Lin ^c  

^a UNIVERSITY OF IOWA   (United States)

^b SIMON FRASER UNIVERSITY   (Canada)

^c MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CONVEX OPTIMIZATION; FUNCTIONS; INFORMATION SCIENCE; OPTIMIZATION; STOCHASTIC SYSTEMS;

CONVEX FUNCTIONS; CONVEX OPTIMIZATION PROBLEMS; COORDINATE ASCENT; DIMENSIONAL VECTORS; EFFICIENT IMPLEMENTATION; EMPIRICAL RISK MINIMIZATION; IMPROVED CONVERGENCE; LINEAR CONVERGENCE RATE;

GRADIENT METHODS;

EID: 84937904560     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (23)

1. A. Beck and M. Teboulle. A fast iterative shrinkage-threshold algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1): 183-202, 2009.
2. A. Beck and L. Tetruashvili. On the convergence of block coordinate descent type methods. SIAM Journal on Optimization, 13(4): 2037-2060, 2013.
3. K.-W. Chang, C.-J. Hsieh, and C.-J. Lin. Coordinate descent method for large-scale l2-loss linear support vector machines. Journal of Machine Learning Research, 9: 1369-1398, 2008.
4. O. Fercoq and P. Richtárik. Accelerated, parallel and proximal coordinate descent. Manuscript, arXiv: 1312.5799, 2013.
5. M. Hong, X. Wang, M. Razaviyayn, and Z. Q. Luo. Iteration complexity analysis of block coordinate descent methods. Manuscript, arXiv: 1310.6957, 2013.
6. C.-J. Hsieh, K.-W. Chang, C.-J. Lin, S.-S. Keerthi, and S. Sundararajan. A dual coordinate descent method for large-scale linear svm. In Proceedings of the 25th International Conference on Machine Learning (ICML), pages 408-415, 2008.
7. R. Johnson and T. Zhang. Accelerating stochastic gradient descent using predictive variance reduction. In Advances in Neural Information Processing Systems 26, pages 315-323. 2013.
8. Y. T. Lee and A. Sidford. Efficient accelerated coordinate descent methods and faster algorithms for solving linear systems. arXiv: 1305.1922.
9. Q. Lin, Z. Lu, and L. Xiao. An accelerated proximal coordinate gradient method and its application to regularized empirical risk minimization. Technical Report MSR-TR-2014-94, Microsoft Research, 2014. (arXiv: 1407.1296).
10. Z. Lu and L. Xiao. On the complexity analysis of randomized block-coordinate descent methods. Accepted by Mathematical Programming, Series A, 2014. (arXiv: 1305.4723).
11. Z. Q. Luo and P. Tseng. On the convergence of the coordinate descent method for convex differentiable minimization. Journal of Optimization Theory & Applications, 72(1): 7-35, 2002.
12. Y. Nesterov. Introductory Lectures on Convex Optimization: A Basic Course. Kluwer, Boston, 2004.
13. Y. Nesterov. Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM Journal on Optimization, 22(2): 341-362, 2012.
14. P. Richtárik and M. Takáč. Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function. Mathematical Programming, 144(1): 1-38, 2014.
15. N. Le Roux, M. Schmidt, and F. Bach. A stochastic gradient method with an exponential convergence rate for finite training sets. In Advances in Neural Information Processing Systems 25, pages 2672-2680. 2012.
16. A. Saha and A. Tewari. On the non-asymptotic convergence of cyclic coordinate descent methods. SIAM Jorunal on Optimization, 23: 576-601, 2013.
17. M. Schmidt, N. Le Roux, and F. Bach. Minimizing finite sums with the stochastic average gradient. Technical Report HAL 00860051, INRIA, Paris, France, 2013.
18. S. Shalev-Shwartz and A. Tewari. Stochastic methods for l1 regularized loss minimization. In Proceedings of the 26th International Conference on Machine Learning (ICML), pages 929-936, Montreal, Canada, 2009.
19. S. Shalev-Shwartz and T. Zhang. Stochastic dual coordinate ascent methods for regularized loss minimization. Journal of Machine Learning Research, 14: 567-599, 2013.
20. S. Shalev-Shwartz and T. Zhang. Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization. Proceedings of the 31st International Conference on Machine Learning (ICML), JMLR W&CP, 32(1): 64-72, 2014.
21. P. Tseng. Convergence of a block coordinate descent method for nondifferentiable minimization. Journal of Optimization Theory and Applications, 140: 513-535, 2001.
22. P. Tseng. On accelerated proximal gradient methods for convex-concave optimization. Unpublished manuscript, 2008.
23. L. Xiao and T. Zhang. A proximal stochastic gradient method with progressive variance reduction. Technical Report MSR-TR-2014-38, Microsoft Research, 2014. (arXiv: 1403.4699).