Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization

31st International Conference on Machine Learning, ICML 2014

Volumn 1, Issue , 2014, Pages 111-119

  Shnlev Shwartz, Shai ^a   Zhang, Tong ^b,c  

^a HEBREW UNIVERSITY OF JERUSALEM   (Israel)

^b RUTGERS UNIVERSITY   (United States)

^c BAIDU INC   (China)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; ITERATIVE METHODS; LEARNING SYSTEMS; OPTIMIZATION; REGRESSION ANALYSIS;

COORDINATE ASCENT; INNER-OUTER ITERATIONS; LEARNING OPTIMIZATIONS; LOGISTIC REGRESSIONS; LOSS MINIMIZATION; MULTICLASS SVM; RIDGE REGRESSION; RUNTIMES;

STOCHASTIC SYSTEMS;

EID: 84919796368     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. Baes, Michel. Estimate sequence methods: extensions and approximations. Institute for Operations Research, ETH, Zurich, Switzerland, 2009.
2. Beck, A. and Teboulle, M. A fast iterative shrinkage-Thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1): 183-202, 2009.
3. Collins, M., A. Globerson, Koo, T., Carreras, X., and Bartlett, P. Exponentiated gradient algorithms for conditional random fields and max-margin markov networks. Journal of Machine Learning Research, 9: 1775-1822, 2008.
4. Cotter, Andrew, Shamir, Ohad, Srebro, Nathan, and Sridharan, Karthik. Better mini-batch algorithms via accelerated gradient methods. arXiv preprint arXiv: 1106.4574, 2011.
5. d'Aspremont, Alexandre. Smooth optimization with approximate gradient. SIAM Journal on Optimization, 19(3): 1171-1183, 2008.
6. Devolder, Olivier, Glineur, Francois, and Nesterov, Yuri. First-order methods of smooth convex optimization with inexact oracle. Technical Report 2011/2, CORE, 2011.
7. Duchi, J. and Singer, Y. Efficient online and batch learning using forward backward splitting. The Journal of Machine Learning Research, 10: 2899-2934, 2009.
8. Duchi, John, Shalev-Shwartz, Shai, Singer, Yoram, and Tewari, Ambuj. Composite objective mirror descent. In Proceedings of the 23rd Annual Conference on Learning Theory, pp. 14-26, 2010.
9. Ghadimi, Saeed and Lan, Guanghui. Optimal stochastic approximation algorithms for strongly convex stochastic composite optimization i: A generic algorithmic framework. SIAM Journal on Optimization, 22(4): 1469-1492, 2012.
10. Hu, Chonghai, Pan, Weike, and Kwok, James T. Accelerated gradient methods for stochastic optimization and online learning. In Advances in Neural Information Processing Systems, pp. 781-789, 2009.
11. Lacoste-Julien, S., Jaggi, M., Schmidt, M., and Pletscher, P. Stochastic block-coordinate frank-wolfe optimization for structural svms. arXiv preprint arXiv.l2Q7.4747 , 2012.
12. Langford, J., Li, L., and Zhang, T. Sparse online learning via truncated gradient. In NIPS, pp. 905-912, 2009.
13. Le Roux, Nicolas, Schmidt, Mark, and Bach, Francis. A Stochastic Gradient Method with an Exponential Convergence Rate for Strongly-Convex Optimization with Finite Training Sets. arXiv preprint arXiv: 1202.6258, 2012.
14. Nesterov, Y. Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM Journal on Optimization, 22(2): 341-362, 2012.
15. Nesterov, Yurii. Smooth minimization of non-smooth functions. Mathematical Programming, 103(1): 127-152, 2005.
16. Nesterov, Yurii. Gradient methods for minimizing composite objective function, 2007.
17. Richtarik, Peter and Takac, Martin. Iteration complexity of randomized block-coordinate descent methods for minimizing a composite function. Mathematical Programming, pp. 1-38, 2012.
18. Schmidt, Mark, Roux, Nicolas Le, and Bach, Francis. Convergence rates of inexact proximal-gradient methods for convex optimization. Technical Report arXiv: l 109.2415, arXiv, 2011.
19. Shalev-Shwartz, S. and Tewari, A. Stochastic methods for 11-regularized loss minimization. 77ie Journal of Machine Learning Research, 12: 1865-1892, 2011.
20. Shalev-Shwartz, Shai and Tewari, Ambuj. Stochastic methods for U regularized loss minimization. In 1CML, pp. 117, 2009.
21. Shalev-Shwartz, Shai and Zhang, Tong. Stochastic dual coordinate ascent methods for regularized loss minimization. Journal of Machine Learning Research, 14: 567-599, Feb 2013.
22. Shalev-Shwartz, Shai and Zhang, Tong. Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization, arxiv: 1309.2375, 2013.
23. Shalev-Shwartz, Shai, Singer, Yoram, and Srebro, Nathan. Pegasos: Primal Estimated sub-GrAdient SOlver for SVM. In ICML, pp. 807-814, 2007.
24. Xiao, Lin. Dual averaging method for regularized stochastic learning and online optimization. Journal of Machine Learning Research, 11: 2543-2596, 2010.
25. Zhang, Tong. On the dual formulation of regularized linear systems. Machine Learning, 46: 91-129, 2002.