Stochastic Dual Coordinate Ascent methods for regularized loss minimization

Journal of Machine Learning Research

Volumn 14, Issue 1, 2013, Pages 567-599

  Shalev Shwartz, Shai ^a   Zhang, Tong ^b  

^a HEBREW UNIVERSITY OF JERUSALEM   (Israel)

^b RUTGERS UNIVERSITY   (United States)

Author keywords

Computational complexity; Logistic regression; Optimization; Regularized loss minimization; Ridge regression; Stochastic Dual Coordinate Ascent; Support vector machines

Indexed keywords

CONVERGENCE ANALYSIS; COORDINATE ASCENT; LOGISTIC REGRESSIONS; LOSS MINIMIZATION; RIDGE REGRESSION; STOCHASTIC GRADIENT DESCENT; SUPERVISED MACHINE LEARNING; THEORETICAL GUARANTEES;

COMPUTATIONAL COMPLEXITY; LOGISTICS; OPTIMIZATION; REGRESSION ANALYSIS; SUPPORT VECTOR MACHINES;

STOCHASTIC SYSTEMS;

EID: 84875134236     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (19)

1. L. Bottou and O. Bousquet. The tradeoffs of large scale learning. In Advances in Neural Information Processing Systems (NIPS), pages 161-168, 2008.
2. M. Collins, A. Globerson, T. Koo, X. Carreras, and P. Bartlett. Exponentiated gradient algorithms for conditional random fields and max-margin markov networks. Journal of Machine Learning Research, 9:1775-1822, 2008.
3. Y. Le Cun and L. Bottou. Large scale online learning. In Advances in Neural Information Processing Systems 16: Proceedings of the 2003 Conference, volume 16, page 217. MIT Press, 2004.
4. C. J. Hsieh, K. W. Chang, C. J. Lin, S. S. Keerthi, and S. Sundararajan. A dual coordinate descent method for large-scale linear SVM. In Proceedings of the International Conference on Machine Learning (ICML), pages 408-415, 2008.
5. D. Hush, P. Kelly, C. Scovel, and I. Steinwart. QP algorithms with guaranteed accuracy and run time for support vector machines. Journal of Machine Learning Research, 7:733-769, 2006.
6. T. Joachims. Making large-scale support vector machine learning practical. In B. Schölkopf, C. Burges, and A. Smola, editors, Advances in Kernel Methods - Support Vector Learning. MIT Press, 1998.
7. S. Lacoste-Julien, M. Jaggi, M. Schmidt, and P. Pletscher. Stochastic block-coordinate frank-wolfe optimization for structural svms. arXiv preprint:1207.4747, 2012.
8. N. Le Roux, M. Schmidt, and F. Bach. A stochastic gradient method with an exponential convergence rate for strongly-convex optimization with finite training sets. In Advances in Neural Information Processing Systems (NIPS): Proceedings of the 2012 Conference, 2012. arXiv preprint:1202.6258.
9. Z. Q. Luo and P. Tseng. On the convergence of coordinate descent method for convex differentiable minimization. J. Optim. Theory Appl., 72:7-35, 1992.
10. O. Mangasarian and D. Musicant. Successive overrelaxation for support vector machines. IEEE Transactions on Neural Networks, 10, 1999.
11. N. Murata. A statistical study of on-line learning. Online Learning and Neural Networks. Cambridge University Press, Cambridge, UK, 1998.
12. Y. Nesterov. Efficiency of coordinate descent methods on huge-scale optimization problems. SIAM Journal on Optimization, 22(2):341-362, 2012.
13. J. C. Platt. Fast training of Support Vector Machines using sequential minimal optimization. In B. Schölkopf, C. Burges, and A. Smola, editors, Advances in Kernel Methods - Support Vector Learning. MIT Press, 1998.
14. H. Robbins and S. Monro. A stochastic approximation method. The Annals of Mathematical Statistics, pages 400-407, 1951.
15. S. Shalev-Shwartz and N. Srebro. SVM optimization: Inverse dependence on training set size. In International Conference on Machine Learning (ICML), pages 928-935, 2008.
16. 1 regularized loss minimization. In Proceedings of the International Conference on Machine Learning (ICML), page 117, 2009.
17. S. Shalev-Shwartz, Y. Singer, and N. Srebro. Pegasos: Primal Estimated sub-GrAdient SOlver for SVM. In Proceedings of the International Conference on Machine Learning (ICML), pages 807-814, 2007.
18. K. Sridharan, N. Srebro, and S. Shalev-Shwartz. Fast rates for regularized objectives. In Advances in Neural Information Processing Systems (NIPS), 2009.
19. T. Zhang. Solving large scale linear prediction problems using stochastic gradient descent algorithms. In Proceedings of the Twenty-First International Conference on Machine Learning (ICML), 2004.