Stochastic primal-dual coordinate method for regularized empirical risk minimization

32nd International Conference on Machine Learning, ICML 2015

Volumn 1, Issue , 2015, Pages 353-361

  Zhang, Yuchen ^a   Xiao, Lin ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; CONVEX OPTIMIZATION; OPTIMIZATION; STOCHASTIC SYSTEMS;

CONVERGENCE RATES; CONVEX OPTIMIZATION PROBLEMS; CONVEX-CONCAVE SADDLE POINT PROBLEMS; EMPIRICAL RISK MINIMIZATION; LINEAR PREDICTORS; OPTIMIZATION METHOD; PROBLEM STRUCTURE; STATE OF THE ART;

LEARNING SYSTEMS;

EID: 84969524464     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. Bertsekas, D. P. Incremental proximal methods for large scale convex optimization. Mathematical Programming, Ser.B, 129: 163-195, 2011.
2. Blatt, D., Hero, A. O., and Gauchman, H. A convergent incremental gradient method with a constant step size, S1AM Journal on Optimization, 18 (1): 29-51, 2007.
3. Bottou, L. Large-scale machine learning with stochastic gradient descent. In Lechevallier, Y. and Saporta, G. (eds.), Proceedings of the 19th International Conference on Computational Statistics, pp. 177-187, Paris, France, August 2010. Springer.
4. Bousquet, O. and Elisseeff, A. Stability and generalization. Journal of Machine Learning Research, 2: 499-526, 2002.
5. Chambolle, A. and Pock, T. A first-order primal-dual algorithm for convex problems with applications to imaging. Journal of Mathematical Imaging and Vision, 40 (1): 120-145, 2011.
6. Duchi, J. and Singer, Y. Efficient online and batch learning using forward backward splitting. Journal of Machine Learning Research, 10: 2873-2898, 2009.
7. Fan, R.-E. and Lin, C.-J. LIBSVM data: Classification, regression and multi-label. URL: http://www.csie.ntu.edu.twr cjlin/libsvmtools/datasets, 2011.
8. Hastie, T., Tibshirani, R., and Friedman, J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York, 2nd edition, 2009.
9. Hiriart-Urruty, J.-B. and Lemarechal, C. Fundamentals of Convex Analysis. Springer, 2001.
10. Hsieh, C.-J., Chang, K.-W., Lin, C.-J., Keerthi, S., and Sundararajan, S. A dual coordinate descent method for large-scale linear svm. In Proceedings of the 25th International Conference on Machine Learning (ICML), pp. 408-415, 2008.
11. Langford, J., Li, L., and Zhang, T. Sparse online learning via truncated gradient. Journal of Machine Learning Research, 10: 777-801, 2009.
12. Lin, Q., Lu, Z., and Xiao, L. An accelerated proximal coordinate gradient method and its application to regularized empirical risk minimization. Technical Report MSR-TR-2014-94, Microsoft Research, 2014.
13. Nedic, A. and Bertsekas, D. P. Incremental subgradient methods for nondifferentiable optimization. SIAM Journal on Optimization, 12 (1): 109-138, 2001.
14. Nemirovski, A., Juditsky, A., Lan, G., and Shapiro, A. Robust stochastic approximation approach to stochastic programming. SIAM Journal on Optimization, 19 (4): 1574-1609, 2009.
15. Nesterov, Y. Introductory Lectures on Convex Optimization: A Basic Course. Kluwer, Boston, 2004.
16. Nesterov, Y. Smooth minimization of nonsmooth functions. Mathematical Programming, 103: 127-152, 2005.
17. Nesterov, Y. Gradient methods for minimizing composite functions. Mathematical Programming, Ser. B, 140: 125-161, 2013.
18. Nocedal, J. and Wright, S. J. Numerical Optimization. Springer, New York, 2nd edition, 2006.
19. Polyak, B. T. and Juditsky, A. Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization, 30: 838-855, 1992.
20. Roux, N. L., Schmidt, M., and Bach, F. A stochastic gradient method with an exponential convergence rate for finite training sets. In Advances in Neural Information Processing Systems 25, pp. 2672-2680. 2012.
21. Shalev-Shwartz, S. and Zhang, T. Stochastic dual coordinate ascent methods for regularized loss minimization. Journal of Machine Learning Research, 14: 567-599, 2013a.
22. Shalev-Shwartz, S. and Zhang, T. Accelerated proximal stochastic dual coordinate ascent for regularized loss minimization. arXiv: 1309.2375, 2013b.
23. Shamir, O. and Zhang, T. Stochastic gradient descent for non-smooth optimization: Convergence results and optimal averaging schemes. arXiv.1212.1824, 2012.
24. Tseng, P. An incremental gradient (-projection) method with momentum term and adaptive stepsiz rule. SIAM Journal on Optimization, 8 (2): 506-531, 1998.
25. Xiao, L. Dual averaging methods for regularized stochastic learning and online optimization. Journal of Machine Learning Research, 11: 2534-2596, 2010.
26. Zhang, T. Solving large scale linear prediction problems using stochastic gradient descent algorithms. In Proceedings of the 21st International Conference on Machine Learning (ICML), pp. 116-123, Banff, Alberta, Canada, 2004.
27. Zhang, Y. and Xiao, L. Stochastic primal-dual coordinate method for regularized empirical risk minimization. arXiv preprint arXiv: 14093257, 2014.