Trading computation for communication: Distributed stochastic dual coordinate ascent

Advances in Neural Information Processing Systems

Volumn , Issue , 2013, Pages

  Yang, Tianbao ^a  

^a NEC LABORATORIES AMERICA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; COMMUNICATION; OPTIMIZATION; STOCHASTIC SYSTEMS; TIME VARYING NETWORKS;

ALTERNATING DIRECTION METHODS; BETTER PERFORMANCE; COMPETITIVE PERFORMANCE; COORDINATE ASCENT; DISTRIBUTED FRAMEWORK; DISTRIBUTED OPTIMIZATION; STOCHASTIC GRADIENT DESCENT METHOD; THEORETICAL GUARANTEES;

DISTRIBUTED COMPUTER SYSTEMS;

EID: 84898970556     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (24)

1. A. Agarwal and J. C. Duchi. Distributed delayed stochastic optimization. In CDC, pages 5451-5452, 2012.
2. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn., 3:1-122, 2011.
3. J. K. Bradley, A. Kyrola, D. Bickson, and C. Guestrin. Parallel Coordinate Descent for L1- Regularized Loss Minimization. In ICML, 2011.
4. C. T. Chu, S. K. Kim, Y. A. Lin, Y. Yu, G. R. Bradski, A. Y. Ng, and K. Olukotun. Map-Reduce for machine learning on multicore. In NIPS, pages 281-288, 2006.
5. W. Deng and W. Yin. On the global and linear convergence of the generalized alternating direction method of multipliers. Technical report, 2012.
6. M. Eberts and I. Steinwart. Optimal learning rates for least squares svms using gaussian kernels. In NIPS, pages 1539-1547, 2011.
7. D. Gabay and B. Mercier. A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Comput. Math. Appl., 2:17-40, 1976.
8. C.-J. Hsieh, K.-W. Chang, C.-J. Lin, S. S. Keerthi, and S. Sundararajan. A dual coordinate descent method for large-scale linear svm. In ICML, pages 408-415, 2008.
9. H. Du III, J. M. Phillips, A. Saha, and S. Venkatasubramanian. Protocols for learning classifiers on distributed data. JMLR- Proceedings Track, 22:282-290, 2012.
10. S. Lacoste-Julien, M. Jaggi, M. W. Schmidt, and P. Pletscher. Stochastic block-coordinate frank-wolfe optimization for structural svms. CoRR, abs/1207.4747, 2012.
11. J. Langford, A. Smola, and M. Zinkevich. Slow learners are fast. In NIPS, pages 2331-2339. 2009.
12. Z. Q. Luo and P. Tseng. On the convergence of the coordinate descent method for convex differentiable minimization. Journal of Optimization Theory and Applications, pages 7-35, 1992.
13. G. Mann, R. McDonald, M. Mohri, N. Silberman, and D. Walker. Efficient Large-Scale distributed training of conditional maximum entropy models. In NIPS, pages 1231-1239. 2009.
14. H. Ouyang, N. He, L. Tran, and A. G. Gray. Stochastic alternating direction method of multipliers. In ICML, pages 80-88, 2013.
15. N. L. Roux, M. W. Schmidt, and F. Bach. A stochastic gradient method with an exponential convergence rate for finite training sets. In NIPS, pages 2672-2680, 2012.
16. S. Shalev-Shwartz and T. Zhang. Stochastic Dual Coordinate Ascent Methods for Regularized Loss Minimization. JMLR, 2013.
17. S. Smale and D.-X. Zhou. Estimating the approximation error in learning theory. Anal. Appl. (Singap.), 1(1):17-41, 2003.
18. K. Sridharan, S. Shalev-Shwartz, and N. Srebro. Fast rates for regularized objectives. In NIPS, pages 1545-1552, 2008.
19. T. Suzuki. Dual averaging and proximal gradient descent for online alternating direction multiplier method. In ICML, pages 392-400, 2013.
20. M. Takác, A. S. Bijral, P. Richtárik, and N. Srebro. Mini-batch primal and dual methods for svms. In ICML, 2013.
21. C. H. Teo, S. Vishwanthan, A. J. Smola, and Q. V. Le. Bundle methods for regularized risk minimization. JMLR, pages 311-365, 2010.
22. K. I. Tsianos, S. Lawlor, and M. G. Rabbat. Communication/computation tradeoffs in consensus-based distributed optimization. In NIPS, pages 1952-1960, 2012.
23. C. Zhang, H. Lee, and K. G. Shin. Efficient distributed linear classification algorithms via the alternating direction method of multipliers. In AISTAT, pages 1398-1406, 2012.
24. M. Zinkevich, M. Weimer, A. Smola, and L. Li. Parallelized stochastic gradient descent. In NIPS, pages 2595-2603, 2010.