Communication complexity of distributed convex learning and optimization

Advances in Neural Information Processing Systems

Volumn 2015-January, Issue , 2015, Pages 1756-1764

  Arjevani, Yossi ^a   Shamir, Ohad ^a  

^a WEIZMANN INSTITUTE OF SCIENCE   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

OPTIMIZATION;

COMMUNICATION COMPLEXITY; COMMUNICATION ROUNDS; COMPUTATIONAL POWER; DISTRIBUTED METHODS; OBJECTIVE FUNCTIONS; STATISTICAL DATAS;

INFORMATION SCIENCE;

EID: 84965119190     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. A. Agarwal, O. Chapelle, M. Dudík, and J. Langford. A reliable effective terascale linear learning system. CoRR, abs/1110.4198, 2011.
2. M.-F. Balcan, A. Blum, S. Fine, and Y. Mansour. Distributed learning, communication complexity and privacy. In COLT, 2012.
3. M.-F. Balcan, V. Kanchanapally, Y. Liang, and D. Woodruff. Improved distributed principal component analysis. In NIPS, 2014.
4. R. Bekkerman, M. Bilenko, and J. Langford. Scaling up machine learning: Parallel and distributed approaches. Cambridge University Press, 2011.
5. S. P. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein. Distributed optimization and statistical learning via ADMM. Foundations and Trends in Machine Learning, 3(1):1-122, 2011.
6. K. Clarkson and D. Woodruff. Numerical linear algebra in the streaming model. In STOC, 2009.
7. A. Cotter, O. Shamir, N. Srebro, and K. Sridharan. Better mini-batch algorithms via accelerated gradient methods. In NIPS, 2011.
8. O. Dekel, R. Gilad-Bachrach, O. Shamir, and L. Xiao. Optimal distributed online prediction using mini-batches. Journal of Machine Learning Research, 13:165-202, 2012.
9. J. Duchi, A. Agarwal, and M. Wainwright. Dual averaging for distributed optimization: Convergence analysis and network scaling. IEEE Trans. Automat. Contr., 57(3):592-606, 2012.
10. R. Frostig, R. Ge, S. Kakade, and A. Sidford. Competing with the empirical risk minimizer in a single pass. arXiv preprint arXiv:1412.6606, 2014.
11. M. Jaggi, V. Smith, M. Takác, J. Terhorst, S. Krishnan, T. Hofmann, and M. Jordan. Communication-efficient distributed dual coordinate ascent. In NIPS, 2014.
12. J. Lee, T. Ma, and Q. Lin. Distributed stochastic variance reduced gradient methods. CoRR, 1507.07595, 2015.
13. D. Mahajan, S. Keerthy, S. Sundararajan, and L. Bottou. A parallel SGD method with strong convergence. CoRR, abs/1311.0636, 2013.
14. Y. Nesterov. Introductory lectures on convex optimization: A basic course. Springer, 2004.
15. Y. Nesterov. Smooth minimization of non-smooth functions. Mathematical programming, 103(1):127-152, 2005.
16. B. Recht, C. Ré, S. Wright, and F. Niu. Hogwild: A lock-free approach to parallelizing stochastic gradient descent. In NIPS, 2011.
17. P. Richtárik and M. Takác. Distributed coordinate descent method for learning with big data. CoRR, abs/1310.2059, 2013.
18. O. Shamir. Fundamental limits of online and distributed algorithms for statistical learning and estimation. In NIPS, 2014.
19. O. Shamir and N. Srebro. On distributed stochastic optimization and learning. In Allerton Conference on Communication, Control, and Computing, 2014.
20. O. Shamir, N. Srebro, and T. Zhang. Communication-efficient distributed optimization using an approximate newton-type method. In ICML, 2014.
21. T. Tao. Topics in random matrix theory, volume 132. American Mathematical Soc., 2012.
22. J. Tsitsiklis and Z.-Q. Luo. Communication complexity of convex optimization. J. Complexity, 3(3):231-243, 1987.
23. T. Yang. Trading computation for communication: Distributed SDCA. In NIPS, 2013.
24. Y.-L. Yu. Better approximation and faster algorithm using proximal average. In NIPS, 2013.
25. Y. Zhang, J. Duchi, and M. Wainwright. Communication-efficient algorithms for statistical optimization. Journal of Machine Learning Research, 14:3321-3363, 2013.
26. Y. Zhang and L. Xiao. Communication-efficient distributed optimization of self-concordant empirical loss. In ICML, 2015.
27. M. Zinkevich, M. Weimer, A. Smola, and L. Li. Parallelized stochastic gradient descent. In NIPS, 2010.