Communication/computation tradeoffs in consensus-based distributed optimization

Advances in Neural Information Processing Systems

Volumn 3, Issue , 2012, Pages 1943-1951

  Tsianos, Konstantinos I ^a   Lawlor, Sean ^a   Rabbat, Michael G ^a  

^a MCGILL UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

COMPLETE GRAPHS; CONVEX MINIMIZATION; DISTRIBUTED OPTIMIZATION; EXPANDER GRAPHS; METRIC LEARNING; OPTIMAL NUMBER; THEORY AND PRACTICE;

ALGORITHMS; COMMUNICATION; OPTIMIZATION;

GRAPH THEORY;

EID: 84877782920     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (17)

1. O. Reingold, S. Vadhan, and A.Wigderson, "Entropy waves, the zig-zag graph product, and new constant-degree expanders," Annals of Mathematics, vol. 155, no. 2, pp. 157-187, 2002.
2. Y. Nesterov, "Primal-dual subgradient methods for convex problems," Mathematical Programming Series B, vol. 120, pp. 221-259, 2009.
3. R. Bekkerman, M. Bilenko, and J. Langford, Scaling up Machine Learning, Parallel and Distributed Approaches. Cambridge University Press, 2011.
4. J. Duchi, A. Agarwal, and M. Wainwright, "Dual averaging for distributed optimization: Convergence analysis and network scaling," IEEE Transactions on Automatic Control, vol. 57, no. 3, pp. 592-606, 2011.
5. A. Nedic and A. Ozdaglar, "Distributed subgradient methods for multi-agent optimization," IEEE Transactions on Automatic Control, vol. 54, no. 1, January 2009.
6. B. Johansson, M. Rabi, and M. Johansson, "A randomized incremental subgradient method for distributed optimization in networked systems," SIAM Journal on Control and Optimization, vol. 20, no. 3, 2009.
7. S. S. Ram, A. Nedic, and V. V. Veeravalli, "Distributed stochastic subgradient projection algorithms for convex optimization," Journal of Optimization Theory and Applications, vol. 147, no. 3, pp. 516-545, 2011.
8. A. Agarwal and J. C. Duchi, "Distributed delayed stochastic optimization," in Neural Information Processing Systems, 2011.
9. K. I. Tsianos and M. G. Rabbat, "Distributed dual averaging for convex optimization under communication delays," in American Control Conference (ACC), 2012.
10. F. Chung, Spectral Graph Theory. AMS, 1998.
11. E. P. Xing, A. Y. Ng, M. I. Jordan, and S. Russell, "Distance metric learning, with application to clustering with side-information," in Neural Information Processing Systems, 2003.
12. K. Q. Weinberger and L. K. Saul, "Distance metric learning for large margin nearest neighbor classification," Journal of Optimization Theory and Applications, vol. 10, pp. 207-244, 2009.
13. K. Q. Weinberger, F. Sha, and L. K. Saul, "Convex optimizations for distance metric learning and pattern classification," IEEE Signal Processing Magazine, 2010.
14. S. Shalev-Shwartz, Y. Singer, and A. Y. Ng, "Online and batch learning of pseudo-metrics," in ICML, 2004, pp. 743-750.
15. M. A. Zinkevich, M. Weimer, A. Smola, and L. Li, "Parallelized stochastic gradient descent," in Neural Information Processing Systems, 2010.
16. R. McDonald, K. Hall, and G. Mann, "Distributed training strategies for the structured perceptron," in Annual Conference of the North American Chapter of the Association for Computational Linguistics, 2012, pp. 456-464.
17. G. Mann, R. McDonald, M. Mohri, N. Silberman, and D. D. Walker, "Efficient large-scale distributed training of conditional maximum entropy models," in Neural Information Processing Systems, 2009, pp. 1231-1239.