Asynchronous distributed ADMM for consensus optimization

31st International Conference on Machine Learning, ICML 2014

Volumn 5, Issue , 2014, Pages 3689-3697

  Zhang, Ruiliang ^a   Kwok, James T ^a  

^a HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; BIG DATA; LEARNING SYSTEMS; OPTIMIZATION;

ALTERNATING DIRECTION METHOD OF MULTIPLIERS; BOUNDED DELAYS; CONVERGENCE RATES; DISTRIBUTED OPTIMIZATION; FASTER CONVERGENCE; MACHINE LEARNING PROBLEM; OPTIMIZATION PROBLEMS; SIMPLE STRUCTURES;

PROBLEM SOLVING;

EID: 84919796967     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. Agarwal, A. and Duchi, J.C. Distributed delayed stochastic optimization. In Advances in Neural Information Processing Systems 24, 2011.
2. Albrecht, J.R., Tuttle, C, Snoeren, A.C., and Vahdat, A. Loose synchronization for large-scale networked systems. In Proceedings of the USENIX Annual Technical Conference, pp. 301-314, 2006.
3. Banerjee, O., El Ghaoui, L., and d'Aspremont, A. Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. Journal of Machine Learning Research, 9:485-516, 2008.
4. Berry, M.W., Browne, M., Langville, A.N., Pauca, V.P, and Plemmons, R.J. Algorithms and applications for approximate nonnegative matrix factorization. Computational Statistics & Data Analysis, 52(1): 155-173, 2007.
5. Bertsekas, D.P and Tsitsiklis, J.N. Parallel and Distributed Computation. Prentice Hall, 1989.
6. Boyd, S., Parikh, N., Chu, E., Peleato, B, and Eckstein, J. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1):1-122, 2011.
7. Dean, J. and Ghemawat, S. MapReduce: Simplified data processing on large clusters. Communications of the ACM, 51(1): 107-113, 2008.
8. He, B. and Yuan, X. On the 0(1/n) convergence rate of the Douglas-Rachford alternating direction method. SIAM Journal on Numerical Analysis, 50(2):700-709, 2012.
9. Ho, Q., Cipar, J., Cui, H., Lee, S., Kim, J.K., Gibbons, P.B., Gibson, G.A., Ganger, G., and Xing, E. More effective distributed ML via a stale synchronous parallel parameter server. In Advances in Neural information Processing Systems 26, pp. 1223-1231, 2013.
10. Kim, S., Sohn, K.-A., and Xing, E.P. A multivariate regression approach to association analysis of a quantitative trait network. Bioinformatics, 25(12):i204-i212, 2009.
11. Langford, J., Smola, A., and Zinkevich, M. Slow learners are fast. In Advances in Neural Information Processing Systems 22, 2009.
12. Li, M., Andersen, D.G., and Smola, A. Distributed delayed proximal gradient methods. In NIPS Workshop on Optimization for Machine Learning, 2013.
13. Ling, Q., Xu, Y, Yin, W., and Wen, Z. Decentralized low-rank matrix completion. In Proceedings of the International Conference on Acoustics, Speech and Signal Processing, pp. 2925-2928, 2012.
14. Lubell-Doughtie, P. and Sondag, J. Practical distributed classification using the alternating direction method of multipliers algorithm. In Proceedings of the International Conference on Big Data, 2013.
15. Mota, J, Xavier, J, Aguiar, P, and Puschel, Markus. D-admm: A communication-efficient distributed algorithm for separable optimization. IEEE Transactions on Signal Processing, 61(10):2718-2723, 2013.
16. Niu, F., Recht, B., Ré, C, and Wright, S.J. Hogwild: A lock-free approach to parallelizing stochastic gradient descent. In Advances in Neural Information Processing Systems 24, 2011.
17. Ouyang, H., He, N., Tran, L., and Gray, A.G. Stochastic alternating direction method of multipliers. In Proceedings of the 30th International Conference on Machine Learning, pp. 80-88, 2013.
18. Shalev-Shwartz, S., Singer, Y, and Srebro, N. Pegasos: Primal estimated sub-gradient solver for SVM. In Proceedings of the 24th International Conference on Machine Learning, pp. 807-814, 2007.
19. Suri, S. and Vassilvitskii, S. Counting triangles and the curse of the last reducer. In Proceedings of the 20th International Conference on World Wide Web, pp. 607-614, 2011.
20. Suzuki, T. Dual averaging and proximal gradient descent for online alternating direction multiplier method. In Proceedings of the 30th International Conference on Machine Learning, pp. 392-400, 2013.
21. Tibshirani, R.J. and Taylor, J. The solution path of the generalized lasso. Annals of Statistics, 39(3): 1335-1371, 2011.
22. Wei, E. and Ozdaglar, A. Distributed alternating direction method of multipliers. In Proceedings of the 51st Annual Conference on Decision and Control, pp. 5445-5450, 2012.
23. Wei, E. and Ozdaglar, A. On the 0(1/k) convergence of asynchronous distributed alternating direction method of multipliers. Preprint arXiv: 1307.8254, 2013.
24. Zhang, X., Burger, M., and Osher, S. A unified primal-dual algorithm framework based on Bregman iteration. Journal of Scientific Computing, 46(1):20-46, 2011.
25. Zhu, H., Cano, A., and Giannakis, G.B. Distributed consensus-based demodulation: Algorithms and error analysis. IEEE Transactions on Wireless Communications, 9(6):2044-2054,2010.