Fast stochastic alternating direction method of multipliers

31st International Conference on Machine Learning, ICML 2014

Volumn 1, Issue , 2014, Pages 78-86

  Zhong, Leon Wenliang ^a   Kwok, James T ^a  

^a HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; ITERATIVE METHODS; LEARNING SYSTEMS;

ALTERNATING DIRECTION METHOD OF MULTIPLIERS; CONVERGENCE RATES; CONVEX PROBLEMS; FUSED LASSOS; NUMBER OF ITERATIONS;

STOCHASTIC SYSTEMS;

EID: 84919792988     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. Agarwal, A., Bartlett, P.L., Ravikumar, P., and Wainwright, M.J. Information-Theoretic lower bounds on the oracle complexity of stochastic convex optimization. IEEE Transactions on Information Theory, 58(5): 3235-3249, 2012.
2. Bach, R, Jenatton, R., Mairal, J., and Obozinski, G. Convex optimization with sparsity-inducing norms. In Sra, S., Nowozin, S., and Wright, S.J. (eds.), Optimization for Machine Learning, pp. 19-53. MIT Press, 2011.
3. Banerjee, O., El Ghaoui, L., and d'Aspremont, A. Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. Journal of Machine Learning Research, 9: 485-516, 2008.
4. Barbero, A. and Sra, S. Fast Newton-Type methods for total variation regularization. In Proceedings of the 28th International Conference on Machine Learning, pp. 313-320, New York, NY, USA, 2011.
5. Bottou, L. Stochastic learning. In Advanced Lectures on Machine Learning, pp. 146-168. Springer Verlag, 2004.
6. Boyd, S. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1): 1-122, 2010.
7. Deng, W. and Yin, W. On the global and linear convergence of the generalized alternating direction method of multipliers. Technical Report TR12-14, Rice University, 2012.
8. Duchi, J. and Singer, Y. Efficient online and batch learning using forward backward splitting. Journal of Machine Learning Research, 10: 2873-2908, 2009.
9. Gabay, D. and Mercier, B. A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Computers & Mathematics with Applications, 2(l): 17-40, 1976.
10. Glowinski, R. and Marrocco, A. Sur 1'approximation, par elements finis d'ordre un, et la resolution, par penalisation-dualite, d'une classe de problems de dirich-let non lineares. Revue Francaise d'Automatique, Infor-matique, et Recherche Operationelle, 9: 41-76, 1975.
11. He, B. S. and Yuan, X. M. On the 0(l/t) convergence rate of alternating direction method. SIAM Journal on Numerical Analysis, 22(4): 1431-1448, 2012.
12. Hong, M. and Luo, Z.-Q. On the linear convergence of the alternating direction method of multipliers. Technical Report arXiv: 1208.3922, University of Minnesota, 2012.
13. Kim, S., Sohn, K.-A., and Xing, E.P. A multivariate regression approach to association analysis of a quantitative trait network. Bioinformatics, 25(12): i204-i212,.2009.
14. Le Roux, N., Schmidt, M., and Bach, F. A stochastic gradient method with an exponential convergence rate for finite training sets. In Advances in Neural Information Processing Systems 25, pp. 2672-2680, 2012.
15. Liu, J., Yuan, L., and Ye, J. An efficient algorithm for a class of fused lasso problems. In Proceedings of the 16th International Conference on Knowledge Discovery and Data Mining, pp. 323-332, Washington, DC, USA, 2010.
16. Mairal, J. Optimization with first-order surrogate function-s. In Proceedings of the 30th International Conference on Machine Learning, Atlanta, GA, USA, 2013.
17. Ouyang, H., He, N., Tran, L., and Gray, A. Stochastic alternating direction method of multipliers. In Proceedings of the 30th International Conference on Machine Learning, Atlanta, GA, USA, 2013.
18. Qin, Z. and Goldfarb, D. Structured sparsity via alternating direction methods. Journal of Machine Learning Research, 13: 1435-1468, 2012.
19. Shalev-Shwartz, S. and Zhang, T. Proximal stochastic dual coordinate ascent. arXiv.T211.2717 http://arXiv.T211.2717, 2013a.
20. Shalev-Shwartz, S. and Zhang, T. Stochastic dual coordinate ascent methods for regularized loss minimization. Journal of Machine Learning Research, 14: 437-469, 2013b.
21. Suzuki, T. Dual averaging and proximal gradient descen-T for online alternating direction multiplier method. In Proceedings of the 30th International Conference on Machine Learning, pp. 392-100, Atlanta, GA, USA, 2013.
22. Tibshirani, R. J. and Taylor, J. The solution path of the generalized lasso. Annals of Statistics, 39(3): 1335-1371, 2011.
23. Wang, H. and Banerjee, A. Online alternating direction method. In Proceedings of the 29th International Conference on Machine Learning, pp. 1119-1126, Edinburgh, Scotland, UK, 2012.
24. Xiao, L. Dual averaging methods for regularized stochastic learning and online optimization. Journal of Machine Learning Research, 11: 2543-2596, 2010.
25. Zhang, X., Burger, M., and Osher, S. A unified primal-dual algorithm framework based on Bregman iteration. Journal of Scientific Computing, 46(l): 20-46, January 2011.