Efficient distributed linear classification algorithms via the alternating direction method of multipliers

Journal of Machine Learning Research

Volumn 22, Issue , 2012, Pages 1398-1406

  Zhang, Caoxie ^a   Lee, Honglak ^a   Shin, Kang G ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALUMINUM; ARTIFICIAL INTELLIGENCE; CLASSIFICATION (OF INFORMATION); LEARNING ALGORITHMS; NEWTON-RAPHSON METHOD; STOCHASTIC SYSTEMS;

ALTERNATING DIRECTION METHOD OF MULTIPLIERS; ALTERNATING DIRECTION METHODS; COORDINATE DESCENT; EXPERIMENTAL EVALUATION; LARGE-SCALE LEARNING; LINEAR CLASSIFICATION; OPTIMIZING ALGORITHM; STOCHASTIC GRADIENT DESCENT;

ALGORITHMS;

EID: 84899031306     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (20)

1. L. Bottou and Y. LeCun. Large scale online learning. In Advances in Neural Information Processing Systems, 2004.
2. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. 3(1):1-122, 2011.
3. E. Y. Chang, K. Zhu, H. Wang, and H. Bai. PSVM: Parallelizing support vector machines on distributed computers. In Advances in Neural Information Processing Systems, 2008.
4. J. Eckstein. Parallel alternating direction multiplier decomposition of convex programs. Journal of Optimization Theory and Applications, 80(1):39-62, 1994.
5. J. Eckstein and D. P. Bertsekas. On the douglas-rachford splitting method and the proximal point algorithm for maximal monotone operators. Mathematical Programming, 55(1):293-318, 1992.
6. R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research, 9:1871-1874, 2008.
7. P. A. Forero, A. Cano, and G. B. Giannakis. Consensus-based distributed support vector machines. Journal of Machine Learning Research, 11:1663-1707, 2010.
8. D. Gabay and B. Mercier. A dual algorithm for the solution of nonlinear variational problems via finite element approximation. Computers and Mathematics with Applications, 2(1):17-40, 1976.
9. C.-J. Hsieh, K.-W. Chang, C.-J. Lin, S. S. Keerthi, and S. Sundararajan. A dual coordinate descent method for large-scale linear SVM. In Proceedings of the 25th International Conference on Machine Learning, 2008.
10. S. S. Keerthi, S. Sundararajan, K.-W. Chang, C.-J. Hsieh, and C.-J. Lin. A sequential dual method for large scale multi-class linear svms. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2008.
11. J. Langford, L. Li, and A. Strehl. Vowpal Wabbit online learning project. Technical report, 2007.
12. C.-J. Lin, R. C. Weng, and S. S. Keerthi. Trust region Newton method for large-scale logistic regression. Journal of Machine Learning Research, 9:627-650, 2008.
13. Z. Q. Luo and P. Tseng. On the convergence of the coordinate descent method for convex differentiable minimization. Journal of Optimization Theory and Applications, 72:7-35, 1992.
14. MPI Forum. MPI: A Message-Passing Interface Standard, version 2.2, 2009.
15. R. Rifkin and A. Klautau. In defense of one-vsall classification. Journal of Machine Learning Research, 5:101-141, 2004.
16. S. Shalev-Shwartz, Y. Singer, and N. Srebro. Pegasos: Primal estimated sub-gradient solver for SVM. In Proceedings of the 24th International Conference on Machine Learning, 2007.
17. S. Shalev-Shwartz and N. Srebro. SVM optimization: inverse dependence on training set size. In Proceedings of the 25th international Conference on Machine Learning, pages 928-935, 2008.
18. Q. Shi, J. Petterson, G. Dror, J. Langford, A. Smola, and S. Vishwanathan. Hash kernels for structured data. Journal of Machine Learning Research, 10:2615-2637, 2009.
19. H.-F. Yu, C.-J. Hsieh, K.-W. Chang, and C.-J. Lin. Large linear classification when data cannot fit in memory. In Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2010.
20. M. Zinkevich, A. J. Smola, M. Weimer, and L. Li. Parallelized stochastic gradient descent. In Advances in Neural Information Processing Systems, 2010.