Coordinate descent method for large-scale L2-loss linear support vector machines

Journal of Machine Learning Research

Volumn 9, Issue , 2008, Pages 1369-1398

  Chang, Kai Wei ^a   Hsieh, Cho Jui ^a   Lin, Chih Jen ^a  

^a NATIONAL TAIWAN UNIVERSITY   (Taiwan)

Author keywords

Coordinate descent; Document classification; Linear support vector machines

Indexed keywords

ARTIFICIAL INTELLIGENCE; CLASSIFICATION (OF INFORMATION); COMPUTATIONAL LINGUISTICS; ECHO SUPPRESSION; IMAGE RETRIEVAL; INFORMATION RETRIEVAL SYSTEMS; LEARNING SYSTEMS; NATURAL LANGUAGE PROCESSING SYSTEMS; VECTORS;

COORDINATE DESCENT; DOCUMENT CLASSIFICATION; LINE SEARCHES; LINEAR RATE; LINEAR SUPPORT VECTOR MACHINES; LINEAR SVM; LOSS FUNCTIONS; NATURAL LANGUAGE PROCESSING; SPARSE DATA; STATE-OF-THE-ART METHODS;

SUPPORT VECTOR MACHINES;

EID: 48849104146     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (20)

1. Dimitri P. Bertsekas. Nonlinear Programming. Athena Scientific, Belmont, MA 02178-9998, second edition, 1999.
2. Bernhard E. Boser, Isabelle Guyon, and Vladimir Vapnik. A training algorithm for optimal margin classifiers. In Proceedings of the Fifth Annual Workshop on Computational Learning Theory, pages 144-152. ACM Press, 1992.
3. Leon Bottou. Stochastic gradient descent examples, 2007. http://leon.bottou.org/projects/sgd.
4. Leon Bottou and Olivier Bousquet. The tradeoffs of large scale learning. In J.C. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20. MIT Press, Cambridge, MA, 2008.
5. Iain S. Duff, Roger G. Grimes, and John G. Lewis. Sparse matrix test problems. ACM Transactions on Mathematical Software, 15:1-14, 1989.
6. Luigi Grippo and Marco Sciandrone. Globally convergent block-coordinate techniques for unconstrained optimization. Optimization Methods and Software, 10:587-637, 1999.
7. Cho-Jui Hsieh, Kai-Wei Chang, Chih-Jen Lin, S. Sathiya Keerthi, and Sellamanickam Sundararajan. A dual coordinate descent method for large-scale linear SVM. In Proceedings of the Twenty Fifth International Conference on Machine Learning (ICML), 2008. URL http://www.csie.ntu.edu.tw/~cjlin/papers/ cddual.pdf. Software available at http://www.csie.ntu.edu.tw/~cjlin/liblinear.
8. Thorsten Joachims. Training linear SVMs in linear time. In Proceedings of the ACM Conference on Knowledge Discovery and Data Mining (KDD). ACM, 2006.
9. S. Sathiya Keerthi and Dennis DeCoste. A modified finite Newton method for fast solution of large scale linear SVMs. Journal of Machine Learning Research, 6:341-361, 2005.
10. David D. Lewis, Yiming Yang, Tony G. Rose, and Fan Li. RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research, 5:361-397, 2004.
11. Chih-Jen Lin, Ruby C. Weng, and S. Sathiya Keerthi. Trust region Newton method for large-scale logistic regression. Journal of Machine Learning Research, 9:627-650. URL http://www.csie.ntu.edu.tw/~cjlin/papers/logistic. pdf. Software available at http://www.csie.ntu.edu.tw/~cjlin/liblinear.
12. Zhi-Quan Luo and Paul Tseng. On the convergence of coordinate descent method for convex differentiable minimization. Journal of Optimization Theory and Applications, 72(1):7-35, 1992.
13. Olvi L. Mangasarian. A finite Newton method for classification. Optimization Methods and Software, 17(5):913-929, 2002.
14. Dukík Miroslav, Steven J. Phillips, and Robert E. Schapire. Performance guarantees for regularized maximum entropy density estimation. In Proceedings of the 17th Annual Conference on Computational Learning Theory, pages 655-662, New York, 2004. ACM press.
15. Gunnar Rätsch, Sebastian Mika, and Manfred K. Warmuth. On the convergence of leveraging. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14, pages 487-494. MIT Press, Cambridge, MA, 2002.
16. Nicol N. Schraudolph. A fast, compact approximation of the exponential function. Neural Computation, 11:853-862, 1999.
17. Shai Shalev-Shwartz, Yoram Singer, and Nathan Srebro. Pegasos: primal estimated sub-gradient solver for SVM. In Proceedings of the 24th International Conference on Machine Learning (ICML), 2007.
18. Alex J. Smola, S V N Vishwanathan, and Quoc Le. Bundle methods for machine learning. In J.C. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20. MIT Press, Cambridge, MA, 2008.
19. Tong Zhang. Solving large scale linear prediction problems using stochastic gradient descent algorithms. In Proceedings of the 21th International Conference on Machine Learning (ICML), 2004.
20. Tong Zhang and Frank J. Oles. Text categorization based on regularized linear classification methods. Information Retrieval, 4(1):5-31, 2001.