Scalable training of sparse linear SVMs

Proceedings - IEEE International Conference on Data Mining, ICDM

Volumn , Issue , 2012, Pages 775-784

  Yuan, Guo Xun ^a   Ma, Kwan Liu ^a  

^a University of California   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALTERNATING DIRECTION METHODS; BENCHMARK DATA; DISTRIBUTED SYSTEMS; LARGE DATA; LARGE PROBLEMS; LARGE-SCALE DATUM; LEARNING PROCEDURES; LINEAR SUPPORT VECTOR MACHINES; SOFT-THRESHOLDING; SPARSE MODELS; STATE-OF-THE-ART APPROACH; TRAINING ALGORITHMS; TRAINING PROCESS; VARIABLE SELECTION;

DATA MINING;

SUPPORT VECTOR MACHINES;

EID: 84874082603     PISSN: 15504786     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDM.2012.157     Document Type: Conference Paper

References (29)

1. J. H. Friedman, T. Hastie, and R. Tibshirani, "Regularization paths for generalized linear models via coordinate descent," Journal of Statistical Software, vol. 33, no. 1, pp. 1-22, 2010.
2. S. J. Wright, R. D. Nowak, and M. A. Figueiredo, "Sparse reconstruction by separable approximation," IEEE TSP, vol. 57, pp. 2479-2493, 2009.
3. A. Beck and M. Teboulle, "A fast iterative shrinkagethresholding algorithm for linear inverse problems," SIAM Journal on Imaging Sciences, vol. 2, no. 1, pp. 183-202, 2009.
4. K. Koh, S.-J. Kim, and S. Boyd, "An interior-point method for large-scale l1-regularized logistic regression," JMLR, vol. 8, pp. 1519-1555, 2007.
5. J. Liu, J. Chen, and J. Ye, "Large-scale sparse logistic regression," in ACM SIGKDD, 2009.
6. G.-X. Yuan, C.-H. Ho, and C.-J. Lin, "An improved GLMNET for l1-regularized logistic regression," JMLR, vol. 13, pp. 1999-2030, 2012.
7. G.-X. Yuan, C.-H. Ho, and C.-J. Lin, "Recent advances of large-scale linear classification," Proc. of the IEEE, 2012.
8. K.-W. Chang and D. Roth, "Selective block minimization for faster convergence of limited memory large-scale linear models," in ACM SIGKDD, 2011.
9. J. Langford, L. Li, and T. Zhang, "Sparse online learning via truncated gradient," JMLR, vol. 10, pp. 771-801, 2009.
10. J. Duchi, E. Hazan, and Y. Singer, "Adaptive subgradient methods for online learning and stochastic optimization," JMLR, vol. 12, pp. 2121-2159, 2010.
11. L. Xiao, "Dual averaging methods for regularized stochastic learning and online optimization," JMLR, vol. 11, pp. 2543-2596, 2010.
12. P. A. Forero, A. Cano, and G. B. Giannakis, "Consensusbased distributed support vector machines," JMLR, vol. 11, pp. 1663-1707, 2010.
13. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, "Distributed optimization and statistical learning via the alternating direction method of multipliers," Foundations and Trends in Machine Learning, vol. 3, no. 1, pp. 1-122, 2011.
14. H.-F. Yu, C.-J. Hsieh, K.-W. Chang, and C.-J. Lin, "Large linear classification when data cannot fit in memory," ACM TKDD, vol. 5, no. 4, pp. 23:1-23:23, 2012.
15. R. Tomioka, T. Suzuki, and M. Sugiyama, "Super-linear convergence of dual augmented Lagrangian algorithm for sparse learning," JMLR, vol. 12, pp. 1537-1586, 2011.
16. Y. Wang, G. Zhou, L. Caccetta, and W. Liu, "An alternative Lagrange-dual based algorithm for sparse signal reconstruction," IEEE TSP, vol. 59, pp. 1895-1901, April 2011.
17. P. Gong and C. Zhang, "A fast dual projected newton method for l1-regularized least squares," in IJCAI, 2011.
18. C.-J. Hsieh, K.-W. Chang, C.-J. Lin, S. S. Keerthi, and S. Sundararajan, "A dual coordinate descent method for largescale linear SVM," in ICML, 2008.
19. S. S. Keerthi, S. Sundararajan, K.-W. Chang, C.-J. Hsieh, and C.-J. Lin, "A sequential dual method for large scale multiclass linear SVMs," in ACM SIGKDD, 2008.
20. Z.-Q. Luo and P. Tseng, "On the convergence of coordinate descent method for convex differentiable minimization," JOTA, vol. 72, no. 1, pp. 7-35, 1992.
21. R. Tibshirani, "Regression shrinkage and selection via the lasso," Journal of the Royal Statistical Society Series B, vol. 58, pp. 267-288, 1996.
22. X. Zhang, M. Burger, X. Bresson, and S. Osher, "Bregmanized nonlocal regularization for deconvolution and sparse reconstruction," SIAM Journal on Imgine Sciences, vol. 3, pp. 253-276, 2010.
23. B. S. He and X. M. Yuan, "On nonergodic convergence rate of Douglas-Rachford alternating direction method of multipliers," tech. rep., 2011. Department of Mathematics, Hong Kong Baptist University.
24. J. Eckstein and D. Bertsekas, "On the Douglas-Rachford splitting method and the proximal point algorithm for maximal monotone operators," Mathematical Programming, vol. 55, no. 1, pp. 293-318, 1992.
25. R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin, "LIBLINEAR: A library for large linear classification," JMLR, vol. 9, pp. 1871-1874, 2008.
26. H.-F. Yu, H.-Y. Lo, H.-P. Hsieh, J.-K. Lou, T. G. McKenzie, J.-W. Chou, P.-H. Chung, C.-H. Ho, C.-F. Chang, Y.-H. Wei, et al., "Feature engineering and classifier ensemble for KDD Cup 2010," in JMLR W&CP, 2011.
27. G.-X. Yuan, K.-W. Chang, C.-J. Hsieh, and C.-J. Lin, "A comparison of optimization methods and software for largescale l1-regularized linear classification," JMLR, vol. 11, pp. 3183-3234, 2010.
28. C.-C. Chang and C.-J. Lin, LIBSVM: a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
29. E. Gabriel, G. E. Fagg, G. Bosilca, T. Angskun, J. J. Dongarra, J. M. Squyres, V. Sahay, P. Kambadur, B. Barrett, A. Lumsdaine, et al., "Open MPI: Goals, concept, and design of a next generation MPI implementation," Recent Advances in Parallel Virtual Machine and Message Passing Interface, pp. 353-337, 2004.