An improved algorithm for the solution of the regularization path of support vector machine

IEEE Transactions on Neural Networks

Volumn 21, Issue 3, 2010, Pages 451-462

  Ong, Chong Jin ^a,b   Shao, Shiyun ^a   Yang, Jianbo ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^b SINGAPORE MIT ALLIANCE   (Singapore)

Author keywords

Numerical solutions of SVM; Parametric solution of SVM; Regularization path; Support vector machine (SVM)

Indexed keywords

ASCENDING VALUE; DATA POINTS; DATA SETS; IMPROVED ALGORITHM; INCREMENTAL UPDATES; KERNEL SPACE; LINEAR PROGRAMMING PROBLEM; LINEAR PROGRAMS; MATLAB ENVIRONMENT; NOMINAL FEATURE; NUMERICAL SOLUTION; OPTIMALITY CONDITIONS; PARAMETRIC SOLUTIONS; REAL WORLD DATA; REGULARIZATION PARAMETERS; SPEED-UPS; TRACKING PROBLEM; TRACKING PROCESS;

ALGORITHMS; IMAGE RETRIEVAL; LINEARIZATION; MATLAB; NUMERICAL ANALYSIS; VECTOR SPACES; VECTORS;

SUPPORT VECTOR MACHINES;

ALGORITHM; ANIMAL; ARTICLE; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; DATA BASE; HUMAN; INFORMATION PROCESSING; INFORMATION RETRIEVAL; STATISTICAL MODEL;

ALGORITHMS; ANIMALS; ARTIFICIAL INTELLIGENCE; AUTOMATIC DATA PROCESSING; COMPUTER SIMULATION; DATABASE MANAGEMENT SYSTEMS; HUMANS; INFORMATION STORAGE AND RETRIEVAL; MODELS, STATISTICAL;

EID: 77649273984     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2009.2039000     Document Type: Article

References (24)

1. C.-C. Chang and C.-J. Lin, LIBSVM: A Library for Support Vector Machines, 2001 [Online]. Available: http://www.csie.ntu.edu.tw/~cjlin/libsvm
2. T. Joachims, "Making large-scale SVM learning practical," in Advances in Kernel Methods: Support Vector Learning, B. Scholkopf, C. Burges, and A. Smola, Eds. Cambridge, MA: MIT Press, 1998.
3. J. C. Platt, "Fast training of support vector machines using sequential minimal optimization," in Advances in Kernel Methods: Support Vector Learning, B. Scholkopf, C. Burges, and A. Smola, Eds. Cambridge, MA: MIT Press, 1998.
4. S. Keerthi, S. Shevade, C. Bhattacharyya, and K. Murthy, "Improvements to Platt's SMO algorithm for SVM classifier design," Neural Comput., vol. 13, no. 3, pp. 637-649, 2001.
5. M. Collins, A. Globerson, T. Koo, X. Carreras, and P. L. Bartlett, "Exponentiated gradient algorithms for conditional random fields and maxmargin Markov networks," J. Mach. Learn. Res., vol. 9, pp. 1775-1822, 2008.
6. A. J. Smola, S. N. Vishwanathan, and Q. V. Le, "Bundle methods for machine learning," in Advances in Neural Information Processing Systems (NIPS). Cambridge, MA: MIT Press, 2008.
7. S. S. Keerthi and D. DeCoste, "A modified finite newton method for fast solution of large scale linear SVMs," J. Mach. Learn. Res., vol. 6, pp. 341-361, 2005.
8. C.-J. Hsieh, K.-W. Chang, C.-J. Lin, S. S. Keerthi, and S. Sundararajan, "A dual coordinate descent method for large-scale linear SVM," in Proc. 25th Int. Conf. Mach. Learn., New York, 2008, pp. 408-415.
9. T. Hastie, S. Rosset, R. Tibshirani, and J. Zhu, "The entire regularization path for the support vector machine," J. Mach. Learn. Res., vol. 5, pp. 1391-1415, Oct. 2004.
10. S. Rosset and J. Zhu, "Piecewise linear regularized solution paths," Ann. Stat., vol. 35, pp. 1012-1030, 2007.
11. L. Gunter and J. Zhu, "Efficient computation and model selection for the support vector regression," Neural Comput., vol. 19, pp. 1633-1655, 2007.
12. G. Wang, D.-Y. Yeung, and F. H. Lochovsky, "A kernel path algorithm for support vector machines," in Proc. 24th Int. Conf. Mach. Learn., New York, 2007, pp. 951-958.
13. G. Wang, D.-Y. Yeung, and F. H. Lochovsky, "A new solution path algorithm in support vector regression," IEEE Trans. Neural Netw., vol. 19, no. 10, pp. 1753-1767, Oct. 2008.
14. B. Scholkopf and A. J. Smola, Learning With Kernels. Cambridge, MA: MIT Press, 2002.
15. G. Strang, Introduction to Linear Algebra, 3rd ed. Cambridge, U.K.: Wellesley/Cambridge Univ. Press, 2003.
16. R. Fan, P. Chen, and C. Lin, "Working set selection using second order information for training support vector machines," J. Mach. Learn. Res., vol. 6, pp. 1889-1918, Nov. 2005.
17. D. Decoste and Wagstaff, "Alpha seeding for support vector machines," in Proc. 6th ACM SIGKDD Int. Conf. Knowl. Disc. Data Mining, New York, 2000, pp. 345-349.
18. M. Lee, S. Keerthi, C. Ong, and D. DeCoste, "An efficient method for computing leave-one-out error in support vector machines with Gaussian kernels," IEEE Trans. Neural Netw., vol. 15, no. 3, pp. 750-757, May 2004.
19. P. Businger and G. Golub, "Linear least squares solutions by householder transformations," Numer. Math., vol. 7, pp. 269-276, 1965.
20. H. G. Golub and C. F. V. Loan, Matrix Computations. Baltimore, MD: The John Hopkins Univ. Press, 1989.
21. L. Trefethen and D. Bau, Numerical Linear Algebra. Philadelphia, PA: SIAM, 1997.
22. A. Asuncion and D. J. Newman, UCI Machine Learning Repository, Univ. California at Irvine, Irvine, CA, 2007 [Online]. Available: http://www.ics.uci. edu/~mlearn/MLRepository.html
23. P. Gill, W. Murray, and M. Wright, Numerical Linear Algebra and Optimization, Vol. 1. Reading, MA: Addison-Wesley, 1991.
24. N. Megiddo, "Linear programming in linear time when the dimension is fixed," J. Assoc. Comput. Mech., vol. 31, pp. 114-127, 1983.