Global convergence of SMO algorithm for support vector regression

IEEE Transactions on Neural Networks

Volumn 19, Issue 6, 2008, Pages 971-982

  Takahashi, Norikazu ^a   Guo, Jun ^a   Nishi, Tetsuo ^b  

^a KYUSHU UNIVERSITY   (Japan)

^b WASEDA UNIVERSITY   (Japan)

Author keywords

Convergence; Quadratic programming (QP); Sequential minimal optimization (SMO); Support vector regression (SVR)

Indexed keywords

CONVERGENCE; CONVEX QUADRATIC PROGRAMMING; EFFICIENT IMPLEMENTATIONS; FINITE NUMBERS; GLOBAL CONVERGENCES; OPTIMAL SOLUTIONS; OPTIMALITY CONDITIONS; SEQUENTIAL MINIMAL OPTIMIZATION (SMO); SEQUENTIAL MINIMAL OPTIMIZATION ALGORITHMS; SMO ALGORITHMS; SUB-PROBLEMS; SUPPORT VECTOR REGRESSION (SVR); TRAINING SAMPLES;

ALGORITHMS; OPTIMIZATION; QUADRATIC PROGRAMMING; REGRESSION ANALYSIS; SEQUENTIAL SWITCHING; VECTORS;

CONVERGENCE OF NUMERICAL METHODS;

ALGORITHM; ANIMAL; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; HUMAN; SIGNAL PROCESSING;

ALGORITHMS; ANIMALS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; HUMANS; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; SIGNAL PROCESSING, COMPUTER-ASSISTED;

EID: 49149114060     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2007.915116     Document Type: Article

References (26)

1. V. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
2. E. Osuna, R. Freund, and F. Girosi, "An improved training algorithm for support vector machines," in Proc. IEEE Workshop Neural Netw. Signal Process., 1997, pp. 511-519.
3. J. C. Platt, "Fast training of support vector machines using sequential minimal optimization," in Advances in Kernel Methods: Support Vector Machines, B. Schölkopf, C. Burges, and A. Smola, Eds. Cambridge, MA: MIT Press, 1998.
4. T. Joachims, "Making large-scale support vector machine learning practical," in Advances in Kernel Methods: Support Vector Machines B. Schölkopf, C. Burges, and A. Smola, Eds. Cambridge, MA: MIT Press, 1998.
5. S. S. Keerthi, S. K. Shevade, C. Bhattacharyya, and K. R. K. Murthy, "Improvements to Platt's SMO algorithm for SVM classifier design," Neural Comput., vol. 13, pp. 637-649, 2001.
6. C.-W. Hsu and C.-J. Lin, "A simple decomposition method for support vector machines," Mach. Learn., vol. 46, pp. 291-314, 2002.
7. C.-C. Chang, C.-W. Hsu, and C.-J. Lin, "The analysis of decomposition methods for support vector machines," IEEE Trans. Neural Netw., vol. 11, no. 4, pp. 1003-1008, Jul. 2000.
8. C.-J. Lin, "On the convergence of the decomposition method for support vector machines," IEEE Trans. Neural Netw., vol. 12, no. 6, pp. 1288-1298, Nov. 2001.
9. C.-J. Lin, "A formal analysis of stopping criteria of decomposition methods for support vector machines," IEEE Trans. Neural Netw., vol. 13, no. 5, pp. 1045-1052, Sep. 2002.
10. N. List and H. U. Simon, "A general convergence theorem for the decomposition method," in Proc. 17th Annu. Conf. Learn. Theory, 2004, pp. 363-377.
11. N. Takahashi and T. Nishi, "Global convergence of decomposition learning methods for support vector machines," IEEE Trans. Neural Netw., vol. 17, no. 6, pp. 1362-1369, Nov. 2006.
12. C.-J. Lin, "Asymptotic convergence of an SMO algorithm without any assumption," IEEE Trans. Neural Netw., vol. 13, no. 1, pp. 248-250, Jan. 2002.
13. S. S. Keerthi and E. G. Gilbert, "Convergence of a generalized SMO algorithm for SVM classifier design," Mach. Learn., vol. 46, pp. 351-360, 2002.
14. N. Takahashi and T. Nishi, "Rigorous proof of termination of SMO algorithm for support vector machines," IEEE Trans. Neural Netw., vol. 16, no. 3, pp. 774-776, May 2005.
15. P.-H. Chen, R.-E. Fan, and C.-J. Lin, "A study on SMO-type decomposition methods for support vector machines," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 893-908, Jul. 2006.
16. C.-C. Chang and C.-J. Lin, "LIBSVM: A library for support vector machines," 2001 [Online]. Available: http://www.csie.ntu.edu.tw/ cjlin/libsvm/
17. S.-P. Liao, H.-T. Lin, and C.-J. Lin, "A note on the decomposition methods for support vector regression," Neural Comput., vol. 14, pp. 1267-1281, 2002.
18. A. J. Smola and B. Schölkopf, "A tutorial on support vector regression," Royal Holloway College, London, U.K., NeuroCOLT Tech. Rep. NC-TR-98-030, 1998.
19. S. K. Shevade, S. S. Keerthi, C. Bhattacharyya, and K. R. K. Murthy, "Improvements to the SMO algorithm for SVM regression," IEEE Trans. Neural Netw., vol. 11, no. 5, pp. 1188-1193, Sep. 2000.
20. G. W. Flake and S. Lawrence, "Efficient SVM regression training with SMO," Mach. Learn., vol. 46, pp. 271-290, 2002.
21. J. Guo, N. Takahashi, and T. Nishi, "A novel sequential minimal optimization algorithm for support vector regression," in Lecture Notes in Computer Science, Oct. 2006, vol. 4232, pp. 827-836.
22. R.-E. Fan, P.-H. Chen, and C.-J. Lin, "Working set selection using second order information for training support vector machines," J. Mach. Learn. Res., vol. 6, pp. 1889-1918, 2005.
23. D. Hush, P. Kelly, C. Scovel, and I. Steinwart, "QP algorithms with guaranteed accuracy and run time for support vector machines," J. Mach. Learn. Res., vol. 7, pp. 733-769, 2006.
24. T. Glasmachers and C. Igel, "Maximum-gain working set selection for SVMs," J. Mach. Learn. Res., vol. 7, pp. 1437-1466, 2006.
25. A. Asuncion and D. Newman, "UCI machine learning repository," 2007 [Online]. Available: http://www.ics.uci.edu/~ mlearn/MLRepository.html
26. "StatLib-datasets archive," [Online]. Available: http:// lib.stat.cmu.edu/datasets/