Simple proof of convergence of the SMO algorithm for different SVM variants

IEEE Transactions on Neural Networks and Learning Systems

Volumn 23, Issue 7, 2012, Pages 1142-1147

  Lopez, Jorge ^a   Dorronsoro, José R ^a  

^a UNIVERSIDAD AUTÓNOMA DE MADRID   (Spain)

Author keywords

Classification; decomposition methods; regression; sequential minimum optimization (SMO); support vector machines

Indexed keywords

ASYMPTOTIC CONVERGENCE; DECOMPOSITION METHODS; MINIMUM NORM PROBLEMS; ONE-CLASS SVMS; REGRESSION; SEQUENTIAL MINIMUM OPTIMIZATIONS; SUPPORT VECTOR CLASSIFICATION; SUPPORT VECTOR REGRESSION (SVR);

ALGORITHMS; CLASSIFICATION (OF INFORMATION); OPTIMIZATION;

SUPPORT VECTOR MACHINES;

EID: 84875884987     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2012.2195198     Document Type: Article

References (20)

1. J. C. Platt, "Fast training of support vector machines using sequential minimal optimization," in Advances in Kernel Methods: Support Vector Learning. Cambridge, MA: MIT Press, 1999, pp. 185-208.
2. S. S. Keerthi, S. K. Shevade, C. Bhattacharyya, and K. R. K. Murthy, "Improvements to platt's SMO algorithm for SVM classifier design," Neural Comput., vol. 13, no. 3, pp. 637-649, 2001.
3. C.-C. Chang and C.-J. Lin. (2001). LIBSVM: A Library for Support Vector Machines [Online]. Available: http://www.csie.ntu.edu.tw/cjlin/libsvm
4. S. S. Keerthi and E. G. Gilbert, "Convergence of a generalized SMO algorithm for SVM classifier design," Mach. Learn., vol. 46, no. 1, pp. 351-360, 2002.
5. N. Takahashi and T. Nishi, "Rigorous proof of termination of SMO algorithm for support vector machines," IEEE Trans. Neural Netw., vol. 16, no. 3, pp. 774-776, May 2005.
6. C.-J. Lin, "On the convergence of the decomposition method for support vector machines," IEEE Trans. Neural Netw., vol. 12, no. 6, pp. 1288-1298, Nov. 2001.
7. C.-J. Lin, "Asymptotic convergence of an SMO algorithm without any assumptions," IEEE Trans. Neural Netw., vol. 13, no. 1, pp. 248-250, Jan. 2002.
8. P.-H. Chen, R.-E. Fan, and C.-J. Lin, "A study on SMO-type decomposition methods for support vector machines," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 893-908, Jul. 2006.
9. A. J. Smola and B. Schölkopf, "A tutorial on support vector regression," Statist. Comput., vol. 14, no. 3, pp. 199-222, 2003.
10. S. K. Shevade, S. S. Keerthi, C. Bhattacharyya, and K. R. K. Murthy, "Improvements to SMO algorithm for SVM regression," IEEE Trans. Neural Netw., vol. 11, no. 5, pp. 1188-1193, Sep. 2000.
11. G. W. Flake and S. Lawrence, "Efficient SVM regression training with SMO," Mach. Learn., vol. 46, nos. 1-3, pp. 271-290, 2002.
12. N. Takahashi and T. Nishi, "Global convergence of SMO algorithm for support vector regression," IEEE Trans. Neural Netw., vol. 19, no. 6, pp. 971-982, Jun. 2008.
13. B. Schölkopf, J. C. Platt, J. Shawe-Taylor, A. J. Smola, and R. C. Williamson, "Estimating the support of a high-dimensional distribution," Neural Comput., vol. 13, no. 7, pp. 1443-1471, 2001.
14. T. Joachims, "Making large-scale support vector machine learning practical," in Advances in Kernel Methods: Support Vector Learning. Cambridge, MA: MIT Press, 1999, pp. 169-184.
15. E. G. Gilbert, "An iterative procedure for computing the minimum of a quadratic form on a convex set," SIAM J. Control, vol. 4, no. 1, pp. 61-80, 1966.
16. C. J. C. Burges and D. J. Crisp, "Uniqueness theorems for kernel methods," Neurocomputing, vol. 55, nos. 1-2, pp. 187-220, 2003.
17. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge, U.K.: Cambridge Univ. Press, 2003.
18. J. López, A. Barbero, and J. R. Dorronsoro, "Clipping algorithms for solving the nearest point problem over reduced convex hulls," Pattern Recog., vol. 44, no. 3, pp. 607-614, 2011.
19. J. López and J. R. Dorronsoro, "Convergence of algorithms for solving the nearest point problem in scaled convex hulls," in Proc. Int. Joint Conf. Neural Netw., 2011, pp. 413-420.
20. N. List and H. U. Simon, "SVM-optimization and steepest-descent line search," in Proc. 21st Annu. Conf. Learn. Theory, 2009, pp. 1-11.