Feasibility and finite convergence analysis for accurate on-line ν-support vector machine

IEEE Transactions on Neural Networks and Learning Systems

Volumn 24, Issue 8, 2013, Pages 1304-1315

  Gu, Bin ^a   Sheng, Victor S ^b  

^a NANJING UNIVERSITY OF INFORMATION SCIENCE AND TECHNOLOGY   (China)

^b UNIVERSITY OF CENTRAL ARKANSAS   (United States)

Author keywords

Active set method; feasibility analysis; finite convergence analysis; incremental support vector classification; online learning

Indexed keywords

ACTIVE SET METHODS; FEASIBILITY ANALYSIS; FINITE CONVERGENCE; ONLINE LEARNING; SUPPORT VECTOR CLASSIFICATION;

MATRIX ALGEBRA;

SUPPORT VECTOR MACHINES;

EID: 84880921984     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2013.2250300     Document Type: Article

References (32)

1. B. Schölkopf, A. J. Smola, R. C. Williamson, and P. L. Bartlett, "New support vector algorithms," Neural Comput., vol. 12, no. 5, pp. 1207-1245, May 2000.
2. P.-H. Chen, C.-J. Lin, and B. Schölkopf, "A tutorial on ?-support vector machines," Appl. Stochastic Model. Bus. Ind., vol. 21, no. 2, pp. 111-136, Feb. 2005.
3. V. Vapnik, Statistical Learning Theory. New York, USA: Wiley, 1998.
4. B. Schölkopf and A. J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. Cambridge, MA, USA: MIT Press, 2001.
5. W. Karush, "Minima of functions of several variables with inequalities as side constraints," M.S. thesis, Dept. Math., Univ. Chicago, Chicago, IL, USA, 1939.
6. G. Cauwenberghs and T. Poggio, "Incremental and decremental support vector machine learning," in Advances in Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2001, pp. 409-415.
7. B. Gu, J.-D. Wang, Y.-C. Yu, G.-S. Zheng, Y.-F. Huang, and T. Xu, "Accurate on-line ?-support vector learning," Neural Netw., vol. 27, no. 3, pp. 51-59, Mar. 2012.
8. C.-C. Chang and C.-J. Lin, "Training ?-support vector classifiers: Theory and algorithms," Neural Comput., vol. 13, no. 9, pp. 2119-2147, Sep. 2001. (Pubitemid 33595045)
9. K. Crammer, O. Dekel, J. Keshet, S. Shalev-Shwartz, and Y. Singer, "Online passive-aggressive algorithms," J. Mach. Learn. Res., vol. 7, no. 3, pp. 551-585, Mar. 2006. (Pubitemid 43668115)
10. C. P. Diehl and G. Cauwenberghs, "Svm incremental learning, adaptation and optimization," in Proc. Int. Joint Conf. Neural Netw., vol. 4. Jul. 2003, pp. 2685-2690.
11. T.-T. Frieb, N. Cristianini, and C. Campbell, "The kernel-adatron algorithm: A fast and simple learning procedure for support vector machines," in Proc. 15th Int. Conf. Mach. Learn., 1998, pp. 188-196.
12. A. S. Householder, The Theory of Matrices in Numerical Analysis. New York, USA: Dover, 1974.
13. A. Gretton and F. Desobry, "On-line one-class support vector machines. An application to signal segmentation," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., vol. 2. Apr. 2003, pp. 709-712.
14. B. Gu, J. D. Wang, and H. Y. Chen, "On-line off-line ranking support vector machine and analysis," in Proc. Int. Joint Conf. Neural Netw., 2008, pp. 1365-1370.
15. P. Laskov, C. Gehl, S. Krüger, K.-R. Müller, K. Bennett, and E. Parrado-Hern, "Incremental support vector learning: Analysis, implementation and applications," J. Mach. Learn. Res., vol. 7, no. 12, pp. 1909-1936, Dec. 2006. (Pubitemid 44477117)
16. O. L. Mangasarian and D. R. Musicant, "Successive overrelaxation for support vector machines," IEEE Trans. Neural Netw., vol. 10, no. 5, pp. 1032-1037, Sep. 1998.
17. M. Martin, "On-line support vector machine regression," in Proc. 13th Eur. Conf. Mach. Learn., 2002, pp. 282-294.
18. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
19. N. A. Syed, H. Liu, and K. K. Sung, "Incremental learning with support vector machines," in Proc. Int. Joint Conf. Artif. Intell., Aug. 1999, pp. 317-321.
20. M. Karasuyama and I. Takeuchi, "Multiple incremental decremental learning of support vector machines," IEEE Trans. Neural Netw., vol. 21, no. 7, pp. 1048-1059, Jul. 2010.
21. C.-J. Lin, "On the convergence of the decomposition method for support vector machines," IEEE Trans. Neural Netw., vol. 12, no. 6, pp. 1288-1298, Jun. 2001. (Pubitemid 33106034)
22. T. Hastie, S. Rosset, R. Tibshirani, J. Zhu, and N. Cristianini, "The entire regularization path for the support vector machine," J.Mach. Learn. Res., vol. 5, no. 10, pp. 1391-1415, Oct. 2004.
23. E. Allgower, and K. Georg, "Continuation and path following," Acta Numer., vol. 2, pp. 1-64, Jan. 1992.
24. B. Gu, J.-D. Wang, G.-S. Zheng, and Y.-C. Yu, "Regularization path for ?-support vector classification," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 5, pp. 800-811, May 2012.
25. Z.-L. Wu and C.-H. Li, "Trace solution paths for SVMs via parametric quadratic programming," in Proc. KDD Workshop Data Mining Using Matrices Tensors, 2008, pp. 1-10.
26. M. J. Best, "An algorithm for the solution of the parametric quadratic programming problem," in Applied Mathematics and Parallel Computing: Festschrift for Klaus Ritter, H. Fischer, B. Riedemller, and S. Schffler, Eds. New York, USA: Springer-Verlag, 1996, pp. 57-76.
27. K. Ritter, "On parametric linear and quadratic programming problems," in Proc. Int Congr. Math. Program. Math. Program., 1984, pp. 307-335.
28. M. J. Best, "Equivalence of some quadratic programming algorithms," Math. Program., vol. 30, no. 1, pp. 71-87, Sep. 1984.
29. E. Wong, "Active-set methods for quadratic programming," Ph.D. dissertation, Dept. Math., Univ. California, San Diego, CA, USA, 2011.
30. J. Lopez and J. R. Dorronsoro, "Simple proof of convergence of the SMO algorithm for different SVM variants," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 7, pp. 1142-1147, Jul. 2012.
31. B. Gärtner, J. Giesen, and M. Jaggi, "An exponential lower bound on the complexity of regularization paths," JoCG, vol. 3, no. 1, pp. 168-195, 2012.
32. D. Geebelen, J. A. K. Suykens, and J. Vandewalle, "Reducing the number of support vectors of svm classifiers using the smoothed separable case approximation," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 4, pp. 682-688, Apr. 2012.