On-line SVM learning via an incremental primal-dual technique

Optimization Methods and Software

Volumn 28, Issue 2, 2013, Pages 256-275

  Couellan, Nicolas P ^a   Trafalis, Theodore B ^b  

^a INSTITUT DE MATHÉMATIQUES DE TOULOUSE   (France)

^b University of Oklahoma   (United States)

Author keywords

incremental gradient technique; interior point methods; machine learning; nonlinear programming; primal dual method; quadratic programming; SVM

Indexed keywords

COMPUTATIONAL RESULTS; CPU TIME; DATA CLASSIFICATION; DECOMPOSITION PROPERTY; GENERALIZATION PERFORMANCE; INCREMENTAL GRADIENT TECHNIQUE; INTERIOR-POINT METHOD; NON-LINEAR OPTIMIZATION PROBLEMS; ON-LINE TRAINING ALGORITHM; PATTERN RECOGNITION PROBLEMS; PREDICTION ACCURACY; PRIMAL-DUAL; PRIMAL-DUAL METHODS; SVM; TRAINING DATA SETS; TRAINING METHODS;

ALGORITHMS; LEARNING SYSTEMS; NONLINEAR PROGRAMMING; PATTERN RECOGNITION; QUADRATIC PROGRAMMING;

SUPPORT VECTOR MACHINES;

EID: 84870862968     PISSN: 10556788     EISSN: 10294937     Source Type: Journal    
DOI: 10.1080/10556788.2011.633705     Document Type: Article

References (31)

1. Bertsekas, D. P. 1995. Incremental least squares methods and the extended Kalman filter. In Tech. Rep, Cambridge, MA: Department of Electrical Engineering and Computer Science, MIT.
2. Bertsekas, D. P. 1996. A new class of incremental gradient methods for least squares problems. In Tech. Rep, Cambridge, MA: Department of Electrical Engineering and Computer Science, MIT.
3. Bersekas, D. P. 2010. Incremental gradient, subgradient, and proximal methods for convex optimization: A survey. In Tech. Rep, Cambridge, MA: LIDS-2848, Laboratory for Information and Decision Systems, MIT.
4. Cauwenberghs, G., and, Poggio, T. 2001. Incremental and decremental support vector machine learning. In Advances in Neural Information Processing Systems 13, Edited by: Leen, T. K., Dietterich, T. G. and Tresp, V. 409-415. Cambridge: MIT.
5. Cristianini, N., and, Shawe-Taylor, J. 2000. An Introduction to Support Vector Machine. Cambridge, UK: Cambridge University Press.
6. Couellan, N. P. 1997. Primal-dual techniques for nonlinear programming and applications to artificial neural network training. Norman, OK: School of Industrial Engineering, University of Oklahoma. Ph.D. Diss
7. Couellan, N. P., and, Trafalis, T. B. 2011. An incremental primal-dual method for nonlinear programming with special structure. In Optim. Lett. doi:10.1007/s11590-011-0393-0
8. Davidon, W. C. 1976. New least-square algorithms. J. Optim. Theory Appl, 18 (2): 187-197.
9. Dennis, J. E., and, Schnabel, R. B. 1996. Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Philadelphia, PA: Classics in Applied Mathematics, SIAM.
10. Diehl, C., and, Cauwenberghs, G. SVM Incremental Learning, Adaptation and Optimization. Proceedings of the 2003 International Joint Conference on Neural Networks. pp. 2685-2690. Portland, OR. Special Series on Incremental Learning
11. Dietterich, T. G. 1998. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Comput, 10 (7): 1895-1924.
12. El-Bakry, A. S., Tapia, R. A., Tsuchiya, T., and, Zhang, Y. 1996. On the formulation and theory of the Newton interior point method for nonlinear programming. J. Optim. Theory Appl, 89 (3): 507-541.
13. Frank, A., and, Asuncion, A. 2010. UCI Machine Learning Repository. Irvine, CA: University of California, School of Information and Computer Science. Available at http://archive.ics.uci.edu/ml
14. Gentile, C. 2001. A new approximate maximal margin classification algorithm. J. Mach. Learn. Res, 2: 213-242.
15. Hastie, T., Tibshirani, R., and, Friedman, J. 2009. The Elements of Statistical Learning-Data Mining, Inference, and Prediction, 2, Springer Series in Statistics New York: Springer.
16. Kivinen, J., Smola, A. J., and, Williamson, R. C. 2004. Online learning with kernels. IEEE Trans. Signal Process, 52 (8): 2165-2176.
17. Matlab, The Math-Works, Inc., Natick, MA, 1994-2010. Available at http://www.mathworks.com
18. Li, Y., and, Long, P. M. 2002. The relaxed online maximum margin algorithm. Mach. Learn, 46 (1): 361-387.
19. Lee, Y. J., Mangasarian, O. L., and, Wolberg, W. H. 2000. Breast cancer survival and chemotherapy: A support vector machine analysis. In Data Mining Institute Tech. Rep. 99-10, Vol. 55, 1-10. Providence, RI: American Mathematical Society. DIMACS Series in Discrete Mathematics and Theoretical Computer Science
20. Fung, G. M., and, Mangasarian, O. L. 2006. Breast tumor susceptibility to chemotherapy via support vector machines. Comput. Manage. Sci, 3: 103-112.
21. Mangasarian, O. L., and, Musicant, D. R. 1999. Successive overrelaxation for support vector machines. IEEE Trans. Neural Netw, 10: 1032-1037.
22. Nadeau, C., and, Bengio, Y. 2003. Inference for the generalization error. Mach. Learn, 52 (3): 239-281.
23. Schlkopf, B., and, Smola, A. 2002. Learning with Kernels. Cambridge, MA: MIT.
24. Sderstrom, T., and, Stoica, P. 1989. System Identification. Englewood Cliffs, NJ: Prentice-Hall International (UK).
25. Balakrishna, P., Raman, S., Santosa, B., and, Trafalis, T. B. 2008. Support vector regression for determining the minimum zone sphericity. Int. J. Adv. Manuf. Technol, 35: 916-923.
26. Gilbert, R. C., Raman, S., Trafalis, T. B., Obeidat, S. M., and, Aguirre-Cruz, J. A. 2009. Mathematical foundations for form inspection and adaptive sampling. J. Manuf. Sci. Eng, 131: 041001-1-041001-8.
27. Trafalis, T. B., Adrianto, I., and, Richman, M. B. 2007. Active learning with support vector machines for tornado prediction. In ICCS 2007, Part I, LNCS 4487, Edited by: Shi, Y., Albade, G. D., Dongara, J. and Sloot, P. M.A. 1130-1137. Berlin: Springer.
28. Trafalis, T. B., and, Couellan, N. P. 1995. An incremental nonlinear primal-dual algorithm and applications to artificial neural networks training. In Large Scale Systems: Theory and Applications, 1995: A Postprint Volume from the 7th IFAC/IFORS/IMACS Symposium, Edited by: Roberts, P. D. and Ellis, J. E. Oxford, UK: Pergamon.
29. Tseng, P. 1995. Incremental gradient(-projection) method with momentum term and adaptive stepsize rule. In Tech. Rep, Seattle, WA: Department of Mathematics, University of Washington.
30. Vapnik, V. 1998. Statistical Learning Theory. New York: Wiley.
31. Yamashita, H. 1992. A globally convergent primal-dual interior point method for constrained optimization. In Tech. Rep, Shinjuku, Tokyo, Japan: Mathematical System Institute, Inc.