Regularized nonsmooth Newton method for multi-class support vector machines

Optimization Methods and Software

Volumn 22, Issue 1, 2007, Pages 225-236

  Zhong, Ping ^a,b   Fukushima, Masao ^b  

^a CHINA AGRICULTURAL UNIVERSITY   (China)

^b KYOTO UNIVERSITY   (Japan)

Author keywords

Box constrained variational inequality problem; D gap function; Multi class classification; Nonsmooth Newton method; Regularization methods

Indexed keywords

ALGORITHMS; BENCHMARKING; CLASSIFICATION (OF INFORMATION); CONSTRAINT THEORY; OPTIMIZATION; PROBLEM SOLVING;

BOX CONSTRAINED VARIATIONAL INEQUALITY PROBLEM; D-GAP FUNCTION; MULTI-CLASS CLASSIFICATION; NONSMOOTH NEWTON METHOD; REGULARIZATION METHODS;

LEARNING SYSTEMS;

EID: 33845321077     PISSN: 10556788     EISSN: 10294937     Source Type: Journal    
DOI: 10.1080/10556780600834745     Document Type: Conference Paper

References (25)

1. Bottou, L., Cortes, C., Denker, J.S., Drucker, H., Guyon, I., Jackel, L.D., LeCun, Y., Müller, U.A., Sackinger, E., Simard, P. and Vapnik, V., 1994, Comparison of classifier methods: a case study in handwriting digit recognition. In: IAPR (Ed.), Proceedings of the International Conference on Pattern Recognition (IEEE Computer Society Press), pp. 77-82.
2. Vapnik, V., 1998, Statistical Learning Theory (New York: Wiley).
3. Hastie, T.J. and Tibshirani, R.J., 1998, Classification by pairwise coupling. In: M.I. Jordan, M.J. Kearns and S.A. Solla (Eds), Advances in Neural Information Processing Systems, Vol. 10 (Cambridge, MA: MIT Press), pp. 507-513.
4. Kreßel, U., 1999, Pairwise classification and support vector machines. In: B. Schölkopf, C.J.C. Surges and A.J. Smola (Eds), Advances in Kernel Methods: Support Vector Learning (Cambridge, MA: MIT Press), pp. 255-268.
5. Platt, J., Cristianini, N. and Shawe-Taylor, J., 2000, Large margin DAGs for multiclass classification. In: S.A. Solla, T.K. Leen and K.-R. Müller (Eds), Advances in Neural Information Processing Systems, Vol. 12 (Cambridge, MA: MIT Press), pp. 547-553.
6. Dietterich, T.G. and Bakiri, G., 1995, Solving multi-class learning problems via error-correcting output codes. Journal of Artificial Intelligence Research, 2, 263-286.
7. Allwein, E.L., Schapire, R.E. and Singer, Y., 2001, Reducing multiclass to binary: a unifying approach for margin classifiers. Journal of Machine Learning Research, 1, 113-141.
8. Angulo, C., Parra, X. and Català, A., 2003, K-SVCR. A support vector machine for multi-class classification. Neurocomputing, 55, 57-77.
9. Zhong, P. and Fukushima, M., 2006, A new multi-class support vector algorithm. Optimization Methods and Software, 21, 359-372.
10. Weston, J. and Watkins, C., 1998, Multi-class support vector machines. CSD-TR-98-04, Royal Holloway, University of London, Egham, UK.
11. Bennett, K.P., 1999, Combining support vector and mathematical programming methods for classification. In: B. Schölkopf, C.J.C. Burges and A.J. Smola (Eds), Advances in Kernel Methods: Support Vector Learning (Cambridge, MA: MIT Press), pp. 307-326.
12. Lee, Y., Lin, Y. and Wahba, G., 2001, Multicategory support vector machines. Computing Science and Statistics, 33, 498-512.
13. Crammer, K. and Singer, Y., 2002, On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research, 2, 265-292.
14. Kanzow, C. and Fukushima, M., 1998, Solving box constrained variational inequalities by using the natural residual with D-gap function globalization. Operations Research Letters, 23, 45-51.
15. Peng, J.M., 1997, Equivalence of variational inequality problems to unconstrained optimization. Mathematical Programming, 78, 347-356.
16. Yamashita, N., Taji, K. and Fukushima, M., 1997, Unconstrained optimization reformulations of variational inequality problems. Journal of Optimization Theory and Applications, 92, 439-456.
17. Fukushima, M., 1992, Equivalent differentiable optimization problems and descent methods for asymmetric variational inequality problems. Mathematical Programming, 53, 99-110.
18. Qi, L., 1993, Convergence analysis of some algorithms for solving nonsmooth equations. Mathematics of Operations Research, 18, 227-244.
19. Facchinei, F. and Pang, J.S., 2003, Finite-Dimensional Variational Inequalities and Complementarity Problems, Vol. 1 (New York: Springer Verlag).
20. Subramanian, P.K., 1988, A note on least two norm solutions of monotone complementarity problems. Applied Mathematics Letters, 1, 395-397.
21. 0-complementarity problem. In : M. Fukushima and L. Qi (Eds), Reformulation: Nonsmooth, Piecewise Smooth, Semismooth and Smoothing Methods (Kluwer Academic Publishers), pp. 371-379.
22. Qi, H.D., 1999, Tikhonov regularization methods for variational inequality problems. Journal of Optimization Theory and Applications, 102, 193-201.
23. Blake, C.L. and Merz, C.J., 1998, UCI repository of machine learning databases. University of California.Available online at: www http://www.ics.uci.edu/~mlearn/MLRepository.html
24. Orippo, L., Lampariello, F. and Lucidi, S., 1986, A nonmonotone line search technique for Newton's method. SIAM Journal on Numerical Analysis, 23, 707-716.
25. Lee, Y-J. and Mangasarian, O.L., 2001, RSVM: reduced support vector machines. Proceedings of the First SIAM International Conference on Data Mining, Chicago. Available online at: ftp://ftp.cs.wise.edu/pub/dmi/techreports/00-07. ps