ε-SSVR: A smooth support vector machine for ε-insensitive regression

IEEE Transactions on Knowledge and Data Engineering

Volumn 17, Issue 5, 2005, Pages 678-685

  Lee, Yuh Jye ^a   Hsieh, Wen Feng ^a   Huang, Chien Ming ^b  

^a NATIONAL TAIWAN UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Taiwan)

^b NATIONAL CHUNG CHENG UNIVERSITY   (Taiwan)

Author keywords

insensitive loss function; smooth support vector regression; Kernel method; Newton armijo algorithm; Support vector machine

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; NONLINEAR SYSTEMS; OPTIMIZATION; PROBLEM SOLVING; VECTORS;

Ε-INSENSITIVE LOSS FUNCTION; Ε-SMOOTH SUPPORT VECTOR REGRESSION; KERNEL METHOD; NEWTON-ARMIJO ALGORITHMS; SUPPORT VECTOR MACHINE (SVM);

REGRESSION ANALYSIS;

EID: 19944407892     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2005.77     Document Type: Article

References (35)

1. D.P. Bertsekas, Nonlinear Programming. Belmont, Mass.: Athena Scientific, 1995.
2. C.J.C. Surges, "A Tutorial on Support Vector Machines for Pattern Recognition," Data Mining and Knowledge Discovery, vol. 2, no. 2, pp. 121-167, 1998.
3. C.-C. Chang and C.-J. Lin, LIBSVM: A Library for Support Vector Machines, 2001, software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
4. B. Chen and P.T. Harker, "Smooth Approximations to Nonlinear Complementarity Problems," SIAM J. Optimization, vol. 7, pp. 403-420, 1997.
5. C. Chen and O.L. Mangasarian, "Smoothing Methods for Convex Inequalities and Linear Complementarity Problems," Math. Programming, vol. 71, no. 1, pp. 51-69, 1995.
6. C. Chen and O.L. Mangasarian, "A Class of Smoothing Functions for Nonlinear and Mixed Complementarity Problems," Computational Optimization and Applications, vol. 5, no. 2, pp. 97-138, 1996.
7. X. Chen, L. Qi, and D. Sun, "Global and Superlinear Convergence of the Smoothing Newton Method and Its Application to General Box Constrained Variational Inequalities," Math. of Computation, vol. 67, pp. 519-540, 1998.
8. X. Chen and Y. Ye, "On Homotopy-Smoothing Methods for Variational Inequalities," SIAM J. Control and Optimization, vol. 37, pp. 589-616, 1999.
9. P.W. Christensen and J.-S. Pang, "Frictional Contact Algorithms Based on Semismooth Newton Methods," Reformulation: Non-smooth, Piecewise Smooth, Semismooth and Smoothing Methods, M. Fukushima and L. Qi, eds., pp. 81-116, Kluwer Academic Publishers, 1999.
10. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines. Cambridge: Cambridge Univ. Press, 2000.
11. DELVE, Data for Evaluating Learning in Valid Experiments, Comp-Activ Dataset, http://www.cs.toronto.edu/~delve/data/comp-activ/desc.html, 2005.
12. DELVE, Data for Evaluating Learning in Valid Experiments, Kin-family Dataset, http://www.cs.toronto.edu/~delve/data/kin/desc.html, 2005.
13. J.E. Dennis and R.B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Englewood Cliffs, N.J.: Prentice-Hall, 1983.
14. H. Drucker, C.J.C. Burges, L. Kaufman, A. Smola, and V. Vapnik, "Support Vector Regression Machines," Advances in Neural Information Processing Systems -9-, M.C. Mozer, M.I. Jordan, and T. Petsche, eds., pp. 155-161, Cambridge, Mass.: MIT Press, 1997.
15. M. Fukushima and L. Qi, Reformulation: Nonsmooth, Piecewise Smooth, Semismooth, and Smoothing Methods. Dordrecht, The Netherlands: Kluwer Academic Publishers, 1999.
16. G. Fung and O.L. Mangasarian, "Proximal Support Vector Machine Classifiers," Proc. KDD-2001: Knowledge Discovery and Data Mining, F. Provost and R. Srikant, eds., pp. 77-86, 2001, ftp://ftp.cs.wisc.edu/pub/dmi/ tech-reports/01-02.ps.
17. S.-Y. Huang and Y.-J. Lee, "Reduced Support Vector Machines: A Statistical Theory/" Preprint, Inst. of Statistical Science, Academia Sinica, 2004, http://www.stat.sinica.edu.tw/syhuang/.
18. light, 2002, http://svmlight.joachims.org.
19. Y.-J. Lee and O.L. Mangasarian, "RSVM: Reduced Support Vector Machines," Technical Report 00-07, Data Mining Inst., Computer Sciences Dept., Univ. of Wisconsin, Madison, Wisconsin, July 2000,
20. also Proc. First SIAM Int'l Conf. Data Mining, 2001, ftp://ftp.cs.wisc. edu/pub/dmi/tech-reports/00-07.ps.
21. Y.-J. Lee and O.L. Mangasarian, "SSVM: A Smooth Support Vector Machine," Computational Optimization and Applications, vol. 20, pp. 5-22, 2001,
22. also Data Mining Inst., Univ. of Wisconsin, Technical Report 99-03, ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/99-03.ps.
23. K.-M. Lin and C.-J. Lin, "A Study on Reduced Support Vector Machines," IEEE Trans. Neural Networks, vol. 14, no. 6, pp. 1449-1459, 2003.
24. O.L. Mangasarian, "Generalized Support Vector Machines," Advances in Large Margin Classifiers, A. Smola, P. Bartlett, B. Schölkopf, and D. Schuurmans, eds., pp. 135-146, Cambridge, Mass: MIT Press, 2000, ftp://ftp.cs.wisc.edu/math-prog/tech-reports/98-14.ps.
25. O.L. Mangasarian and D.R. Musicant, "Robust Linear and Support Vector Regression," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 9, pp. 950-955, 2000, ftp://ftp.cs.wisc. edu/pub/dmi/tech-reports/ 99-09.ps.
26. O.L. Mangasarian and D.R. Musicant, "Large Scale Kernel Regression via Linear Programming," Machine Learning, vol. 46, pp. 255-269, 2002, ftp://ftp.cs.wisc.edu/pub/dmi/tech-reports/99-02.ps.
27. MATLAB, User's Guide. Natick, Mass.: The MathWorks, Inc., 1994-2001, http://www.mathworks.com.
28. C.L. Blake and C.J. Merz UCI Repository of Machine Learning Databases, 1998, http://www.ics.uci.edu/~mlearn/MLRepository.htm.
29. D.R. Musicant and A. Feinberg, "Active Set Support Vector Regression," IEEE Trans. Neural Networks, vol. 15, no. 2, pp. 268-275, 2004.
30. M. Stone, "Cross-Validatory Choice and Assessment of Statistical Predictions," J. Royal Statistical Soc., vol. 36, pp. 111-147, 1974.
31. P. Tseng, "Analysis of a Non-Interior Continuation Method Based on Chen-Mangasarian Smoothing Functions for Complementarity Problems," Reformulation: Nonsmooth, Piecewise Smooth, Semi-smooth, and Smoothing Methods, M. Fukushima and L. Qi, eds., pp. 381-404, Dordrecht, Netherlands: Kluwer Academic Publishers, 1999.
32. V.N. Vapnik, The Nature of Statistical Learning Theory. New York: Springer, 1995.
33. R.C. Whaley, A. Petitet, and J.J. Dongarra, "Automated Empirical Optimization of Software and the ATLAS Project," Parallel Computing, vol. 27, nos. 1-2, pp. 3-35, 2001,
34. also available as Univ. of Tennessee LAPACK Working Note #147, UT-CS-00-448, www.netlib.org/lapack/lawns/lawn147.ps, 2000.
35. I.H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. San Francisco: Morgan Kaufmann, 1999.