Support vector machine soft margin classifiers: Error analysis

Journal of Machine Learning Research

Volumn 5, Issue , 2004, Pages 1143-1175

  Chen, Di Rong ^a   Wu, Qiang ^b   Ying, Yiming ^b   Zhou, Ding Xuan ^b  

^a BEIHANG UNIVERSITY   (China)

^b CITY UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Approximation error; Misclassification error; q norm soft margin classifier; Regularization error; Support vector machine classification

Indexed keywords

ERROR ANALYSIS;

APPROXIMATION ERRORS; MISCLASSIFICATION ERROR; REGULARIZATION ERRORS; SOFT MARGINS; SUPPORT VECTOR MACHINE CLASSIFICATION;

SUPPORT VECTOR MACHINES;

EID: 84879394399     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (36)

1. N. Aronszajn. Theory of reproducing kernels. Trans. Amer. Math. Soc. 68 (1950), 337-404, 1950.
2. A. R. Barron. Complexity regularization with applications to artificial neural networks. G. Roussa, editor, in Nonparametric Functional Estimation, Kluwer, Dortrecht, pages 561-576, 1990.
3. P. L. Bartlett. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Transactions on Information Theory, 44: 525-536, 1998.
4. P. L. Bartlett, O. Bousquet, and S. Mendelson. Local Rademacher complexities. Annals Statis., 2004, to appear. P. L. Bartlett, M. I. Jordan, and J. D.McAuliffe. Convexity, classification, and risk bounds. Preprint, 2003.
5. B. E. Boser, I. Guyon, and V. Vapnik. A training algorithm for optimal margin classifiers. In Proceedings of the Fifth Annual Workshop of Computational Learning Theory, 5, Pittsburgh, ACM, pages 144-152, 1992.
6. O. Bousquet and A. Elisseeff. Stability and generalization. Journal of Machine Learning Research, 2: 499-526, 2002.
7. C. Cortes and V. Vapnik. Support-vector networks. Machine Learning, 20: 273-297, 1995.
8. N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines. Cambridge University Press, 2000.
9. F. Cucker and S. Smale. On the mathematical foundations of learning. Bull. Amer. Math. Soc., 39: 1-49, 2001.
10. F. Cucker and D. X. Zhou. Learning Theory: An Approximation Theory Viewpoint. Monograph manuscript in preparation for Cambridge University Press, 2004.
11. L. Devroye, L. Györfi, and G. Lugosi. A Probabilistic Theory of Pattern Recognition. Springer-Verlag, New York, 1997.
12. T. Evgeniou, M. Pontil, and T. Poggio. Regularization networks and support vector machines. Adv. Comput. Math., 13: 1-50, 2000.
13. W. S. Lee, P. Bartlett, and R.Williamson. The importance of convexity in learning with least square loss. IEEE Transactions on Information Theory, 44: 1974-1980, 1998.
14. Y. Lin. Support vector machines and the Bayes rule in classification. Data Mining and Knowledge Discovery 6: 259-275, 2002.
15. S. Mendelson. Improving the sample complexity using global data. IEEE Transactions on Information Theory 48: 1977-1991, 2002.
16. S. Mukherjee, R. Rifkin, and T. Poggio. Regression and classification with regularization. In D. D. Denison, M. H. Hansen, C. C. Holmes, B. Mallick, and B. Yu, eds., Lecture Notes in Statistics: Nonlinear Estimation and Classification, Springer-Verlag, New York, pages 107-124, 2002.
17. P. Niyogi and F. Girosi. On the relationship between generalization error, hypothesis complexity, and sample complexity for radial basis functions. Neural Comp., 8: 819-842, 1996.
18. M. Pontil. A note on different covering numbers in learning theory. J. Complexity 19: 665-671, 2003.
19. F. Rosenblatt. Principles of Neurodynamics. Spartan Book, New York, 1962.
20. J. Shawe-Taylor, P. L. Bartlett, R. C. Williamson, and M. Anthony. Structural risk minimization over data-dependent hierarchies. IEEE Transations on Information Theory 44: 1926-1940, 1998.
21. S. Smale and D. X. Zhou. Estimating the approximation error in learning theory. Anal. Appl., 1: 17-41, 2003.
22. S. Smale and D. X. Zhou. Shannon sampling and function reconstruction from point values. Bull. Amer. Math. Soc. , 41: 279-305, 2004.
23. I. Steinwart. On the influence of the kernel on the consistency of support vector machines. Journal of Machine Learning Research, 2: 67-93, 2001.
24. I. Steinwart. Support vector machines are universally consistent. J. Complexity, 18: 768-791, 2002.
25. A. Tikhonov and V. Arsenin. Solutions of Ill-posed Problems. W. H. Winston, 1977.
26. A. W. van der Vaart and J. A. Wellner. Weak Convergence and Empirical Processes. Springer-Verlag, New York, 1996.
27. V. Vapnik. Estimation of Dependences Based on Empirical Data. Springer-Verlag, New York, 1982.
28. V. Vapnik. Statistical Learning Theory. John Wiley and Sons, 1998.
29. G. Wahba. Spline models for observational data. SIAM, 1990.
30. G.Wahba. Support vector machines, reproducing kernel Hilbert spaces and the Randomized GACV. In Schölkopf, Burges and Smola, eds., Advances in Kernel Methods-Support Vector Learning, MIT Press, pages 69-88, 1999.
31. R. C. Williamson, A. J. Smola, and B. Schölkopf. Generalization performance of regularization networks and support vector machines via entropy numbers of compact operators. IEEE Transactions on Information Theory 47: 2516-2532, 2001.
32. Q. Wu and D. X. Zhou. Analysis of support vector machine classification. Preprint, 2004.
33. T. Zhang. Statistical behavior and consistency of classification methods based on convex risk minimization. Annals Statis., 32: 56-85, 2004.
34. D. X. Zhou. The covering number in learning theory. J. Complexity, 18: 739-767, 2002.
35. D. X. Zhou. Capacity of reproducing kernel spaces in learning theory. IEEE Transactions on Information Theory, 49: 1743-1752, 2003a.
36. D. X. Zhou. Density problem and approximation error in learning theory. Preprint, 2003b.