A comparative study of multi-class support vector machines in the unifying framework of large margin classifiers

Applied Stochastic Models in Business and Industry

Volumn 21, Issue 2, 2005, Pages 199-214

  Guermeur, Yann ^a   Elisseeff, André ^b   Zelus, Dominique ^c  

^a LORIA   (France)

^b MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

^c CIBIO   (Argentina)

Author keywords

Extended VC dimensions; Generalization error bounds; Large margin classifiers; Multi class support vector machines (M SVMs)

Indexed keywords

DATA ACQUISITION; FUNCTIONS; MATHEMATICAL MODELS; PROBABILITY DISTRIBUTIONS; STATISTICAL METHODS; THEOREM PROVING;

EXTENDED VC DIMENSIONS; GENERALIZATION ERROR BOUNDS; LARGE MARGIN CLASSIFIERS; MULTI-CLASS SUPPORT VECTOR MACHINES (M-SVMS);

LEARNING SYSTEMS;

EID: 17444410319     PISSN: 15241904     EISSN: None     Source Type: Journal    
DOI: 10.1002/asmb.534     Document Type: Article

References (49)

1. Vapnik VN, Chervonenkis AYa. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications 1971; 16:264-280.
2. Ben-David S, Cesa-Bianchi N, Haussler D, Long PM. Characterizations of learnability for classes of {0,...,n}-valued functions. Journal of Computer and System Sciences 1995; 50:74-86.
3. Alon N, Ben-David S, Cesa-Bianchi N, Haussler D. Scale-sensitive dimensions, uniform convergence, and learnability. Journal of the ACM 1997; 44:615-631.
4. Vapnik VN. Estimation of Dependences Based on Empirical Data. Springer: New York, 1982.
5. Shawe-Taylor J, Bartlett PL, Williamson RC, Anthony M. Structural risk minimization over data-dependent hierarchies. Technical Report NC-TR-96-053, NeuroCOLT, 1996.
6. Bartlett PL. The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Transactions on Information Theory 1998; 44(2):525-536.
7. Shawe-Taylor J, Anthony M. Sample sizes for multiple-output threshold networks. Network: Computation in Neural Systems 1991; 2:107-117.
8. Anthony M. Probabilistic analysis of learning in artificial neural networks: the PAC model and its variants. Neural Computing Surveys 1997; 1:1-47.
9. Guermeur Y, Elisseeff A, Paugam-Moisy H. Estimating the sample complexity of a multi-class discriminant model. In ICANN'99. IEE: London, 1999; 310-315.
10. Elisseeff A, Guermeur Y, Paugam-Moisy H. Margin error and generalization capabilities of multi-class discriminant models. Technical Report NC-TR-99-051-R, NeuroCOLT2, 1999 (revised in 2001).
11. Guermeur Y. A simple unifying theory of multi-class support vector machines. Technical Report RR-4669, INRIA, 2002.
12. Platt JC, Cristianini N, Shawe-Taylor J. Large margin DAGs for multiclass classification. In NIPS'12, 2000; 547-553.
13. Richard MD, Lippmann RP. Neural network classifiers estimate Bayesian a posteriori probabilities. Neural Computation 1991; 3:461-483.
14. Vapnik VN. Statistical Learning Theory. Wiley: New York, 1998.
15. Pollard D. Empirical processes: theory and applications. In NFS-CBMS Regional Conference Series in Probability and Statistics, vol. 2. Institute of Math. Stat. and Am. Stat. Assoc., 1990.
16. Haussler D. Decision theoretic generalizations of the PAC model for neural net and other learning applications. Information and Computation 1992; 100:78-150.
17. Kearns MJ, Schapire RE. Efficient distribution-free learning of probabilistic concepts. In Proceedings of the 31st Annual Symposium on Foundations of Computer Science, vol. 1. IEEE Computer Society Press: Silver Spring, MD, 1990; 382-391.
18. Kearns MJ, Schapire RE. Efficient distribution-free learning of probabilistic concepts. Journal of Computer and System Sciences 1994; 48(3):464-497.
19. Gurvits L. A note on a scale-sensitive dimension of linear bounded functionals in Banach spaces. Theoretical Computer Science 2001; 261(1):81-90.
20. Dudley RM. Universal Donsker classes and metric entropy. The Annals of Probability 1987; 15(4): 1306-1326.
21. Natarajan BK. On learning sets and functions. Machine Learning 1989; 4:67-97.
22. Boser B, Guyon I, Vapnik V. A training algorithm for optimal margin classifiers. In COLT'92, 1992; 144-152.
23. Cortes C, Vapnik VN. Support-vector networks. Machine Learning 1995; 20:273-297.
24. Schölkopf B, Burges C, Vapnik V. Extracting support data for a given task. In ICKDDM'95, 1995; 252-257.
25. Mayoraz E, Alpaydin E. Support vector machines for multi-class classification. Technical Report 98-06, IDIAP, 1998.
26. Saitoh S. Theory of Reproducing Kernels and its Applications. Longman Scientific & Technical: Harlow, England, 1988.
27. Wahba G. Spline models for observational data. In SIAM, vol. 59 of CBMS-NSF Regional Conference Series in Applied Mathematics, 1990.
28. Wahba G. Support vector machines, reproducing Kernel Hubert spaces, and randomized GACV. In Advances in Kernel Methods, Support Vector Learning, Schölkopf B, Burges CJC, Smola AJ (eds). MIT Press: Cambridge, MA, 1999; 69-88.
29. Aizerman M, Braverman E, Rozonoer L. Theoretical foundations of the potential function method in pattern recognition learning. Automation and Remote Control 1964; 25:821-837.
30. Weston J, Watkins C. Multi-class support vector machines. Technical Report CSD-TR-98-04, Royal Holloway, Department of Computer Science, University of London, 1998.
31. Bredensteiner EJ, Bennett KP. Multicategory classification by support vector machines. Computational Optimization and Applications 1999; 12(1/3):53-79.
32. Guermeur Y, Elisseeff A, Paugam-Moisy H. A new multi-class SVM based on a uniform convergence result. In IJCNN'00, vol. IV, 2000; 183-188.
33. Hsu C-W, Lin C-J. A comparison of methods for multi-class support vector machines. IEEE Transactions on Neural Networks 2002; 13:415-425.
34. Crammer K, Singer Y. On the algorithmic implementation of multidass kernel-based vector machines. Journal of Machine Learning Research 2001; 2:265-292.
35. Crammer K, Singer Y. On the learnability and design of output codes for multidass problems. In Proceedings of the Thirteen Annual Conference on Computational Learning Theory (COLT), 2000; 35-46.
36. Lee Y, Lin Y, Wahba G. Multicategory support vector machines. Technical Report 1043, Department of Statistics, University of Wisconsin, Madison, 2001.
37. Lee Y. Multicategory support vector machines, theory, and application to the classification of microarray data and satellite radiance data. Technical Report 1063, Department of Statistics, University of Wisconsin, Madison, 2002.
38. Guermeur Y, Zelus D. Combining protein secondary structure prediction models with ensemble methods of optimal complexity. In JOBIM'01, 2001; 97-104.
39. Guermeur Y. Combining discriminant models with new multi-class SVMs. Pattern Analysis and Applications 2002; 5(2): 168-179.
40. Fletcher R. Practical Methods of Optimization. Wiley: New York, 1987.
41. Bartlett PL, Shawe-Taylor J. Generalization performance of support vector machines and other pattern classifiers. In Advances in Kernel Methods, Support Vector Learning, Schölkopf B, Burges CJC, Smola A (eds). MIT Press: Cambridge, MA, 1999; 43-54.
42. Williamson RC, Smola AJ, Schölkopf B. Generalization performance of regularization networks and support vector machines via entropy numbers of compact operators. IEEE Transactions on Information Theory 2001; 47(6): 2516-2532.
43. Smola AJ. Learning with kernels. Ph.D. Thesis, Technische Universität Berlin, 1998.
44. Williamson RC, Smola AJ, Schölkopf B. Entropy numbers of linear function classes. In COLT'00, 2000; 309-319.
45. Schölkopf B, Smola AJ. Learning with Kernels, Support Vector Machines, Regularization, Optimization and Beyond. MIT Press: Cambridge, MA, 2002.
46. Carl B, Stephani I. Entropy, Compactness, and the Approximation of Operators. Cambridge University Press: Cambridge, UK, 1990.
47. Boucheron S, Lugosi G, Massart P. A sharp concentration inequality with applications. Technical Report NC2-TR-1999-057, NeuroCOLT2, 1999.
48. Bousquet O. Concentration inequalities and empirical processes theory applied to the analysis of learning algorithms. Ph.D. Thesis, Ecole Polytechnique, 2002.
49. Bartlett PL, Boucheron S, Lugosi G. Model selection and error estimation. Machine Learning 2002; 48:85-113.