Generalization error for multi-class margin classification

Electronic Journal of Statistics

Volumn 1, Issue , 2007, Pages 307-330

  Shen, Xiaotong ^a   Wang, Lifeng ^b  

^a UNIVERSITY OF MINNESOTA   (United States)

^b UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

Convex and nonconvex losses; Import vector machines; Small n and large p; Sparse learning; Support vector machines; learning

Indexed keywords

EID: 59349112940     PISSN: 19357524     EISSN: None     Source Type: Journal    
DOI: 10.1214/07-EJS069     Document Type: Article

References (33)

1. Alexander, K. (1984) Probability inequalities for empirical processes and a law of the iterated logarithm The Annals of Probability, 12, 1041-1067.
2. Bartlett, P., and Shawe-Taylor, J. (1999). Generalization performance of support vector machines and other pattern classifiers. In B. Scholkopf, et. al (ed.) Advances in Kernel Methods -Support Vector Learning, 43-54, MIT Press.
3. Bartlett, P. L., Jordan, M. I. and McAuliffe, J. D. (2006) Convexity, classification, and risk bounds. J. Amer. Statist. Assoc., 101, 138-156.
4. Bradley, P. S. and Mangasarian, O. L. (1998) Feature selection via concave minimization and support vector machines. in J. Shaclik (ed.), Machine Learning Proc the Fifteenth International Conf., Morgan Kaufmann, San Francisco, Ca, 82-90.
5. Bredensteiner, E. J. and Bennett, K. P. (1999) Multicategory classification by support vector machines. Comput. Optim. and Appli., 12, 35-46.
6. Corduneanu, A. and Jaakkola, T. (2003) On information regularization. Proceedings of the Ninth Annual Conference on Uncertainty in Artificial Intelligence.
7. Cortes, C., and Vapnik, V. (1995). Support-vector networks. Machine Learning. 20, 73-297.
8. Crammer, K., and Singer, Y. (2001). On the algorithmic implementation of multi-class kernel-based vector machines. J. Machine Learning Res., 2, 265-292.
9. Cucker, F. and Smale, S. (2002) On the Mathematical foundations of learning. Bulletin Amer. Math. Soc., 39, 1-49.
10. Gu, C. (2000). Multidimension smoothing with splines. Smoothing and Regression: Approaches, Computation and Application, edited by M.G. Schimek.
11. Guermeur, Y. (2002). Combining discriminant models with new multi-class SVMs. Pattern Analy. Appli., 5, 168-179.
12. Lafferty, J. and Wasserman, L. (2006) Challenges in Statistical Machine Learning. Statistica Sinica, 16, 307-323.
13. Lee, Y., Lin, Y. and Wahba, G. (2004) Multicategory support vector machines, theory, and application to the classification of microarray data and satellite radiance data. J. Amer. Statist. Assoc., 99, 67-81.
14. Lin, Y. (2000). Some asymptotic properties of the support vector machine. Technical report 1029, Department of Statistics, University of Wisconsinmadison.
15. Lin, Y. (2002) A Note on margin-based loss functions in classification. Statistics and Probability Letters, 68, 73-82.
16. Liu, Y. and Shen, X. (2006). On multicategory ψ-learning and support vector machine. J. Amer. Statist. Assoc., 101, 500-509.
17. Mercer, J. (1909). Functions of positive and negative type and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society, London A, 209 415-446.
18. Pollard, D. (1984). Convergence of Stochastic Processes. Springer-Verlag, New York.
19. Shen, X., Tseng, G., Zhang, X. and Wong, W. (2003) On ψ-Learning. J. Amer. Statist. Assoc., 98, 724-734.
20. Shen, X. and Wong, W. (1994). Convergence rate of sieve estimates. Ann. Statist., 22, 580-615.
21. Tewari, A., and Bartlett, P. (2005). On the consistency of multiclass classification methods. In Proceedings of the 18th Annual Conference on Learning Theory, 3559, 143-157. Springer.
22. 1 penalty. Technical Report MI 2004-14, University of Leiden.
23. Vapnik, V. (1998). Statistical Learning Theory, Wiley, New York.
24. Van De Geer, S. (1993). Hellinger-consistency of certain nonparametric maximum likelihood estimators. Ann. of Statist., 21,14-44.
25. Van der Vaart, A. W. and Wellner, J. A. (2000) Weak Convergence and Empirical Processes with Application to Statistics, Springer, New York.
26. Weston, J., and Watkins, C. (1999). Support vector machines for multiclass pattern recognition. Proceedings of the Seventh European Symposium On Artificial Neural Networks.
27. Wang, L., Zhu, J. and Zou, H. (2006) The doubly regularized support vector machine. Statistica Sinica, 16, 617-634.
28. 1-norm multi-class support vector machines: methodology and theory. J. Amer. Statist. Asso, 102, 595-602.
29. Zhang, T. (2004a). Statistical behavior and consistency of classification methods based on convex risk minimization. Ann. Statist., 32, 56-85.
30. Zhang, T. (2004b). Statistical analysis of some multi-category large Margin classification Methods. J. Machine Learning Res., 5, 1225-1251.
31. Zou, H. and Hastie, T. (2005). Regularization and Variable Selection via the Elastic Net. J. R. Statist. Soc. B, 67, 301-320.
32. Zou, H., Zhu, J. and Hastie, T. (2005) The margin vector, admissible losses and multi-class margin-based classifiers. Technical Report, University of Minnesota.
33. Zhu, J. and Hastie, T. (2005) Kernel logistic regression and the import vector machine. J. Comput. and Graph. Statist., 14, 185-205.