Multicategory large-margin unified machines

Journal of Machine Learning Research

Volumn 14, Issue , 2013, Pages 1349-1386

  Zhang, Chong ^a   Liu, Yufeng ^a  

^a University of North Carolina at Chapel Hill   (United States)

Author keywords

Hard classification; Large margin; Soft classification; Support vector machine

Indexed keywords

CLASS PROBABILITIES; CLASSIFICATION TASKS; COMPETITIVE PERFORMANCE; CONDITIONAL PROBABILITIES; LARGE-MARGIN; LOGISTIC REGRESSIONS; MULTI-CATEGORY CLASSIFICATION; SOFT CLASSIFICATION;

ESTIMATION; LOGISTICS; PROBABILITY;

SUPPORT VECTOR MACHINES;

EID: 84878914736     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (38)

1. N. Aronszajn. Theory of reproducing kernels. Transactions of the American Mathematical Society, 68(3):337-404, 1950.
2. P. L. Bartlett, M. I. Jordan, and J. D. McAuliffe. Convexity, classification, and risk bounds. Journal of the American Statistical Association, 101(473):138-156, 2006.
3. B. E. Boser, I. M. Guyon, and V. N. Vapnik. A training algorithm for optimal margin classifiers. In Proceedings of the Fifth Annual Workshop on Computational Learning Theory, COLT '92, pages 144-152, 1992. ISBN 0-89791-497-X.
4. C. Cortes and V. Vapnik. Support vector networks. Machine Learning, 20:273-297, 1995.
5. K. Crammer, Y. Singer, N. Cristianini, J. Shawe-taylor, and B. Williamson. On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research, 2:265-292, 2001.
6. Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119-139, 1997.
7. J. Friedman, T. Hastie, and R. Tibshirani. Additive logistic regression: a statistical view of boosting. Annals of Statistics, 28(2):337-407, 2000.
8. J. Friedman, T. Hastie, and R. Tibshirani. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1):1-22, 2010.
9. H. Huang, Y. Liu, Y. Du, C. Perou, D. N. Hayes, M. Todd, and J. S. Marron. Multiclass distance weighted discrimination. Journal of Computational and Graphical Statistics, 2013. Forthcoming.
10. Y. Lee, Y. Lin, and G.Wahba. Multicategory support vector machines, theory, and application to the classification of microarray data and satellite radiance data. Journal of the American Statistical Association, 99(465):67-81, 2004.
11. X. Lin, G. Wahba, D. Xiang, F. Gao, R. Klein, and B. Klein. Smoothing spline anova models for large data sets with bernoulli observations and the randomized gacv. Annals of Statistics, 28(6): 1570-1600, 2000.
12. Y. Liu. Fisher consistency of multicategory support vector machines. In Eleventh International Conference on Artificial Intelligence and Statistics, pages 289-296, 2007.
13. Y. Liu and X. Shen. Multicategory y-learning. Journal of the American Statistical Association, 101 (474):500-509, 2006.
14. Y. Liu and M. Yuan. Reinforced multicategory support vector machines. Journal of Computational and Graphical Statistics, 20(4):901-919, 2011.
15. Y. Liu, X. Shen, and H. Doss. Multicategory y-learning and support vector machine: computational tools. Journal of Computational and Graphical Statistics, 14(1):219-236, 2005.
16. Y. Liu, H. H. Zhang, and Y.Wu. Soft or hard classification? large margin unified machines. Journal of the American Statistical Association, 106(493):166-177, 2011.
17. J. S. Marron, M. Todd, and J. Ahn. Distance weighted discrimination. Journal of the American Statistical Association, 102(480):1267-1271, 2007.
18. S. Y. Park, Y. Liu, D. Liu, and P. Scholl. Multicategory composite least squares classifiers. Statistical Analysis and Data Mining, 3(4):272-286, 2010.
19. X. Shen andW. H.Wong. Convergence rate of sieve estimates. Annals of Statistics, 22(2):580-615, 1994.
20. X. Shen, G. C. Tseng, X. Zhang, and W. H. Wong. On y-learning. Journal of the American Statistical Association, 98(463):724-734, 2003.
21. I. Steinwart and C. Scovel. Fast rates for support vector machines using gaussian kernels. Annals of Statistics, 35(2):575-607, 2007.
22. Y. Tang and H. H. Zhang. Multiclass proximal support vector machines. Journal of Computational and Graphical Statistics, 15(2):339-355, 2006.
23. A. Tewari and P. L. Bartlett. On the consistency of multiclass classification methods. Journal of Machine Learning Research, 8:1007-1025, 2007.
24. P. Tseng. Convergence of a block coordinate descent method for nondifferentiable minimization. Journal of Optimization Theory and Applications, 109(3):475-494, 2001.
25. V. Vapnik. Statistical Learning Theory. Wiley, 1998.
26. R. G. Verhaak, K. A. Hoadley, E. Purdom, V. Wang, Y. Qi, M. D. Wilkerson, C. R. Miller, L. Ding, T. Golub, J. P. Mesirov, G. Alexe, M. Lawrence, M. O'Kelly, P. Tamayo, B. A. Weir, S. Gabriel, W. Winckler, S. Gupta, L. Jakkula, H. S. Feiler, J. G. Hodgson, C. D. James, J. N. Sarkaria, C. Brennan, A. Kahn, P. T. Spellman, R. K. Wilson, T. P. Speed, J. W. Gray, M. Meyerson, G. Getz, C. M. Perou, D. N. Hayes, and Cancer Genome Atlas Research Network. Integrated genomic analysis identifies clinically relevant subtypes of glioblastoma characterized by abnormalities in pdgfra, idh1, egfr, and nf1. Cancer Cell, 17(1):98-110, 2010.
27. G.Wahba. Support vector machines, reproducing kernel hilbert spaces and the randomized GACV. In Advances in Kernel Methods Support Vector Learning, pages 69-87. MIT Press, 1999.
28. G.Wahba. Soft and hard classification by reproducing kernel hilbert space methods. In Proceedings of the National Academy of Sciences, volume 99, pages 16524-16530, 2002.
29. J. Wang, X. Shen, and Y. Liu. Probability estimation for large margin classifiers. Biometrika, 95 (1):149-167, 2008.
30. L. Wang and X. Shen. On l1-norm multi-class support vector machines: methodology and theory. Journal of the American Statistical Association, 102(478):595-602, 2007.
31. J. Weston and C. Watkins. Support vector machines for multi-class pattern recognition. In Proceedings of the Seventh European Symposium on Artificial Neural Networks, volume 4, pages 219-224, 1999.
32. T.-F. Wu, C.-J. Lin, and R. C. Weng. Probability estimates for multi-class classification by pairwise coupling. Journal of Machine Learning Research, 5:975-1005, 2004.
33. Y. Wu and Y. Liu. Robust truncated-hinge-loss support vector machines. Journal of the American Statistical Association, 102(479):974-983, 2007.
34. T. Zhang. Statistical behavior and consistency of classification methods based on convex risk minimization. Annals of Statistics, 32:56-85, 2004a.
35. T. Zhang. Statistical analysis of some multi-category large margin classification methods. Journal of Machine Learning Research, 5:1225-1251, 2004b.
36. J. Zhu and T. Hastie. Kernel logistic regression and the import vector machine. Journal of Computational and Graphical Statistics, 14(1):185-205, 2005.
37. J. Zhu, H. Zou, S. Rosset, and T. Hastie. Multi-class adaboost. Statistics and Its Interface, 2(3): 349-360, 2009.
38. H. Zou, J. Zhu, and T. Hastie. New multicategory boosting algorithms based on multicategory fisher-consistent losses. Annals of Applied Statistics, 2(4):1290-1306, 2008.