Statistical approaches to combining binary classifiers for multi-class classification

Neurocomputing

Volumn 74, Issue 5, 2011, Pages 680-688

  Shiraishi, Yuichi ^a   Fukumizu, Kenji ^b  

^a UNIVERSITY OF TOKYO   (Japan)

^b INSTITUTE OF STATISTICAL MATHEMATICS   (Japan)

Author keywords

Combining binary classifiers; Group lasso; Meta learning; Multi class classification; Stacking

Indexed keywords

COMBINING BINARY CLASSIFIERS; GROUP LASSO; METALEARNING; MULTI-CLASS CLASSIFICATION; STACKING;

LOCOMOTIVES; RAILROAD CARS;

CLASSIFIERS;

ACCURACY; ANALYTIC METHOD; ARTICLE; CLASSIFICATION; CLASSIFIER; LOGISTIC REGRESSION ANALYSIS; MATHEMATICAL ANALYSIS; MATHEMATICAL COMPUTING; MATHEMATICAL PARAMETERS; PRIORITY JOURNAL; PROBABILITY; STATISTICAL ANALYSIS; SUPPORT VECTOR MACHINE;

EID: 78650718992     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2010.09.004     Document Type: Article

References (43)

1. Allwein E.L., Schapire R.E., Singer Y. Reducing multiclass to binary: a unifying approach for margin classifiers. Journal of Machine Learning Research 2000, 1:113-141.
2. Bennett P.N., Dumais S.T., Horvitz E. The combination of text classifiers using reliability indicators. Information Retrieval 2005, 8:67-100.
3. C.L. Blake, C.J. Merz, UCI repository of machine learning databases, Technical Report, Department of Information Computational Science, University of California, Irvine, 1998.
4. Bradley R.A., Terry M.E. Rank analysis of incomplete block designs: I. The methods of paired comparisons. Biometrika 1952, 39:324-345.
5. Bredensteiner E.J., Bennett K.P. Multicategory classification by support vector machines. Computational Optimization and Applications 1999, 12:53-79.
6. Breiman L. Stacked regressions. Machine Learning 1996, 24:49-64.
7. Burges C.J.C., Schölkopf B. Improving the accuracy and speed of support vector machines. Advances in Neural Information Processing Systems 1997, vol. 9:375-381. MIT Press.
8. Cortes C., Vapnik V. Multicategory classification by support vector machines. Machine Learning 1995, 12:273-297.
9. Crammer K., Singer Y. On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research 2001, 2:265-292.
10. Crammer K., Singer Y. On the learnability and design of output codes for multiclass problems. Machine Learning 2002, 47:201-233.
11. David H.A. The Method of Paired Comparisons 1988, Oxford University Press, pp. 945-952. second ed.
12. Dekel O., Singer Y. Multiclass learning by probabilistic embeddings. Advances in Neural Information Processing Systems 2002, 945-952. MIT Press.
13. Demšar J. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 2006, 7:1-30.
14. Dietterich T.G., Bakiri G. Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research 1995, 2:263-286.
15. K. Duan, S.S. Keerthi, Which is the best multiclass SVM method? An empirical study, in: Multiple Classifier Systems, 2005, pp. 278-285.
16. Džeroski S., Ženko B. Is combining classifiers with stacking better than selecting the best one?. Machine Learning 2004, 54:255-273.
17. J. Friedman, Another approach to polychotomous classification, Technical Report, Department of Statistics, Stanford University, 1996.
18. Gama J., Brazdil P. Cascade generalization. Machine Learning 2000, 41:315-343.
19. Hastie T., Tibshirani R. Classification by pairwise coupling. Annals of Statistics 1998, 26:451-471.
20. Hsu C.W., Lin C.J. A comparison of methods for multiclass support vector machines. IEEE Transactions on Neural Networks 2002, 13:415-425.
21. Huang T.K., Weng R.C., Lin C.J. Generalized Bradley-Terry models and multi-class probability estimates. Journal of Machine Learning Research 2006, 7:85-115.
22. Keerthi S.S., Chapelle O., DeCoste D. Building support vector machines with reduced classifier complexity. Journal of Machine Learning Research 2006, 7:1493-1515.
23. Kittler J., Hatef M., Duin R.P.W., Matas J. On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 1998, 20:226-239.
24. LeBlanc M., Tibshirani R. Combining estimates in regression and classification. Journal of the American Statistical Association 1996, 91:1641-1650.
25. Lee Y., Lin Y., Wahba G. Multicategory support vector machines: theory and application to the classification of microarray data and satellite radiance data. Journal of the American Statistical Association 2004, 99:67-81.
26. Meier L., van de Geer S., Bühlmann P. The group lasso for logistic regression. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 2008, 70:53-71.
27. Melvin I., Ie E., Weston J., Noble W.S., Leslie C. Multi-class protein classification using adaptive codes. Journal of Machine Learning Research 2007, 8:1557-1581.
28. Platt J.C. Fast training of support vector machines using sequential minimal optimization. Advances in Kernel Methods-Support Vector Learning 1999, 185-208. MIT Press.
29. Platt J.C. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in Large Margin Classifiers 1999, 61-74. MIT Press.
30. Platt J.C., Cristianini N., Shawe-Taylor J. Large margin dags for multiclass classification. Advances in Neural Information Processing Systems 2000, 547-553. MIT Press.
31. Rätsch G., Smola A.J., Mika S. Adapting codes and embeddings for polychotomies. Advances in Neural Information Processing Systems 2003, 513-520. MIT Press.
32. Rifkin R., Klautau A. In defense of one-vs-all classification. Journal of Machine Learning Research 2004, 5:101-141.
33. T. Takenouchi, S. Ishii, Multiclass classification as a decoding problem, in: Proceedings of the 9th Workshop on Information-Based Induction Sciences, 2006, pp. 244-249 (in Japanese).
34. Tewari A., Bartlett P. On the consistency of multiclass classification methods. Journal of Machine Learning Research 2007, 8:1007-1025.
35. Tibshirani R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B 1996, 58:267-288.
36. Todorovski L., Džeroski S. Combining classifiers with meta decision trees. Machine Learning 2003, 50:223-249.
37. J. Weston, C. Watkins, Multi-class support vector machines, Technical Report, Department of Computer Science, University of London, 1998.
38. Wolpert D.H. Stacked generalization. Neural Networks 1992, 5:241-259.
39. D.H. Wolpert, W.G. Macready, Combining stacking with bagging to improve a learning algorithm, Technical Report, Santa Fe Institute, 1996.
40. Wu T.F., Lin C.J., Weng R.C. Probability estimates for multi-class classification by pairwise coupling. Journal of Machine Learning Research 2004, 5:975-1005.
41. Yuan M., Lin Y. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 2006, 68:49-67.
42. Zhang T. Statistical analysis of some multi-category large margin classification methods. Journal of Machine Learning Research 2004, 5:1225-1251.
43. Zhu J., Hastie T. Kernel logistic regression and import vector machine. Journal of Computational and Graphical Statistics 2005, 14:185-205.