Reducing multiclass to binary: A unifying approach for margin classifiers

Journal of Machine Learning Research

Volumn 1, Issue 2, 2001, Pages 113-141

  Allwein, Erin L ^a   Schapire, Robert E ^b   Singer, Yoram ^c  

^a SOUTHWEST RESEARCH INSTITUTE   (United States)

^b AT AND T LABS RESEARCH   (United States)

^c HEBREW UNIVERSITY OF JERUSALEM   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE BOOSTING; DECISION TREES; LOGISTIC REGRESSION; SUPPORT VECTOR MACHINES; SUPPORT VECTOR REGRESSION;

BINARY PROBLEMS; CLASSIFICATION LEARNING; DECISION-TREE ALGORITHM; ERROR-CORRECTING; GENERAL METHOD; GENERALIZATION ERROR; MARGIN CLASSIFIERS; MULTI-CLASS CATEGORIZATION;

LEARNING SYSTEMS;

EID: 24044435942     PISSN: 15324435     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (28)

1. Bartlett, P. L. (1998). The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network. IEEE Transactions on Information Theory, 44(2), 525-536.
2. Breiman, L. (1997a). Arcing the edge. Tech. rep. 486, Statistics Department, University of California at Berkeley.
3. Breiman, L. (1997b). Prediction games and arcing classifiers. Tech. rep. 504, Statistics Department, University of California at Berkeley.
4. Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984). Classification and Regression Trees. Wadsworth & Brooks.
5. Collins, M., Schapire, R. E., & Singer, Y. (2000). Logistic regression, AdaBoost and Bregman distances. In Proceedings of the Thirteenth Annual Conference on Computational Learning Theory.
6. Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273-297.
7. Crammer, K., & Singer, Y. (2000). On the learnability and design of output codes for multiclass problems. In Proceedings of the Thirteenth Annual Conference on Computational Learning Theory.
8. Csiszár, I., & Tusnády, G. (1984). Information geometry and alternating minimization procedures. Statistics and Decisions, Supplement Issue, 1, 205-237.
9. Della Pietra, S., Della Pietra, V., & Lafferty, J. (1997). Inducing features of random fields. IEEE Transactions Pattern Analysis and Machine Intelligence, 19(4), 1-13.
10. Dietterich, T. G., & Bakiri, G. (1995). Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research, 2, 263-286.
11. Freund, Y. (1999). An adaptive version of the boost by majority algorithm. In Proceedings of the Twelfth Annual Conference on Computational Learning Theory, pp. 102-113.
12. Freund, Y., & Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1), 119-139.
13. Friedman, J., Hastie, T., & Tibshirani, R. (2000). Additive logistic regression: a statistical view of boosting. The Annals of Statistics, 38(2), 337-374.
14. Guruswami, V., & Sahai, A. (1999). Multiclass learning, boosting, and error-correcting codes. In Proceedings of the Twelfth Annual Conference on Computational Learning Theory, pp. 145-155.
15. Hastie, T., & Tibshirani, R. (1998). Classification by pairwise coupling. The Annals of Statistics, 26(2), 451-471.
16. Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301), 13-30.
17. Höffgen, K.-U., & Simon, H.-U. (1992). Robust trainability of single neurons. In Proceedings of the Fifth Annual ACM Workshop on Computational Learning Theory, pp. 428-439.
18. Kearns, M., & Mansour, Y. (1996). On the boosting ability of top-down decision tree learning algorithms. In Proceedings of the Twenty-Eighth Annual ACM Symposium on the Theory of Computing.
19. Lafferty, J. (1999). Additive models, boosting and inference for generalized divergences. In Proceedings of the Twelfth Annual Conference on Computational Learning Theory, pp. 125-133.
20. Mason, L., Baxter, J., Bartlett, P., & Frean, M. (1999). Functional gradient techniques for combining hypotheses. In Smola, A. J., Bartlett, P. J., Schölkopf, B., & Schuurmans, D. (Eds.), Advances in Large Margin Classifiers. MIT Press.
21. Merz, C. J., & Murphy, P. M. (1998). UCI repository of machine learning databases. www.ics.uci.edu/~mlearn/MLRepository.html.
22. Quinlan, J. R. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann.
23. Rätsch, G., Onoda, T., & Müller, K.-R. (to appear). Soft margins for AdaBoost. Machine Learning.
24. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In Rumelhart, D. E., & McClelland, J. L. (Eds.), Parallel Distributed Processing - Explorations in the Microstructure of Cognition, chap. 8, pp. 318-362. MIT Press.
25. Schapire, R. E., Freund, Y., Bartlett, P., & Lee, W. S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics, 26(5), 1651-1686.
26. Schapire, R. E., & Singer, Y. (1999). Improved boosting algorithms using confidence-rated predictions. Machine Learning, 37(3), 297-336.
27. Schölkopf, B., Smola, A., Williamson, R., & Bartlett, P. (1998). New support vector algorithms. Tech. rep. NC2-TR-1998-053, NeuroColt2.
28. Vapnik, V. N. (1995). The Nature of Statistical Learning Theory. Springer.