Unifying multi-class AdaBoost algorithms with binary base learners under the margin framework

Pattern Recognition Letters

Volumn 28, Issue 5, 2007, Pages 631-643

  Sun, Yijun ^a   Todorovic, Sinisa ^b   Li, Jian ^c  

^a UNIVERSITY OF FLORIDA   (United States)

^b UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^c University of Florida   (United States)

Author keywords

AdaBoost; Margin theory; Multi class classification problem

Indexed keywords

ALGORITHMS; BINARY SEQUENCES; DATA REDUCTION; DATABASE SYSTEMS; GRADIENT METHODS;

ADABOOST ALGORITHM; MARGIN THEORY; MULTI CLASS CLASSIFICATION PROBLEM;

LEARNING SYSTEMS;

EID: 33846109957     PISSN: 01678655     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patrec.2006.11.001     Document Type: Article

References (20)

1. Allwein E.L., Schapire R.E., and Singer Y. Reducing multiclass to binary: a unifying approach for margin classifiers. J. Mach. Learn. Res. 1 (2000) 113-141
2. Blake, C., Merz, C., 1998. UCI repository of machine learning databases.
3. Breiman L. Prediction games and arcing algorithms. Neural Comput. 11 7 (1999) 1493-1517
4. Crammer K., and Singer Y. On the learnability and design of output codes for multiclass problems. Proc. 13th Annual Conf. Computational Learning Theory (2000), Stanford University, CA, USA 35-46
5. Dietterich T.G. An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization. Mach. Learn. 40 2 (2000) 139-157
6. Dietterich T.G., and Bakiri G. Solving multiclass learning problems via error-correcting output codes. J. Artificial Intell. Res. 2 (1995) 263-286
7. Freund Y., and Schapire R.E. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55 1 (1997) 119-139
8. Friedman J. Greedy function approximation: a gradient boosting machine. Ann. Statist. 29 5 (2001) 1189-1232
9. Friedman J., Hastie T., and Tibshirani R. Additive logistic regression: a statistical view of boosting. Ann. Statist. 28 2 (2000) 337-407
10. Grove, A.J., Schuurmans, D., 1998. Boosting in the limit: maximizing the margin of learned ensembles. In: Proc. 15th Natl. Conf. on Artificial Intelligence, Madison, WI, USA, pp. 692-699.
11. Guruswami, V., Sahai, A., 1999. Multiclass learning, boosting, and error-correcting codes. In: Proc. 12th Annual Conf. Computational Learning Theory, Santa Cruz, California, pp. 145-155.
12. LeCun Y., Boser B., Denker J.S., Hendersen D., Howard R., Hubbard W., and Jackel L.D. Backpropagation applied to handwritten zip code recognition. Neural Comput. 1 4 (1989) 541-551
13. Mason L., Bartlett J., Baxter P., and Frean M. Functional gradient techniques for combining hypotheses. In: Scholkopf B., Smola A., Bartlett P., and Schuurmans D. (Eds). Advances in Large Margin Classifiers (2000), MIT Press, Cambridge, MA, USA 221-247
14. Quinlan J.R. C4.5: Programs for Machine Learning (1993), Morgan Kaufmann
15. Quinlan, J., 1996. Bagging, boosting, and C4.5. In: Proc. 13th Natl. Conf. Artificial Intelligence and 8th Innovative Applications of Artificial Intelligence Conf., Portland, OR, USA, pp. 725-730.
16. Rudin C., Daubechies I., and Schapire R.E. The dynamics of AdaBoost: cyclic behavior and convergence of margins. J. Mach. Learn. Res. 5 (2004) 1557-1595
17. Schapire R.E. Using output codes to boost multiclass learning problems. Proc. 14th Intl. Conf. Machine Learning (1997), Nashville, TN, USA 313-321
18. Schapire R., and Singer Y. Improved boosting algorithms using confidence-rated predictions. Mach. Learn. 37 3 (1999) 297-336
19. Schapire R.E., Freund Y., Bartlett P., and Lee W.S. Boosting the margin: a new explanation for the effectiveness of voting methods. Ann. Statist. 26 5 (1998) 1651-1686
20. Sun, Y., Todorovic, S., Li, J. Reducing the overfitting of AdaBoost by controlling its data distribution skewness. Int. J. Pattern Recog. Artificial Intell., in press.