Using boosting to prune bagging ensembles

Pattern Recognition Letters

Volumn 28, Issue 1, 2007, Pages 156-165

  Martínez Muñoz, Gonzalo ^a   Suárez, Alberto ^a  

^a UNIVERSIDAD AUTÓNOMA DE MADRID   (Spain)

Author keywords

Bagging; Boosting; Decision trees; Ensemble pruning; Ensembles; Machine learning

Indexed keywords

CLASSIFICATION (OF INFORMATION); DATA STORAGE EQUIPMENT; ERROR ANALYSIS; IDENTIFICATION (CONTROL SYSTEMS); LEARNING SYSTEMS; TREES (MATHEMATICS);

BAGGING; BOOSTING; DECISION TREES; ENSEMBLE PRUNING;

PATTERN RECOGNITION SYSTEMS;

EID: 33750460241     PISSN: 01678655     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patrec.2006.06.018     Document Type: Article

References (24)

1. Bakker B., and Heskes T. Clustering ensembles of neural network models. Neural Networks 16 2 (2003) 261-269
2. Banfield R.E., Hall L.O., Bowyer K.W., and Kegelmeyer W.P. Ensemble diversity measures and their application to thinning. Inform. Fusion J. 6 1 (2005) 49-62
3. Bauer E., and Kohavi R. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learn. 36 1-2 (1999) 105-139
4. Blake, C.L., Merz, C.J. 1998. UCI repository of machine learning databases. URL Available from: .
5. Breiman L. Bagging predictors. Machine Learn. 24 2 (1996) 123-140
6. Breiman, L. 1996b. Bias, variance, and arcing classifiers. Tech. Rep. 460, Statistics Department, University of California.
7. Breiman L. Arcing classifiers. Ann. Statist. 26 3 (1998) 801-849
8. Breiman L. Random forests. Machine Learn. 45 1 (2001) 5-32
9. Breiman L., Friedman J.H., Olshen R.A., and Stone C.J. Classification and Regression Trees (1984), Chapman & Hall, New York
10. Demir C., and Alpaydin E. Cost-conscious classifier ensembles. Pattern Recognition Lett. 26 14 (2005) 2206-2214
11. Dietterich T.G. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learn. 40 2 (2000) 139-157
12. Domingos P. Knowledge acquisition from examples via multiple models. Proc. 14th Internat. Conf. on Machine Learning (1997), Morgan Kaufmann 98-106
13. Freund, Y., Schapire, R.E. 1995. A decision-theoretic generalization of on-line learning and an application to boosting. In: Proc. 2nd European Conf. on Computational Learning Theory, pp. 23-37.
14. Giacinto G., and Roli F. An approach to the automatic design of multiple classifier systems. Pattern Recognition Lett. 22 1 (2001) 25-33
15. Margineantu D.D., and Dietterich T.G. Pruning adaptive boosting. Proc. 14th Internat. Conf. on Machine Learning (1997), Morgan Kaufmann 211-218
16. Martínez-Muñoz G., and Suárez A. Aggregation ordering in bagging. Proc. IASTED Internat. Conf. on Artificial Intelligence and Applications (2004), Acta Press 258-263
17. Martínez-Muñoz G., and Suárez A. Using all data to generate decision tree ensembles. IEEE Trans. Systems Man Cybernet. Part C 34 4 (2004) 393-397
18. Martínez-Muñoz G., and Suárez A. Switching class labels to generate classification ensembles. Pattern Recognition 38 10 (2005) 1483-1494
19. Prodromidis A.L., and Stolfo S.J. Cost complexity-based pruning of ensemble classifiers. Knowl. Inform. Systems 3 4 (2001) 449-469
20. Rätsch G., Onoda T., and Müller K.-R. Soft margins for AdaBoost. Machine Learn. 42 3 (2001) 287-320
21. Tamon C., and Xiang J. On the boosting pruning problem. Proc. 11th European Conf. on Machine Learning vol. 1810 (2000), Springer, Berlin 404-412
22. Webb G.I. Multiboosting: A technique for combining boosting and wagging. Machine Learn. 40 2 (2000) 159-196
23. Zhou Z.-H., and Tang W. Selective ensemble of decision trees. Lecture Notes in Artificial Intelligence vol. 2639 (2003), Springer, Berlin 476-483
24. Zhou Z.-H., Wu J., and Tang W. Ensembling neural networks: Many could be better than all. Artif. Intell. 137 1-2 (2002) 239-263