RotBoost: A technique for combining Rotation Forest and AdaBoost

Pattern Recognition Letters

Volumn 29, Issue 10, 2008, Pages 1524-1536

  Zhang, Chun Xia ^a   Zhang, Jiang She ^a  

^a XI'AN JIAOTONG UNIVERSITY   (China)

Author keywords

AdaBoost; Bagging; Base learning algorithm; Ensemble method; MultiBoost; Rotation Forest

Indexed keywords

DATA STRUCTURES; ERROR ANALYSIS; LEARNING ALGORITHMS; PARALLEL PROCESSING SYSTEMS;

BAGGING; BASE LEARNING ALGORITHMS; MULTIBOOST; PARALLEL EXECUTION; ROTATION FORESTS;

TREES (MATHEMATICS);

EID: 44449124996     PISSN: 01678655     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patrec.2008.03.006     Document Type: Article

References (36)

1. Banfield R.E., Hall L.O., Bowyer K.W., and Kegelmeyer W.P. A comparison of decision tree ensemble creation techniques. IEEE Trans. Pattern Anal. Machine Intell. 29 1 (2007) 173-180
2. Bauer E., and Kohavi R. An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learn. 36 1-2 (1999) 105-139
3. Blake, C.L., Merz, C.J., 1998. UCI repository of machine learning databases. .
4. Breiman L. Bagging predictors. Machine Learn. 24 2 (1996) 123-140
5. Breiman L. Arcing classifiers. Ann. Statist. 26 3 (1998) 801-849
6. Breiman L. Random forests. Machine Learn. 45 1 (2001) 5-32
7. Breiman L., Friedman J., Olshen R., and Stone C. Classification and Regression Trees (1984), Chapman & Hall, New Work
8. Dietterich T.G. Machine-learning research: Four current directions. AI. Mag. 18 4 (1997) 97-136
9. Dietterich T.G. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting and randomization. Machine Learn. 40 2 (2000) 139-157
10. Freund Y., and Schapire R.E. Experiments with a new boosting algorithm. Proc. 13th Internat. Conf. on Machine Learn. (1996), Morgan Kaufmann, Bari, Italy 148-156
11. Freund Y., and Schapire R.E. A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. System Sci. 55 1 (1997) 119-139
12. Friedman J.H. On bias, variance, 0/1-loss, and the curse-of-dimensionality. Data Min. Knowl. Disc. 1 1 (1997) 55-77
13. Friedman J., Hastie T., and Tibshirani R. Additive logistic regression: A statistical view of boosting. Ann. Statist. 28 2 (2000) 337-407
14. Geman S., Bienenstock E., and Doursat R. Neural networks and the bias/variance dilemma. Neural Comput. 4 1 (1992) 1-58
15. Hansen L., and Salamon P. Neural networks ensembles. IEEE Trans. Pattern Anal. Machine Intell. 12 10 (1990) 993-1001
16. Ho T.K. The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Machine Intell. 20 8 (1998) 832-844
17. James G.M. Variance and bias for general loss functions. Machine Learn. 51 2 (2003) 115-135
18. Jin R., and Zhang J. Multi-class learning by smoothed boosting. Machine Learn. 67 3 (2007) 207-227
19. Kohavi R., and Wolpert D. Bias plus variance decomposition for zero-one loss functions. Proc. 13th Internat. Conf. on Machine Learn. (1996), Morgan Kaufmann, Bari, Italy 275-283
20. Kong E.B., and Dietterich T.G. Error-correcting output coding corrects bias and variance. Proc. 12th Internat. Conf. on Machine Learn. (1995), Morgan Kaufmann, San Francisco 313-321
21. Krogh A., and Vedelsby J. Neural network ensembles, cross validation, and active learning. In: Tesauro G., Touretzky D.S., and Leen T.K. (Eds). Adv. Neural Inform. Proc. Sys vol. 7 (1995), MIT Press, Cambridge MA 231-238
22. Latinne P., Debeir O., and Decaestecker C. Combining different methods and number of weak decision trees. Pattern Anal. Appl. 5 2 (2002) 201-209
23. Leblanc M., and Tibshirani R. Combining estimates in regression and classification. J. Amer. Statist. Assoc. 91 436 (1996) 1641-1650
24. Lim T.S., Loh W.Y., and Shin Y.S. A comparison of prediction accuracy, complexity, and training time of thirty-three old and new classification algorithms. Machine Learn. 40 3 (2000) 203-229
25. Meir R., and Rätsch G. An introduction to boosting and leveraging. Advanced Lectures on Machine Learning. Lect. Notes Comput. Sci. vol. 2600 (2003), Springer-Verlag, Berlin 118-183
26. Nadeau C., and Bengio Y. Inference for the generalization error. Machine Learn. 52 3 (2003) 239-281
27. Optiz D., and Maclin R. Popular ensemble methods: An empirical study. J. Artif. Intell. Res. 11 (1999) 169-198
28. Optiz D.W., and Shavlik J.W. Generating accurate and diverse members of a neural-network ensemble. In: Touretzky D.S., Mozer M.C., and Hasselmo M.M. (Eds). Adv. Neural Inform. Proc. Sys vol. 8 (1996), MIT Press, Cambridge MA 535-541
29. Rodríguez J.J., Alonso C.J., and Prieto O.J. Bias and variance of rotation-based ensembles. In: Cabestany J., Prieto A., and Sandoval D.F. (Eds). IWANN. Lect. Notes Comput. Sci. vol. 3512 (2005), Springer-Verlag, Berlin 779-786
30. Rodríguez J.J., Kuncheva L.I., and Alonso C.J. Rotation forest: A new classifier ensemble method. IEEE Trans. Pattern Anal. Machine Intell. 28 10 (2006) 1619-1630
31. Skurichina M., and Duin R.P.W. Bagging, boosting and the random subspace method for linear classifiers. Pattern Anal. Appl. 5 2 (2002) 121-135
32. Valentini G., and Dietterich T.G. Bias-variance analysis of support vector machines for the development of SVM-based ensemble methods. J. Machine Learn. Res. 5 (2004) 725-775
33. Webb G.I. MultiBoosting: A technique for combining boosting and wagging. Machine Learn. 40 2 (2000) 159-196
34. Webb, G.I., Conilione, P., 2006. Estimating bias and variance from data. .
35. Zhou Z.H., Wu J.X., and Tang W. Ensembling neural networks: Many could be better than all. Artif. Intell. 137 1-2 (2002) 239-263
36. Zhou Z.H., and Yu Y. Ensembling local learners through multimodal perturbation. IEEE Trans. Syst. Man Cybernet. B 35 4 (2005) 725-735