Integrating complementary techniques for promoting diversity in classifier ensembles: A systematic study

Neurocomputing

Volumn 138, Issue , 2014, Pages 347-357

  Nascimento, Diego S C ^a,c   Coelho, André L V ^b   Canuto, Anne M P ^c  

^a Centro Federal de Educacao Tecnologica do Rio Grande do Norte   (Brazil)

^b UNIVERSITY OF FORTALEZA   (Brazil)

^c FEDERAL UNIVERSITY OF RIO GRANDE DO NORTE   (Brazil)

Author keywords

Bagging; Classifier ensembles; Diversity; Feature selection; Heterogeneous models

Indexed keywords

COMPUTER APPLICATIONS; FEATURE EXTRACTION; NEURAL NETWORKS;

BAGGING; CLASSIFIER ENSEMBLES; COMPLEMENTARY TECHNIQUES; CONCEPTUAL FRAMEWORKS; DIVERSITY; HETEROGENEOUS MODELS; RESAMPLING TECHNIQUE; SEQUENTIAL MANNERS;

COMPUTER SIMULATION;

ARTICLE; CLASSIFICATION ALGORITHM; CONTROLLED STUDY; DATA RESAMPLING; FEATURE SELECTION; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; MATHEMATICAL PHENOMENA; MEASUREMENT ACCURACY; PRIORITY JOURNAL;

EID: 84899957756     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2014.01.027     Document Type: Article

References (32)

1. Kuncheva L.I. Combining Pattern Classifiers. Methods and Algorithms 2004, Wiley, Hoboken, New Jersey.
2. Polikar R. Ensemble based systems in decision making. IEEE Circuits Syst. Mag. 2006, 6(3):21-45.
3. G. Brown, L.I. Kuncheva, "Good" and "bad" diversity in majority vote ensembles, in: N. Gayar, J. Kittler, F. Roli (Eds.), Multiple Classifier Systems-MCS 2010, Lecture Notes in Computer Science, vol. 5997, Springer, Berlin, 2010, pp. 124-133.
4. Kuncheva L.I., Whitaker C.J. Measures of diversity in classifier ensembles. Mach. Learn. 2003, 51:181-207.
5. Schapire R.E., Freund Y. Boosting. Foundations and Algorithms 2012, The MIT Press, Cambridge, Massachusetts.
6. M. Re, G. Valentini, Ensemble methods: a review, in: Advances in Machine Learning and Data Mining for Astronomy, 2012, pp. 563-582.
7. Ho T.K. The random subspace method for constructing decision forests. IEEE Trans. Pattern Anal. Mach. Intell. 1998, 20(8):832-844.
8. Achlioptas D. Database-friendly random projections. Johnson-Lindenstrauss with binary coins. J. Comput. Syst. Sci. 2003, 66(4):671-687.
9. A. Bertoni, G. Valentini, Discovering significant structures in clustered bio-molecular data through the bernstein inequality, in: Proceedings of the 11th International Conference, KES 2007 and XVII Italian Workshop on Neural Networks Conference on Knowledge-Based Intelligent Information and Engineering Systems: Part III, Springer-Verlag, Berlin, Heidelberg, 2007, pp. 886-891.
10. Q.L. Zhao, Y.J. Huang, M. Xu, Incremental learning by heterogeneous bagging ensemble, in: Proceedings of the 6th International Conference on Advanced Data Mining and Applications, ADMA 10, vol. II, Springer-Verlag, Berlin, Heidelberg, 2010, pp. 1-12.
11. Q.-L. Zhao, Y.-H. Jiang, M. Xu, Incremental learning based on ensemble pruning, in: 8th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), vol. 1, 2011, pp. 377-381.
12. Brown G., Wyatt J., Harris R., Yao X. Diversity creation methods. a survey and categorisation. J. Inf. Fusion 2005, 6:5-20.
13. Breiman L. Random forests. Mach. Learn. 2001, 45:5-32.
14. Dietterich T.G. An experimental comparison of three methods for constructing ensembles of decision trees. bagging, boosting and randomization. Mach. Learn. 2000, 40(2):139-158.
15. T.G. Dietterich, Ensemble methods in machine learning, in: Proceedings of the 1st International Workshop on Multiple Classifier Systems, Springer, London, UK, 2000, pp. 1-15.
16. Canuto A., Abreu M., Oliveira L., Xavier J., Santos A. Investigating the influence of the choice of the ensemble members in accuracy and diversity of selection-based and fusion-based methods for ensembles. Pattern Recognit. Lett. 2007, 28(4):472-486.
17. L.E.A. Santana, L. Silva, A.M.P. Canuto, F. Pintro, K.O. Vale, A comparative analysis of genetic algorithm and ant colony optimization to select attributes for an heterogeneous ensemble of classifiers, in: IEEE Congress on Evolutionary Computation (CEC), 2010, pp. 1-8.
18. Coelho A.L.V., Nascimento D.S.C. On the evolutionary design of heterogeneous bagging models. Neurocomputing 2010, 73:3319-3322.
19. Nascimento D.S.C., Coelho A.L.V. Ensembling heterogeneous learning models with boosting. Neural Information Processing-ICONIP 2009, Part I, Lecture Notes in Computer Science 2009, vol. 5863:512-519. Springer, Berlin.
20. Breiman L. Bagging predictors. Mach. Learn. 1996, 24:123-140.
21. D.S.C. Nascimento, A.M.P. Canuto, L.M.M. Silva, A.L.V. Coelho, Combining different ways to generate diversity in bagging models: an evolutionary approach, in: International Joint Conference on Neural Networks (IJCNN), 2011, pp. 2235-2242.
22. Guyon I., Gunn S., Nikravesh M., Zadeh L.A. Feature Extraction. Foundations and Applications 2006, Springer, Berlin.
23. Eiben A.E., Smith J.E. Introduction to Evolutionary Computing 2007, Springer, Berlin. 2nd ed.
24. Chen P., Popovich P. Correlation. Parametric and Nonparametric Measures 2002, Sage Publications, Thousand Oaks, California.
25. Witten I.H., Frank E. Data Mining. Practical Machine Learning Tools and Techniques 2005, Elsevier, San Francisco, California. 2nd ed.
26. Geman S., Bienenstock E., Doursat R. Neural networks and the bias/variance dilemma. Neural Comput. 1992, 4:1-58.
27. Opitz D., Maclin R. Popular ensemble methods. an empirical study. J. Artif. Intell. Res. 1999, 11:169-198.
28. A. Asunción, D.J. Newman, UCI Machine Learning Repository, University of California at Irvine, 2007. http://ics.uci.edu/~mlearn/MLRepository.html.
29. Kotsiantis S.B., Pintelas P.E. Combining bagging and boosting. Int. J. Comput. Intell. 2004, 1(4):324-333.
30. Demšar J. Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 2006, 7:1-30.
31. Webb G.I. Multiboosting. a technique combining boosting and wagging. Mach. Learn. 2000, 40:159-196.
32. Brazdil P., Carrier C.G., Soares C., Vilalta R. Metalearning. Applications to Data Mining 2009, Springer, Berlin.