A design heuristic for hybrid classification ensembles in machine learning

Intelligent Data Analysis

Volumn 16, Issue 2, 2012, Pages 233-246

  Baumgartner, Dustin ^a   Serpen, Gursel ^a  

^a The University of Toledo   (United States)

Author keywords

ensemble classifier; ensemble design heuristic; global local learner; heterogeneous homogeneous diversity; hybrid ensemble; Machine learning

Indexed keywords

ENSEMBLE CLASSIFIERS; ENSEMBLE DESIGN HEURISTIC; GLOBAL-LOCAL; HETEROGENEOUS-HOMOGENEOUS DIVERSITY; HYBRID ENSEMBLE; MACHINE-LEARNING;

DESIGN;

LEARNING SYSTEMS;

EID: 84858204091     PISSN: 1088467X     EISSN: 15714128     Source Type: Journal    
DOI: 10.3233/IDA-2012-0521     Document Type: Article

References (35)

1. M.Aksela and J. Laaksonen, Using diversity of errors for selectingmembers of a committee classifier, Pattern Recognition 39 (2006), 608-623. (Pubitemid 43199459)
2. A. Asuncion and D.J. Newman, UCI Machine Learning Repository. Retrieved from [http://www.ics.uci.edu/~mlearn/MLRepository.html]. Irvine, CA: University of California, School of Information and Computer Science, 2007.
3. R.E. Banfield, L.O. Hall, K.W. Bowyer, D. Bhadoria and W.P. Kegelmeyer, A comparison of decision tree ensemble creation techniques, IEEE Transactions on Pattern Analysis and Machine Intelligence 29(1) (2007), 173-180. (Pubitemid 46415955)
4. D. Baumgartner and G. Serpen, Large experiment and evaluation tool forWEKAclassifiers, 5th International Conference on Data Mining, Las Vegas, 2009, 340-346.
5. S. Bian and W. Wang, On diversity and accuracy of homogeneous and heterogeneous ensembles, International Journal of Hybrid Intelligent Systems 4 (2007), 103-128.
6. R.R. Bouckaert, E. Frank, M. Hall, R. Kirkby, P. Reutemann, A. Seewald et al., WEKA Manual for Version 3-6-0, Retrieved from http://www.cs.waikato.ac.nz/ ~ml/weka/index.html. University of Waikato, New Zealand, 18 December 2008.
7. V.L. Brailovsky, O. Barzilay and R. Shahave, On global, local, mixed and neighborhood kernels for support vector machines, Pattern Recognition Letters 20 (1999), 1183-1190. (Pubitemid 32261896)
8. L. Breiman, Bagging predictors, Machine Learning 24(2) (1996), 123-140. (Pubitemid 126724382)
9. L. Breiman, Random forests, Machine Learning 45 (2001), 5-32. (Pubitemid 32933532)
10. A.M. Canuto, M.C. Abreu, L.M. Oliveira, J.C. Xavier and A.M. Santos, Investigating the influence of the choice of the ensemble members in accuracy and diversity of selection-based and fusion-based methods for ensembles, Pattern Recognition Letters 28 (2007), 472-486. (Pubitemid 44918135)
11. R. Caruana, A. Niculescu-Mizil, G. Crew and A. Ksikes, Ensemble selection from libraries of models, 21st International Conference on Machine Learning (2004), 137-144. (Pubitemid 40290801)
12. R. Caruana and A. Niculescu-Mizil, Data mining in metric space: An empirical analysis of supervixed learning performance criteria, 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2004), 69-78. Seattle. (Pubitemid 40114917)
13. A. Chandra and X. Yao, Evolving hybrid ensembles of learning machines for better generalisation, Neurocomputing 69 (2006), 686-700. (Pubitemid 43230374)
14. P. Cunningham and J. Carney, Diversity versus quality in classification ensembles based on feature selection. Technical Report TCD-CS-2000-02, Trinity College, Department of Computer Science, Dublin, 2000.
15. J. Demsar, Statistical comparisons of classifiers over multiple data sets, Journal of Machine Learning 7 (2006), 1-30. (Pubitemid 43022939)
16. O.J. Dunn, Multiple comparisons among means, Journal of the American Statistical Association 56 (1961), 52-64.
17. Y. Freund and R.E. Schapire, Experiments with a new boosting algorithm, 13th International Conference on Machine Learning (1996), 148-156.
18. M. Friedman, A comparison of alternative tests of significance for the problem of m rankings, Annals of Mathematical Statistics 11 (1940), 56-92.
19. R.L. Iman and J.M. Davenport, Approximations of the critical region of the Friedman statistic, Communications in Statistics (1980), 571-595.
20. D.J. Hand and V. Vinciotti, Local versus global models for classification problems: Fitting models where it matters, The American Statistician 57(2) (2003), 124-131. (Pubitemid 36594034)
21. T.K. Ho, Random decision forests, 3rd International Conference on Document Analysis Recognition (1995), 278-282.
22. T.K. Ho, Multiple classifier combination: Lessons and the next steps, in: Hybrid Methods in Pattern Recognition, A. Kandel and H. Bunke, eds, 2002, pp. 171-198.
23. T.R. Hoens and N.V. Chawla, Generating Diverse Ensembles to Counter the Problem of Class Imbalance, Advances in Knowledge Discovery and Data Mining, Lecture Notes in Computer Science (2010)(6119), 488-499.
24. R.L. Iman and J.M. Davenport, Approximations of the critical region of the Friedman statistic, Communications in Statistics (1980), 571-595.
25. S.B.Kotsiantis and P.E. Pintelas,Ahybrid decision support tool-Using ensembles of classifiers, InternationalConference on Enterprise Information Systems 2 (2004), 448-456.
26. L.I. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms, Hoboken, NJ: John Wiley & Sons, Inc, 2004.
27. L.I.Kuncheva, That elusive diversity in classifier ensembles, Lecture Notes in Computer Science 2652 (2003), 1126-1138.
28. T.M. Mitchell, Machine Learning, McGraw Hill, 1997.
29. R. Polikar, Ensemble based systems in decision making, IEEE Circuits and Systems Magazine 6(3) (2006), 21-45.
30. K. Seewald and J. Furnkranz, An evaluation of grading classifiers, 4th International Conference on Advances in Intelligence Data Analysis (2001), 115-124. (Pubitemid 33348491)
31. D. Skalak, The sources of increased accuracy for two proposed boosting algorithms. AAAI '96 Workshop on Integrating Multiple Learned Models for Improving and Scaling Machine Learning Algorithms, (1996
32. E.K. Tang, P.N. Suganthan and X. Yao, An analysis of diversity measures, Machine Learning 65 (2006), 247-271. (Pubitemid 44451200)
33. H.Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques (2 ed.), San Francisco: Morgan Kaufmann, 2005.
34. D.H. Wolpert, Stacked generalization, Neural Networks 5(2) (1992), 241-260.
35. W.B.Yates andD. Partridge,Use of methodological diversity to improve neural network generalisation, NeuralComputing & Applications 4(2) (1996), 114-128.