Multiobjective genetic classifier selection for random oracles fuzzy rule-based classifier ensembles: How beneficial is the additional diversity?

Knowledge-Based Systems

Volumn 54, Issue , 2013, Pages 3-21

  Trawiński, Krzysztof ^a   Cordón, Oscar ^a,b   Quirin, Arnaud ^c   Sánchez, Luciano ^d  

^a EUROPEAN CENTRE FOR SOFT COMPUTING   (Spain)

^b UNIVERSITY OF GRANADA   (Spain)

^c Galician Research and Development Center in Advanced Telecommunications GRADIANT   (Spain)

^d UNIVERSITY OF OVIEDO   (Spain)

Author keywords

Bagging; Diversity measures; Evolutionary multiobjective optimization; Fuzzy rule based classifier ensembles; Genetic classifier selection; High complexity datasets; NSGA II; Random oracles

Indexed keywords

ECONOMIC AND SOCIAL EFFECTS; FUZZY INFERENCE; FUZZY RULES; MULTIOBJECTIVE OPTIMIZATION;

BAGGING; DIVERSITY MEASURE; EVOLUTIONARY MULTIOBJECTIVE OPTIMIZATION; FUZZY RULE-BASED CLASSIFIER; GENETIC CLASSIFIERS; HIGH COMPLEXITY; NSGA-II; RANDOM ORACLE;

CLASSIFICATION (OF INFORMATION);

EID: 84901724783     PISSN: 09507051     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.knosys.2013.08.006     Document Type: Article

References (62)

1. L.I. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms, Wiley, 2004.
2. K. Trawiński, O. Cordón, A. Quirin, L. Sánchez, Random oracles fuzzy rule-based multiclassifiers for high complexity datasets, in: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Hyderabad, India, 2013, in press.
3. L.I. Kuncheva, J.J. Rodríguez, Classifier ensembles with a random linear oracle, IEEE Transactions on Knowledge and Data Engineering 19 (2007) 500-508.
4. K. Trawiński, O. Cordón, A. Quirin, On designing fuzzy rule-based multiclassification systems by combining FURIA with bagging and feature selection, International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 19 (2011) 589-633.
5. J.C. Hühn, E. Hüllermeier, FURIA: an algorithm for unordered fuzzy rule induction, Data Mining and Knowledge Discovery 19 (2009) 293-319.
6. J.C. Hühn, E. Hüllermeier, An analysis of the FURIA algorithm for fuzzy rule induction, in: Advances in Machine Learning I, 2010, pp. 321-344.
7. L. Breiman, Bagging predictors, Machine Learning 24 (1996) 123-140.
8. X. Yao, Y. Liu, Making use of population information in evolutionary artificial neural networks, IEEE Transactions on Systems Man and Cybernetics Part B: Cybernetics 28 (1998) 417-425.
9. Z.-H. Zhou, J. Wu, W. Tang, Ensembling neural networks: many could be better than all, Artificial Intelligence 137 (2002) 239-263.
10. D. Partridge, W. Yates, Engineering multiversion neural-net systems, Neural Computation 8 (1996) 869-893.
11. K. Trawiński, A. Quirin, O. Cordón, A study on the use of multi-objective genetic algorithms for classifier selection in FURIA-based fuzzy multiclassifiers, International Journal of Computational Intelligence Systems 5 (2012) 231-253.
12. K. Deb, A. Pratap, S. Agarwal, T. Meyarivan, A fast and elitist multiobjective genetic algorithm: NSGA-II, IEEE Transactions on Evolutionary Computation 6 (2002) 182-197.
13. C. Coello, G. Lamont, D.V. Veldhuizen, Evolutionary Algorithms for Solving Multi-Objective Problems, second ed., Springer, 2007.
14. J.J. Rodríguez, L.I. Kuncheva, Naïve bayes ensembles with a random oracle, Lecture Notes in Computer Science, 4472, Springer-Verlag, 2007, pp. 450-458.
15. C.L. Blake, C.J. Merz, UCI repository of machine learning databases, 1998. .
16. J. Alcalá-Fdez, A. Fernández, J. Luengo, J. Derrac, S. García, Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework, Journal of Multiple-Valued Logic and Soft Computing 17 (2011) 255-287.
17. T. Dietterich, An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting, and randomization, Machine Learning 40 (2000) 139-157.
18. R. Banfield, L. Hall, K. Bowyer, W. Kegelmeyer, A comparison of decision tree ensemble creation techniques, IEEE Transactions on Pattern Analysis and Machine Intelligence 29 (2007) 173-180.
19. D. Optiz, R. Maclin, Popular ensemble methods: an empirical study, Journal of Artificial Intelligence Research 11 (1999) 169-198.
20. R. Schapire, The strength of weak learnability, Machine Learning 5 (1990) 197-227.
21. Z. Zhou, Ensembling local learners through multimodal perturbation, IEEE Transactions of Systems, Man, and Cybernetics, Part B: Cybernetics 35 (2005) 725-735.
22. A. Tsymbal, M. Pechenizkiy, P. Cunningham, Diversity in search strategies for ensemble feature selection, Information Fusion 6 (2005) 83-98.
23. L. Xu, A. Krzyzak, C. Suen, Methods of combining multiple classifiers and their application to handwriting recognition, IEEE Transactions on Systems, Man, and Cybernetics 22 (1992) 418-435.
24. T. Ho, The random subspace method for constructing decision forests, IEEE Transactions on Pattern Analysis and Machine Intelligence 20 (1998) 832-844.
25. L. Breiman, Random forests, Machine Learning 45 (2001) 5-32.
26. J.J. Rodriguez, L.I. Kuncheva, C.J. Alonso, Rotation forest: a new classifier ensemble method, IEEE Transactions on Pattern Analysis and Machine Intelligence 28 (2006) 1619-1630.
27. P.P. Bonissone, J.M. Cadenas, M.C. Garrido, R.A. Díaz-Valladares, A fuzzy random forest, International Journal of Approximate Reasoning 51 (2010) 729-747.
28. H. Takahashi, H. Honda, Lymphoma prognostication from expression profiling using a combination method of boosting and projective adaptive resonance theory, Journal of Chemical Engineering of Japan 39 (2006) 767-771.
29. J. Canul-Reich, L. Shoemaker, L.O. Hall, Ensembles of fuzzy classifiers, in: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), London, 2007, pp. 1-6.
30. C. Marsala, Data mining with ensembles of fuzzy decision trees, in: IEEE Symposium on Computational Intelligence and Data Mining, Nashville (USA), 2009, pp. 348-354.
31. C.Z. Janikow, Fuzzy decision trees: issues and methods, IEEE Transactions on Systems, Man, and Cybernetics, Part B 28 (1998) 1-14.
32. J. Aguilera, M. Chica, M. del Jesus, F. Herrera, Niching genetic feature selection algorithms applied to the design of fuzzy rule based classification systems, in: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), London, 2007, pp. 1794-1799.
33. D.E. Goldberg, Genetic Algorithms in Search Optimization and Machine Learning, Addison-Wesley, 1989.
34. Y. Nojima, H. Ishibuchi, Designing fuzzy ensemble classifiers by evolutionary multiobjective optimization with an entropy-based diversity criterion, in: International Conference on Hybrid Intelligent Systems and Conference on Neuro-Computing and Evolving Intelligence, 2006, 4pp (CD-ROM).
35. Y. Nojima, H. Ishibuchi, Genetic rule selection with a multi-classifier coding scheme for ensemble classifier design, International Journal of Hybrid Intelligent Systems 4 (2007) 157-169.
36. H. Ishibuchi, Y. Nojima, Evolutionary multiobjective optimization for the design of fuzzy rule-based ensemble classifiers, International Journal of Hybrid Intelligent Systems 3 (2006) 129-145.
37. R. Schapire, Y. Freund, P. Bartlett, W. Lee, Boosting the margin: a new explanation for the effectiveness of voting methods, in: International Conference on Machine Learning, Nashville, 1997, pp. 322-330.
38. B. Dasarathy, B. Sheela, A composite classifier system design: concepts and methodology, Proceedings of IEEE 67 (1979) 708-713.
39. E. Dos Santos, R. Sabourin, P. Maupin, A dynamic overproduce-and-choose strategy for the selection of classifier ensembles, Pattern Recognition 41 (2008) 2993-3009.
40. O. Cordón, A. Quirin, Comparing two genetic overproduce-and-choose strategies for fuzzy rule-based multiclassification systems generated by bagging and mutual information-based feature selection, International Journal of Hybrid Intelligent Systems 7 (2010) 45-64.
41. L. Oliveira, M. Morita, R. Sabourin, F. Bortolozzi, Multi-objective genetic algorithms to create ensemble of classifiers, Lecture Notes in Computer Science 3410 (2005) 592-606.
42. E. Dos Santos, R. Sabourin, P. Maupin, Single and multi-objective genetic algorithms for the selection of ensemble of classifiers, in: International Joint Conference on Neural Networks (IJCNN), Vancouver, 2006, pp. 3070-3077.
43. D. Ruta, B. Gabrys, Classifier selection for majority voting, Information Fusion 6 (2005) 63-81.
44. R.R. Yager, D.P. Filev, Essentials of Fuzzy Modeling and Control, Wiley Interscience, New York NY, USA, 1994.
45. J. Casillas, O. Cordon, F. Herrera, L. Magdalena, Interpretability Issues in Fuzzy Modeling, Springer-Verlag, Berlin, Heidelberg, 2003.
46. J.M. Alonso, L. Magdalena, G. González-Rodríguez, Looking for a good fuzzy system interpretability index: an experimental approach, International Journal of Approximate Reasoning 51 (2009) 115-134.
47. O. Cordón, F. Herrera, F. Hoffmann, L. Magdalena, Genetic Fuzzy Systems Evolutionary Tuning and Learning of Fuzzy Knowledge Bases, World Scientific, 2001.
48. O. Cordón, F. Gomide, F. Herrera, F. Hoffmann, L. Magdalena, Ten years of genetic fuzzy systems: current framework and new trends, Fuzzy Sets and Systems 141 (2004) 5-31.
49. F. Herrera, Genetic fuzzy systems: taxonomy current research trends and prospects, Evolutionary Intelligence 1 (2008) 27-46.
50. O. Cordón, A historical review of evolutionary learning methods for mamdani-type fuzzy rule-based systems: designing interpretable genetic fuzzy systems, International Journal of Approximate Reasoning 52 (2011) 894-913.
51. H. Ishibuchi, T. Nakashima, T. Morisawa, Voting in fuzzy rule-based systems for pattern classification problems, Fuzzy Sets and Systems 103 (1999) 223-238.
52. O. Cordón, M. del Jesus, F. Herrera, A proposal on reasoning methods in fuzzy rule-based classification systems, International Journal of Approximate Reasoning 20 (1999) 21-45.
53. J. Kittler, M. Hatef, R. Duin, J. Matas, On combining classifiers, IEEE Transactions on Pattern Analysis and Machine Intelligence 20 (1998) 226-238.
54. T. Dietterich, Approximate statistical test for comparing supervised classification learning algorithms, Neural Computation 10 (1998) 1895-1923.
55. J. Demšar, Statistical comparisons of classifiers over multiple data sets, Journal of Machine Learning Research 7 (2006) 1-30.
56. S. García, F. Herrera, An extension on statistical comparisons of classifiers over multiple data sets for all pairwise comparisons, Journal of Machine Learning Research 9 (2008) 2677-2694.
57. S. García, A. Fernández, J. Luengo, F. Herrera, Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power, Information Sciences 180 (2010) 2044-2064.
58. I.H. Witten, E. Frank, Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems), Morgan Kaufman Publishers Inc., 2005.
59. H. Ishibuchi, T. Nakashima, M. Nii, Classification and Modeling With Linguistic Information Granules, Springer, 2005.
60. K. Trawiński, O. Cordón, L. Sánchez, A. Quirin, A genetic fuzzy linguistic combination method for fuzzy rule-based multiclassifiers, IEEE Transactions on Fuzzy Systems (2013). in press.
61. L.I. Kuncheva, C.J. Whitaker, Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy, Machine Learning 51 (2003) 181-207.
62. V. López, A. Fernández, M.J. del Jesus, F. Herrera, A hierarchical genetic fuzzy system based on genetic programming for addressing classification with highly imbalanced and borderline data-sets, Knowledge-Based Systems 38 (2013) 85-104. Special Issue on Advances in Fuzzy Knowledge Systems: Theory and Application.