A study on the use of multiobjective genetic algorithms for classifier selection in furia-based fuzzy multiclassifiers

International Journal of Computational Intelligence Systems

Volumn 5, Issue 2, 2012, Pages 231-253

  Trawiński, Krzysztof ^a,b   Cordón, Oscar ^a,b   Quirin, Arnaud ^a,b  

^a EUROPEAN CENTRE FOR SOFT COMPUTING   (Spain)

^b Calle Gonzalo Gutiérrez Quirós S/N ^*  (Austria)

Author keywords

Bagging; Diversity measures; Evolutionary multiobjective optimization; FURIA; Fuzzy rule based multiclassification systems; Genetic selection of individual classifiers; NSGA II

Indexed keywords

EVOLUTIONARY ALGORITHMS; FUZZY INFERENCE; FUZZY RULES; GENETIC ALGORITHMS; LEARNING ALGORITHMS; LEARNING SYSTEMS; MULTIOBJECTIVE OPTIMIZATION;

BAGGING; DIVERSITY MEASURE; EVOLUTIONARY MULTIOBJECTIVE OPTIMIZATION; FURIA; INDIVIDUAL CLASSIFIERS; MULTI-CLASSIFICATION; NSGA-II;

OPTIMIZATION;

EID: 84865770445     PISSN: 18756891     EISSN: 18756883     Source Type: Journal    
DOI: 10.1080/18756891.2012.685272     Document Type: Article

References (56)

1. Kuncheva, L., 2004. Combining Pattern Classifiers:Methods and Algorithms, Wiley.
2. Verzi, S.J., Heileman, G.L., and Georgiopoulos, M., 2006. Boosted ARTMAP:Modifications to fuzzy ARTMAP motivated by boosting theory. Neural Networks, 19 (4):446–468.
3. Pedrycz, W., and Kwak, K.C., 2006. Boosting of granular models. Fuzzy Sets and Systems, 157 (22):2934–2953.
4. Takahashi, H., and Honda, H., 2005. A new reliable cancer diagnosis method using boosted fuzzy classifier with a SWEEP operator method. Journal of Chemical Engineering of Japan, 38 (9):763–773.
5. Canul-Reich, J., Shoemaker, L., and Hall, L.O., 2007. “ Ensembles of fuzzy classifiers ”. In IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), 1–6. UK:London.
6. Nojima, Y., and Ishibuchi, H., 2007. Genetic rule selection with a multi-classifier coding scheme for ensemble classifier design. International Journal of Hybrid Intelligent Systems, 4 (3):157–169.
7. C., Marsala. Data mining with ensembles of fuzzy decision trees. In IEEE Symposium on Computational Intelligence and Data Mining, 348–354, 2009 Nashville USA.
8. Bonissone, P. P., Cadenas, J. M., Garrido, M. C., and Díaz-Valladares, R. A., 2010. A fuzzy random forest. International Journal of Approximate Reasoning, 51 (7):729–747.
9. O., Cordón, A., Quirin, and L., Sánchez. A first study on bagging fuzzy rule-based classification systems with multicriteria genetic selection of the component classifiers. In Third International Workshop on Genetic and Evolving Fuzzy Systems (GEFS), pages 11–16, Witten-Bommerholz Germany, 2008.
10. Cordón, O., and Quirin, A., 2010. Comparing two genetic overproduce-and-choose strategies for fuzzy rule-based multiclassification systems generated by bagging and mutual information-based feature selection. International Journal of Hybrid Intelligent Systems, 7 (1):45–64.
11. K., Trawiński, A., Quirin, and O., Cordón. Bi-criteria genetic selection of bagging fuzzy rule-based multiclassification systems. In IFSA/EUSFLAT Conf., pages 1514–1519, 2009.
12. Trawiński K., Quirin A., Cordón O., On the combination of accuracy and diversity measures for genetic selection of bagging fuzzy rule-based multi-classification systems International Conference on Intelligent Systems Design and Applications (ISDA) 2009 121 127
13. Partridge, D., and Yates, W.B., 1996. Engineering multi-version neural-net systems. Neural Computation, 8 (4):869–893.
14. Cordón, O., Herrera, F., Hoffmann, F., and Magdalena, L., 2001. Genetic Fuzzy Systems. Evolutionary Tuning and Learning of Fuzzy Knowledge Bases, World Scientific.
15. Cordón, O., Gomide, F., Herrera, F., Hoffmann, F., and Magdalena, L., 2004. Ten years of genetic fuzzy systems:Current framework and new trends. Fuzzy Sets and Systems, 141 (1):5–31.
16. Herrera F., Genetic fuzzy systems:taxonomy, current research trends and prospects Evolutionary Intelligence 1 1 27–46 January 2008
17. Trawiński, K., Cordón, O., and Quirin, A., 2011. On designing fuzzy multiclassifier systems by combining furia with bagging and feature selection, International Journal of Uncertainty. Fuzziness and Knowledge-Based Systems, 19 (4):589–633.
18. Hühn, J. C., and Hüllermeier, E., 2009. FURIA:an algorithm for unordered fuzzy rule induction. Data Mining and Knowledge Discovery, 19 (3):293–319.
19. Hühn, J. C., and Hüllermeier, E., 2010. An analysis of the FURIA algorithm for fuzzy rule induction. In Advances in Machine Learning I 321–344.
20. Ho, T., 1998. The random subspace method for constructing decision forests. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20 (8):832–844.
21. Battiti, R., 1994. Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on Neural Networks, 5 (4):537–550.
22. Feo, T.A., and Resende, M.G.C., 1995. Greedy randomized adaptive search procedures. Journal of Global Optimization, 6:109–133.
23. Ishibuchi, H., Nakashima, T., and Nii, M., 2004. Classification and Modeling with Linguistic Information Granules:Advanced Approaches to Linguistic Data Mining (Advanced Information Processing), Secaucus, NJ, USA:Springer-Verlag New York, Inc.
24. Coello, C.A., Lamont, G.B., and Van Veldhuizen, D.A., 2007. Evolutionary Algorithms for Solving Multi-Objective Problems, 2nd Edition, Springer.
25. Ludmila, I. Kuncheva, and Christopher, J. Whitaker. 2003. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine Learning, 51 (2):181–207.
26. Ruta, D., and Gabrys, B., 2005. Classifier selection for majority voting. Information Fusion, 6 (1):63–81.
27. Tsymbal, A., Pechenizkiy, M., and Cunningham, P., 2005. Diversity in search strategies for ensemble feature selection. Information Fusion, 6 (1):83–98.
28. Deb, K., Pratap, A., Agarwal, S., and Meyarivan, T., 2002. A fast and elitist multiobjective genetic algorithm:NSGA-II. IEEE Transactions on Evolutionary Computation, 6:182–197.
29. E.M., Dos Santos, R., Sabourin, P., Maupin. Single and multi-objective genetic algorithms for the selection of ensemble of classifiers. In International Joint Conference on Neural Networks (IJCNN), pages 3070–3077, Vancouver, 2006.
30. Dos Santos, E.M., Sabourin, R., and Maupin, P., 2008. A dynamic overproduce-and-choose strategy for the selection of classifier ensembles. Pattern Recognition, 41 (10):2993–3009.
31. Dietterich, T.G., 2000. An experimental comparison of three methods for constructing ensembles of decision trees:bagging, boosting, and randomization. Machine Learning, 40 (2):139–157.
32. R.E. Banfield, L.O. Hall, K.W. Bowyer, W.P., Kegelmeyer. A comparison of decision tree ensemble creation techniques, IEEE Transactions on Pattern Analysis and Machine Intelligence, 29 1:173–180, 2007.
33. Optiz, D., and Maclin, R., 1999. Popular ensemble methods:an empirical study. Journal of Artificial Intelligence Research, 11:169–198.
34. Breiman, L., 1996. Bagging predictors. Machine Learning, 24 (2):123–140.
35. Schapire, R., 1990. The strength of weak learnability. Machine Learning, 5 (2):197–227.
36. Zhou, Z.H., 2005. Ensembling local learners through multimodal perturbation. IEEE Transactions of Systems, Man, and Cybernetics Part B:Cybernetics, 354:725–735.
37. Xu, L., Krzyzak, A., and Suen, C.Y., 1992. Methods of combining multiple classifiers and their application to handwriting recognition. IEEE Transactions on Systems, Man and Cybernetics, 22 (3):418–435.
38. Breiman, L., 2001. Random forests. Machine Learning, 451:5–32.
39. del Jesus, M.J., Hoffmann, F., Navascues, L.J., and Sánchez, L., 2004. Induction of fuzzy-rule-based classifiers with evolutionary boosting algorithms. IEEE Transactions on Fuzzy Systems, 12 (3):296–308.
40. Sánchez, L., and Otero, J., 2007. Boosting fuzzy rules in classification problems under single-winner inference. International Journal of Intelligent Systems, 229:1021–1034.
41. Takahashi, H., and Honda, H., 2006. Lymphoma prognostication from expression profiling using a combination method of boosting and projective adaptive resonance theory. Journal of Chemical Engineering of Japan, 39 (7):767–771.
42. Janikow, C. Z., 1998. Fuzzy decision trees:issues and methods. IEEE Transactions on Systems, Man, and Cybernetics Part B, 28 (1):1–14.
43. J.J., Aguilera, M., Chica, M.J., del Jesus, and F., Herrera. Niching genetic feature selection algorithms applied to the design of fuzzy rule based classification systems. In IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pages 1794–1799, London UK, 2007.
44. Y., Nojima, H., Ishibuchi Designing fuzzy ensemble classifiers by evolutionary multiobjective optimization with an entropy-based diversity criterion. In International Conference on Hybrid Intelligent Systems and Conference on Neuro-Computing and Evolving Intelligence, Auckland, New Zealand, 2006.
45. Ishibuchi, H., and Nojima, Y., 2006. Evolutionary multiobjective optimization for the design of fuzzy rule-based ensemble classifiers. International Journal of Hybrid Intelligent Systems, 3 (3):129–145.
46. R., Schapire, Y., Freund, P., Bartlett, W., Lee. Boosting the margin:A new explanation for the effectiveness of voting methods. In International Conference on Machine Learning, pages 322–330, Nashville USA, 1997.
47. Dasarathy, B.V., and Sheela, B.V., 1979. A composite classifier system design:Concepts and methodology. Proceedings of IEEE, 67 (5):708–713.
48. Oliveira, L.S., Morita, M., Sabourin, R., and Bortolozzi, F., 2005. Multi-objective genetic algorithms to create ensemble of classifiers. Lecture Notes in Computer Science, 3410:592–606.
49. W. W., Cohen. Fast effective rule induction. In In Proceedings of the Twelfth International Conference on Machine Learning, pages 115–123. Morgan Kaufmann, 1995.
50. Ishibuchi, H., Nakashima, T., and Morisawa, T., 1999. Voting in fuzzy rule-based systems for pattern classification problems. Fuzzy Sets and Systems, 103 (2):223–238.
51. Cordón, O., del Jesus, M.J., and Herrera, F., 1999. A proposal on reasoning methods in fuzzy rule-based classification systems. International Journal of Approximate Reasoning, 20:21–45.
52. P., Panov, S., Džeroski. Combining bagging and random subspaces to create better ensembles. In IDA'07:Proceedings of the 7th international conference on Intelligent data analysis, pages 118–129, Berlin Heidelberg, 2007. Springer-Verlag.
53. Zitzler, E., and Thiele, L., 1999. Multiobjective evolutionary algorithms:a comparative case study and the strength pareto approach. IEEE Transactions on Evolutionary Computation, 3:257–271.
54. Quinlan, J. R., 1993. C4.5:programs for machine learning, San Francisco, CA, USA:Morgan Kaufmann Publishers Inc.
55. Dietterich, T.G., 1998. Approximate statistical test for comparing supervised classification learning algorithms. Neural Computation, 10 (7):1895–1923.
56. Knowles, J., and Corne, D., 2002. “ On metrics for comparing nondominated sets ”. In In 2002 Congress on Evolutionary Computation CEC 02 Vol. 1, 711–716.