On designing fuzzy rule-based multiclassification systems by combining FURIA with bagging and feature selection

International Journal of Uncertainty, Fuzziness and Knowldege-Based Systems

Volumn 19, Issue 4, 2011, Pages 589-603

  Trawiski, Krzysztof ^a,b   Cordón, Oscar ^a,b   Quirin, Arnaud ^a,b  

^a EUROPEAN CENTRE FOR SOFT COMPUTING   (Spain)

^b Calle Gonzalo Gutiérrez Quirós S N   (Spain)

Author keywords

bagging; classifier ensembles; feature selection; FURIA; fuzzy rule based classification systems; fuzzy rule based multiclassification systems; MIFS; Multiclassification systems

Indexed keywords

CLASSIFICATION (OF INFORMATION); CLUSTERING ALGORITHMS; DECISION TREES; FUZZY INFERENCE; FUZZY RULES;

BAGGING; CLASSIFIERS ENSEMBLE; FEATURES SELECTION; FUZZY RULE BASED; FUZZY RULE BASED CLASSIFICATION SYSTEMS; FUZZY RULE-BASED MULTICLASSIFICATION SYSTEM; FUZZY UNORDERED RULE INDUCTION ALGORITHM; MIFS; MULTI-CLASSIFICATION; MULTICLASSIFICATION SYSTEM; RULE INDUCTION ALGORITHMS;

FEATURE SELECTION;

EID: 79960017575     PISSN: 02184885     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0218488511007155     Document Type: Article

References (54)

1. J. J. Aguilera, M. Chica, M. J. del Jesus and F. Herrera, Niching genetic feature selection algorithms applied to the design of fuzzy rule based classification systems, in IEEE Int. Conf. on Fuzzy Systems (FUZZ-IEEE), London (UK), 2007, pp. 1794-1799.
2. R. E. Banfield, L. O. Hall, K. W. Bowyer and W. P. Kegelmeyer, A comparison of decision tree ensemble creation techniques, IEEE Trans. Pattern Analysis and Machine Intelligence 29(1) (2007) 173-180. (Pubitemid 46415955)
3. R. Battiti, Using mutual information for selecting features in supervised neural net learning, IEEE Trans. Neural Networks 5(4) (1994) 537-550.
4. P. P. Bonissone, J. M. Cadenas, M. C. Garrido and R. A. Díaz-Valladares, A fuzzy random forest, International Journal of Approximate Reasoning 51(7) (2010) 729-747.
5. P. P. Bonissone, J. M. Cadenas, M. C. Garrido and R. A. Díaz-Valladares, A fuzzy random forest: Fundamental for design and construction, in Proc. 12th Int. Conf. Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU'08), Málaga (Spain), 2008, pp. 1231-1238.
6. A. P. Bradley, The use of the area under the ROC curve in the evaluation of machine learning algorithms, Pattern Recognition 30 (1997) 1145-1159. (Pubitemid 127406521)
7. L. Breiman, Bagging predictors, Machine Learning 24(2) (1996) 123-140. (Pubitemid 126724382)
8. L. Breiman, Random forests, Machine Learning 45(1) (2001) 5-32. (Pubitemid 32933532)
9. J. Canul-Reich, L. Shoemaker, and L.O. Hall, Ensembles of fuzzy classifiers, in IEEE Int. Conf. Fuzzy Systems (FUZZ-IEEE), London (UK), 2007, pp. 1-6.
10. W. W. Cohen, Fast effective rule induction, in Proc. Twelfth Int. Conf. Machine Learning (Morgan Kaufmann, 1995), pp. 115-123.
11. O. Cordón, M. J. del Jesus and F. Herrera, A proposal on reasoning methods in fuzzy rule-based classification systems, Int. J. Approximate Reasoning 20 (1999) 21-45.
12. O. Cordón and A. Quirin, Comparing two genetic overproduce-And-choose strategies for fuzzy rule-based multiclassification systems generated by bagging and mutual information-based feature selection, Int. J. Hybrid Intelligent Systems 7(1) (2010) 45-64.
13. O. Cordón, A. Quirin and L. Sánchez, A first study on bagging fuzzy rule-based classification systems with multicriteria genetic selection of the component classifiers, in Third Int. Workshop on Genetic and Evolving Fuzzy Systems (GEFS), Witten-Bommerholz (Germany), 2008, pp. 11-16.
14. O. Cordón, A. Quirin and L. Sánchez, On the use of bagging, mutual informationbased feature selection and multicriteria genetic algorithms to design fuzzy rule-based classification ensembles, in Int. Conf. Hybrid Intelligent Systems (HIS), Barcelona (Spain), 2008, pp. 549-554.
15. M. J. del Jesus, F. Hoffmann, L. J. Navascues and L. Sánchez, Induction of fuzzy-rulebased classifiers with evolutionary boosting algorithms, IEEE Trans. Fuzzy Systems 12(3) (2004) 296-308.
16. T. G. Dietterich, Approximate statistical test for comparing supervised classification learning algorithms, Neural Computation 10(7) (1998) 1895-1923. (Pubitemid 128463689)
17. T. G. Dietterich, An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization, Machine Learning 40(2) (2000) 139-157.
18. T. A. Feo and M. G. C. Resende, Greedy randomized adaptive search procedures, J. Global Optimization 6 (1995) 109-133.
19. J. Gentle, W. Härdle and Y. Mori, Handbook of computational statistics: Concepts and methods, Metrika 67(2) (2008) 245-246.
20. T. Ho, The random subspace method for constructing decision forests, IEEE Trans. Pattern Analysis and Machine Intelligence 20(8) (1998) 832-844. (Pubitemid 128741345)
21. F. Hoffmann, Combining boosting and evolutionary algorithms for learning of fuzzy classification rules, Fuzzy Sets and Systems 141(1) (2004) 47-58.
22. J. Huang and C. X. Ling, Using AUC and accuracy in evaluating learning algorithms, IEEE Trans. Knowledge and Data Engineering 17 (2005) 299-310. (Pubitemid 40320164)
23. J. C. Hühn and E. Hüllermeier, FURIA: An algorithm for unordered fuzzy rule induction, Data Mining and Knowledge Discovery 19(3) (2009) 293-319.
24. J. C. Hühn and E. Hüllermeier, An analysis of the FURIA algorithm for fuzzy rule induction, in Advances in Machine Learning I, 2010, pp. 321-344.
25. H. Ishibuchi, T. Nakashima and T. Morisawa, Voting in fuzzy rule-based systems for pattern classification problems, Fuzzy Sets and Systems 103(2) (1999) 223-238. (Pubitemid 129502321)
26. H. Ishibuchi, T. Nakashima, and M. Nii, Classification and Modeling with Linguistic Information Granules: Advanced Approaches to Linguistic Data Mining (Advanced Information Processing) (Springer-Verlag, New York, Inc., Secaucus, NJ, USA, 2004
27. H. Ishibuchi and Y. Nojima, Evolutionary multiobjective optimization for the design of fuzzy rule-based ensemble classifiers, Int. J. Hybrid Intelligent Systems 3(3) (2006) 129-145.
28. C. Z. Janikow, Fuzzy decision trees: Issues and methods, IEEE Transactions on Systems, Man, and Cybernetics, Part B 28(1) (1998) 1-14. (Pubitemid 128747981)
29. L. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms (Wiley, 2004
30. C. Marsala, Data mining with ensembles of fuzzy decision trees, in IEEE Symp. Computational Intelligence and Data Mining, Nashville (USA), 2009, pp. 348-354.
31. Y. Nojima and H. Ishibuchi, Designing fuzzy ensemble classifiers by evolutionary multiobjective optimization with an entropy-based diversity criterion, in Int. Conf. Hybrid Intelligent Systems and Conference on Neuro-Computing and Evolving Intelligence, Auckland, New Zealand, 2006.
32. Y. Nojima and H. Ishibuchi, Genetic rule selection with a multi-classifier coding scheme for ensemble classifier design, Int. J. Hybrid Intelligent Systems 4(3) (2007) 157-169.
33. D. Optiz and R. Maclin, Popular ensemble methods: An empirical study, J. Artificial Intelligence Res. 11 (1999) 169-198. (Pubitemid 129628763)
34. P. Panov and S. Džeroski, Combining bagging and random subspaces to create better ensembles, in IDA'07: Proc. 7th Int. Conf. Intelligent Data Analysis, Springer-Verlag, Berlin, Heidelberg, 2007, pp. 118-129.
35. D. Partridge and W. B. Yates, Engineering multiversion neural-net systems, Neural Computation 8(4) (1996) 869-893. (Pubitemid 126449950)
36. W. Pedrycz, A. Breuer and N. J. Pizzi, Fuzzy adaptive logic networks as hybrid models of quantitative software engineering, Intelligent Automation and Soft Computing 12(2) (2006) 189-209. (Pubitemid 43501652)
37. W. Pedrycz and K.C. Kwak, Boosting of granular models, Fuzzy Sets and Systems 157(22) (2006) 2934-2953. (Pubitemid 44528754)
38. J. R. Quinlan, C4.5: Programs for Machine Learning (Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1993
39. L. Sánchez and J. Otero, Boosting fuzzy rules in classification problems under singlewinner inference, Int. J. Intelligent Systems 22(9) (2007) 1021-1034. (Pubitemid 47359495)
40. E. M. Dos Santos, R. Sabourin and P. Maupin, A dynamic overproduce-And-choose strategy for the selection of classifier ensembles, Pattern Recognition 41(10) (2008) 2993-3009.
41. R. Schapire, The strength of weak learnability, Machine Learning 5(2) (1990) 197-227.
42. R. Scherer, Boosting ensemble of relational neuro-fuzzy systems, Lecture Notes in Computer Science 4029 (2006) 306-313. (Pubitemid 44089819)
43. C. E. Shannon and W. Weaver, The Mathematical Theory of Communication (University of Illlinois Press, 1949
44. J. Stefanowski, An experimental study of methods combining multiple classifiers -diversified both by feature selection and bootstrap sampling, in K. T. Atanassov, J. Kacprzyk, M. Krawczak, and E. Szmidt (eds.), Issues in the Representation and Processing of Uncertain and Imprecise Information, Akademicka OficynaWydawnicza EXIT, Warsaw, 2005, pp. 337-354.
45. H. Takahashi and H. Honda, A new reliable cancer diagnosis method using boosted fuzzy classifier with a SWEEP operator method, J. Chem. Eng. Jpn. 38(9) (2005) 763-773. (Pubitemid 41535253)
46. H. Takahashi and H. Honda, Lymphoma prognostication from expression profiling using a combination method of boosting and projective adaptive resonance theory, J. Chem. Eng. Jpn. 39(7) (2006) 767-771. (Pubitemid 44111218)
47. K. Trawiński, A. Quirin and O. Cordón, A study on the use of multi-objective genetic algorithms for classifier selection in furia-based fuzzy multiclassifers, Int. J. Compu-tational Intelligence Systems, in press.
48. K. Trawiński, A. Quirin and O. Cordón, Bi-criteria genetic selection of bagging fuzzy rule-based multiclassification systems, in IFSA World Congress-EUSFLAT Conference (IFSA/EUSFLAT), Lisbon (Portugal), 2009, pp. 1514-1519.
49. K. Trawiński, A. Quirin and O. Cordón, On the combination of accuracy and diversity measures for genetic selection of bagging fuzzy rule-based multiclassification systems, in Int. Conf. Intelligent Systems Design and Applications (ISDA), Pisa (Italy), IEEE Computer Society, 2009, pp. 121-127.
50. C.-A. Tsai, T.-C. Lee, I.-C. Ho, U.-C. Yang, C.-H. Chen and J. J. Chen, Multi-class clustering and prediction in the analysis of microarray data, Math. Biosci. 193(1) (2005) 79-100. (Pubitemid 40175025)
51. A. Tsymbal, M. Pechenizkiy and P. Cunningham, Diversity in search strategies for ensemble feature selection, Information Fusion 6(1) (2005) 83-98.
52. S. J. Verzi, G. L. Heileman and M. Georgiopoulos, Boosted ARTMAP: Modifications to fuzzy ARTMAP motivated by boosting theory, Neural Networks 19(4) (2006) 446-468. (Pubitemid 43850232)
53. L. Xu, A. Krzyzak and C. Y. Suen, Methods of combining multiple classifiers and their application to handwriting recognition, IEEE Transactions on Systems, Man, and Cybernetics 22(3) (1992) 418-435.
54. Z. H. Zhou, Ensembling local learners through multimodal perturbation. IEEE Trans. Systems, Man, and Cybernetics, Part B: Cybernetics 35(4) (2005) 725-735.