Extracting rules from multilayer perceptrons in classification problems: A clustering-based approach

Neurocomputing

Volumn 70, Issue 1-3, 2006, Pages 384-397

  Hruschka, Eduardo R ^a   Ebecken, Nelson F F ^b  

^a CATHOLIC UNIVERSITY OF SANTOS   (Brazil)

^b FEDERAL UNIVERSITY OF RIO DE JANEIRO   (Brazil)

Author keywords

Clustering; Genetic algorithms; Rule extraction from neural networks

Indexed keywords

DATA MINING; GENETIC ALGORITHMS; KNOWLEDGE BASED SYSTEMS; MAPPING; PATTERN RECOGNITION;

CLUSTERING-BASED APPROACH; RULE EXTRACTION; VECTOR MAPPINGS;

MULTILAYER NEURAL NETWORKS;

ACCURACY; ARTICLE; ARTIFICIAL NEURAL NETWORK; BREAST CANCER; CLUSTER ANALYSIS; CONTROLLED STUDY; DIABETES MELLITUS; GENETIC ALGORITHM; GENOTYPE; INFORMATION PROCESSING; IRIDACEAE; MATHEMATICAL COMPUTING; METEOROLOGY; PRIORITY JOURNAL; PROBLEM SOLVING; PROGENY;

EID: 33750967077     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2005.12.127     Document Type: Article

References (70)

1. Alexander J.A., and Mozer M.C. Template-based algorithm for connectionist rule extraction. In: Tesauro G., Touretzky D., and Leen T. (Eds). Advances in Neural Information Processing Systems vol. 7 (1995), MIT Press, Cambridge, MA
2. Andrews R., Diederich J., Tickle A.B., and Survey A. Critique of techniques for extracting rules from trained artificial neural networks. Knowledge Based Sys. 8 6 (1995) 373-389
3. Arabie P., and Hubert L.J. An Overview of Combinatorial Data Analysis. In: Arabie P., Hubert L.J., and DeSoete G. (Eds). Clustering and Classification (1999), World Scientific, Singapore (Chapter 1)
4. R. Baron, Knowledge extraction from neural networks: A survey, in: Report no. 94-17, Laboratoire de l'Informatique du Parallélisme, Ecole Normale Supérieure de Lyon, 1994.
5. Benítez J.M., Castro J.L., and Requena I. Are artificial neural networks black boxes?. IEEE Trans. Neural Networks 8 5 (1997) 1156-1164
6. M.W. Craven, J.W. Shavlik, Extracting tree-structured representations of trained networks, Advances in Neural Information Processing Systems, vol. 8, MIT Press, Cambridge, MA, 1996, pp. 24-30.
7. M.W. Craven, J.W. Shavlik, Using sampling and queries to extract rules from trained neural networks, in: Proceedings of the 11th International Conference on Machine Learning, San Francisco CA, 1994.
8. Das S., and Mozer M. Dynamic on-line clustering and state extraction: an approach to symbolic learning. Neural Networks 11 1 (1998) 53-64
9. Duch W., Adamczak R., and Grabczewski K. Extraction of logical rules from training data using backpropagation networks. Neural Process. Lett. 7 (1998) 1-9
10. W. Duch, R. Adamczak, K. Grabczewski, M. Ishikawa, H. Ueda, Extraction of crisp logical rules using constrained backpropagation networks, in: Proceedings of the European Symposium on Artificial Neural Networks, Bruge, 1997.
11. W. Duch, R. Adamczak, K. Grabczewski, Optimization of Logical Rules Derived by Neural Procedures, in: Proceedings of the International Joint Conference on Neural Networks, Washington. DC, July, 1999.
12. Duch W., Adamczak R., Grabczewski K., and Zal G. Hybrid neural-global minimization method of logical rule extraction. J. Adv. Comput. Intell. 3 (1999) 348-356
13. Duch W., Adamczak R., and Grabczwski K. A new methodology of extraction. optimization and application of crisp and fuzzy logical rules, IEEE Trans. Neural Networks 11 2 (2000) 1-31
14. Everitt B.S., Landau S., and Leese M. Cluster Analysis (2001), Arnold Publishers, London
15. Falkenauer E. Genetic Algorithms and Grouping Problems (1998), Wiley, New York
16. Fu L. Rule generation from neural networks. IEEE Trans. Syst. Man Cybern 24 8 (1994) 1114-1124
17. Fu L. Neural Networks in Computer Intelligence. first ed (1994), McGraw-Hill, New York, USA
18. Fu L. Knowledge-based connectionism for revising domain theories. IEEE Trans. Syst. Man Cybern. 23 1 (1993) 173-182
19. Gallant S.I. Connectionist expert systems. Commun. ACM 31 2 (1988) 152-169
20. Gallant S.I. Neural Network Learning and Expert Systems (1994), MIT Press, Cambridge, MA
21. Garcez A.S.D., Broda K., and Gabbay D.M. Symbolic knowledge extraction from trained neural networks: a sound approach. Artif. Intell. 125 (2001) 155-207
22. Goldberg D.E. Genetic Algorithms in Search, Optimization and Machine Learning (1989), Addison Wesley, Longman
23. M. Golea, On the complexity of rule-extraction from neural networks and network-querying, in: Proceedings of the Rule Extraction from Trained Artificial Neural Networks Workshop, University of Sussex, Brighton, UK, 1996, pp. 51-59.
24. Hayashi Y., Setiono R., and Yoshida K. A comparison between two neural network rule extraction techniques for the diagnosis of hepatobiliary disorders. Artif. Intell. Med. 20 (2000) 205-216
25. Haykin S.S. Neural Networks: A Comprehensive Foundation (1998), Prentice Hall, Englwood Eliffs, NJ
26. C. Hermann, A. Their, Backpropagation for neural DNF- and CNF-networks, Technical Report, FG Intellektik, TH Darmstadt, Germany, 1996.
27. E.R. Hruschka, N.F.F. Ebecken, A clustering genetic algorithm for extracting rules from multilayer perceptrons trained in classification problems. In: A. Zanasi, et al. (ed.), Data Mining, vol. III, Southampton,UK, 2002. pp. 451-460.
28. Hruschka E.R., and Ebecken N.F.F. A clustering genetic algorithm for extracting rules from supervised neural network models in data mining tasks. Int. J. Comput. Syst. Signals (IJCSS) 1 1 (2000) 17-29 (Special Issue: Knowledge Discovery from Structured and Unstructured Data)
29. E.R. Hruschka, N.F.F. Ebecken, Applying a clustering genetic algorithm for extracting rules from a supervised neural network, in: Proceedings of the IEEE International Joint Conference on Neural Networks, Como, Italy, 2000.
30. E.R. Hruschka, N.F.F. Ebecken, Rule extraction from neural networks: modified RX algorithm, in: Proceedings of the IEEE International Joint Conference on Neural Networks, Washington DC, USA, 1999.
31. E.R. Hruschka, N.F.F. Ebecken, Using a clustering genetic algorithm for rule extraction from artificial neural networks, in: Proceedings of the First IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks, San Antonio, TX, USA, 2000.
32. Hruschka E.R., and Ebecken N.F.F. A genetic algorithm for cluster analysis. Intell. Data Anal. 7 1 (2003) 15-25
33. E.R. Hruschka, N.F.F. Ebecken, Rules from supervised neural network in data mining tasks. In: J.M. Abbe, J.I. da Silva Filho, (eds.), Frontiers in Artificial Intelligence and Applications, vol. 71, Amsterdam, 2001, pp. 84-100.
34. Y. Jiang, Z. Zhou, Z. Chen, Z., Rule learning based on neural network ensemble, in: Proceedings of the International Joint Conference on Neural Networks, Honolulu, 2002, pp. 1416-1420.
35. R. Kane, M. Milgram, Extraction of semantic rules from trained neural networks, in: Proceedings of the ICNN, San Francisco, CA, 1993.
36. L. Kaufman, P. J. Rousseeuw, Finding Groups in Data-An Introduction to Cluster Analysis, Wiley Series in Probability and Mathematical Statistics, 1990.
37. Keedwell E., Narayanan A., and Savic D. Creating rules from trained neural networks using genetic algorithms. Int. J. Comput. Syst. Signals (IJCSS), 1 1 (2000) 30-42
38. B. Liu, W. Hsu, Ma Yiming, Pruning and summarizing the discovered associations, in: Proceedings of the ACM International Conference on Knowledge Discovery & Data Mining, San Diego, CA, USA, 1999, pp. 125-134.
39. B. Liu, W. Hsu, S. Chen, Ma Yiming, Analyzing the Subjective Interestingness of Association Rules, IEEE Intell. Syst. (2000) 47-55.
40. Liu G.L. Introduction to Combinatorial Mathematics (1968), McGraw Hill, New York
41. Lu H., Setiono R., and Liu H. Effective data mining using neural networks. IEEE Trans Knowledge Data Eng 8 6 (1996) 957-961
42. T.B. Ludermir, Extracting Rules from Feedforward Boolean Neural Networks, in: Proceedings of the V Brazilian Symposium on Neural Networks, 1998, pp. 61-65.
43. McMillan C., Mozer M.C., and Smolenski P. Rule Induction through Integrated Symbolic and Subsymbolic Processing. In: Moody J.E., Hanson S.J., and Lippmann R.P. (Eds). Advances in Neural Processing Systems vol. IV (1992), Morgan Kaufmann Publishers, Los Allos 969-976
44. C. J. Merz, P. M. Murphy, UCI repository of machine learning databases, Department of Information and Computer Science, University of California, Irvine. www.ics.uci.edu.
45. Nakhaeizadeh G., Kressel U., Keh S., et al. Dataset descriptions and results. In: Michie D., Spiegelhalter D.J., and Taylor C.C. (Eds). Machine Learning, Neural and Statistical Classification, Ellis Horwood Series in Artificial Intelligence (1994), Bookcraft, Midsomer Norton 131-174
46. H. Narazaki, I. Shigaki, T. Watanabe, A Method for Extracting Approximate Rules from Neural Network, in: Proceedings of the Fourth IEEE International Conference on Fuzzy Systems and Second International Fuzzy Engineering Symposium, Yokohama, Japan, 1995.
47. Narazaki H., Watanabe T., and Yamamoto M. Re-organizing knowledge in neural networks: an explanatory mechanism for neural networks in data classification problems. IEEE Trans. Syst. Man Cybern. 26 1 (1996) 107-117
48. Omlin C.W., and Giles C.L. Rule revision with recurrent neural networks. IEEE Trans Knowledge Data Eng 8 1 (1996) 183-188
49. V. Palade, S. Bumbaru, M.G. Negoita, A method for compiling neural networks into fuzzy rules using genetic algorithms and hierarchical approach, in: Proceedings of the Second IEEE International Conference on Knowledge-Based Intelligent Electronic Systems 2 (1998), pp. 353-358.
50. Palade V., Neagu D., and Patton R.J. Interpretation of trained neural networks by rule extraction. In: Reusch B. (Ed). Lecture Notes in Computer Science (LNCS 2206), Computational Intelligence: Theory and Applications (2001), Springer, Berlin 152-161
51. V. Palade, D. Neagu, G. Puscasu, Rule extraction from neural networks by interval propagation, in: Proceedings of the Fourthh IEEE International Conference on Knowledge-Based Intelligent Engineering Systems, Brighton-UK, 2000, pp. 217-220.
52. V. Palade, M. G. Negoita, V. Ariton, Genetic algorithm optimization of knowledge extraction from neural networks, in: Proceedings of the Sixth IEEE International Conference on Neural Information Processing (ICONIP99), vol. 2, Perth, Australia, 1999, pp. 752-758.
53. Y. Park, M. Song, A genetic algorithm for clustering problems, in: Proceedings of the Genetic Programming Conference, University of Wisconsin, July, 1998.
54. Sethi I., and Yoo J. Multi-valued logic mapping of neurons in feed-forward networks. Eng Intell. Syst. 4 4 (1996) 243-253
55. Sethi I., and Yoo J.H. Symbolic approximation of feedforward neural networks. In: Gelsema E.S., and Kanal L.N. (Eds). Pattern Recognition in Practice vol. IV (1994), North-Holland, Amsterdam 313-324
56. Setiono R. Extracting rules from neural networks by pruning and hidden-unit splitting. Neural Computation 9 1 (1997) 205-225
57. Setiono R., and Leow K. FERNN: an algorithm for fast extraction of rules from neural networks. Appl. Intell. 12 1-2 (2000) 15-25
58. Setiono R., Leow W.K., and Zuarada J.M. Extraction of rules from artificial neural networks for nonlinear regression. IEEE Trans. Neural Networks 13 3 (2002) 564-577
59. Setiono R., and Liu H. Neurolinear: from neural networks to oblique decision rules. Neurocomputing 16 3 (1997) 225-235
60. Setiono R., and Liu H. Symbolic representation of neural networks. IEEE Comput 29 3 (1996) 71-77
61. J. Shavlik, An overview of research at wisconsin on knowledge-based neural networks, in: Proceedings of the International Conference on Neural Networks, Washington, DC, 1996, pp. 65-69.
62. Smith M. Neural networks for statistical modeling. first ed (1996), International Thomson Computer Press, USA
63. S. Snyders, C. Omlin, Rule Extraction from Knowledge-Based Neural Networks with Adaptive Inductive Bias, in: Proceedings of the Eighth International Conference on Neural Information Processing (ICONIP), vol. 1, 2001, pp. 143-148.
64. Taha I., and Ghosh J. Symbolic interpretation of artificial neural networks. IEEE Trans. Knowledge Data Eng. 11 3 (1999) 448-463
65. I. Taha, J. Ghosh, Three techniques for extracting rules from feed-forward networks, Intelligent Engineering Systems Through Artificial Neural Networks, vol. 6, ASME Press, New York, 1996.
66. I. Taha, J. Ghosh, Evaluation and ordering of rules extracted from feedforward networks, in: Proceedings of the IEEE International Conference on Neural Networks, 1997, pp. 221-226.
67. S. Thrun, Extracting provably correct rules from artificial neural networks, Technical Report IAI-93-5, Institut für Informatik III, Universität Bonn, 1993.
68. Towell G.G., and Shavlik J. Extracting refined rules from knowledge-based neural networks. Mach. Learn. 13 (1993) 71-101
69. A. Vahed, C.W. Omlin, Rule extraction from recurrent neural networks using a symbolic machine learning algorithm, in: Proceedings of the Sixth International Conference on Neural Information Processing, Perth, Australia, 1999.
70. T. Weijters, A.V.D. Bosh, J.V.D. Herik, Intelligible neural networks with BP_SOM, in: Proceedings of the Ninth Dutch Conference on Artificial Intelligence, 1997, pp. 27-36.