Extracting symbolic knowledge from recurrent neural networks-A fuzzy logic approach

Fuzzy Sets and Systems

Volumn 160, Issue 2, 2009, Pages 145-161

  Kolman, Eyal ^a   Margaliot, Michael ^a  

^a TEL AVIV UNIVERSITY   (Israel)

Author keywords

All permutations fuzzy rule base; Formal language; Hybrid intelligent systems; Knowledge extraction; Knowledge based neurocomputing; Neuro fuzzy systems; Recurrent neural networks; Regular grammar; Rule extraction; Rule generation

Indexed keywords

ARTIFICIAL INTELLIGENCE; ELECTRIC FAULT LOCATION; FORMAL LANGUAGES; FUZZY LOGIC; FUZZY NEURAL NETWORKS; FUZZY RULES; FUZZY SETS; FUZZY SYSTEMS; INTELLIGENT SYSTEMS; KNOWLEDGE BASED SYSTEMS; LINGUISTICS; NETWORK PROTOCOLS; NEURAL NETWORKS; REINFORCEMENT LEARNING; SENSOR NETWORKS;

ALL PERMUTATIONS FUZZY RULE-BASE; HYBRID INTELLIGENT SYSTEMS; KNOWLEDGE EXTRACTION; KNOWLEDGE-BASED NEUROCOMPUTING; NEURO-FUZZY SYSTEMS; REGULAR GRAMMAR; RULE EXTRACTION; RULE GENERATION;

RECURRENT NEURAL NETWORKS;

EID: 54549100404     PISSN: 01650114     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.fss.2008.05.005     Document Type: Article

References (67)

1. Andrews R., Diederich J., and Tickle A.B. Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowledge-Based Systems 8 (1995) 373-389
2. Angeline P.J., Saunders G.M., and Pollack J.B. An evolutionary algorithm that constructs recurrent neural networks. Neural Comput. 5 (1994) 54-65
3. Benitez J.M., Castro J.L., and Requena I. Are artificial neural networks black boxes?. IEEE Trans. Neural Networks 8 (1997) 1156-1164
4. Bishop C.M. Neural Networks for Pattern Recognition (1995), Oxford University Press, Oxford, UK
5. Blanco A., Delgado M., and Pegalajar M.C. Fuzzy automaton induction using neural network. Internat. J. Approx. Reason. 27 (2001) 1-26
6. Castro J.L., Mantas C.J., and Benitez J.M. Interpretation of artificial neural networks by means of fuzzy rules. IEEE Trans. Neural Networks 13 (2002) 101-116
7. Chomsky N. Three models for the description of language. IRE Trans. Inform. Theory IT-2 (1956) 113-124
8. Chomsky N., and Schützenberger M.P. The algebraic theory of context-free languages. In: Bradford P., and Hirschberg D. (Eds). Computer Programming and Formal Systems (1963), North-Holland, Amsterdam 118-161
9. I. Cloete, J.M. Zurada (Eds.), Knowledge-Based Neurocomputing, MIT Press, Cambridge, MA, 2000.
10. Dubois D., Nguyen H.T., Prade H., and Sugeno M. Introduction: the real contribution of fuzzy systems. In: Nguyen H.T., and Sugeno M. (Eds). Fuzzy Systems: Modeling and Control (1998), Kluwer, Dordrecht 1-17
11. Frasconi P., Gori M., Maggini M., and Soda G. Representation of finite-state automata in recurrent radial basis function networks. Mach. Learn. 23 (1996) 5-32
12. Fu L.M. Learning capacity and sample complexity on expert networks. IEEE Trans. Neural Networks 7 (1996) 1517-1520
13. Gallant S.I. Connectionist expert systems. Commun. ACM 31 (1988) 152-169
14. Giles C.L., Lawrence S., and Tsoi A.C. Noisy time series prediction using recurrent neural networks and grammatical inference. Mach. Learn. 44 (2001) 161-183
15. Giles C.L., Miller C.B., Chen D., Chen H.H., Sun G.Z., and Lee Y.C. Learning and extracting finite state automata with second-order recurrent neural networks. Neural Comput. 4 (1992) 393-405
16. Gori M., Maggini M., Martinelli E., and Soda G. Inductive inference from noisy examples using the hybrid finite state filter. IEEE Trans. Neural Networks 9 (1998) 571-575
17. Guillaume S. Designing fuzzy inference systems from data: an interpretability-oriented review. IEEE Trans. Fuzzy Systems 9 3 (2001) 426-443
18. Hopcroft J.E., Motwani R., and Ullman J.D. Introduction to Automata Theory, Languages and Computation. second ed. (2001), Addison-Wesley, Reading, MA
19. Ishikawa M. Structural learning and rule discovery. In: Cloete I., and Zurada J.M. (Eds). Knowledge-Based Neurocomputing (2000), Kluwer, Dordrecht 153-206
20. Jacobsson H. Rule extraction from recurrent neural networks: a taxonomy and review. Neural Comput. 17 (2005) 1223-1263
21. Jang J.-S.R., and Sun C.-T. Functional equivalence between radial basis function networks and fuzzy inference systems. IEEE Trans. Neural Networks 4 (1993) 156-159
22. Jang J.-S.R., Sun C.-T., and Mizutani E. Neuro-Fuzzy and Soft Computing: A Computational Approach to Learning and Machine Intelligence (1997), Prentice-Hall, Englewood Cliffs, NJ
23. Jin Y., Von Seelen W., and Sendhoff B. On generating FC3 fuzzy rule systems from data using evolution strategies. IEEE Trans. Systems Man Cybernet. 29 (1999) 829-845
24. Kleene S.C. Representation of events in nerve nets and finite automata. In: Shannon C.E., and McCarthy J. (Eds). Automata Studies vol. 34 (1956), Princeton University Press, Prentice 3-41
25. R. Kohavi, A study of cross-validation and bootstrap for accuracy estimation and model selection, in: Proc. 14th Internat. Joint Conf. on Artificial Intelligence (IJCAI), 1995, pp. 1137-1145.
26. Kolen J.F. Fool's gold: extracting finite state machines from recurrent network dynamics. In: Cowan J.D., Tesauro G., and Alspector J. (Eds). Advances in Neural Information Processing Systems 6 (1994), Morgan Kaufmann, Los Altos, CA 501-508
27. Kolen J.F., and Pollack J.B. The observer's paradox: apparent computational complexity in physical systems. J. Experiment. Theoret. Artificial Intelligence 7 (1995) 253-277
28. Kolman E., and Margaliot M. Are artificial neural networks white boxes?. IEEE Trans. Neural Networks 16 (2005) 844-852
29. Kolman E., and Margaliot M. Knowledge extraction from neural networks using the all-permutations fuzzy rule base: the LED display recognition problem. IEEE Trans. Neural Networks 18 (2007) 925-931
30. E. Kolman, M. Margaliot, A new approach to knowledge-based design of recurrent neural networks, IEEE Trans. Neural Networks, to appear, Available: 〈www.eng.tau.ac.il/~michaelm〉 (Online).
31. V. Kreinovich, C. Langrand, H. T. Nguyen, A statistical analysis for rule base reduction, in: Proc. Second Internat. Conf. on Intelligent Technologies (InTech'2001), Bangkok, Thailand, 2001, pp. 47-52.
32. Kuan C.-M., and Liu T. Forecasting exchange rates using feedforward and recurrent neural networks. J. Appl. Econometrics 10 (1995) 347-364
33. Lee S.-W., and Song H.-H. A new recurrent neural network architecture for visual pattern recognition. IEEE Trans. Neural Networks 8 (1997) 331-340
34. Manolios P., and Fanelly R. First order recurrent neural networks and deterministic finite state automata. Neural Comput. 6 (1994) 1155-1173
35. Mantas C.J., Puche J.M., and Mantas J.M. Extraction of similarity based fuzzy rules from artificial neural networks. Internat J. Approx. Reason. 43 (2006) 202-221
36. Margaliot M., and Langholz G. Hyperbolic optimal control and fuzzy control. IEEE Trans. Systems Man Cybernet. 29 (1999) 1-10
37. Margaliot M., and Langholz G. New Approaches to Fuzzy Modeling and Control-Design and Analysis (2000), World Scientific, Singapore
38. McCulloch W.S., and Pitts W.H. A logical calculus of the ideas imminent in nervous activity. Bull. Math. Biophys. 5 (1943) 115-137
39. L.R. Medsker, L.C. Jain, Recurrent Neural Networks: Design and Applications, CRC Press, Boca Raton, 1999.
40. Miller C.B., and Giles C.L. Experimental comparison of the effect of order in recurrent neural networks. Internat. J. Pattern Recognition Artificial Intelligence 7 (1993) 849-872
41. Mitra S., and Hayashi Y. Neuro-fuzzy rule generation: survey in soft computing framework. IEEE Trans. Neural Networks 11 (2000) 748-768
42. Mitra S., and Pal S. Fuzzy multi-layer perceptron, inferencing and rule generation. IEEE Trans. Neural Networks 6 (1995) 51-63
43. Novak V. Are fuzzy sets a reasonable tool for modeling vague phenomena?. Fuzzy Sets and Systems 156 (2005) 341-348
44. Omlin C.W., and Giles C.L. Extraction and insertion of symbolic information in recurrent neural networks. In: Honavar V., and Uhr L. (Eds). Artificial Intelligence and Neural Networks: Step toward Principled Integration (1994), Academic Press, New York 271-299
45. Omlin C.W., and Giles C.L. Extraction of rules from discrete-time recurrent neural networks. Neural Networks 9 (1996) 41-52
46. Omlin C.W., and Giles C.L. Symbolic knowledge representation in recurrent neural networks: insights from theoretical models of computation. In: Cloete I., and Zurada J.M. (Eds). Knowledge-Based Neurocomputing (2000), Kluwer, Dordrecht 63-115
47. Omlin C.W., Thornber K.K., and Giles C.L. Fuzzy finite-state automata can be deterministically encoded into recurrent neural networks. IEEE Trans. Fuzzy Systems 6 (1998) 76-89
48. Pal S.K., and Mitra S. Multilayer perceptron, fuzzy sets, and classification. IEEE Trans. Neural Networks 3 (1992) 683-697
49. Pedrycz W., and Rocha A.F. Fuzzy-set based models of neurons and knowledge-based networks. IEEE Trans. Fuzzy Systems 1 (1993) 254-266
50. Robinson T., Hochberg M., and Renals S. The use of recurrent networks in continuous speech recognition. In: Lee C.H., Soong F.K., and Paliwal K.K. (Eds). Automatic Speech and Speaker Recognition: Advanced Topics (1996), Kluwer Academic, Dordrecht 233-258
51. Rodriguez P. Simple recurrent networks learn context-free and context-sensitive languages by counting. Neural Comput. 13 (2001) 2093-2118
52. Rodriguez P., Wiles J., and Elman J.L. A recurrent neural network that learns to count. Connection Sci. 11 (1999) 5-40
53. Samuel A.L. Some studies in machine learning using the game of checkers. IBM J. Res. Development 3 (1959) 211-229
54. S. Snyders, C.W. Omlin, What inductive bias gives good neural network training performance?, in: Proc. Internat. Joint Conf. on Neural Networks (IJCNN'00), Como, Italy, 2000, pp. 445-450.
55. Sousa J.W.C., and Kaymak U. Fuzzy Decision Making in Modeling and Control (2002), World Scientific, Singapore
56. Tickle A.B., Andrews R., Golea M., and Diederich J. The truth will come to light: directions and challenges in extracting the knowledge embedded within trained artificial neural networks. IEEE Trans. Neural Networks 9 (1998) 1057-1068
57. Tiňo P., and Köteles M. Extracting finite-state representation from recurrent neural networks trained on chaotic symbolic sequences. Neural Comput. 10 (1999) 284-302
58. Tiňo P., and Vojtec V. Extracting stochastic machines from recurrent neural networks trained on complex symbolic sequences. Neural Network World 8 (1998) 517-530
59. M. Tomita, Dynamic construction of finite-state automata from examples using hill-climbing, in: Proc. Fourth Annual Conf. on Cognitive Science, 1982, pp. 105-108.
60. G. Towell, J. Shavlik, M. Noordewier, Refinement of approximate domain theories by knowledge based neural network, in: Proc. Eighth National Conf. on Artificial Intelligence, 1990, pp. 861-866.
61. Tron E., and Margaliot M. Mathematical modeling of observed natural behavior: a fuzzy logic approach. Fuzzy Sets and Systems 146 (2004) 437-450
62. Watrous R.L., and Kuhn G.M. Induction of finite-state languages using second-order recurrent networks. Neural Comput. 4 (1992) 406-414
63. Williams R.J., and Zipser D. A learning algorithm for continually running fully recurrent neural networks. Neural Comput. 1 (1989) 270-280
64. Zadeh L.A. Fuzzy sets. Information and Control 8 (1965) 338-353
65. Zadeh L.A. The concept of a linguistic variable and its application to approximate reasoning. Inform. Sci. 30 (1975) 199-249
66. Zadeh L.A. Fuzzy logic=computing with words. IEEE Trans. Fuzzy Systems 4 (1996) 103-111
67. Zeng Z., Goodman R.M., and Smyth P. Learning finite state machines with self-clustering recurrent networks. Neural Comput. 5 (1993) 976-990