On-line identification and reconstruction of finite automata with generalized recurrent neural networks

Neural Networks

Volumn 16, Issue 1, 2003, Pages 101-120

  Gabrijel, Ivan ^a   Dobnikar, Andrej ^a  

^a UNIVERSITY OF LJUBLJANA   (Slovenia)

Author keywords

Finite automata; On line learning; On line rule extraction; Recurrent neural networks; Supervised learning; System identification

Indexed keywords

ALGORITHMS; FINITE AUTOMATA; LEARNING SYSTEMS; ONLINE SYSTEMS;

ON-LINE RULE-EXTRACTION METHOD;

NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; EXTRACTION; LEARNING; MOLECULAR DYNAMICS; PRIORITY JOURNAL; TRAINING;

EID: 0037254330     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(02)00221-6     Document Type: Review

References (52)

1. Angeline P.J., Saunders G.M., Pollack J.B. Complete induction of recurrent neural networks. Proceedings of the Third International Conference on Evolutionary Programming. 1994.
2. Arai K., Nakano R. Stable behavior in a recurrent neural network for a finite state machine. Neural Networks. 13:2000;667-680.
3. Bäck T., Fogel D.B., Michalewicz Z. Evolutionary computation 1 - Basic algorithms and operators. 2000;Institute of Physics Publishing, Bristol and Philadelphia.
4. Basye K., Dean T., Kaelbling L.P. Learning dynamics: System identification for perceptually challenged agents. Artificial Intelligence. 72:(1-2):1995;139-171.
5. Blair A.D., Pollack J.B. Analysis of dynamical recognizers. Neural Computation. 9:(5):1997;1127-1142.
6. Casey M. The dynamics of discrete-time computation, with application to recurrent neural networks and finite state machine extraction. Neural Computation. 8:(6):1996;1135-1178.
7. Casey M. Correction to proof that recurrent neural networks can robustly recognize only regular languages. Neural Computation. 10:1998;1167-1169.
8. Cleeremans A., Servan-Schreiber D., McClelland J.L. Finite state automata and simple recurrent networks. Neural Computation. 1:1989;372-381.
9. Das S., Mozer M. Dynamic on-line clustering and state extraction: An approach to symbolic learning. Neural Networks. 11:(1):1998;53-64.
10. Dobnikar A. Learning feedforward neural networks with synaptic correlations. Electrotechnical Review. 62:(5):1995;295-298.
11. Doya K. Recurrent networks: Supervised learning. Arbib M.A. Handbook of brain theory and neural networks. 1998;796-800 Cambridge, MA, MIT Press.
12. Frasconi P., Gori M., Maggini M., Soda G. Representation of finite state automata in recurrent radial basis function networks. Machine Learning. 23:1996;5-32.
13. Freund Y., Kearns M., Ron D., Rubinfeld R., Schapire R.E., Sellie L. Efficient learning of typical finite automata from random walks. Information and Computation. 138:(1):1997;23-48.
14. Friedman A.D., Menon P.R. Theory and Design of Switching Circuits. 1975;Computer Science Press, Woodland Hills, CA.
15. Gabrijel I. Recurrent neural networks and finite automata, Internal report. 2000;Faculty of Computer and Information Science, Ljubljana, Slovenia.
16. Gabrijel I., Dobnikar A. Adaptive RBF neural network. Pearson D.W. Proceedings of the Second International ICSC Symposium on Soft Computing. 1997;164-170 ICSC Academic Press, Millet, Alberta, Canada.
17. Gabrijel I., Dobnikar A. On-line identification and rule extraction of finite state automata with recurrent neural networks. Kurkova V., Steele N.C., Neruda R., Karny M. Proceedings of the International Conference on Artificial Neural Nets and Genetic Algorithms. 2001;78-81 Springer, Vienna, Austria.
18. Gabrijel I., Dobnikar A., Steele N. RBF neural networks and fuzzy logic based control - A case study. Borne P., Ksouri M., El Kamel A. Proceedings of the Second IMACS International Multiconference on Computational Engineering in Systems Applications - CESA'98. Vol. 2:1998;284-289 Nabeul-Hammamet, Tunisia.
19. Gers F.A., Schmidhuber J. Long short-term memory learns context free and context sensitive languages. Kurkova V., Steele N.C., Neruda R., Karny M. Proceedings of the International Conference on Artificial Neural Nets and Genetic Algorithms. 2001;134-137 Springer, Wien, Austria.
20. Haykin S. Adaptive filter theory. 1986;Prentice-Hall, Englewood Cliffs, NJ.
21. Haykin S. Neural networks - A comprehensive foundation. 2nd ed:1999;Prentice-Hall, Englewood Cliffs, NJ
22. Hochreiter S., Schmidhuber J. Long short-term memory. Neural Computation. 9:(8):1997;1735-1780.
23. Hochreiter S., Schmidhuber J. LSTM can solve hard long time lag problems. Mozer M.C., Jordan M.I., Petsche T. Advances in neural information processing systems. Vol. 9:1997;473-479 Cambridge, MA, MIT Press.
24. Hopcroft J.E., Ullman J.D. Introduction to automata, theory, languages and computation. 1979;Addison-Wesley, Reading, MA.
25. Horne B.G., Giles C.L. An experimental comparison of recurrent neural networks. Tesauro G., Touretzky D., Leen T. Neural information processing systems. Vol. 7:1995;697 MIT Press, Cambridge, MA.
26. Horne B.G., Hush D.R. Bounds on the complexity of recurrent neural network implementations of finite state machines. Neural Networks. 9:(2):1996;243-252.
27. Levin A.U., Narendra K.S. Identification of nonlinear dynamical systems using neural networks. Omidvar O., Elliott D.L. Neural systems for control. 1997;129-160 Academic Press, San Diego, CA.
28. Ljung L. System identification: Theory for the user. 1987;Prentice-Hall, Englewood Cliffs, NJ.
29. Narendra K.S. Adaptive control using neural networks. Miller W.T. III, Sutton R.S., Werbos P.J. Neural networks for control. 1990;115-142 MIT Press, Cambridge, MA.
30. Obradovic D. On-line training of recurrent neural networks with continuous topology adaptation. IEEE Transaction on Neural Networks. 7:(1):1996;222-228.
31. Omlin C.W., Giles C.L. Extraction of rules from discrete-time recurrent neural networks. Neural Networks. 9:(1):1996;41-52.
32. Rabiner L.R., Gold B. Theory and application of digital signal processing. 1975;Prentice-Hall, Englewood Cliffs, NJ.
33. Ron D., Rubinfeld R. Exactly learning automata of small cover time. Machine Learning. 27:(1):1997;69-96.
34. Robinson, A. J., & Fallside, F (1987). The utility driven dynamic error propagation network (Report No. CUED/F-INFENG/TR.1). Cambridge, England: Cambridge University Engineering Department.
35. Shah S., Palmieri F., Datum M. Optimal filtering algorithms for fast learning in feedforward neural networks. Neural Networks. 5:(5):1992;779-787.
36. Siegelmann H.T., Sontag E.D. Turing computability with neural nets. Applied Mathematics Letters. 4:(6):1991;77-80.
37. Sima J., Wiedermann J. Theory of neuromata. Journal of the ACM. 45:1998;155-178.
38. Sontag E.D. Automata and neural networks. Arbib M.A. The handbook of brain theory and neural networks. 1998;119-123 MIT Press, Cambridge, MA.
39. Sperduti A. On the computational power of recurrent neural networks for structures. Neural Networks. 10:(3):1997;395-400.
40. Starke P.H. Abstract automata. 1972;North-Holland Publishing Company, Amsterdam.
41. Su H.T., Samad T. Neuro-control design: Optimization aspects. Omidvar O., Elliott D.L. Neural systems for control. 1997;259-288 Academic Press, San Diego, CA.
42. Sundareshan M.K., Condarcure T.A. Recurrent neural-network training by a learning automaton approach for trajectory learning and control system design. IEEE Transactions on Neural Networks. 9:(3):1998;354-368.
43. Tickle A.B., Andrews A., Golea M., Diederich J. Rule extraction from trained artificial neural networks. Browne A. Neural Network Analysis, Architectures and Applications. 1997;61-99 Institute of Physics Publishing, Bristol.
44. Tomita M. Dynamic construction of finite automata from examples using hill-climbing. Proceedings of the Fourth Annual Cognitive Science Conference. 1982;105-108. Ann Arbor, MI.
45. Touzet C., Giambiasi N. The connectionist sequential machine: A general model of sequential networks. Proceedings on the ACNN'92 Conference. Canberra, Australia. 1992.
46. Trakhtenbrot B.A., Barzdin Y.M. Finite automata - Behavior and synthesis. 1973;North-Holland Publishing Company, Amsterdam.
47. Tsutsumi K. On redundancy in neural architecture: Dynamics of a simple module-based neural network and initial-state independence. Neural Networks. 12:1999;1075-1085.
48. Unnikrishnan K.P., Venugopal K.P. Alopex: A correlation-based algorithm for feedforward and recurrent neural networks. Neural Computation. 6:1994;469-490.
49. Venugopal K.P., Pandya A.S., Sudhakar R. A recurrent neural network controller and learning algorithm for the on-line learning control of autonomous underwater vehicles. Neural Networks. 7:(5):1994;833-846.
50. Werbos P.J. Backpropagation through time: What it does and how to do it. Proceedings of the IEEE. 78:(10):1990;1550-1560.
51. Williams R.J., Zipser D. A learning algorithm for continually running fully recurrent neural networks. Neural Computation. 1:1989;270-280.
52. Williams R.J., Zipser D. Gradient-based learning algorithms for recurrent networks and their computational complexity. Chauvin Y., Rumelhart D.E. Backpropagation: Theory, architectures, and applications. 1995;433-486 Lawrence Erlbaum Associates, Hillsdale, NJ.