Stable behavior in a recurrent neural network for a finite state machine

Neural Networks

Volumn 13, Issue 6, 2000, Pages 667-680

  Arai, K ^a   Nakano, R ^a  

^a Nippon Telegraph and Telephone Corporation   (Japan)

Author keywords

Bayesian learning; Finite state machine; Internal representation; Recurrent neural network; Stability

Indexed keywords

ASYMPTOTIC STABILITY; COMPUTER SIMULATION; FINITE AUTOMATA; FUNCTIONS; LEARNING SYSTEMS; PROBABILITY;

BAYESIAN LEARNING; FINITE STATE MACHINE; INTERNAL REPRESENTATION;

RECURRENT NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; BAYES THEOREM; CLUSTER ANALYSIS; FINITE ELEMENT ANALYSIS; LEARNING; MATHEMATICAL ANALYSIS; PRIORITY JOURNAL;

ARTIFICIAL INTELLIGENCE; BAYES THEOREM; COMPUTER SIMULATION; NEURAL NETWORKS (COMPUTER);

EID: 0034233325     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(00)00037-X     Document Type: Article

References (33)

1. Alquezar R., Sanfeliu A. An algebraic framework to represent finite state machines in single-layer recurrent neural networks. Neural Computation. 7:1995;931-949.
2. Arai, K., & Nakano, R., (1996). Annealed RNN learning of finite state automata. In Proceedings of ICANN (pp. 519-524).
3. Arai, K., & Nakano, R., (1996). Adaptive β scheduling learning method of finite state automata by recurrent neural networks. In Proceedings of ICONIP, Vol. 1 (pp. 351-354).
4. Bengio Y., Simard P., Frasconi P. Learning long-term dependencies with gradient descent is difficult. IEEE Neural Networks. 5:(2):1994;157-166.
5. Bishop C.M. Neural networks for pattern recognition. 1995;385-439 Oxford University Press, New York.
6. Blair A.D., Pollack J.B. Analysis of dynamical recognizers. Neural Computation. 9:1997;1127-1142.
7. Carrasco R.C., Forcada M.L., Neco R.P. Stable encoding of finite-state machines in discrete neural nets with sigmoidal units. Neural Computation. 2000;. (in press).
8. Casey M. The dynamics of discrete-time computation, with application to recurrent neural networks and finite state machine extraction. Neural Computation. 8:1996;1135-1178.
9. Cleeremans A., Servan-Schreiber D., McClelland J. Finite state automata and simple recurrent neural networks. Neural Computation. 1:(3):1989;372-381.
10. Das S., Mozer M.C. A unified gradient-descent/clustering architecture for finite state machine induction. Advances in Neural Information Processing Systems. 6:1994;19-26.
11. Das S., Mozer M. Dynamic on-line clustering and state extraction: an approach to symbolic learning. Neural Networks. 11:(1):1998;53-64.
12. Elman J.L. Finding structure in time. Cognitive Science. 14:1990;179-211.
13. Forcada M.L., Carrrasco R.C. Learning the initial state of a second-order recurrent neural network during regular-language inference. Neural Computation. 7:(5):1995;923-930.
14. Frasconi P., Gori M., Maggini M., Soda G. Representation of finite state automata in recurrent radial basis function networks. Machine Learning. 23:1996;5-32.
15. Giles C.L., Sun G.Z., Chen H.H., Lee Y.C., Chen D. Higher order recurrent neural networks and grammatical inference. Advances in Neural Information Processing Systems. 2:1990;380-386.
16. Giles C.L., Miller C.B., Chen D., Chen H.H., Sun G.Z., Lee Y.C. Learning and extracting finite state automata with second-order recurrent neural networks. Neural Computation. 4:1992;393-405.
17. Gill P.E., Murray W., Wright M.H. Practical optimization. 1981;83-154 Academic Press, London.
18. Goudreau M.W., Giles C.L., Chakradhar S.T., Chen D. First-order versus second-order single-layer recurrent neural networks. IEEE Neural Networks. 5:(3):1994;511-513.
19. Hertz J., Krogh A., Palmer R.G. Introduction to the theory of neural computation. 1991;415-447 Addison-Wesley, Reading, MA.
20. Hinton, G. E., (1987). Learning translation invariant recognition in massively parallel networks. In Proceedings of PARLE Conference on Parallel Architecture and Languages, Europe (pp. 1-13).
21. Kolen J.R. Fool's gold: extracting finite state machines from recurrent neural network dynamics. Advances in Neural Information Processing Systems. 6:1994;508-515.
22. MacKay D.J.C. Bayesian interpolation. Neural Computation. 4:1992;415-447.
23. Manolis P., Fenelli R. First-order recurrent neural networks and deterministic finite state automata. Neural Computation. 6:1994;1155-1173.
24. Minsky M.L. Computation: finite and infinite machine. 1967;32-66 Prentice Hall, Englewood Cliffs, NJ.
25. Omlin C.W., Giles C.L. Stable encoding of large finite-state automata in recurrent neural networks with sigmoidal discriminant. Neural Computation. 8:1996;675-696.
26. Omlin C.W., Giles C.L. Constructing deterministic finite-state automata in recurrent neural networks. Journal of the ACM. 43:(6):1996;937-972.
27. Pallack J.B. The induction of dynamical recognizers. Machine Learning. 7:1991;227-252.
28. Tino P., Sajda J. Learning and extracting initial Mealy automata with a modular neural network model. Neural Computation. 7:1995;822-844.
29. Tomita, M., (1982). Dynamic construction of finite automata from example using hill-climbing. In Proceedings of the Fourth Annual Cognitive Science Conference (pp. 105-108).
30. Walker A.M. On the asymptotic behavior of posterior distribution. Journal of the Royal Statistical Society, B. 31:(1):1969;80-88.
31. Watrous R., Kuhn G.M. Induction of finite-state languages using second-order recurrent networks. Neural Computation. 4:1992;406-414.
32. Williams R.L., Zipser D. A learning algorithm for continually running fully recurrent neural networks. Neural Computation. 1:(2):1989;270-380.
33. Zeng Z., Goodman R.M., Smyth P. Learning finite state machines with self-clustering recurrent networks. Neural Computation. 5:1993;976-990.