Constructing deterministic finite-state automata in recurrent neural networks

Journal of the ACM

Volumn 43, Issue 6, 1996, Pages 937-972

  Omlin, Christian W ^a   Giles, C Lee ^a,b  

^a NEC LABORATORIES AMERICA   (United States)

^b UNIVERSITY OF MARYLAND   (United States)

Author keywords

Algorithms; Automata; Connectionism; Knowledge encoding; Neural networks; Nonlinear dynamics; Recurrent neural networks; Rules; Stability; Theory; Verification

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; DYNAMICS; ENCODING (SYMBOLS); FINITE AUTOMATA; KNOWLEDGE ACQUISITION; NUMERICAL ANALYSIS; PATTERN RECOGNITION; PERFORMANCE; STABILITY;

ABSTRACT DEVICES; ARBITRARY LENGTH; ARITHMETIC STRUCTURES; CONNECTIONISM; DETERMINISTIC FINITE STATE AUTOMATA; KNOWLEDGE ENCODING; LOGIC STRUCTURES; RECURRENT NEURAL NETWORKS; ROBUST STRING CLASSIFICATION; SIGMOIDAL DISCRIMINANT FUNCTION;

NEURAL NETWORKS;

EID: 0030286473     PISSN: 00045411     EISSN: None     Source Type: Journal    
DOI: 10.1145/235809.235811     Document Type: Article

References (32)

1. ALON, N., DEWDNEY, A. K., AND OTT, T. J. 1991. Efficient simulation of finite automata by neural nets, JACM 38, 2 (Apr.), 495-514.
2. BARNSLEY, M. 1988. Fractals Everywhere. Academic Press, San Diego, Calif.
3. CASEY, M. 1996. The dynamics of discrete-time computation, with application to recurrent neural networks and finite state machine extraction, Neural Comput. 8, 6, 1135-1178.
4. ELMAN, J. 1990. Finding structure in time. Cogn. Sci. 14, 179-211.
5. FRASCONI, P., GORI, M., MAGGINI, M., AND SODA, G. 1996. Representation of finite state automata in recurrent radial basis function networks, Mach. Learn. 23, 5-32.
6. FRASCONI, P., GORI, M., MAGGINI, M., AND SODA, G. 1991. A unified approach for integrating explicit knowledge and learning by example in recurrent networks. In Proceedings of the International Joint Conference on Neural Networks, vol. 1. IEEE, New York, p. 811.
7. FRASCONI, P., GORI, M., AND SODA, G. 1993. Injecting nondeterministic finite state automata into recurrent networks. Tech. Rep. Dipartimento di Sistemi e Informatica, Università di Firenze, Italy, Florence, Italy.
8. GEMAN, S., BIENENSTOCK, E., AND DOURSTAT, R. 1992. Neural networks and the bias/variance dilemma, Neural Comput. 4, 1, 1-58.
9. GILES, C., CHEN, D., MILLER, C., CHEN, H., SUN, G., AND LEE, Y. 1991. Second-order recurrent neural networks for grammatical inference. In Proceedings of the International Joint Conference on Neural Networks 1991, vol. II. IEEE, New York, pp. 273-281.
10. GILES, C., KUHN, G., AND WILLIAMS, R. 1994. Special issue on Dynamic recurrent neural networks: Theory and applications, IEEE Trans. Neural Netw. 5, 2.
11. GILES, C., MILLER, C., CHEN, D., CHEN, H., SUN, G., AND LEE, Y. 1992. Learning and extracting finite state automata with second-order recurrent neural networks. Neural Comput. 4, 3, 380.
12. GILES, C., AND OMLIN, C. 1992. Inserting rules into recurrent neural networks. In Neural Networks for Signal Processing II, Proceedings of the 1992 IEEE Workshop (S. Kung, F. Fallside, J. A. Sorenson, and C. Kamm, eds.) IEEE, New York, pp. 13-22.
13. GILES, C., AND OMLIN, C. 1993. Rule refinement with recurrent neural networks. In Proceedings IEEE International Conference on Neural Networks (ICNN'93), vol. II. IEEE, New York, pp. 801-806.
14. HAYKIN, S. 1994. Neural Networks, A Comprehensive Foundation. MacMillan, New York.
15. HIRSCH, M. 1989. Convergent activation dynamics in continuous-time neural networks. Neural Netw. 2, 331-351.
16. HIRSCH, M. 1994. Saturation at high gain in discrete time recurrent networks. Neural Netw. 7, 3, 449-453.
17. HOPCROFT, J., AND ULLMAN, J. 1979. Introduction to Automata Theory, Languages, and Computation. Addison-Wesley, Reading, Mass.
18. HORNE, B., AND HUSH, D. 1996. Bounds on the complexity of recurrent neural network implementations of finite state machines. Neural Netw. 9, 2, 243-252.
19. MACLIN, R., AND SHAVLIK, J. 1993. Using knowledge-based neural networks to improve algorithms: Refining the Chou-Fasman algorithm for protein folding. Mach. Learn. 11, 195-215.
20. MEAD, C. 1989. Analog VLSI and Neural Systems. Addison-Wesley, Reading, Mass.
21. MINSKY, M. 1967. Computation: Finite and Infinite Machines. Prentice-Hall, Inc., Englewood Cliffs, N.J., pp. 32-66 (Chap. 3).
22. OMLIN, C., AND GILES, C. 1996a. Rule revision with recurrent neural networks. IEEE Trans. Knowl. Data Eng. 8, 1, 183-188.
23. OMLIN, C., AND GILES, C. 1996b. Stable encoding of large finite-state automata in recurrent neural networks with sigmoid discriminants. Neural Comput. 8, 4, 675-696.
24. OMLIN, C., AND GILES, C. 1992. Training second-order recurrent neural networks using hints, in Proceedings of the 9th International Conference on Machine Learning (San Mateo, Calif.), D. Sleeman and P. Edwards, eds. Morgan-Kaufmann, San Mateo, Calif., pp. 363-368.
25. POLLACK, J. 1991. The induction of dynamical recognizers. Mach. Learn. 7, 227-252.
26. SERVAN-SCHREIBER, D., CLEEREMANS, A., AND MCCLELLAND, J. 1991. Graded state machine: The representation of temporal contingencies in simple recurrent networks. Mach. Learn. 7, 161.
27. SHAVLIK, J. 1994. Combining symbolic and neural learning. Mach. Learn. 14, 3, 321-331.
28. SHEU, B. J. 1995. Neural Information Processing and VLSI. Kluwer Academic Publishers, Boston, Mass.
29. TINO, P., HORNE, B. AND GILES, C. 1995. Fixed points in two-neuron discrete time recurrent networks: Stability and bifurcation considerations. Tech. Rep. UMIACS-TR-95-51. Institute for Advanced Computer Studies, Univ. Maryland, College Park, Md.
30. TOWELL, G., SHAVLIK, J., AND NOORDEWIER, M. 1990. Refinement of approximately correct domain theories by knowledge-based neural networks. In Proceedings of the 8th National Conference on Artificial Intelligence (San Mateo, Calif.) Morgan-Kaufmann, San Mateo, Calif., p. 861.
31. WATROUS, R., AND KUHN, G. 1992. Induction of finite-state languages using second-order recurrent networks. Neural Comput. 4, 3, 406.
32. ZENG, Z., GOODMAN, R., AND SMYTH, P. 1993. Learning finite state machines with self-clustering recurrent networks. Neural Comput. 5, 6, 976-990.