Bounds on the complexity of recurrent neural network implementations of finite state machines

Neural Networks

Volumn 9, Issue 2, 1996, Pages 243-252

  Horne, Bill G ^a   Hush, Don R ^b  

^a NEC LABORATORIES AMERICA   (United States)

^b NONE

Author keywords

automata theory; circuit complexity; finite state machines; logic synthesis; neural networks; recurrent neural networks

Indexed keywords

COMPUTATIONAL COMPLEXITY; COMPUTER CIRCUITS; FINITE AUTOMATA; LOGIC DESIGN; SET THEORY;

CIRCUIT COMPLEXITY; LOGIC SYNTHESIS; RECURRENT NEURAL NETWORKS;

NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; PRIORITY JOURNAL;

EID: 0030110968     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/0893-6080(95)00095-X     Document Type: Article

References (27)

1. Alon, N., Dewdney, A., & Ott, T. (1991). Efficient simulation of finite automata by neural nets. Journal of the Association of Computing Machinery, 38(2), 495-514.
2. Back, A., & Tsoi, A. (1991). FIR and IIR synapses, a new neural network architecture for time series modeling. Neural Computation, 3(3), 375-385.
3. Elman, J. (1990). Finding structure in time. Cognitive Science, 14, 179-211.
4. Giles, C., Sun, G., Chen, H., Lee, Y., & Chen, D. (1990). Higher order recurrent networks and grammatical inference. In D. Touretzky (Ed.) (pp. 380-387). Advances in Neural Information Processing Systems 2. Morgan Kaufmann.
5. Hopcroft, J., & Ullman, J. (1979). Introduction to automata theory, languages, and computation. Reading, MA: Addison-Wesley.
6. Hopfield, J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Science USA, 79, 2554-2558.
7. Horne, B., & Hush, D. (1994). On the node complexity of neural networks. Neural Networks, 7(9), 1413-1426.
8. Jordan, M., & Rumelhart, D. (1992). Forward models: supervised learning with a distal teacher. Cognitive Science, 16, 307-354.
9. Kilian, J., & Siegelmann, H. (1993). On the power of sigmoid neural networks. In Proceedings of the Sixth ACM Workshop on Computational Learning Theory. (pp. 137-143). ACM Press.
10. Lupanov, O. (1972). Circuits using threshold elements. Soviet Physics - Doklady, 17(2), 91-93.
11. Lupanov, O. (1973). The synthesis of circuits from threshold elements. Problemy Kibernetiki, 26, 109-140 (in Russian).
12. McCulloch, W., & Pitts, W. (1943), A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5, 115-133.
13. Minsky, M. (1967). Computation: finite and infinite machines. Englewood Cliffs, NJ: Prentice-Hall.
14. Muroga, S. (1971). Threshold logic and its applications. New York: John Wiley.
15. Narendra, K., & Parthasarathy, K. (1990). Identification and control of dynamical systems using neural networks. IEEE Transactions on Neural Networks, 1, 4-27.
16. Nerrand, O., Rouseel-Ragot, P., Personnaz, L., Dreyfus, G., & Marcos, S. (1993). Neural networks and nonlinear adaptive filtering: Unifying concepts and new algorithms. Neural Computation, 5, 165-199.
17. Parberry, I. (1994). Circuit complexity and neural networks. Cambridge, MA: MIT Press.
18. Red'kin, N. (1972). Realization of boolean functions in a certain class of threshold element circuits. Automation and Remote Control, 1252-1256.
19. Robinson, A., & Fallside, F. (1988). Static and dynamic error propagation networks with application to speech coding. In D. Anderson (Ed.), Neural information processing systems (pp. 632-641). New York: American Institute of Physics.
20. Siegelmann, H., & Sontag, E. (1991). Neural networks are universal computing devices. Technical Report SYCON-91-08, Rutgers Center for Systems and Control.
21. Siegelmann, H., & Sontag, E. (1992). On the computational power of neural networks. In Proceedings of the Fifth ACM Workshop on Computational Learning Theory (pp. 440-449). ACM Press.
22. Siegelmann, H., & Sontag, E. (1994). Analog computation via neural networks. Theoretical Computer Science, 131, 331-360.
23. Siegelmann, H., Sontag, E., & Giles, C. (1992). The complexity of language recognition by neural networks. In J. van Leeuwen (Ed.), Algorithms, software, architecture (Proceedings of IFIP 12th World Computer Congress) (pp. 329-335). Amsterdam: North-Holland.
24. Watrous, R., & Kuhn, G. (1992). Induction of finite-state languages using second-order recurrent networks. Neural Computation, 4(3), 406-414.
25. Williams, R., & Zipser, D. (1989). A learning algorithm for continually running fully recurrent neural networks. Neural Computation, 1(2), 270-280.
26. Winder, R. (1963). Bounds on threshold gate realizability. IEEE Transactions on Electronic Computers, EC-12, 561-564.
27. Winder, R. (1970). Threshold logic asymptotes. IEEE Transactions on Computers, C-19(4), 349-353.