Computational capabilities of recurrent NARX neural networks

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 27, Issue 2, 1997, Pages 208-215

  Siegelmann, Hava T ^b   Horne, Bill G ^c   Giles, C Lee ^a,c,d  

^a IEEE

^b TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^c NEC LABORATORIES AMERICA   (United States)

^d UNIVERSITY OF MARYLAND   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION THEORY; AUTOMATA THEORY; COMPUTATIONAL METHODS; COMPUTER SIMULATION; FEEDBACK; FUNCTIONS; LEARNING SYSTEMS; MATHEMATICAL MODELS; NONLINEAR SYSTEMS; TURING MACHINES;

EXOGENOUS INPUTS; MULTILAYER PERCEPTRON; NONLINEAR AUTOREGRESSIVE MODELS; RECURRENT NETWORKS;

NEURAL NETWORKS;

EID: 0031124173     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/3477.558801     Document Type: Article

References (30)

1. W. McCulloch and W. Pitts, "A logical calculus of the ideas immanent in nervous activity," Bull. Math. Biophys., vol. 5, pp. 115-133, 1943.
2. M. Minsky, Computation: Finite and Infinite Machines. Englewood Cliffs, NJ: Prentice-Hall, 1967, ch. 3.
3. N. Alon, A. Dewdney, and T. Ott, "Efficient simulation of finite automata by neural nets," J. Assoc. Comput. Mach., vol. 38, no. 2, pp. 495-514, 1991.
4. B. Horne and D. Hush, "Bounds on the complexity of recurrent neural network implementations of finite state machines," Neural Networks, vol. 9, no. 2, pp. 243-252, 1996.
5. H. Siegelmann, E. Sontag, and C. Giles, "The complexity of language recognition by neural networks," in Algorithms, Software, Architecture (Proc. IFIP 12th World Computer Congr.), J. van Leeuwen, Ed. North-Holland: Amsterdam, 1992; pp. 329-335.
6. J. Kilian and H. Siegelmann, "On the power of sigmoid neural networks," in Proc. 6th ACM Workshop on Computational Learning Theory. New York: ACM, 1993, pp. 137-143.
7. H. Siegelmann and E. Sontag, "Turing computability with neural nets," Appl. Math. Lett., vol. 4, no. 6, pp. 77-80, 1991.
8. _, "On the computational power of neural networks," J. Comput. Syst. Sci.. vol. 50, no. 1, pp. 132-150, 1995.
9. _, "Analog computation via neural networks," Theor. Comput. Sci.. vol. 131, pp. 331-360, 1994.
10. I. Leontaritis and S. Billings, "Input-output parametric models for nonlinear systems: Part I: deterministic nonlinear systems," Int. J. Control, vol. 41, no. 2, pp. 303-328, 1985.
11. S. Chen, S. Billings, and P. Grant, "Non-linear system identification using neural networks," Int. J. Control, vol. 51, no. 6, pp. 1191-1214, 1990.
12. K. Narendra and K. Parthasarathy, "Identification and control of dynamical systems using neural networks," IEEE Trans. Neural Networks, vol. 1, pp. 4-27, Mar. 1990.
13. H.-T. Su and T. McAvoy, "Identification of chemical processes using recurrent networks," in Proc. American Controls Conf., vol. 3, 1991, pp. 2314-2319.
14. H.-T. Su, T. McAvoy, and P. Werbos, "Long-term predictions of chemical processes using recurrent neural networks: A parallel training approach," Industrial Engineering and Chemical Research, vol. 31, pp. 1338-1352, 1992.
15. S. Venkataraman, "On encoding nonlinear oscillations in neural networks for locomotion," in Proc. 8th Yale Workshop Adaptive and Learning Systems, 1994, pp. 14-20.
16. S.-Z. Qin, H.-T. Su, and T. McAvoy, "Comparison of four neural net learning methods for dynamic system identifcation," IEEE Trans. Neural Networks, vol. 3, no. 1, pp. 122-130, 1992.
17. C. Giles and B. Horne, "Representation and learning in recurrent neural network architectures," in Proc. 8th Yale Workshop Adaptive and Learning Systems, 1994, pp. 128-134.
18. B. Horne and C. Giles, "An experimental comparison of recurrent neural networks," in Advances in Neural Information Processing Systems 7, G. Tesauro, D. Touretzky, and T. Leen, Eds. Cambridge, MA: MIT Press, 1995, pp. 697-704.
19. Y. Benigo, P. Simard, and P. Frasconi, "Learning long-term dependencies with gradient descent is difficult," IEEE Trans. Neural Networks, vol. 5, no. 2, pp. 157-166, 1994.
20. T. Lin. B. Horne, P. Tiňo, and C. Giles, "Learning long-term dependencies is not as difficult with NARX recurrent neural networks," IEEE Trans. Neural Networks, vol. 7, no. 6, 1996.
21. G. Cybenko, "Approximation by superpositions of a signoidal function," Mathematics of Control, Signals, and Systems, vol. 2, no. 4, pp. 303-314, 1989.
22. K. Funahashi, "On the approximate realization of continuous mappings by neural networks," Neural Networks, vol. 2, no. 3, pp. 183-192, 1989.
23. T. Booth, "Sequential machines and automata theory," New York, NY: Wiley, 1967.
24. Z. Kohavi, Switching and Finite Automata Theory, 2nd ed. New York: McGraw-Hill, 1978.
25. C. Giles, B. Horne, and T. Lin, "Learning a class of large finite state machines with a ecurrent neural network," Neural Networks, vol. 8, no. 9, p. 1359, 1996..
26. J. Pollack, "The induction of dynamical recognizers," Mach. Learn., vol. 7, no. 2/3, pp. 227-252, 1991.
27. A. Back and A. Tsoi, "FIR and IIR synapses, a new neural network architecture for time series modeling," Neural Computation, vol. 3, no. 3, pp. 375-385, 1991.
28. B. de Vries and J. Principe, "The gamma model-A new neural model for temporal processing," Neural Networks, vol. 5, pp. 565-576, 1992.
29. P. Frasconi, M. Gori, and G. Soda, "Local feedback multilayered networks," Neural Computation, vol. 4, pp. 120-130, 1992.
30. C. W. Omlin and C. L. Giles, "Constructing deterministic finite-state automata in recurrent neural networks," J. Assoc. Comput. Mach., 1997.