Recurrent neural networks are universal approximators

International Journal of Neural Systems

Volumn 17, Issue 4, 2007, Pages 253-263

  Schäfer, Anton Maximilian ^a   Zimmermann, Hans Georg ^b  

^a UNIVERSITY OF OSNABRÜCK   (Germany)

^b SIEMENS AG   (Germany)

Author keywords

Dynamical systems; Recurrent neural networks; System identification; Universal approximation

Indexed keywords

APPROXIMATION THEORY; DYNAMICAL SYSTEMS; ERROR CORRECTION; IDENTIFICATION (CONTROL SYSTEMS); SPECTRUM ANALYSIS;

APPROXIMATION ABILITY; NORMALIZED RECURRENT NEURAL NETWORKS; UNIVERSAL APPROXIMATION;

RECURRENT NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; NONLINEAR SYSTEM; SIGNAL PROCESSING; TIME;

MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; SIGNAL PROCESSING, COMPUTER-ASSISTED; TIME FACTORS;

EID: 34547898032     PISSN: 01290657     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0129065707001111     Document Type: Conference Paper

References (23)

1. H. G. Zimmermann, R. Grothmann, A. M. Schaefer and Ch. Tietz, Identification and forecasting of large dynamical systems by dynamical consistent neural networks, in S. Haykin, J. Principe, T. Sejnowski and J. McWhirter (eds.), New Directions in Statistical Signal Processing: From Systems to Brain, MIT Press (2006), pp. 203-242.
2. S. Haykin. Neural Networks: A Comprehensive Foundation (Macmillan, New York, 1994).
3. S. Haykin, J. Principe, T. Sejnowski and J. McWhirter, New Directions in Statistical Signal Processing: From Systems to Brain (MIT Press, Cambridge, MA, 2006).
4. J. F. Kolen and S. C. Kremer, A Field Guide to Dynamical Recurrent Networks, IEEE Press (2001).
5. L. R. Medsker and L. C. Jain, Recurrent neural networks: Design and application, Vol. 1, Comp. Intelligence (CRC Press international, 1999).
6. A. Soofi and L. Cao, Modeling and Forecasting Financial Data, Techniques of Nonlinear Dynamics, Kluwer Academic Publishers (2002).
7. H. G. Zimmermann and R. Neuneier, Neural network architectures for the modeling of dynamical systems, in J. F. Kolen and St. Kremer (eds.), A Field Guide to Dynamical Recurrent Networks, IEEE Press (2001), pp. 311-350.
8. D. E. Rumelhart, G. E. Hinton and R. J. Williams, Learning internal representations by error propagation, in D. E. Rumelhart and J. L. McClelland et al. (eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1, MIT Press, Cambridge, MA (1986), pp. 318-362.
9. P. J. Werbos, Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences, PhD thesis, Harvard University (1974).
10. H. G. Zimmermann, R. Neuneier and R. Grothmann, Modeling of dynamical systems by error correction neural networks, in A. Soofi and L. Cao (eds.), Modeling and Forecasting Financial Data, Techniques of Nonlinear Dynamics, Kluwer Academic Publishers (2002), pp. 237-263.
11. K. Hornik, M. Stinchcombe and H. White, Multilayer feedforward networks are universal approximators, Neural Networks 2 (1989) 359-366.
12. G. Cybenko, Approximation by superpositions of a sigmoidal function, in Mathematics of Control, Signals and Systems, Springer, New York (1989), pp. 303-314.
13. K. I. Funahashi, On the approximate realization of continuous mappings by neural networks, Neural Networks 2 (1989) 183-192.
14. B. Hammer, On the approximation capability of recurrent neural networks, in International Symposium on Neural Computation (1998).
15. T. Chen and H. Chen, Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and applications to dynamical systems, IEEE Transactions on Neural Networks 6(4) (1995) 911-917.
16. A. D. Back and T. Chen, Universal approximation of multiple nonlinear operators by neural networks, Neural Computation 14(11) (2002) 2561-2566.
17. J. Kilian and H. T. Siegelmann, The dynamic universality of sigmoidal neural networks, Information and Computation 128(1) (1996) 48-56.
18. W. Maass, T. Natschlger and H. Markram, Real-time computing without stable states: A new framework for neural computation based on perturbations, Neural Computation 14(11) (2002) 2531-2560.
19. E. Sontag, Neural nets as systems model and controllers, in Proceedings of the Seventh Yale Workshop on Adaptive and Learning Systems, Yale University (1992), pp. 73-79.
20. D. P. Mandic and J. A. Chambers, Recurrent neural networks for prediction: Learning algorithms, architectures and stability, in S. Haykin (ed.), Adaptive and Learning Systems for Signal Processing, Communications and Control, John Wiley & Sons, Chichester (2001).
21. R. Neuneier and H. G. Zimmermann, How to train neural networks, in G. B. Orr and K. R. Mueller (eds.), Neural Networks: Tricks of the Trade, Springer Verlag, Berlin (1998), pp. 373-423.
22. A. M. Schaefer, S. Udluft and H. G. Zimmermann, Learning long term dependencies with recurrent neural networks, in Proceedings of the International Conference on Artificial Neural Networks (ICANN-06), Athens (2006).
23. H. G. Zimmermann, L. Bertolini, R. Grothmann, A. M. Schaefer and Ch. Tietz, A technical trading indicator based on dynamical consistent neural networks, in Proceedings of the International Conference on Artificial Neural Networks (ICANN-06), Athens (2006).