Complete memory structures for approximating nonlinear discrete-time mappings

IEEE Transactions on Neural Networks

Volumn 8, Issue 6, 1997, Pages 1397-1409

  Stiles, Bryan Waitsel ^a,b   Sandberg, Irwin W ^a,b   Ghosh, Joydeep ^b  

^a IEEE   (United States)

^b The University of Texas at Austin   (United States)

Author keywords

Approximation theory; Discrete time systems; Functional analysis; Modeling; Multidimensional systems; Neural networks; Nonlinear systems; Universal approximators

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; COMPUTER SIMULATION; MATHEMATICAL MODELS; NEURAL NETWORKS; NONLINEAR SYSTEMS;

FUNCTIONAL ANALYSIS; MULTIDIMENSIONAL SYSTEM; NONLINEAR DISCRETE TIME SYSTEMS;

DATA STRUCTURES;

EID: 0031269630     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.641463     Document Type: Article

References (34)

1. G. Cybenko, "Approximations by superpositions of a sigmoidal function," Math. Contr., Signals, Syst., vol. 2, pp. 303-314, 1989.
2. J. Park and I. W. Sandberg, "Universal approximation using radial basis function networks," Neural Computa., vol. 3, no. 2, pp. 246-257, Summer 1991.
3. Y. Shin and J. Ghosh, "Ridge polynomial networks," IEEE Trans. Neural Networks, vol. 6, pp. 610-622, May 1995.
4. _, "Function approximation using higher-order connectionist networks," Computer and Vision Res. Center, Univ. Texas, Austin, Tech. Rep. TR-92-12-87, May 1992.
5. W. Rudin, Principles of Mathematical Analysis, 3rd ed. New York: McGraw-Hill, 1976.
6. A. N. Kolmogorov, "On the representations of continuous functions of many variables by superpositions of continuous functions of one variable and addition," Dokl. Akade. Nauk USSR, vol. 114, no. 5, pp. 953-956, 1957.
7. D. A. Sprecher, "A numerical implementation of Kolmogorov's theorems," Neural Networks, vol. 9, no. 5, pp. 765-771, 1996.
8. A. R. Barron, "Universal approximation bounds for superpositions of a sigmoidal function theory," IEEE Trans. Inform. Theory, vol. 39, pp. 930-945, May 1993.
9. K. S. Narendra and K. Parthasarathy, "Identification and control of dynamical systems using neural networks," IEEE Trans. Neural Networks, vol. 1, pp. 4-27, Mar. 1990.
10. B. de Vries and J. C. Principe, "The gamma model - A new neural-net model for temporal processing," Neural Networks, vol. 5, pp. 565-576, 1992.
11. A. Waibel, "Modular construction of time-delay neural networks for speech recognition," Neural Computa., vol. 1, no. 1, pp. 39-46, 1989.
12. A. D. Back and A. C. Tsoi, "A comparison of discrete-time operator models for nonlinear system identification," in Advances in Neural Information Processing Systems: Proc. 1994 Conf., vol. 7, pp. 883-890.
13. W. S. McCulloch and W. Pitts, "A logical calculus of the ideas immanent in nervous activity," Bull. Math. Biophys., vol. 9, pp. 115-133, 1943.
14. H. T. Siegelmann, B. G. Horne, and C. L. Giles, "Computational capabiliites of recurrent NARX neural networks," to be published in IEEE Trans. Syst., Man, Cybern. Also, Univ. Maryland, College Park, MD, Tech. Reps. UMIACS-TR-95-78 and CS-TR-3500.
15. H. T. Siegelmann and E. D. Sontag, "On the computational power of neural networks," J. Comput. Syst. Sci., vol. 50, no. 1, pp. 132-150, 1995.
16. I. W. Sandberg, "Structure theorems for nonlinear systems," Multidimensional Syst. Signal Processing, vol. 2, pp. 267-286, 1991. (See also the errata in vol. 3, p. 101, 1992.)
17. _, "Multidimensional nonlinear systems and structure theorems," J. Circuits, Syst., and Computers, vol. 2, no. 4, pp. 383-388, 1992.
18. I. W. Sandberg and L. Xu, "Network approximation of input-output maps and functionals," J. Circuits, Syst., Signal Processing, vol. 15, no. 6, pp. 711-725, 1996.
19. T. Chen and H. Chen, "Approximation of continuous functionals by neural networks with application to dynamical systems," IEEE Trans. Neural Networks, vol. 4, pp. 910-918, Nov. 1993.
20. _, "Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems," IEEE Trans. Neural Networks, vol. 6, pp. 918-928, July 1995.
21. Y. Bengio, P. Simard, and P. Frasconi, "Learning long-term dependencies with gradient descent is difficult," IEEE Trans. Neural Networks, vol. 5, pp. 157-166, Mar. 1994.
22. T. Lin, B. G. Horne, P. Tiňo, and C. L. Giles, "Learning long-term dependencies in NARX recurrent neural networks," IEEE Trans. Neural Networks, vol. 7, pp. 1329-1338, Nov. 1996.
23. Y. Bengio and P. Frasconi, "Input-output HMM's for sequence processing," IEEE Trans. Neural Networks, vol. 7, pp. 1231-1248, Sept. 1996.
24. S. Grossberg, Studies of Mind and Brain. Dordrecht, The Netherlands: Reidel, 1982.
25. B. W. Stiles and J. Ghosh, "Habituation based neural networks for spatio-temporal classification," Neurocomputing, vol. 15, no. 3/4, ppl 273-307, 1997.
26. _, "Some limitations of linear memory architectures for signal processing," in Proc. 1996 Int. Workshop on Neural Networks for Identification, Contr., Robot., Signal/Image Processing, Venice, Italy, 1996, pp. 102-110.
27. _, "Nonlinear memory functions for modeling discrete-time systems," Center for Vision and Image Sci., Univ. Texas Austin, Tech. Rep. UT-CVIS-TR-96-004, available http://www.lans.ece.utexas.edu under "technical reports."
28. M. H. Stone, "A generalized Weierstrass approximation theorem," in R. C. Buck, Ed., Studies in Modern Analysis. The Math. Assoc. Amer., 1962.
29. I. W. Sandberg and L. Xu, "Uniform approximation and gamma networks," Neural Networks, vol. 10, no. 5, pp. 781-784, 1997.
30. D. Robin, P. Abbas, and L. Hug, "Neural response to auditory patterns," J. Acoust. Soc. Amer., vol. 87, no. 4, pp. 1673-1682, 1990.
31. J. H. Byrne and K. J. Gingrich, "Mathematical model of cellular and molecular processes contributing to associative and nonassociative learning in Aplysia," in Neural Models of Plasticity, J. H. Byrne and W. O. Berry, Eds. San Diego, CA: Academic, 1989, pp. 58-70.
32. J. Ghosh, L. Deuser, and S. Beck, "A neural-network-based hybrid system for detection, characterization, and classification of short-duration oceanic signals," IEEE J. Oceanic Eng., vol. 17, pp. 351-363, Oct. 1992.
33. B. Stiles and J. Ghosh, "A habituation based neural network for spatiotemporal classification," in Neural Networks for Signal Processing V, Proc. 1995 IEEE Workshop, Cambridge, MA, Sept. 1995, pp. 135-144.
34. F. J. Pineda, "Recurrent backpropagation and the dynamical approach to adaptive neural computation," Neural Computa., vol. 1, no. 2, pp. 161-172, 1989.