Volterra models and three-layer perceptrons

IEEE Transactions on Neural Networks

Volumn 8, Issue 6, 1997, Pages 1421-1433

  Marmarelis, Vasilis Z ^a,b   Zhao, Xiao ^a,c  

^a IEEE   (United States)

^b University of Southern California ^*  (United States)

^c SOUTHWEST RESEARCH INSTITUTE   (United States)

Author keywords

Laguerre kernel expansion; Nonlinear system modeling; Polynomial activation functions; Separable Volterra network; Three layer perceptrons; Volterra kernels; Volterra models

Indexed keywords

BACKPROPAGATION; BOOLEAN FUNCTIONS; COMPUTER SIMULATION; MATHEMATICAL MODELS; NONLINEAR SYSTEMS; POLYNOMIALS;

LAGUERRE EXPANSION TECHNIQUE; LAGUERRE KERNEL EXPANSION; POLYNOMIAL ACTIVATION FUNCTIONS; VOLTERRA KERNELS; VOLTERRA MODELS;

FEEDFORWARD NEURAL NETWORKS;

EID: 0031273027     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.641465     Document Type: Article

References (39)

1. E. K. Blum and L. K. Li, "Approximation theory and feedforward networks," Neural Networks, vol. 4, pp. 511-515, 1991.
2. T. Chen and H. Chen, "Universal aaproximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems," IEEE Trans. Neural Networks, vol. 6, pp. 911-916, 1995.
3. S. Chen, G. J. Gibson, and C. F. N. Cowan, "Adaptive channel equalization using a polynomial perceptron structure," in Proc. Inst. Elec. Eng., 1990, vol. 137, pp. 257-264.
4. M. S. Chen and M. T. Manry, "Conventional modeling of the multilayer perceptron using polynomial basis functions," Neural Networks, vol. 4, pp. 164-166, 1993.
5. G. Cybenko, "Approximation by superpositions of a sigmoidal function," Math. Contr., Signals, Syst., vol. 2, pp. 303-314, 1989.
6. G. W. David and M. L. Gasperi, "ANN modeling of Volterra systems," in Proc. IJCNN'91, Seattle, WA, 1991, pp. II 727-734.
7. R. J. P. de Figueiredo, "Implications and applications of Kolmogorov's superposition theorem," IEEE Trans. Automat. Contr., vol. 25, pp. 1227-1231, 1980.
8. _, "A generalized Fock space framework for nonlinear system and signal analysis," IEEE Trans. Circuits Syst., vol. CAS-30, pp. 637-647, Sept. 1983 (Special invited issue on nonlinear circuits and sytems).
9. _, "A new nonlinear functional analytic framework for modeling artificial neural networks," (invited paper), in Proc. 1990 IEEE Int. Symp. Circuits Syst., New Orleans, LA, May 1990, pp. 723-726.
10. _, "An optimal mutlilayer neural interpolating (OMNI) net in a generalized Fock space setting," (invited paper), in Proc. IJCNN, Baltimore, MD, June 1992.
11. K.-I. Funahashi, "On the approximate realization of continuous mappings by neural networks," Neural Networks, vol. 2, pp. 183-192, 1989.
12. G. Govind and P. A. Ramamoorthy, "Multilayered neural networks and Volterra series: The missing link," in Proc. Int. Conf. Syst. Eng., 1990, pp. 633-636.
13. M. H. Hassoun, Fundamentals of Artificial Neural Networks. Cambridge, MA: MIT Press, 1995.
14. K. Hornik, M. Stinchcombe, and H. White, "Multilayer feedforward networks are universal approximators," Neural Networks, vol. 2, pp. 359-366, 1989.
15. W. G. Knecht, "Nonlinear noise filtering and beamforming using the perceptron and its Volterra approximation," IEEE Trans. Speech Audio Processing., vol. 2, pp. 55-62, 1994.
16. A. N. Kolmogorov, "On the representation of continuous functions of several variables by superposition of continuous functions of one variable and addition," Dokl. Akad. Nauk, SSSR, vol. 114, pp. 953-956, 1957; AMS Transl., vol. 2, pp. 55-59, 1963.
17. M. J. Korenberg, "Parallel cascade identification and kernel estimation for nonlinear systems," Ann. Biomed. Eng., vol. 19, pp. 429-455, 1991.
18. S. Y. Kung, Digital Neural Networks. Englewood Cliffs, NJ: Prentice-Hall, 1993.
19. M. Leshno, V. Lin, A. Pinkus, and H. White, "Multilayer feedforward networks with a nonpolynomial activation function can approximate any function," Neural Networks, vol. 6, pp. 861-867, 1993.
20. G. G. Lorentz, "The 13th problem of Hilbert," in Proc. Symp. Pure Math., F. E. Browder, Ed., vol. 28. Providence, RI: Amer. Math. Soc., 1976, pp. 419-430.
21. P. Z. Marmarelis and V. Z. Marmarelis, Analysis of Physiological Systems: The White Noise Approach. New York: Plenum, 1978. Russian translation: Mir Press, Moscow, 1981. Chinese translation: Academy of Sciences Press, Beijing, 1990.
22. V. Z. Marmarelis, Ed., Advanced Methods of Physiological System Modeling, vol. I. Los Angeles: Univ. Southern California, Biomedical Simulations Resource, 1978.
23. _, Advanced Methods of Physiological System Modeling, vol. II. New York: Plenum, 1989.
24. _, Advanced Methods of Physiological System Modeling, vol. III. New York: Plenum, 1994.
25. V. Z. Marmarelis, "Signal transformation and coding in neural systems," IEEE Trans. Biomed. Eng., vol. 36, pp. 15-24, 1989.
26. _, "Identification of nonlinear biological systems using Laguerre expansions of kernels," Ann. Biomed. Eng., vol. 21, pp. 573-589, 1993.
27. _, "Nonlinear modeling of physiological systems using principal dynamic modes," in Advanced Methods of Physiological System Modeling, vol. III. New York: Plenum, 1994, pp. 1-28.
28. V. Z. Marmarelis and M. E. Orme, "Modeling of neural systems by use of neuronal modes, "IEEE Trans. Biomed. Eng., vol. 40, pp. 1149-1158, 1993.
29. V. Z. Marmarelis, M. C. Citron, and C. P. Vivo, "Minimum-order Wiener modeling of spike-output systems," Biol. Cybern., vol. 54, pp. 115-123, 1986.
30. G. Palm, "On representation and approximation of nonlinear systems. Part II: Discrete systems," Biol. Cybern., vol. 34, pp. 49-52, 1979.
31. F. Rosenblatt, Principles of Neurodynamics: Perceptrons and the Theory of Brain Mechanisms. Washington, D.C.: Spartan Books, 1962.
32. D. E. Rumelhart and J. L. McClelland, Eds., Parallel Distributed Processing, vols. I and II. Cambridge, MA: MIT Press, 1986.
33. I. W. Sandberg, "Approximation theorems for discrete-time systems," IEEE Trans. Circuits Syst., vol. 38, pp. 564-566, 1991.
34. S. K. Sin and R. J. P. de Figueiredo, "Efficient learning procedures for optimal interpolative nets," IEEE Trans. Neural Networks, vol. 6, pp. 90-113, 1993.
35. D. F. Specht, "Probabilistic neural networks and the polynomial adaline as complementary techniques for classification," IEEE Trans. Neural Networks, vol. 1, pp. 111-121, 1990.
36. D. A. Sprecher, "An improvement in the superposition theorem of Kolmogorov," J. Math. Anal. Applicat., vol. 38, pp. 208-213, 1972.
37. N. Wiener, Nonlinear Problems in Random Theory. New York: The Technology Press of M.I.T. and Wiley, 1958.
38. J. Wray and G. G. R. Green, "Calculation of the Volterra kernels of nonlinear dynamic systems using an artificial neural network," Biol. Cybern., vol. 71, pp. 187-195, 1994.
39. Z. Xiang, G. Bi, and T. Le-Ngoc, "Polynomial perceptrons and their applications to fading channel equalization and cochannel interference suppression," IEEE Trans. Signal Processing, vol. 42, pp. 2470-2479, 1994.