Representation of nonlinear random transformations by non-Gaussian stochastic neural networks

IEEE Transactions on Neural Networks

Volumn 19, Issue 6, 2008, Pages 1033-1060

  Turchetti, Claudio ^a   Crippa, Paolo ^a   Pirani, Massimiliano ^b   Biagetti, Giorgio ^a  

^a UNIVERSITÀ POLITECNICA DELLE MARCHE   (Italy)

^b SPES scpa ^*  (Italy)

Author keywords

Approximation; Lee Schetzen method; Neural computation; Neural networks; Nonlinear systems; Stochastic processes (SPs)

Indexed keywords

APPLICATION EXAMPLES; APPROXIMATE IDENTITIES; APPROXIMATION; DETERMINISTIC FUNCTIONS; DIFFERENTIAL TRANSFORMATIONS; GAUSSIAN STOCHASTIC PROCESS; INPUT OUTPUTS; LARGE CLASS; LEARNING CAPABILITIES; LEE-SCHETZEN METHOD; LINEAR TRANSFORMATIONS; MEMORY SYSTEMS; NEURAL COMPUTATION; NON-GAUSSIAN; NONLINEAR INPUTS; RANDOM FUNCTIONS; REAL ENVIRONMENTS; STOCHASTIC FUNCTIONS; STOCHASTIC NEURAL NETWORKS; STOCHASTIC PROCESSES (SPS); UNIVERSAL APPROXIMATORS;

ALGEBRA; EDUCATION; GAUSSIAN NOISE (ELECTRONIC); MATHEMATICAL TRANSFORMATIONS; MODELS; NONLINEAR SYSTEMS; RANDOM PROCESSES; STOCHASTIC SYSTEMS;

NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; HUMAN; LANGUAGE; NONLINEAR SYSTEM; NORMAL DISTRIBUTION; PATTERN RECOGNITION; PHYSIOLOGY; STATISTICS; VOICE;

HUMANS; LANGUAGE; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; NORMAL DISTRIBUTION; PATTERN RECOGNITION, AUTOMATED; PATTERN RECOGNITION, PHYSIOLOGICAL; STOCHASTIC PROCESSES; VOICE;

EID: 49149095665     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2007.2000055     Document Type: Article

References (35)

1. G. Cybenko, "Approximation by superposition of sigmoidal function," Math. Control Signals Syst., vol. 2, no. 4, pp. 303-314, Dec. 1989.
2. K. Funahashi, "On the approximate realization of continuous mappings by neural networks," Neural Netw., vol. 2, no. 3, pp. 183-192, May 1989.
3. K. Hornik, M. Stinchcombe, and H. White, "Multilayer feedforward networks are universal approximators," Neural Netw., vol. 2, no. 5, pp. 359-366, Jul. 1989.
4. J. Park and I. W. Sandberg, "Universal approximation using radial-basis-function networks," Neural Comput., vol. 3, no. 2, pp. 246-257, 1991.
5. M. Conti and C. Turchetti, "Approximation of dynamical systems by continuous-time recurrent approximate identity neural networks," Neural Parallel Sci. Comput., vol. 2, no. 3, pp. 299-322, Sep. 1994.
6. R. J. Schilling, J. J. Carroll, Jr., and A. F. Al-Ajlouni, "Approximation of nonlinear systems with radial basis function neural networks," IEEE Trans. Neural Netw., vol. 12, no. 1, pp. 1-15, Jan. 2001.
7. S. Ferrari and R. F. Stengel, "Smooth function approximation using neural networks," IEEE Trans. Neural Netw., vol. 16, no. 1, pp. 24-38, Jan. 2005.
8. G.-B. Huang, P. Saratchandran, and N. Sundararajan, "A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation," IEEE Trans. Neural Netw., vol. 16, no. 1, pp. 57-67, Jan. 2005.
9. J.-M. Wu, Z.-H. Lin, and P.-H. Hsu, "Function approximation using generalized adalines," IEEE Trans. Neural Netw., vol. 17, no. 3, pp. 541-558, May 2006.
10. G.-B. Huang, L. Chen, and C.-K. Siew, "Universal approximation using incremental constructive feedforward networks with random hidden nodes," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 879-892, Jul. 2006.
11. D. Wedge, D. Ingram, D. McLean, C. Mingham, and Z. Bandar, "On global-local artificial neural networks for function approximation," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 942-952, Jul. 2006.
12. L. Scardovi, M. Baglietto, and T. Parisini, "Active state estimation for nonlinear systems: A neural approximation approach," IEEE Trans. Neural Netw., vol. 18, no. 4, pp. 1172-1184, Jul. 2007.
13. C. Turchetti, M. Conti, P. Crippa, and S. Orcioni, "On the approximation of stochastic processes by approximate identity neural networks," IEEE Trans. Neural Netw., vol. 9, no. 6, pp. 1069-1085, Nov. 1998.
14. M. R. Belli, M. Conti, P. Crippa, and C. Turchetti, "Artificial neural networks as approximators of stochastic processes," Neural Netw., vol. 12, no. 45, pp. 647-658, Jun. 1999.
15. C. Turchetti, Stochastic Models of Neural Networks. Amsterdam, The Netherlands: IOS, 2004.
16. P. Crippa, C. Turchetti, and M. Pirani, "Stochastic model of neural computing," in Proc. 8th Int. Conf. Knowl.-Based Intell. Inf. Eng. Syst., Wellington, New Zealand, Sep. 2004, vol. 2, pp. 683-690.
17. S. Amari, K. Kurata, and H. Nagaoka, "Information geometry of Boltzmann machines," IEEE Trans. Neural Netw., vol. 3, no. 2, pp. 260-271, Mar. 1992.
18. B. Igelnik and Y. H. Pao, "Stochastic choice of basis functions in adaptive function approximation and the functional-link net," IEEE Trans. Neural Netw., vol. 6, no. 6, pp. 1320-1329, Nov. 1995.
19. G. B. Giannakis and E. Serpedin, "A bibliography on nonlinear system identification," Signal Process., vol. 81, no. 3, pp. 533-580, Mar. 2001.
20. I. I. Gihman and A. V. Skorohod, The Theory of Stochastic Processes Berlin, Germany: Springer-Verlag, 1974.
21. P. Crippa and C. Turchetti, "Learning of SAINNs from covariance function: Historical learning," in Proc. 7th Int. Conf. Knowl.-Based Intell. Inf. Eng. Syst., Oxford, U.K., Sep. 2003, vol. 1, pp. 177-183.
22. M. Schetzen, The Volterra and Wiener Theories of Nonlinear Systems New York: Wiley, 1989.
23. M. Pirani, S. Orcioni, and C. Turchetti, "Diagonal kernel point estimation of n-th order discrete Volterra-Wiener systems," EURASIP J. Appl. Signal Process., vol. 2004, no. 12, pp. 1807-1816, Sep. 2004.
24. V. J. Mathews and G. L. Sicuranza, Polynomial Signal Processing. New York: Wiley, 2000.
25. K. Gopalan, "Speech coding using Fourier-Bessel expansion of speech signals," in Proc. 27th Annu. Conf. IEEE Ind. Electron. Soc., Nov. 2001, vol. 3, pp. 2199-2203.
26. G. Biagetti, P. Crippa, and C. Turchetti, "Modeling of speech signals based on Bessel-like orthogonal transform," in Proc. Interspeech 2006/ICSLP 9th Int. Conf. Spoken Lang. Process., Pittsburgh, PA, Sep. 2006, pp. 2478-2481.
27. V. Volterra, Theory of Functionals and of Integrals and Integro-Differential Equations. New York: Dover Publications, 1959.
28. G. Palm and T. Poggio, "The Volterra representation and the Wiener expansion: Validity and pitfalls," SIAM J. Appl. Math., vol. 33, no. 2, pp. 195-216, Sep. 1977.
29. I. W. Sandberg, "Uniform approximation with doubly-finite Volterra series," IEEE Trans. Signal Process., vol. 40, no. 6, pp. 1438-1442, Jun. 1992.
30. I. W. Sandberg and L. Xu, "Approximation of myopic systems whose inputs need not be continuous," Multidimension. Syst. Signal Process., vol. 9, no. 2, pp. 207-225, Apr. 1998.
31. I. W. Sandberg, "Separation conditions and approximation of continuous-time approximately-finite-memory systems," IEEE Trans. Circuits Syst. I, Fundam. Theory App., vol. 46, no. 7, pp. 820-826, Jul. 1999.
32. M. Schetzen, "A theory of non-linear system identification," Int. J. Control, vol. 20, no. 4, pp. 577-592, Oct. 1974.
33. J. D. Victor, "The fractal dimension of a test signal: Implications for system identification procedures," Biol. Cybern., vol. 57, no. 6, pp. 421-426, Dec. 1987.
34. R. D. Nowak and B. D. Van Veen, "Random and pseudorandom inputs for Volterra filter identification," IEEE Trans. Signal Process., vol. 42, no. 8, pp. 2124-2135, Aug. 1994.
35. G. Palm and T. Poggio, "Stochastic identification methods for nonlinear systems: An extension of the Wiener theory," SIAM J. Appl. Math., vol. 34, no. 3, pp. 524-534, May 1978.