Echo state networks with filter neurons and a delay&sum readout

Neural Networks

Volumn 23, Issue 2, 2010, Pages 244-256

  Holzmann, Georg ^a,b   Hauser, Helmut ^a  

^a GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

^b TECHNISCHE UNIVERSITÄT BERLIN   (Germany)

Author keywords

Delay learning; Echo state networks; Nonlinear dynamical system modeling; Reservoir computing; Time series prediction

Indexed keywords

ART MODEL; DELAY LEARNING; ECHO STATE NETWORKS; INFINITE IMPULSE RESPONSE FILTER; INHERENT LIMITATIONS; MEMORY CAPACITY; MULTIPLE ATTRACTORS; NETWORK MODELS; NEW STRUCTURES; OUTPUT NEURONS; REAL-WORLD; RESERVOIR COMPUTING; SYNAPTIC CONNECTIONS; TIME-SCALES;

DYNAMICAL SYSTEMS; IIR FILTERS; IMPULSE RESPONSE; LEARNING ALGORITHMS; NEURONS; NONLINEAR DYNAMICAL SYSTEMS; TIME SERIES;

RECURRENT NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; CONTROLLED STUDY; FILTER; LEARNING ALGORITHM; MEMORY; NERVE CELL; NONLINEAR SYSTEM; PRIORITY JOURNAL; SYNAPSE; TIME SERIES ANALYSIS;

ALGORITHMS; COMPUTER SIMULATION; HUMANS; LEARNING; LINEAR MODELS; MEMORY; MUSIC; NEURAL NETWORKS (COMPUTER); NEURONS; NONLINEAR DYNAMICS; PERIODICITY; SYNAPSES; TIME FACTORS;

EID: 73949157176     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2009.07.004     Document Type: Article

References (32)

1. Atiya A.F., and Parlos A.G. New results on recurrent network training: Unifying the algorithms and accelerating convergence. IEEE-NN 11 3 (2000) 697
2. Ausserlechner, H. (2006). Untersuchungen von dropout-concealment-algorithmen. Institute for Electronic Music and Acoustics, Graz, Austria. Diplomarbeit
3. Azaria M., and Hertz D. Time delay estimation by generalized cross correlation methods. Acoustics, Speech, and Signal Processing; IEEE Transactions on Signal Processing 32 (1986) 285-380
4. Bringuier V., Chavane F., Glaeser L., and Frégnac Y. Horizontal propagation of visual activity in the synaptic integration field of area 17 neurons. Science 283 (1999) 695-699
5. Bristow-Johnson, R. (2008). Cookbook formulae for audio eq biquad filter coefficients. Online resource, accessed 14.4.2008
6. Cristianini N., and Shawe-Taylor J. An introduction to support vector machines and other kernel-based learning methods (2000), Cambridge University Press
7. Feldkamp L., Prokhorov D., Eagen C., and Yuan F. Enhanced multi-stream kalman filter training for recurrent networks. Nonlinear Modeling: Advanced Black-Box Techniques (1998) 29-53
8. Gers F.A., Eck D., and Schmidhuber J. Applying LSTM to time series predictable through time-window approaches. Lecture Notes in Computer Science 2130 (2001) 669+
9. Godsill S., and Rayner P. Digital audio restoration (1997), Springer
10. Goodman D., Lockhart G., Wasern O., and Wai-Choong W. Waveform substitution techniques for recovering missing speech segments in packet voice communications. Acoustics, Speech, and Signal Processing, IEEE Transactions 34 (1986) 1440-1448
11. Huang Y., Benesty J., and Chen J. Acoustic MIMO signal processing (2006), Springer-Verlag, Berlin, Heidelberg
12. Jaeger, H. (2001). The "echo state" approach to analysing and training recurrent neural networks. Tech. rep. no. 148. German National Research Center for Information Technology Bremen
13. Jaeger, H. (2002a). Short-term memory in echo state networks. Tech. rep. no. 152. German National Research Center for Information Technology Bremen
14. Jaeger, H. (2002b). A tutorial on training recurrent neural networks, covering bppt, rtrl, ekf and the "echo state network" approach. Tech. rep. no. 159. German National Research Center for Information Technology Bremen
15. Jaeger H. Adaptive nonlinear system identification with echo state networks. Advances in neural information processing systems 15 (2003), MIT Press 593-600
16. Jaeger H., and Hass H. Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science 304 (2004) 78-80
17. Jaeger H., Lukoševičius M., Popovici D., and Siewert U. 2007 special issue: Optimization and applications of echo state networks with leaky-integrator neurons. Neural Networks 20 3 (2007) 335-352
18. Knapp C., and Carter G. The generalized correlation method for estimation of time delay. Acoustics, Speech, and Signal Processing; IEEE Transactions on Signal Processing (1976)
19. Maass W., and Bishop C. Pulsed neural networks (2001), MIT Press
20. Maass W., Natschlaeger T., and Markram H. Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation 14 11 (2002) 2531-2560
21. Niedzwiecki M., and Cisowski K. Smart copying - a new approach to reconstruction of audio signals. IEEE Transactions on Signal Processing 49 (2001) 2272-2282
22. Paugam-Moisy H., Martinez R., and Bengio S. Delay learning and polychronization for reservoir computing. Neurocomputing 71 7-9 (2008) 1143-1158
23. Pedersen, K. I., Fleury, B. H., & Mogensen, P. E. (1997). High resolution of electromagnetic waves in time-varying radiochannels. In Personal, indoor and mobile radio communications, 1997. 'Waves of the year 2000', Vol. 2 (pp. 650-654)
24. Schmidhuber J., Wierstra D., Gagliolo M., and Gomez F. Training recurrent networks by evolino. Neural Computation 19 3 (2007) 757-779
25. Siegelmann H.T., and Sontag E.D. Turing computability with neural nets. Applied Mathematics Letters 4 (1991) 77-80
26. Smith J.O. Introduction to digital filters with audio applications (2007), W3K Publishing Online book, accessed 14.4.2008
27. Smith J.O. Mathematics of the discrete fourier transform (dft) (2007), W3K Publishing Online book, accessed 24.4.2008
28. Swadlow H. Physiological properties of individual cerebral axons studied in vivo for as long as one year. Journal of Neurophysiology 54 (1985) 1346-1362
29. Verstraeten D., Schrauwen B., D'Haene M., and Stroobandt D. 2007 special issue: An experimental unification of reservoir computing methods. Neural Networks 20 3 (2007) 391-403
30. Wustlich W., and Siewert U. Echo-state networks with band-pass neurons: Towards generic time-scale-independent reservoir structures. PLANET Intelligent Systems GmbH (2007) posted on the reservoir computing mailing list, accessed 23.3.2008
31. Wyffels, F., Schrauwen, B., Verstraeten, D., & Stroobandt, D. (2008). Band-pass reservoir computing (pp. 3203-3208)
32. Xue Y., Yang L., and Haykin S. 2007 special issue: Decoupled echo state networks with lateral inhibition. Neural Networks 20 3 (2007) 365-376