Reservoir computing and extreme learning machines for non-linear time-series data analysis

Neural Networks

Volumn 38, Issue , 2013, Pages 76-89

  Butcher, J B ^a   Verstraeten, D ^b   Schrauwen, B ^b   Day, C R ^a   Haycock, P W ^a  

^a KEELE UNIVERSITY   (United Kingdom)

^b GHENT UNIVERSITY   (Belgium)

Author keywords

Extreme learning machine; Non linearity; Reservoir computing; Reservoir with random static projections; Short term memory; Time series data

Indexed keywords

EXTREME LEARNING MACHINE; NON-LINEARITY; RESERVOIR COMPUTING; SHORT TERM MEMORY; TIME-SERIES DATA;

BRAIN; DATA HANDLING; KNOWLEDGE ACQUISITION; NETWORK ARCHITECTURE; RECURRENT NEURAL NETWORKS;

TIME SERIES ANALYSIS;

ALGORITHM; ARTICLE; CALCULATION; DATA ANALYSIS; LINEAR SYSTEM; MACHINE LEARNING; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; NONLINEAR SYSTEM; PRIORITY JOURNAL; SHORT TERM MEMORY; SIGNAL NOISE RATIO; SIMULATION; TIME SERIES ANALYSIS;

ARTIFICIAL INTELLIGENCE; MATHEMATICAL COMPUTING; NONLINEAR DYNAMICS; RANDOM ALLOCATION; STATISTICS AS TOPIC;

EID: 84871651156     PISSN: 08936080     EISSN: 18792782     Source Type: Journal    
DOI: 10.1016/j.neunet.2012.11.011     Document Type: Article

References (29)

1. Antonelo, E., Schrauwen, B., & Stroobandt, D. (2008). Mobile robot control in the road sign problem using reservoir computing networks. In Proceedings of the IEEE international conference on robotics and automation (pp. 911-916).
2. Büsing L., Schrauwen B., Legenstein R. Connectivity, dynamics and memory in reservoir computing with binary and analog neurons. Neural Computation 2010, 22(5):1272-1311.
3. Butcher, J.B. (2012). Reservoir computing with high non-linear separation and long-term memory for time-series data analysis, Ph.D. Thesis, Institute for the Environment, Physical Sciences and Applied Mathematics (EPSAM), Keele University, Staffordshire, ST5 5BG, UK.
4. Butcher, J.B., Verstraeten, D., Schrauwen, B., Day, C.R., & Haycock, P.W. (2010a). Extending reservoir computing with random static projections: a hybrid between OP-ELM and RC. In European symposium on artificial neural networks, ESANN (pp. 303-308).
5. Butcher, J.B., Verstraeten, D., Schrauwen, B., Day, C.R., & Haycock, P.W. (2010b). Pruning reservoirs with random static projections. In IEEE International workshop on machine learning for signal processing, MLSP (pp. 250-255). http://dx.doi.org/10.1109/MLSP.2010.5589251.
6. Butcher, J.B., Verstraeten, D., Schrauwen, B., Day, C.R., & Haycock, P.W. (2012). Defect detection in reinforced concrete using random neural architectures. In Computer aided civil and infrastructure engineering (submitted for publication).
7. Chen F., Ou T. Sales forecasting system based on Gray extreme learning machine with Taguchi method in retail industry. Expert Systems with Applications 2011, 38(3):1336-1345.
8. Doddington G., Schalk T. Speech recognition: turning theory to practice. IEEE Spectrum 1981, 18(9):26-32.
9. Gallicchio C., Micheli A. Architectural and Markovian factors of echo state networks. Neural Networks 2011, 24(5):440-456.
10. Holzmann, G. (2009). Reservoir computing: a powerful black-box framework for nonlinear audio processing. In 12th international conference on digital audio effects, DAFx-09. Como, Italy.
11. Huang G., Wang D. Advances in extreme learning machines (elm2010). Neurocomputing 2011, 74(16):2411-2412.
12. Huang G., Zhu Q., Siew C. Extreme learning machines: theory and applications. Neurocomputing 2006, 70(1-3):489-501.
13. Jaeger, H. (2001). The "echo state" approach to analysing and training recurrent neural networks, Tech. rep., German National Research Institute for Computer Science.
14. Jaeger, H. (2002a). Adaptive nonlinear system identification with echo state networks. In Neural information processing systems, NIPS.
15. Jaeger, H. (2002b). A tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the "echo state network" approach, Tech. Rep. GMD Report 159, German National Research Center for Information Technology.
16. Jaeger H., Haas H. Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 2004, 304(5667):78-80.
17. Jaeger H., Lukoševičius M., Popovici D., Siewert U. Optimisation and applications of echo state networks with leaky-integrator neurons. Neural Networks 2007, 20(3):335-352.
18. Lukoševičius M., Jaeger H. Reservoir computing approaches to recurrent neural network training. Computer Science Review 2009, 3(3):127-149.
19. Montgomery D., Peck E. Introduction to linear regression analysis. Probability and mathematical statistics 1982, Wiley, USA.
20. Moore E. On the reciprocal of the general algebraic matrix. Bulletin of The American Mathematical Society 1920, 26:394-395.
21. Penrose R. A generalised inverse for matrices. Cambridge philosophical society, Vol. 51 1955, 406-413).
22. Rees D. Essential statistics 2000, Chapman and Hall, London, UK. 4th Ed.
23. Singh R., Balasundaram S. Application of extreme learning machine method for time series analysis. International Journal of Intelligent Technology 2007, 2(4):256-262.
24. van Heeswijk M., Miche Y., Lindh-Knuutila T., Hilbers P., Honkela T., Oja E., et al. Adaptive ensemble models of extreme learning machines for time series prediction. Proceedings of the 19th international conference on artificial neural networks 2009, 305-314. Springer-Verlag, Berlin, Heidelberg.
25. Verstraeten, D. (2009). Reservoir computing: computation with dynamical systems, Ph.D. Thesis, Electronics and Information Systems department, University of Ghent, Belgium.
26. Verstraeten, D., Dambre, J., Dutoit, X., & Schrauwen, B. (2010). Memory versus non-linearity in reservoirs. In International joint conference on neural networks, IJCNN (pp. 2669-2676).
27. Verstraeten D., Schrauwen B., D'Haene M., Stroobandt D. An experimental unification of reservoir computing methods. Neural Networks 2007, 20(3):391-403.
28. Verstraeten, D., Schrauwen, B., & Stroobandt, D. (2006). Reservoir-based techniques for speech recognition. In Proceedings of the international joint conference on neural networks, IJCNN (pp. 1050-1053).
29. Verstraeten D., Schrauwen B., Stroobandt D., Van Campenhout J. Isolated word recognition with a liquid state machine: a case study. Information Processing Letters 2005, 95(6):521-528.