Taming the reservoir: Feedforward training for recurrent neural networks

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2012, Pages

  Obst, Oliver ^a   Riedmiller, Martin ^b  

^a CSIRO   (Australia)

^b UNIVERSITY OF FREIBURG   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ECHO STATE NETWORKS; FEED-FORWARD; HIDDEN LAYERS; HIGH VARIABILITY; LIQUID STATE MACHINES; RESERVOIR COMPUTING; TIME SERIES PREDICTION; TRAINING DATA; TRAINING METHODS;

ARTIFICIAL INTELLIGENCE; TIME SERIES;

RECURRENT NEURAL NETWORKS;

EID: 84865072849     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2012.6252506     Document Type: Conference Paper

References (14)

1. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, Learning internal representations by error propagation. Cambridge, MA, USA: MIT Press, 1986, pp. 318-362.
2. H. Jaeger and H. Haas, "Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication," Science, vol. 304, no. 5667, pp. 78-80, 2004. [Online]. Available: http://www.sciencemag.org/cgi/ content/abstract/304/5667/78
3. W. Maass, T. Natschläger, and H. Markram, "Real-time computing without stable states: A new framework for neural computation based on perturbations," Neural Computation, vol. 14, no. 11, pp. 2531-2560, 2002.
4. D. Prokhorov, "Echo state networks: appeal and challenges," in IEEE International Joint Conference on Neural Networks (IJCNN '05), vol. 3, 2005, pp. 1463-1466.
5. J. Boedecker, O. Obst, N. M. Mayer, and M. Asada, "Initialization and self-organized optimization of recurrent neural network connectivity," HFSP Journal, vol. 3, no. 5, pp. 340-349, Oct. 2009. [Online]. Available: http://dx.doi.org/10.2976/1.3240502
6. M. Hermans and B. Schrauwen, "One step backpropagation through time for learning input mapping in reservoir computing applied to speech recognition," in Proceedings of 2010 IEEE International Symposium on Circuits and Systems (ISCAS), Jun. 2010, pp. 521-524.
7. J. Triesch, "A gradient rule for the plasticity of a neuron's intrinsic excitability," in Proceedings of the International Conference on Artificial Neural Networks (ICANN 2005), ser. Lecture Notes in Computer Science, W. Duch, J. Kacprzyk, E. Oja, and S. Zadrozny, Eds. Springer, 2005, pp. 65-70.
8. J. J. Steil, "Online reservoir adaptation by intrinsic plasticity for backpropagation-decorrelation and echo state learning," Neural Networks, vol. 20, no. 3, pp. 353-364, Apr. 2007.
9. O. Obst, J. Boedecker, and M. Asada, "Improving recurrent neural network performance using transfer entropy," in Neural Information Processing. Models and Applications, ser. Lecture Notes in Computer Science, K. W. Wong, B. S. U. Mendis, and A. Bouzerdoum, Eds. Springer, 2010, vol. 6444, pp. 193-200.
10. F. Rosenblatt, "The perceptron: A probabilistic model for information storage and organisation in the brain," Psychological Review, vol. 65, no. 6, pp. 386-408, 1958.
11. M. Riedmiller and H. Braun, "A direct adaptive method for faster backpropagation learning: the RPROP algorithm," in IEEE International Conference on Neural Networks, vol. 1, 1993, pp. 586-591.
12. J. Bergstra and Y. Bengio, "Random search for hyper-parameter optimization," Journal of Machine Learning Research, vol. 13, pp. 281-305, 2012.
13. M. Cottrell, B. Girard, and P. Rousset, "Forecasting of curves using a kohonen classification," Journal of Forecasting, vol. 17, no. 5-6, pp. 429-439, 1998.
14. B. de Vries and J. C. Principe, "The gamma model - a new neural model for temporal processing," Neural Networks, vol. 5, no. 4, pp. 565-576, 1992.