Improving recurrent neural network performance using transfer entropy

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 6444 LNCS, Issue PART 2, 2010, Pages 193-200

  Obst, Oliver ^a,b   Boedecker, Joschka ^c,d   Asada, Minoru ^c,d  

^a CSIRO   (Australia)

^b UNIVERSITY OF SYDNEY   (Australia)

^c OSAKA UNIVERSITY   (Japan)

^d JAPAN SCIENCE AND TECHNOLOGY AGENCY   (Japan)

Author keywords

guided self organisation; information theory; Machine learning; recurrent neural network; reservoir computing

Indexed keywords

HIDDEN LAYERS; INFORMATION TRANSFERS; INPUT AND OUTPUTS; INTRINSIC PLASTICITY; LEARNING GOALS; MACHINE-LEARNING; OFFLINE; ONLINE LEARNING; RECURSIVE LEAST SQUARES; RESERVOIR COMPUTING; RESERVOIR QUALITY; SELF-ORGANISATION; SELF-ORGANISED; SYNTHETIC DATA; TRANSFER ENTROPY; RECURSIVE LEAST SQUARE (RLS); SELF ORGANISATION;

DATA PROCESSING; INFORMATION THEORY; NETWORK PERFORMANCE; REINFORCEMENT LEARNING; COMPUTATION THEORY; LEARNING SYSTEMS;

RECURRENT NEURAL NETWORKS;

EID: 78650211628     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-17534-3_24     Document Type: Conference Paper

References (15)

1. Lukosevicius, M., Jaeger, H.: Reservoir computing approaches to recurrent neural network training. Computer Science Review 3(3), 127-149 (2009)
2. Maass, W., Natschläger, T., Markram, H.: Real-time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation 14(11), 2531-2560 (2002)
3. Jaeger, H., Haas, H.: Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communication. Science 304(5667), 78-80 (2004)
4. Boedecker, J., Obst, O., Mayer, N.M., Asada, M.: Initialization and self-organized optimization of recurrent neural network connectivity. HFSP Journal 3(5), 340-349 (2009)
5. Triesch, J.: A gradient rule for the plasticity of a neuron's intrinsic excitability. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3696, pp. 65-70. Springer, Heidelberg (2005)
6. Steil, J.J.: Online reservoir adaptation by intrinsic plasticity for backpropagation-decorrelation and echo state learning. Neural Networks 20(3), 353-364 (2007)
7. Hebb, D.O.: The organization of behavior: a neuropsychological theory. Lawrence Erlbaum Associates, Mahwah (1949)
8. Prokopenko, M.: Guided self-organization. HFSP Journal 3(5), 287-289 (2009)
9. Steil, J.J.: Backpropagation-decorrelation: Recurrent learning with O(N) complexity. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN), vol. 1, pp. 843-848 (2004)
10. Hayes, M.H.: Chapter 9.4 Recursive Least Squares. In: Statistical Digital Signal Processing and Modeling. Wiley, Chichester (1996)
11. Jaeger, H.: The "echo state" approach to analysing and training recurrent neural networks. Technical Report 148, GMD - German National Research Institute for Computer Science (2001)
12. Jaeger, H.: Adaptive nonlineaer systems identification with echo state networks. In: Advances in Neural Information Processing Systems, pp. 609-615 (2003)
13. Schreiber, T.: Measuring information transfer. Physical Review Letters 85(2), 461-464 (2000)
14. Hajnal, M., Locombining double acute accentrincz, A.: Critical echo state networks. In: Kollias, S.D., Stafylopatis, A., Duch, W., Oja, E. (eds.) ICANN 2006. LNCS, vol. 4131, pp. 658-667. Springer, Heidelberg (2006)
15. Lizier, J.T., Prokopenko, M., Zomaya, A.Y.: Detecting non-trivial computation in complex dynamics. In: Almeida e Costa, F., Rocha, L.M., Costa, E., Harvey, I., Coutinho, A. (eds.) ECAL 2007. LNCS (LNAI), vol. 4648, pp. 895-904. Springer, Heidelberg (2007)