Intrinsic plasticity via natural gradient descent

ESANN 2012 proceedings, 20th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning

Volumn , Issue , 2012, Pages 555-560

  Neumann, Klaus ^a   Steil, Jochen J ^a  

^a BIELEFELD UNIVERSITY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; NEURAL NETWORKS;

ACTIVATION FUNCTIONS; GEOMETRIC PROPERTIES; INPUT DISTRIBUTIONS; INTRINSIC PLASTICITY; NATURAL GRADIENT; PARAMETER SPACES;

LEARNING SYSTEMS;

EID: 84947730351     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (10)

1. Rumelhart, D. E. and Hinton, G. E. and Williams, R. J.: Learning repre- sentations by back-propagating errors. Nature, pages 533-536, volume 323, 1986.
2. Amari Shun-ichi: Differential-Geometrical Methods in Statistics. Lecture Notes in Statistics 28, New-York: Springer-Verlag, 1985.
3. Amari Shun-Ichi: Natural gradient works eficiently in learning. Neural Computation, pages 251-276, volume 10, 1998.
4. Amari, S.-I. and Douglas, S. C.: Why natural gradient? Acoustics, Speech, and Signal Processing, pages 1213-1216, volume 2, 1998.
5. Douglas, C. C. and Hu, J. and Ray, J. and Thorne, D. T. and Tuminaro, R. S.: Natural-gradient adaptation. Unsupervised Adaptive Filtering, Vol. 1: Blind Source Separation, pages 13-61, 2000.
6. Triesch, J.: Synergies beween Intrinsic and Synaptic Plasticity in Individual Model Neurons. NIPS, 2005.
7. Steil, J. J.: Online Reservoir Adaptation by Intrinsic Plasticity for Backprop- agation - Decorrelation and Echo State Learning. Neural Networks, Special Issue on Echo State and Liquid State networks, pages 353-364, 2007.
8. Rao, C. R.: Information and the accuracy attainable in the estimation of statistical parameters. Bull. Calcutta Math. Soc., 1945.
9. Kullback, S. and Leibler, A.: On Information and Sufficiency. Ann. Math. Statist., pages 79-86, volume 22, number 1, 1951.
10. Kostal, L. and Lansky, P.: Classification of stationary neuronal activity according to its information rate. Network Computation in Neural Systems, volume 17, pages 193-210, 2006.