The effiiciency and the robustness of natural gradient descent learning rule

Advances in Neural Information Processing Systems

Volumn , Issue , 1998, Pages 385-391

  Yang, Howard Hua ^a   Amari, Shun Ichi ^b  

^a OREGON HEALTH AND SCIENCE UNIVERSITY   (United States)

^b RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

FISHER INFORMATION MATRIX;

INPUT DIMENSIONS; LEARNING RULES; MULTI LAYER PERCEPTRON; MULTI-LAYER PERCEPTRONS; NATURAL GRADIENT; NUMBER OF HIDDEN NEURONS; SINGLE LAYER;

ALGORITHMS;

EID: 84899033584     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (5)

1. S. Amari. Natural gradient works efficiently in learning. Accepted by Neural Computation, 1997.
2. S. Amari. Neural learning in structured parameter spaces - natural Riemannian gradient. In Advances in Neural Information Processing Systems, 9, ed. M. C. Mozer, M. I. Jordan and T. Petsche, The MIT Press: Cambridge, MA., pages 127-133, 1997.
3. C. Darken and J. Moody. Towards faster stochastic gradient search. In Advances in Neural Information Processing Systems, 4, eds. Moody, Hanson, and Lippmann, Morgan Kaufmann, San Mateo, pages 1009-1016, 1992.
4. D. Saad and S. A. Solla. On-line learning in soft committee machines. Physical Review E, 52:4225-4243, 1995.
5. H. H. Yang and S. Amari. Natural gradient descent for training multi-layer perceptrons. Submitted to IEEE Tr. on Neural Networks, 1997.