Complexity Issues in Natural Gradient Descent Method for Training Multilayer Perceptrons

Neural Computation

Volumn 10, Issue 8, 1998, Pages 2137-2157

  Yang, Howard Hua ^a   Amari, Shun Ichi ^b  

^a OREGON HEALTH AND SCIENCE UNIVERSITY   (United States)

^b RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; LEARNING; NERVE CELL; STATISTICS;

LEARNING; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER); NEURONS; STOCHASTIC PROCESSES;

EID: 0032533046     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976698300017007     Document Type: Article

References (9)

1. Amari, S. (1997). Neural learning in structured parameter spaces - Natural Riemannian gradient. In M. C. Mozer, M. I. Jordan, & T. Petsche (Eds.), Advances in neural information processing systems, 9 (pp. 127-133) Cambridge, MA: MIT Press.
2. Amari, S. (1998). Natural gradient works efficiently in learning. Neural Computation, 10, 251-276.
3. Orr, G. B., & Leen, T. K. (1997). Using curvature information for fast stochastic search. In M. C. Mozer, M. I. Jordan & T. Petsche (Eds.), Advances in neural information processing systems, 9. Cambridge, MA: MIT Press.
4. Saad, D., & Solla, S. A. (1995). On-line learning in soft committee machines. Physical Review E, 52, 4225-4243.
5. Stewart, G. W. (1973). Introduction to matrix computations. New York: Academic Press.
6. Yang, H. H., & Amari, S. (1997a). Natural gradient descent for training multilayer perceptrons. Unpublished manuscript.
7. Yang, H. H., & Amari, S. (1997b). Training multi-layer perceptrons by natural gradient descent. In ICONIP'97 Proceedings, New Zealand.
8. Yang, H. H., & Amari, S. (1998). The efficiency and the robustness of natural gradient descent learning rule. In M. I. Jordan, M. J. Kearns, & S. A. Solla (Eds.), Advances in neural information processing systems, 10. Cambridge, MA: MIT Press.
9. Yang, H. H., Murata, N., & Amari, S. (1998). Statistical inference: Learning in artificial neural networks. Trends in Cognitive Sciences, 2(1), 4-10.