Local minima and plateaus in hierarchical structures of multilayer perceptrons

Neural Networks

Volumn 13, Issue 3, 2000, Pages 317-327

  Fukumizu, K ^a   Amari, S ^a  

^a RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

Author keywords

Error surface; Local minima; Multilayer perceptron; Plateau

Indexed keywords

COMPUTATIONAL GEOMETRY; ERROR ANALYSIS; LEARNING SYSTEMS; MATHEMATICAL MODELS;

LOCAL MINIMA;

MULTILAYER NEURAL NETWORKS;

ARTICLE; GEOMETRY; LEARNING; MODEL; NERVE CELL NETWORK; PERCEPTION; PRIORITY JOURNAL; SIMULATION;

MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER);

EID: 0034018074     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(00)00009-5     Document Type: Article

References (14)

1. Amari S. Natural gradient works efficiently in learning. Neural Computation. 10:1998;251-276.
2. Amari, S., Park, H., & Fukumizu, K. (2000). Adaptive method of realizing natural gradient learning for multilayer perceptrons. Neural Computation. In press.
3. Chen A.M., Lu H., Hecht-Nielsen R. On the geometry of feedforward neural network error surfaces. Neural Computation. 5:1993;910-927.
4. Fukumizu K. A regularity condition of the information matrix of a multilayer perceptron network. Neural Networks. 9:(5):1996;871-879.
5. Hamney L.G.C. XOR has no local minima: a case study in neural network error surface analysis. Neural Networks. 11:(4):1998;669-682.
6. Kůrkova V., Kainen P.C. Functionally equivalent feedforward neural networks. Neural Computation. 6:1994;543-558.
7. Lisboa P.J.G., Perantonis S.J. Complete solution of the local minima in the XOR problem. Network. 2:1991;119-124.
8. Rattray M., Saad D., Amari S. Natural gradient descent for on-line learning. Physical Review Letters. 81:1998;5461-5465.
9. Rüger S.M., Ossen A. The metric of weight space. Neural Processing Letters. 5:1997;63-72.
10. Saad D., Solla S. On-line learning in soft committee machines. Physical Review E. 52:1995;4225-4243.
11. Sprinkhuizen-Kuyper I.G., Boers E.J.W. The error surface of the 2-2-1 XOR network: the finite stationary points. Neural Networks. 11:(4):1998;683-690.
12. Sprinkhuizen-Kuyper, I. G., & Boers, E. J. W. (1996). The error surface of the 2-2-1 XOR network: stationary points with infinite weights. Technical Report 96-10, Department of Computer Science, Leiden University.
13. Sussmann H.J. Uniqueness of the weights for minimal feedforward nets with a given input-output map. Neural Networks. 5:1992;589-593.
14. Yang H.H., Amari S. Complexity issues in natural gradient descent method for training multilayer perceptrons. Neural Computation. 10:1998;2137-2157.