Dynamics of learning near singularities in radial basis function networks

Neural Networks

Volumn 21, Issue 7, 2008, Pages 989-1005

  Wei, Haikun ^a,b,c   Amari, Shun ichi ^a  

^a RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

^b SOUTHEAST UNIVERSITY   (China)

^c KYUSHU INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

Dynamics of learning; Gradient descent learning; Neural networks; Plateau phenomenon; RBF networks; Singularity; Stability analysis

Indexed keywords

ATTITUDE CONTROL; COMPUTER NETWORKS; DYNAMICS; FEEDFORWARD NEURAL NETWORKS; IMAGE SEGMENTATION; NEURAL NETWORKS; POLYNOMIAL APPROXIMATION;

DYNAMICS OF LEARNING; GRADIENT DESCENT LEARNING; PLATEAU PHENOMENON; RBF NETWORKS; SINGULARITY; STABILITY ANALYSIS;

RADIAL BASIS FUNCTION NETWORKS;

ARTICLE; BEHAVIOR; COMPUTER SIMULATION; DYNAMICS; NETWORK LEARNING; ONLINE ANALYTICAL PROCESSING; PRIORITY JOURNAL; PROCESS DESIGN; RADIAL BASIS FUNCTION NETWORK; REGRESSION ANALYSIS;

ANIMALS; BEHAVIOR; COMPUTER SIMULATION; HUMANS; LEARNING; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; NUMERICAL ANALYSIS, COMPUTER-ASSISTED; REGRESSION ANALYSIS; TIME FACTORS;

EID: 51049114453     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2008.06.017     Document Type: Article

References (14)

1. Amari S. Theory of adaptive pattern classifiers. IEEE Transactions on Electronic Computers EC-16 3 (1967) 299-307
2. Amari S., Park H., and Ozeki T. Singularities affect dynamics of learning in neuromanifolds. Neural Computation 18 (2006) 1007-1065
3. Biehl M., and Schwarze H. Learning by online gradient descent. Journal of Physics A-Mathematical and General 28 (1995) 643-656
4. Cousseau, F., Ozeki, T., & Amari, S. (2007). Dynamics of learning in multilayer perceptrons near the singularity. IEEE Transactions on Neural Networks (in press)
5. Freeman J., and Saad D. Dynamics of on-line learning in radial basis function networks. Physical Review E 56 (1997) 907-916
6. Freeman J., and Saad D. Online learning in radial basis function networks. Neural Computation 9 (1997) 1601-1622
7. Fukumizu K. A regularity condition of information matrix of a multilayer perceptron network. Neural Networks 9 (1996) 871-879
8. Fukumizu K., and Amari S. Local minima and plateaus in hierarchical structure of multilayer perceptrons. Neural Networks 13 (2000) 317-327
9. Geman S. Some averaging and stability results for random differential equations. SIAM Journal of Applied Mathematics 36 (1980) 86-105
10. Horn R., and Johnson C. Matrix analysis (1985), Cambridge University Press, Cambridge, England
11. Ostrowski A. A quantitative formulation of Sylvester's law of inertia. Proceedings of the National Academy of Sciences 45 (1959) 740-744
12. Park H., Inoue M., and Okada M. Slow dynamics due to singularities of hierarchical learning machines. Progress of Theoretical Physics Supplement 157 (2005) 275-279
13. Saad D., and Solla A. On-line learning in soft committee machines. Physical Review E 52 4 (1995) 4225-4243
14. Wei H., Zhang J., Cousseau F., Ozeki T., and Amari S. Dynamics of learning near singularities in layered networks. Neural Computation 20 3 (2008) 813-843