Online learning dynamics of radial basis function neural networks near the singularity

IEEE International Conference on Neural Networks - Conference Proceedings

Volumn , Issue , 2006, Pages 4770-4776

  Wei, Haikun ^b,c   Amari, Shun Ichi ^a,c  

^a IEEE

^b SOUTHEAST UNIVERSITY   (China)

^c RIKEN BRAIN SCIENCE INSTITUTE   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; FEEDFORWARD NEURAL NETWORKS; HIERARCHICAL SYSTEMS; LEARNING SYSTEMS; ONLINE SYSTEMS;

HIERARCHICAL MODELS; RADIAL BASIS FUNCTION (RBF);

MULTILAYER NEURAL NETWORKS;

EID: 34547640647     PISSN: 10987576     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (16)

1. S. Amari, H. Park and T. Ozeki, "Singularity Problems and Dynamics," Neural Computation, to be published.
2. M. Biehl and N. Caticha, "Statistical mechanics of on line learning and generalization," In M. A. Arbib, handbook of brain theory and neural networks, Cambridge, MA: MIT Press, 2002.
3. M. Biehl, P. Riegler and C. Wohler, "Transient dynamics of on-line learning in two-layered neural networks," J. Phys. A: Math. Gen. vol. 29, pp. 4769-4780, 1996.
4. F. Cousseau, T. Ozeki and S. Amari, "Dynamics of learning in multilayer perceptrons near the singularity," submitted to IEEE transaction on neural networks.
5. J. Freeman and D. Saad, "Online Learning in Radial Basis Function Networks," Neural Computation, vol. 9, pp. 1601-1622, 1997.
6. J. Freeman and D. Saad, "Dynamics of on-line learning in radial basis function networks," Physical Review E, vol. 56, pp. 907-916, 1997.
7. M. Inoue, H. Park and M. Okada, "On-line learning theory of soft committee machines with correlated hidden units - Steepest gradient descent and natural gradient descent," J. Phys. Soc. Jpn, vol. 72, pp. 805-810, 2003.
8. M. Inoue, H. Park and M. Okada, "Dynamics of the adaptive natural gradient descent method for soft committee machines," Physical Review E, vol. 69, 056120, 2004.
9. M. Marinaro and S. Scarpetta, "On-line learning in RBF neural networks: a stochastic approach," Neural networks, vol. 13, pp. 719-729, 2000.
10. J. Moody and C. Darken, "Fast learning in networks of locallytuned processing units", Neural Computation, vol. 1, pp. 1281-294, 1989.
11. P. Niyogi and F. Girosi, "On the relationship between Generalization Error, Hypothesis Complexity, and Sample Complexity for Radial Basis Function," Neural Computation, vol. 8, pp. 819-842, 1996.
12. H. Park, M. Inoue and M. Okada, "Online learning dynamics of multilayer perceptrons with unidentifiable parameters," J. Phys. A: Math. Gen. vol. 36, pp. 11753-11764, 2003.
13. H. Park, M. Inoue and M. Okada, "Slow dynamics due to singularities of hierarchical learning machines" Progress of Theoretical Physics Supplement, vol. 157, pp. 275-279, 2005.
14. P. Riegler and M. Biehl, "On-line backpropagation in two-layered neural networks," J. Phys. A: Math. Gen. 28, pp. L507-L513, 1995.
15. D. Saad and A. Solla, "On-Line Learning Theory of Soft Committee Machines," Physical Review E, vol. 52, pp. 4225-4243, 1995.
16. D. Saad and A. Solla, "Exact solution for on-line learning in multilayer neural networks," Physical Review Letters, vol. 74, pp. 4337-4340, 1995.