Analysis of natural gradient descent for multilayer neural networks

Physical Review E - Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics

Volumn 59, Issue 4, 1999, Pages 4523-4532

  Rattray, Magnus ^a   Saad, David ^b  

^a UNIVERSITY OF MANCHESTER   (United Kingdom)

^b ASTON UNIVERSITY   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; LEARNING SYSTEMS; STATISTICAL MECHANICS;

NATURAL GRADIENT DESCENT (NGD) ALGORITHMS;

MULTILAYER NEURAL NETWORKS;

EID: 0032613639     PISSN: 1063651X     EISSN: None     Source Type: Journal    
DOI: 10.1103/PhysRevE.59.4523     Document Type: Article

References (22)

1. Online Learning in Neural Networks, edited by D. Saad, Publications of the Newton Institute (Cambridge University Press, London, 1998).
2. S. Amari, Neural Comput. 10, 251 (1998).
3. H. H. Yang and S. Amari, in Advances in Neural Information Processing Systems, edited by M. I. Jordan, M. J. Kearns, and S. A. Solla (MIT Press, Cambridge, MA, 1998), Vol. 10, p. 385.
4. H. H. Yang and S. Amari, Neural Comput. 10, 2137 (1998).
5. M. Rattray, D. Saad, and S. Amari, Phys. Rev. Lett. 81, 5461 (1998).
6. T. Cover and J. Thomas, Elements of Information Theory (Wiley, New York, 1991).
7. G. B. Orr and T. K. Leen in Advances in Neural Information Processing Systems, edited by M. C. Mozer, M. I. Jordan, and T. Petsche (MIT Press, Cambridge, MA, 1997), Vol. 9, p. 606.
8. S. Scarpetta, M. Rattray, and D. Saad (unpublished).
9. D. Saad and S. A. Solla, Phys. Rev. Lett. 74, 4337 (1995)
10. D. SaadS. A. SollaPhys. Rev. E 52, 4225 (1995).
11. D. Barber, D. Saad, and P. Sollich, Europhys. Lett. 34, 151 (1996).
12. P. Riegler and M. Biehl, J. Phys. A 28, L507 (1995).
13. M. Biehl, P. Riegler, and C. Wöhler, J. Phys. A 29, 4769 (1996).
14. A. H. L. West and D. Saad, Phys. Rev. E 56, 3426 (1997).
15. D. Saad and M. Rattray, Phys. Rev. Lett. 79, 2578 (1997)
16. M. Rattray and D. Saad, Phys. Rev. E 58, 6379 (1998).
17. R. Vicente and N. Caticha, J. Phys. A 30, L599 (1997).
18. T. K. Leen, B. Schottky, and D. Saad, in Advances in Neural Information Processing Systems, edited by M. I. Jordan, M. J. Kearns, and S. A. Solla (MIT Press, Cambridge, MA, 1998), Vol. 10, p. 301
19. Phys. Rev. E 59, 985 (1999).
20. S. Amari and N. Murata, Neural Comput. 5, 140 (1993).
21. G. Cybenko, Math. Control, Signals and Systems 2, 303 (1989).
22. M. Rattray and D. Saad, in Online Learning in Neural Networks (Ref. 1), p. 185.