On-line learning with adaptive back-propagation in two-layer networks

Physical Review E - Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics

Volumn 56, Issue 3, 1997, Pages 3426-3445

  West, Ansgar H L ^a,b   Saad, David ^a  

^a ASTON UNIVERSITY   (United Kingdom)

^b UNIVERSITY OF EDINBURGH   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0040708121     PISSN: 1063651X     EISSN: None     Source Type: Journal    
DOI: 10.1103/PhysRevE.56.3426     Document Type: Article

References (20)

1. C. Cybenko, Math. Control Signals Syst. 2, 303 (1989).
2. D. Saad and S. A. Solla, Phys. Rev. E 52, 4225 (1995).PLEEE8
3. T. K. Leen and G. B. Orr, in Advances in Neural Information Processing Systems, edited by J. D. Cowan, G. Tesauro, and J. Alspector (Kaufmann, San Francisco, 1994), Vol. 6, p. 477.
4. A. Prügel-Bennett (unpublished).
5. S. Amari, in Advances in Neural Information Processing Systems, edited by M. C. Mozer, M. I. Jordan, and T. Petsche (MIT Press, Cambridge, 1997), Vol. 9, p. 127.
6. O. Kinouchi and N. Caticha, J. Phys. A 25, 6243 (1992); JPHAC5
7. M. Copelli and N. Caticha, 28, 1615 (1995);
8. M. Copelli, O. Kinouchi, and N. Caticha, Phys. Rev. E 53, 6341 (1996).PLEEE8
9. M. Biehl and P. Riegler, Europhys. Lett. 28, 525 (1994).EULEEJ
10. J. Kim and H. Sompolinsky, Phys. Rev. Lett. 76, 3021 (1996).PRLTAO
11. M. Biehl and H. Schwarze, J. Phys. A 28, 643 (1995).JPHAC5
12. D. Saad and M. Rattray, in Proceedings of the Minerva Workshop on Mesoscopics, Fractals and Neural Networks, Eilat, 1997 (unpublished).
13. D. Barber, D. Saad, and P. Sollich, Europhys. Lett. 34, 151 (1996).EULEEJ
14. A. H. L. West (unpublished).
15. D. Barber, D. Saad, and P. Sollich, Europhys. Lett. p. 288.
16. A. H. L. West and D. Saad, in Advances in Neural Information Processing Systems, edited by D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo (MIT Press, Cambridge, MA, 1996), Vol. 8, p. 323.
17. The term adaptive in ABP therefore refers to the adjustability of (Formula presented) and to the subsequent deformation of the search space. It does not imply an ability of the algorithm to tune its adjustable parameters on-line.
18. Although we find numerically exponents for (Formula presented) smaller than (Formula presented) for larger (Formula presented), it remains unclear if these hold even for (Formula presented). However, for (Formula presented), (Formula presented), and (Formula presented), the power law seems to approach (Formula presented).
19. due to different scaling for (Formula presented).
20. P. Riegler, Ph.D. thesis, University of Würzburg, 1997 (unpublished).