Optimal asymptotic learning rate: Macroscopic versus microscopic dynamics

Physical Review E - Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics

Volumn 59, Issue 1, 1999, Pages 985-991

  Leen, Todd K ^a   Schottky, Bernhard ^b   Saad, David ^b  

^a OREGON HEALTH AND SCIENCE UNIVERSITY   (United States)

^b ASTON UNIVERSITY   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION THEORY; ASYMPTOTIC STABILITY; EIGENVALUES AND EIGENFUNCTIONS; LEARNING SYSTEMS; NEURAL NETWORKS; NUMERICAL METHODS; PARTIAL DIFFERENTIAL EQUATIONS; PERTURBATION TECHNIQUES; PROBABILITY DENSITY FUNCTION; SIMULATED ANNEALING;

ASYMPTOTIC LEARNING RATE; MACROSCOPIC DYNAMICS; MICROSCOPIC DYNAMICS;

LEARNING ALGORITHMS;

EID: 0032615384     PISSN: 1063651X     EISSN: None     Source Type: Journal    
DOI: 10.1103/PhysRevE.59.985     Document Type: Article

References (25)

1. V. Fabian, Ann. Math. Stat. 39, 1327 (1968).
2. L. Goldstein, University of Southern California, Report No. DRB-306, 1987 (unpublished).
3. T. M. Heskes and B. Kappen, in Mathematical Foundations of Neural Networks, edited by J. Taylor (Elsevier, Amsterdam, 1993), p. 199.
4. T. K. Leen and J. E. Moody, in Advances in Neural Information Processing Systems, edited by C. L. Giles, S. J. Hanson, and J. D. Cowan (Morgan Kaufmann, San Mateo, CA, 1993), Vol. 5, p. 451.
5. T. K. Leen and G. B. Orr, in Advances in Neural Information Processing Systems, edited by J. D. Cowan, G. Tesauro, and J. Alspector (Morgan Kaufmann, San Francisco, 1994), Vol. 6, p. 477.
6. T. K. Leen and J. E. Moody, Phys. Rev. E 56, 1262 (1997).
7. W. Wiegerinck, A. Komoda, and T. Heskes, J. Phys. A 27, 4425 (1994).
8. T. Heskes and J. Coolen, J. Phys. A 30, 4983 (1997).
9. M. Biehl and H. Schwarze, J. Phys. A 28, 643 (1995).
10. D. Saad and S. A. Solla, Phys. Rev. Lett. 74, 4337 (1995)
11. D. SaadS. A. SollaPhys. Rev. E 52, 4225 (1995).
12. M. Biehl, P. Riegler, and C. Wöhler, J. Phys. A 29, 4769 (1996).
13. A. H. L. West and D. Saad, Phys. Rev. E 56, 3426 (1997).
14. D. Saad and M. Rattray, Phys. Rev. E 57, 2170 (1998).
15. D. Saad and M. Rattray, Phys. Rev. Lett. 79, 2578 (1997).
16. M. Rattray and D. Saad, Phys. Rev. E 58, 6379(1998).
17. M. Rattray and D. Saad, J. Phys. A 30, L771 (1997).
18. G. B. Orr, Ph.D. thesis, Oregon Graduate Institute, 1996 (unpublished).
19. N. Barkai, Ph.D. thesis, Hebrew University of Jerusalem, 1995 (unpublished).
20. C. W. H. Mace and A. C. C. Coolen, Stat. Comput. 8, 55 (1998).
21. D. Barber, D. Saad, and P. Sollich, Europhys. Lett. 34, 151 (1996).
22. P. Riegler and M. Biehl, J. Phys. A 28, L507 (1995).
23. G. B. Orr and Todd K. Leen, in Advances in Neural Information Processing Systems, edited by M. Mozer, M. Jordan, and T. Petsche (MIT Press, Cambridge, MA, 1997), Vol. 9, p. 606.
24. T. K. Leen, in On-Line Learning in Neural Networks, edited by D. Saad, Publications of the Newton Institute (Cambridge University Press, Cambridge, 1998), p. 43.
25. For annealing as (Formula presented) with (Formula presented), the order parameter equations are a bit more restrictive to deal with than the equations of motion for the fluctuation density. In particular, we were able to solve the asymptotic, linearized, order parameter equations (in terms of exponential integrals) only for (Formula presented), with b an integer, placing some restriction on the values of p considered. However, this technical difficulty might yield under further effort.