Gradient descent learning in and out of equilibrium

Physical Review E - Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics

Volumn 63, Issue 6, 2001, Pages

  Caticha, Nestor ^a   Araújo de Oliveira, Evaldo ^a  

^a UNIVERSITY OF SÃO PAULO   (Brazil)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTIFICIAL NEURAL NETWORK; BAYES THEOREM; BIOPHYSICS; LEARNING; NORMAL DISTRIBUTION; PROCEDURES; STATISTICAL MODEL;

ALGORITHMS; BAYES THEOREM; BIOPHYSICS; LEARNING; LIKELIHOOD FUNCTIONS; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); NORMAL DISTRIBUTION;

ANNEALING; APPROXIMATION THEORY; GRADIENT METHODS; LEARNING ALGORITHMS; MATHEMATICAL MODELS; MAXIMUM LIKELIHOOD ESTIMATION; NEURAL NETWORKS; ONLINE SYSTEMS; THERMODYNAMIC STABILITY;

BAYESIAN ALGORITHMS; GRADIENT DESCENT LEARNING; KULLBACK-LEIBLER INFORMATION LOSS; ONLINE LEARNING; THERMAL EQUILIBRIUM;

STATISTICAL MECHANICS;

EID: 0035363423     PISSN: 1063651X     EISSN: None     Source Type: Journal    
DOI: 10.1103/PhysRevE.63.061905     Document Type: Article

References (20)

1. S. Amari, IEEE Trans. Electron. Comput. 16, 299 (1967).
2. M. Opper, Phys. Rev. Lett. 77, 4671 (1996).
3. M. Opper, On-Line Learning in Neural Networks, edited by D. Saad (Cambridge University Press, Cambridge, England, 1998).
4. S. Solla and O. Winther, On-Line Learning in Neural Networks, edited by D. Saad (Cambridge University, Cambridge, 1998).
5. E. Levin, N. Tishby, and S. Solla, Proc. IEEE 78, 1568 (1990).
6. S. Kullback, Information Theory and Statistics (Wiley, New York, 1959).
7. M. Griniasty, Phys. Rev. E 47, 4496 (1993).
8. O. Kinouchi and N. Caticha, J. Phys. A 25, 6243 (1992).
9. O. Kinouchi and N. Caticha, Phys. Rev. E 54, R54 (1996).
10. C. Van den Broeck and P. Riemann, Phys. Rev. Lett. 76, 2188 (1996).
11. N. Barkai, H. Seung, H. Sompolinsky, Phys. Rev. Lett. 75, 1415 (1995).
12. R. Meir and J. Fontanari, Phys. Rev. A 45, 8874 (1992).
13. O. Kinouchi and N. Caticha, J. Phys. A 26, 6161 (1993).
14. N. Caticha and O. Kinouchi, Philos. Mag. B 77, 5 1565 (1998).
15. M. Biehl, P. Riegler, and M. Stechert, Phys. Rev. E 52, R4624 (1995).
16. O. Kinouchi and N. Caticha, Phys. Rev. E 52, 2878 (1995).
17. E. A. Oliveira, R. Vicente, and N. Caticha (unpublsihed).
18. C. W. H. Mace and A. C. C. Coolen, Statistical and Computation 8, 55 (1998).
19. A. Engel and C. van den Broeck, The Statistical Mechanics of Learning (Cambridge University Press, Cambridge, England, 2000).
20. M. Opper and D. Haussler, Phys. Rev. Lett. 66, 2677 (1991);in Proceedings of the 4th Annual Workshop on Computational Learning Theory (COLT91) (Morgan Kauffman, San Mateo, 1991).