A theoretical comparison of batch-mode, on-line, cyclic, and almost-cyclic learning

IEEE Transactions on Neural Networks

Volumn 7, Issue 4, 1996, Pages 919-925

  Heskes, Tom ^a   Wiegerinck, Wim ^a  

^a RADBOUD UNIVERSITY NIJMEGEN   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

ASYMPTOTIC STABILITY; DIFFERENTIAL EQUATIONS; NEURAL NETWORKS; ONLINE SYSTEMS; RANDOM PROCESSES;

ALMOST CYCLIC LEARNING; BATCH MODE LEARNING; CYCLIC LEARNING; INCREMENTAL LEARNING; ONLINE LEARNING;

LEARNING SYSTEMS;

EID: 0030195986     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.508935     Document Type: Review

References (20)

1. D. Rumelhart, J. McClelland, and the PDP Research Group, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Cambridge, MA: MIT Press, 1986.
2. T. Kohonen, "Self-organized formation of topologically correct feature maps," Biological Cybernetics, vol. 43, pp. 59-69, 1982.
3. C. Kuan and K. Hornik, "Convergence of learning algorithms with constant learning rates," IEEE Trans. Neural Networks, vol. 2, pp. 484-489, 1991.
4. G. Radons, "On stochastic dynamics of supervised learning," J. Phys. A, vol. 26, pp. 3455-3461, 1993.
5. L. Hansen, R. Pathria, and P. Salamon, "Stochastic dynamics of supervised learning," J. Phys. A, vol. 26, pp. 63-71, 1993.
6. T. Heskes, "On Fokker-Planck approximations of on-line learning processes," J. Phys. A, vol. 27, pp. 5145-5160, 1994.
7. S. Amari, "A theory of adaptive pattern classifiers," IEEE Trans. Elec. Comput., vol. EC 16, 299-307, 1967.
8. T. Heskes and B. Kappen, "Learning processes in neural networks," Phys. Rev. A, vol. 44, pp. 2718-2726, 1991.
9. W. Wiegerinck and T. Heskes, "On-line learning with time-correlated patterns," Europhysics Lett., vol. 28, pp. 451-455, 1994.
10. G. Orr and T. Leen, Momentum and Optimal Stochastic Search, M. Mozer, P. Smolensky, D. Touretzky, J. Elman, and A. Weigend, Eds., in Proc. 1993 Connectionist Models Summer School. Hillsdale: Erlbaum, 1993.
11. W. Wiegerinck, A. Komoda, and T. Heskes, "Stochastic dynamics of learning with momentum in neural networks," J. Phys. A, vol. 27, pp. 4425-4437, 1994.
12. Z. Luo, "On the convergence of LMS algorithm with adaptive learning rate for linear feedforward networks," Neural Computation, vol. 3, pp. 226-245, 1991.
13. N. van Kampen, Stochastic Processes in Physics and Chemistry. Amsterdam: North-Holland, 1992.
14. W. Press, S. Teukolsky, W. Vetterling, and B. Flannery, Numerical Recipes in C, 2nd ed. Cambridge: Cambridge Univ. Press, 1992.
15. C. Gardiner, Handbook of Stochastic Methods, 2nd ed. Berlin: Springer, 1985.
16. G. Radons, H. Schuster, and D. Werner, "Fokker-Planck description of learning in backpropagation networks," in Proc. Int. Neural Network Conf. 1990 Paris. Dordrecht: Kluwer Academic, 1990, pp. 993-996.
17. W. Buntine and A. Weigend, "Computing second derivatives in feedforward networks: A review," IEEE Trans. Neural Networks, vol. 5, pp. 480-488, 1994.
18. T. Heskes, E. Slijpen, and B. Kappen, "Learning in neural networks with local minima," Phys. Rev. A, vol. 46, pp. 5221-5231, 1992.
19. _, "Cooling schedules for learning in neural networks," Physical Rev. E, vol. 47, pp. 4457-4464, 1993.
20. E. Barnard, "Optimization for neural nets," IEEE Trans. Neural Networks, vol. 3, pp. 232-240, 1992.