On-Line Learning with Minimal Degradation in Feedforward Networks

IEEE Transactions on Neural Networks

Volumn 6, Issue 3, 1995, Pages 657-668

  de Angulo, Vincente Ruiz ^a   Torras, Carme ^b  

^a JOINT RESEARCH CENTRE   (Italy)

^b CSIC Instituto de Cibernetica de Barcelona   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE SYSTEMS; ALGORITHMS; BACKPROPAGATION; DATA PROCESSING; ERRORS; INPUT OUTPUT PROGRAMS; LEARNING SYSTEMS; OPTIMIZATION; PERFORMANCE;

FAULT TOLERANCE; FEEDFORWARD NETWORKS; FORGETTING; INPUT OUTPUT PATTERNS; LEARNING RATE; LEARNING WITH MINIMAL DEGRADATION;

FEEDFORWARD NEURAL NETWORKS;

EID: 0029308833     PISSN: 10459227     EISSN: 19410093     Source Type: Journal    
DOI: 10.1109/72.377971     Document Type: Article

References (29)

1. R. Ratcliff, “Connectionist models of recognition memory Constraints imposed by learning and forgetting functions,” Psychological Review, vol. 97, no. 2, 235-308, 1990.
2. M. McCloskey and N. J. Cohen, “Catastrophic interference in connectionist networks: The sequential learning problem,” in The Psychology of Learning and Motivation, G. H. Bower Ed. New York: Academic, 1989.
3. R. M. French, “Using semi-distributed representations to overcome catastrophic forgetting in connectionist networks,” Center for Research on Concepts and Cognition, Indiana Univ., CRCC Tech. Rep. 51-1991, 1991.
4. J. M. J. Murre, “Categorization and learning in neural networks,” Ph.D. dissertation, Univ. Leiden, 1991.
5. F. J. Smieja, “Hyperplane ' ‘spin’ dynamics, network plasticity and backpropagation learning,” Tech. Rep. German National Res. Centre for Comput. Sci. (GMD), Nov. 1991.
6. O. P. Brouse and P. Smolensky, “Virtual memories and massive generalization in connectionist combinatorial learning,” in Proc. 11th Annual Conf. Cognitive Science Society, 1989, pp. 380-387.
7. P. A. Hetherington, “The sequential learning problem in connectionist networks,” Master's thesis, Dept. Psych., McGill Univ., Montreal, Quebec, Canada, Nov. 1990.
8. J. K. Kruschke, Message to the Connectionists Electronic Mail Distribution List.
9. J. Moody and C. Darken, “Fast learning in networks of localy tuned processing units,” Neural Computation, vol. 1, no. 2, pp. 281-294, 1989.
10. T. Poggio and F. Girosi, “Networks for approximation and learning,” Proc. IEEE, vol. 78, no. 9, pp. 1481-1497, 1990.
11. J. Platt, “A resource-allocating network for function interpolation,” Neural Computation, vol. 3, no. 2, 1989.
12. D. C. Park, M. A. El-Sharkawi, and R. J. Marks II, “An adaptively trained neural network,” IEEE Trans. Neural Networks, vol. 2, no. 3, May 1991.
13. A. Blum and R. L. Rivest, “Training a 3-node neural network is NP-complete,” in Proc. 1988 Workshop Computational Learning Theory, 1988, pp. 9-18.
14. S. Judd, Neural Network Design and the Complexity of MA: MIT Press, 1990.
15. M. C. Mozer and P. Smolensky, “Skeletonization: A technique for trimming the fat from a network via relevance assessment,” Advances in Neural Information Processing I, D. S. Touretzky, Ed. San Mateo, CA: 107-115.
16. V. Ruiz de Angulo and C. Torras, “The MDL algorithm,” in Proc. IWANN ’91, A. Prieto, Ed., Lecture Notes on Computer Science no. 540. New York: Springer-Verlag, 1991, pp. 162-172.
17. B. Widrow and R. Winter, “Neural nets for adaptative filtering and recognition,” Computer, Mar. 1988.
18. S. Becker and Y. Le Cun, “Improving the convergence of backpropagation learning with second order methods,” in Proc. 1988 Connectionist Models Summer School, D. Touretzky, F. Hinton and T. Sejnowski, Eds., 1988, pp. 29-37.
19. R. Scalatter and A. Zee, “Emergence of grandmother memory in feedforward networks. Learning with noise and forgetfulness,” in Connectionists Models and their Implications, Readings from Cognitive Science, D. Waltz and J. H. Feldman, Eds. Norwood, NJ: Ablex, 1988, 309-323.
20. L. P. Ricotti, S. Ragazzini, and G. Martinelli, “Learning word stress in a suboptimal second order back-propagation neural network,” in Proc. IEEE Int. Conf. Neural Networks, San Diego, vol. I, pp. 355-364.
21. V. Ruiz de Angulo, Ph.D. dissertation, in preparation.
22. G. E. Hinton and T. J. Sejnowski, “Learning and relearning in Boltzman machines,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol 1: Foundations, D. E. Rumelhart and J. L. McCLelland, Eds. Cambridge, MA: MIT Press, 1986.
23. J. Mijhuis, Hofflinger, Van Sc. Haik, and L. Spaanenburg, “Limits to the fault-tolerance of a feedforward neural network with learning,” in Proc. 20th International Symp. Fault Tolerance Computing, Newcastle upon Tyne, June 1990, pp. 228-235.
24. Y. Le Cun, J. S. Denker, and S. A. Solla, “Optimal brain damage,” Advances in Neural Information Processing Systems, David S. Touretzky, Ed. San Mateo, CA: Morgan-Kauffman, 1990.
25. E. D. Karnin, “A simple procedure for pruning back-propagation trained neural networks,” IEEE Trans. Neural Networks, vol. 1, no. 2, June 1990.
26. S. Gavin, “Designing multilayer perceptions from nearest-neighbor systems,” IEEE Trans. Neural Networks, vol. 3, no. 2, Mar. 1992.
27. T. Grossman, R. Meir, and E. Domany, “Learning by internal representations,” Complex Syst. 2, pp. 555-575, 1988.
28. A. Krogh, C. J. Thorbergsson, and J. A. Hertz, “A cost function for internal representation,” in Advances in Neural Information Processing Systems, D. S. Touretzky, Ed. San Mateo, CA: Morgan-Kauffman, 1990.
29. R. Rohwer, “The moving target training algorithm,” in Advances in Neural Information Processing Systems, D. S. Touretzky, Ed. San Mateo, CA: Morgan-Kauffman, 1990.