Catastrophic forgetting in connectionist networks

Trends in Cognitive Sciences

Volumn 3, Issue 4, 1999, Pages 128-135

  French, Robert M ^a  

^a UNIVERSITY OF LIÈGE   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

ACCIDENT; AMNESIA; COGNITION; DISASTER; HUMAN; LEARNING; MATHEMATICAL MODEL; NERVE BLOCK; NERVE CELL NETWORK; PATHOPHYSIOLOGY; REVIEW;

EID: 0032923221     PISSN: 13646613     EISSN: None     Source Type: Journal    
DOI: 10.1016/S1364-6613(99)01294-2     Document Type: Review

References (49)

1. McCloskey, M. and Cohen, N. (1989) Catastrophic interference in connectionist networks: the sequential learning problem, in The Psychology of Learning and Motivation (Vol. 24) (Bower, G.H., ed.), pp. 109-154, Academic Press
2. Ratcliff, R. (1990) Connectionist models of recognition memory: constraints imposed by learning and forgetting functions Psychol. Rev. 97, 285-308
3. Minsky, M. and Papert, S. (1969) Perceptrons, MIT Press
4. Rosenblatt, F. (1958) The perceptron: a probabilistic model for information storage and organization in the brain Psychol. Rev. 65, 386-408
5. Rosenblatt, F. (1962) Principles of Neurodynamics, Spartan, NY, USA
6. Grossberg. S. (1982) Studies of Mind and Brain: Neural Principles of Learning, Perception, Development, Cognition and Motor Control, Reidel, Boston, MA, USA
7. Carpenter, G. and Grossberg, S. (1987) ART 2: self-organization of stable category recognition codes for analog input patterns Appl. Opt. 23, 4919-4930
8. Barnes, J. and Underwood, B. (1959) 'Fate' of first-learned associations in transfer theory J. Exp. Psychol. 58, 97-105
9. Rumelhart, D., Hinton, G. and Williams, R. (1986) Learning representations by back-propagating error Nature 323, 533-536
10. Carpenter G. and Grossberg, S. (1986) Adaptive resonance theory: stable self-organization of neural recognition codes in response to arbitrary list of input patterns, in Proceedings of the Eighth Annual Conference of the Cognitive Science Society, pp. 45-62, Erlbaum
11. Hinton, G., McClelland, J. and Rumelhart, D. (1986) Distributed representations, in Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Vol. 1. Foundations (Rumelhart, D. and McClelland, J., eds), pp. 77-109, MIT Press
12. Hinton G. and Plaut D. (1987) Using fast weights to deblur old memories, in Proceedings of the Ninth Annual Conference of the Cognitive Science Society, pp. 177-186, Erlbaum
13. Sutton, R. (1986) Two problems with backpropagation and other steepest-descent learning procedures for networks, in Proceedings of the Eighth Annual Conference of the Cognitive Society, pp. 823-831, Erlbaum
14. Hetherington, P. (1991) The Sequential Learning Problem, Master's Thesis, Department of Psychology, McGill University, Montreal, Québec, Canada
15. Hetherington, P. and Seidenberg, M. (1989) Is there 'catastrophic interference' in connectionist networks?, in Proceedings of the 11th Annual Conference of the Cognitive Science Society, pp. 26-33, Erlbaum
16. Ebbinghaus, H. (1885) Über das Gedächtnis: Untersuchen zur Experimentellen Psychologie ('On memory') (H.A. Ruger and C.E. Bussenius, transi. 1954), Dover
17. Kortge, C. (1990) Episodic memory in connectionist networks, in Proceedings of the 12th Annual Conference of the Cognitive Science Society, pp. 764-771, Erlbaum
18. French, R.M. (1991) Using semi-distributed representations to overcome catastrophic forgetting in connectionist networks, in Proceedings of the 13th Annual Cognitive Science Society Conference, pp. 173-178, Erlbaum
19. French, R.M. (1992) Semi-distributed representations and catastrophic forgetting in connectionist networks Connection Sei. A, 365-377
20. Brousse, O. and Smolensky, P. (1989) Virtual memories and massive generalization in connectionist combinatorial learning, in Proceedings of the 11th Annual Conference of the Cognitive Science Society, pp. 380-387, Erlbaum
21. McRae, K. and Hetherington, P. (1993) Catastrophic interference is eliminated in pretrained networks, in Proceedings of the 15th Annual Conference of the Cognitive Science Society, pp. 723-728, Erlbaum
22. Sharkey, N. and Sharkey, A. (1995) An analysis of catastrophic interference Connection Sci. 7, 301-329
23. French, R.M. (1994) Dynamically constraining connectionist networks to produce distributed, orthogonal representations to reduce catastrophic interference, in Proceedings of the 16th Annual Conference of the Cognitive Science Society, pp. 335-340, Erlbaum
24. Lewandowsky, S. and Shu-Chen Li (1993) Catastrophic interference in neural networks: causes, solutions, and data, in New Perspectives on Interference and Inhibition in Cognition (Dempster, F.N. and Brainerd, C., eds), pp. 329-361, Academic Press
25. Lewandowsky S. (1991) Gradual unlearning and catastrophic interference: a comparison of distributed architectures, in Relating Theory and Data: Essays on Human Memory in Honor of Bennet B. Murdock (Hockley, W. and Lewandowsky, S., eds), pp. 445-476, Erlbaum
26. Murre, J. (1992) The effects of pattern presentation on interference in backpropagation networks, in Proceedings of the 14th Annual Conference of the Cognitive Science Society, pp. 54-59, Erlbaum
27. Murre, J. (1992) Learning and Categorization in Modular Neural Networks, Erlbaum
28. Krushke, J. (1992) ALCOVE: an exemplar-based model of category learning Psychol. Rev. 99, 22-44
29. Kruschke, J. (1993) Human category learning: implications for backpropagation models Connection Sei. 5, 3-36
30. Sloman, S. and Rumelhart, D. (1992) 'Reducing interference in distributed memories through episodic gating', in Essays in Honor of W. K. Estes (Healy, A., Kosslyn, S. and Shiffrin, R., eds), pp. 227-248, Erlbaum
31. Carpenter, G. and Grossberg, S. (1987) A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer Vision, Graphics and Image Processing 37, 54-115
32. Carpenter, G. (1994) A distributed outstar network for spatial pattern learning Neural Netw. 7, 159-168
33. Metcalfe, J. (1982) A composite holographic associative recall model Psychol. Rev. 89, 627-661
34. Murdock, B. (1983) A distributed memory model for serial-order information Psycho!. Rev. 100, 183-203
35. Kanerva, P. (1989) Sparse Distributed Memory, MIT Press
36. Keeler, J.D. (1988) Comparison between Kanerva's SDM and Hopfield-type neural networks Cognit. Sei. 12, 279-329
37. Robins, A. (1995) Catastrophic forgetting, rehearsal, and pseudorehearsal Connection Sei. 7, 123-146
38. McClelland, J., McNaughton, B. and O'Reilly, R. (1995) Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory Psycho!. Rev. 102, 419-457
39. French, R.M. (1997) Pseudo-recurrent connectionist networks: an approach to the 'sensitivity-stability' dilemma Connection Sei. 9, 353-379
40. Ans, B. and Rousset, S. (1997) Avoiding catastrophic forgetting by coupling two reverberating neural networks Academie des Sciences (Sciences de la vie) 320, 989-997
41. Rueckl, J. (1993) Jumpnet: a multiple-memory connectionist architecture, in Proceedings of the 15th Annual Conference of the Cognitive Science Society, pp. 866-871, Erlbaum
42. French, R.M. and Mareschal, D. (1998) Could category-specific anomia reflect differences in the distributions of features within a unified semantic memory?, in Proceedings of the 20th Annual Cognitive Science Society Conference, pp. 374-379, Erlbaum
43. Ratcliff, R. and Murdock, B. (1976) Retrieval processes in recognition memory Psychol. Rev. 83, 190-214
44. Strong, E. (1912) The effect of length of series upon recognition memory Psychol. Rev. 19, 447-462
45. Murnane, K. and Shiffrin (1991) Interference and the representation of events in memory J. Exp. Psychol: Learn. Mem. Cognit. 17, 355-374
46. Chappell M. and Humphreys, M. (1994) An auto-associative neural network for sparse representations: analysis and application to models of recognition and cued recall Psychol. Rev. 101, 103-128
47. Levy J. and Bairaktaris, D. (1995) Connectionist dual-weight architectures Lang. Cognit. Process. 10, 265-283
48. Fahlman, S. and Libiere, C. (1990) The cascade-correlation learning architecture, in Advances in Neural Information Processing Systems (Vol. 2) (Touretsky, D., ed.), pp. 524-532, Morgan-Kaufmann
49. Tetewsky, S.J., Shultz, T.R. and Takane, Y. (1995) Training regimens and function compatibility: implications for understanding the effects of knowledge on concept learning, in Proceedings of the 17th Annual Conference of the Cognitive Science Society, pp. 304-309, Erlbaum