The consolidation of learning during sleep: Comparing the pseudorehearsal and unlearning accounts

Neural Networks

Volumn 12, Issue 7-8, 1999, Pages 1191-1206

  Robins, A ^a   McCallum, S ^a  

^a UNIVERSITY OF OTAGO   (New Zealand)

Author keywords

Catastrophic forgetting; Consolidation; Pseudorehearsal; Sleep; Unlearning

Indexed keywords

COMPUTATIONAL METHODS; LEARNING SYSTEMS; MATHEMATICAL MODELS; NEUROPHYSIOLOGY; PSYCHOPHYSIOLOGY;

CATASTROPHIC FORGETTING; PSEUDOREHEARSAL; SLEEP; UNLEARNING;

NEURAL NETWORKS;

ARCHITECTURE; ARTIFICIAL NEURAL NETWORK; COGNITION; DREAMING; HIPPOCAMPUS; HIPPOCAMPUS THETA RHYTHM; HUMAN; INFORMATION; LEARNING; LONG TERM MEMORY; MEMORY; MODEL; NEOCORTEX; NONHUMAN; PRIORITY JOURNAL; REM SLEEP; REVIEW; SIMULATION; SLEEP; TRAINING;

EID: 0033213822     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(99)00056-8     Document Type: Article

References (57)

1. Abbot L. Learning in neural network memories. Network. 1:1990;105-122.
2. Ans B., Rousset S. Avoiding catastrophic forgetting by coupling two reverberating neural networks. C. R. Academy of Science Paris, Life Sciences. 320:1997;989-997.
3. Brown D.W. Crick and Mitchison's theory of REM sleep and neural networks. Medical Hypotheses. 40:1993;329-331.
4. Burgess N., Shapiro J.L., Moore M.A. Neural network models of list learning. Network. 2:1991;399-422.
5. Buzaski G. Memory consolidation during sleep - a neurophysiological perspective. Journal of Sleep Research. 7 (suppl. 1):1998;17-23.
6. Christos G.A. Investigation of the Crick-Mitchison reverse-learning dream sleep hypothesis in a dynamical setting. Neural Networks. 9:1996;427-434.
7. Crick F., Mitchison G. The function of dream sleep. Nature. 304:1983;111-114.
8. Crick F., Mitchison G. REM sleep and neural nets. Behavioural Brain Research. 69:1995;147-155.
9. Davis B.D. Sleep and the maintenance of memory. Perspective in Biology and Medicine. 28:1985;457-464.
10. Empson J. Sleep and dreaming. 2nd ed :1993;Harvester Wheatsheaf, Hemel Hempstead, UK.
11. Frean M. A "thermal" perceptron learning rule. Neural Computation. 4:1992;946-957.
12. Frean M., Robins A. Catastrophic forgetting and 'pseudorehearsal' in linear networks. Australian Journal of Intelligent Information Processing Systems. 5:1998;44-49.
13. French R.M. Semi-distributed representations and catastrophic forgetting in connectionist networks. Connection Science. 4:(3/4):1992;365-377.
14. French R.M. Dynamically constraining connectionist networks to produce distributed, orthogonal representations to reduce catastrophic interference. Proceedings of the 16th Annual Cognitive Science Society Conference. 1994;Earlbaum, Hillsdale, NJ. pp. 335-340.
15. French R.M. Interactive connectionist networks: an approach to the 'sensitivity-stability' dilemma. Connection Science. 9:1997;353-380.
16. Gardner E.J., Stroud N., Wallace D.J. Training with noise and the storage of correlated patterns in a neural network model. Journal of Physics A: Mathematics and General. 22:1989;2019-2030.
17. Giuditta A., Ambrosini M.V., Montagnese P., Mandile P., Cotugno M., Zucconi G., Vescia S. The sequential hypothesis of the function of sleep. Behavioural Brain Research. 69:1995;157-166.
18. Globus G.G. Connectionism and sleep. Moffitt A., Kramer M. The functions of dreaming. 1993;119-138 State University of New York Press, Albany, NY.
19. Greenberg R., Pearlman C.A. Cutting the REM nerve: an approach to the adaptive role of REM sleep. Perspectives in Biology and Medicine. 17:1974;513-521.
20. Grossberg S. Competitive learning: From interactive activation to adaptive resonance. Cognitive Science. 11:1987;23-63.
21. Harlow J., Roll S. Frequency of day residue in dreams of young adults. Perceptive Motor Skills. 74:1992;832-834.
22. Hennevin E., Hars B., Maho C., Bloch V. Processing of learned information in paradoxical sleep: Relevance for memory. Behavioural Brain Research. 69:1995;125-135.
23. Hertz J., Krough A., Palmer R.G. Introduction to the theory of neural computation. 1991;Addison-Wesley, Redwood City, CA.
24. Hetherington P.A., Seidenberg M.S. Is there "catastrophic interference" in connectionist networks? Proceedings of the Eleventh Annual Conference of the Cognitive Science Society. 1989;Earlbaum, Hillsdale, NJ. pp. 26-33.
25. Hinton G.E., Plaut D.C. Using fast weights to deblur old memories. Proceedings of the Ninth Annual Conference of the Cognitive Science Society. 1987;Earlbaum, Hillsdale, NJ. pp. 177-186.
26. Hinton G.E., Sejnowski T.J. Learning and relearning in Boltzmann machines. Rumelhart D.E., McClelland J.L., PDP Research Group. Parallel distributed processing: explorations in the microstructure of cognition: foundations. 1:1986;282-317 MIT Press, Cambridge, MA.
27. Hobson A.J., Mamelak A.N., Sutton J.P. Models wanted: must fit dimensions of sleep and dreaming. Moody J.E, Lippman R.P. Advances in neural information processing. Vol. 4:1992;3-10 Morgan Kaufmann, San Mateo, CA.
28. Hopfield J.J., Feinstein D.I., Palmer R.G. Unlearning has a stabilising effect in collective memories. Nature. 304:1983;158-159.
29. Horn D., Levy N., Ruppin E. Memory maintenance via neuronal regulation. Neural Computation. 10:1998;1-18.
30. Kavanau J.L. Memory, sleep, and the evolution of mechanisms of synaptic efficacy maintenance. Neuroscience. 9:1997;7-44.
31. Kortge C.A. Episodic memory in connectionist networks. Proceedings of the 12th Annual Conference of the Cognitive Science Society. 1990;Earlbaum, Hillsdale, NJ. pp. 764-771.
32. Lewandowsky S. Gradual unlearning and catastrophic interference: a comparison of distributed architectures. Hockley W.E., Lewandowsky S. Relating theory and data: essays on human memory in honour of Bennet, B., Murdok. 1991;445-476 Earlbaum, Hillsdale, NJ.
33. Lewandowsky S., Li S. Catastrophic interference in neural networks: causes, solutions, and data. Dempster F.N., Brainerd C. New perspectives on interference and inhibition in cognition. 1995;329-361 Academic Press, San Diego, CA.
34. McClelland J.L., McNaughton B.L., O'Rielly R.C. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological Review. 102:1995;419-457.
35. McCloskey M., Cohen N.J. Catastrophic interference in connectionist networks: The sequential learning problem. Bower G.H. The psychology of learning and motivation. 23:1989;109-164 Academic Press, New York.
36. McRae K., Hetherington P.A. Catastrophic interference is eliminated in pretrained networks. Proceedings of the Fifteenth Annual Meeting of the Cognitive Science Society. 1993;Earlbaum, Hillsdale, NJ. pp. 723-728.
37. Moffitt A., Kramer M. The functions of dreaming. 1993;State University of New York Press, Albany, NY.
38. Murphy, P.M., & Aha, D.W. (1994). UCI repository of machine learning databases [http://www.ics.uci.edu/~mlearn/MLRepository.html]. Irvine, CA: University of California, Department of Information and Computer Science.
39. Murre J. Learning and categorization in modular neural networks. 1992a;Earlbaum, Hillsdale, NJ.
40. Murre J. The effects of pattern presentation on interference in backpropagation networks. Proceedings of the 14th Annual Conference of the Cognitive Science Society. 1992b;Earlbaum, Hillsdale, NJ. pp. 54-59.
41. Nadal J.P., Toulouse G., Changeux J.P., Dehaene S. Networks of formal neurons and memory palimpsets. Europhysics Letters. 1:1986;535-542.
42. Nakao M., Honda I., Musila M., Yamamoto M. Metastable associative network models of dream sleep. Neural Networks. 10:1997;1289-1302.
43. Ratcliff R. Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions. Psychological Review. 97:1990;285-308.
44. Robins A. Catastrophic forgetting, rehearsal, and pseudorehearsal. Connection Science. 7:1995;123-146.
45. Robins A. Consolidation in neural networks and in the sleeping brain. Connection Science. 8:1996;259-275.
46. Robins A. Maintaining stability during new learning in neural networks. Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics (SMC97). 1997;IEEE Computer Society Press, Los Alamos, NM. pp. 3013-3018.
47. Robins A., McCallum S. Catastrophic forgetting and the pseudorehearsal solution in hopfield type networks. Connection Science. 7:1998;121-135.
48. Rumelhart D.E. The architecture of mind: a connectionist approach. Posner M.I. Foundations of cognitive science. 1989;133-159 MIT Press, Cambridge, MA.
49. Sharkey N., Sharkey A. An analysis of catastrophic interference. Connection Science. 7:1995;301-329.
50. Silver, D. & Mercer, R. (1998). The task rehearsal method of sequential learning. Technical Report. Ontario: University of Western Ontario, Department of Computer Science.
51. Smith C. REM sleep and learning: some recent findings. Moffitt A., Kramer M. The functions of dreaming. 1993;341-362 State University of New York Press, Albany, NY.
52. Smith C. Sleep states, memory processes and synaptic plasticity. Behavioural Brain Research. 78:1996;49-56.
53. Sutton J.P., Mamelak A.N., Hobson J.A. Modeling states of waking and sleeping. Psychiatric Annals. 23:1992;137-143.
54. Van Hemmen J. Hebbian learning, its correlation catastrophe, and unlearning. Network. 8:1997;V1-V17.
55. Wilson M.A., McNaughton B.L. Reactivation of hippocampal ensemble memories during sleep. Science. 265:1994;676-679.
56. Wimbauer S., Klemmer N., Van Hemmen J. Universality of unlearning. Neural Networks. 7:1994;261-270.
57. Winson J. The meaning of dreams. Scientific American. November:1990;42-48.