Overcoming catastrophic forgetting in neural networks

Proceedings of the National Academy of Sciences of the United States of America

Volumn 114, Issue 13, 2017, Pages 3521-3526

  Kirkpatrick, James ^a   Pascanu, Razvan ^a   Rabinowitz, Neil ^a   Veness, Joel ^a   Desjardins, Guillaume ^a   Rusu, Andrei A ^a   Milan, Kieran ^a   Quan, John ^a   Ramalho, Tiago ^a   Grabska Barwinska, Agnieszka ^a   Hassabis, Demis ^a   Clopath, Claudia ^a   Kumaran, Dharshan ^a   Hadsell, Raia ^b  

^a DEEPMIND   (United Kingdom)

^b IMPERIAL COLLEGE LONDON   (United Kingdom)

Author keywords

Artificial intelligence; Continual learning; Deep learning; Stability plasticity; Synaptic consolidation

Indexed keywords

ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; BRAIN; CLASSIFICATION ALGORITHM; CONTROLLED STUDY; GAME; HANDWRITING; HUMAN; HUMAN EXPERIMENT; INFORMATION PROCESSING; LEARNING; LEARNING DISORDER; MEMORY CONSOLIDATION; NETWORK LEARNING; PRIORITY JOURNAL; PROBLEM SOLVING; SYNAPSE; TASK PERFORMANCE; TRAINING; ALGORITHM; COMPUTER SIMULATION; MEMORY; RECALL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; HUMANS; LEARNING; MEMORY; MENTAL RECALL; NEURAL NETWORKS (COMPUTER);

EID: 85016395012     PISSN: 00278424     EISSN: 10916490     Source Type: Journal    
DOI: 10.1073/pnas.1611835114     Document Type: Article

References (44)

1. Legg S, Hutter M (2007) Universal intelligence: A definition of machine intelligence Minds Mach 17(4):391-444
2. French RM (1999) Catastrophic forgetting in connectionist networks. Trends Cognit Sci 3(4):128-135
3. McCloskey M, Cohen NJ (1989) Catastrophic interference in connectionist networks: The sequential learning problem. The Psychology of Learning and Motivation, ed GH Bower (Academic, New York), Vol 24, pp 109-165
4. McClelland JL, McNaughton BL, O'Reilly RC (1995) Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychol Rev 102(3): 419-457
5. Kumaran D, Hassabis D, McClelland JL (2016) What learning systems do intelligent agents need? Complementary learning systems theory updated. Trends Cogn Sci 20(7):512-534
6. Ratcliff R (1990) Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions. Psychol Rev 97(2):285-308
7. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25, eds Pereira F, Burges CJC, Bottou L, Weinberger KQ (Curran Assoc, Red Hook, NY), pp 1097-1105
8. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436-444
9. Rusu AA, et al. (2015) Policy distillation. arXiv:1511.06295
10. Parisotto E, Ba JL, Salakhutdinov R (2015) Actor-mimic: Deep multitask and transfer reinforcement learning. arXiv:1511.06342
11. Cichon J, Gan WB (2015) Branch-specific dendritic ca2+ spikes cause persistent synaptic plasticity. Nature 520(7546):180-185
12. Hayashi-Takagi A, et al. (2015) Labelling and optical erasure of synaptic memory traces in the motor cortex. Nature 525(7569):333-338
13. Yang G, Pan F, Gan WB (2009) Stably maintained dendritic spines are associated with lifelong memories. Nature 462(7275):920-924
14. Yang G, et al. (2014) Sleep promotes branch-specific formation of dendritic spines after learning. Science 344(6188):1173-1178
15. Fusi S, Drew PJ, Abbott L (2005) Cascade models of synaptically stored memories Neuron 45(4):599-611
16. Benna MK, Fusi S (2015) Computational principles of biological memory. arXiv:1507 07580
17. Hecht-Nielsen R (1988) Theory of the backpropagating network. Neural Netw 1(Suppl 1):445-448
18. Sussmann HJ (1992) Uniqueness of the weights for minimal feedforward nets with a given input-output map. Neural Networks 5:589-593
19. MacKay DJ (1992) A practical Bayesian framework for backpropagation networks Neural Comput 4(3):448-472
20. Pascanu R, Bengio Y (2013) Revisiting natural gradient for deep networks. arXiv:1301 3584
21. Eskin E, Smola AJ, Vishwanathan S (2004) Laplace propagation. Advances in Neural Information Processing Systems 16, eds Thrun S, Saul LK, Schoelkopf PB (MIT Press, Cambridge, MA), pp 441-448
22. LeCun Y, Cortes C, Burges CJ (1998) The MNIST database of handwritten digits. Available at yann.lecun.com/exdb/mnist/. Accessed March 3, 2017
23. Srivastava RK, Masci J, Kazerounian S, Gomez F, Schmidhuber J (2013) Compete to compute. Advances in Neural Information Processing Systems 26, eds Burges CJC, Bottou L, Welling M, Ghahramani Z, Weinberg KQ (Curran Assoc, Red Hook, NY), Vol 26, pp 2310-2318
24. Goodfellow IJ, Mirza M, Xiao D, Courville A, Bengio Y (2015) An empirical investigation of catastrophic forgeting in gradient-based neural networks. arXiv:1312.6211
25. Mnih V, et al. (2015) Human-level control through deep reinforcement learning Nature 518(7540):529-533
26. Bellemare MG, Naddaf Y, Veness J, Bowling M (2013) The arcade learning environment: An evaluation platform for general agents. J Artif Intell Res 47:253-279
27. Ring MB (1998) Child: A first step towards continual learning. Learning to Learn, eds Thrun S, Pratt L (Kluwer Acadamic, Norwell, MA), pp 261-292
28. Rusu AA, et al. (2016) Progressive neural networks. arXiv:1606.04671
29. Collins AG, Frank MJ (2013) Cognitive control over learning: Creating, clustering, and generalizing task-set structure. Psychol Rev 120(1):190-229
30. O'Reilly RC, Frank MJ (2006) Making working memory work: A computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput 18(2): 283-328
31. Mante V, Sussillo D, Shenoy KV, Newsome WT (2013) Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature 503(7474):78-84
32. Miller EK, Cohen JD (2001) An integrative theory of prefrontal cortex function. Annu Rev Neurosci 24(1):167-202
33. Doya K, Samejima K, Katagiri K-i, Kawato M (2002) Multiple model-based reinforcement learning. Neural Comput 14(6):1347-1369
34. Milan K, et al. (2016) The forget-me-not process. Advances in Neural Information Processing Systems 29, eds Lee DD, Sugiyama M, Luxburg UV, Guyon I, Garnett R (Curran Assoc, Red Hook, NY)
35. French RM, Chater N (2002) Using noise to compute error surfaces in connectionist networks: A novel means of reducing catastrophic forgetting. Neural Comput 14(7):1755-1769
36. Ruvolo PL, Eaton E (2013) ELLA: An efficient lifelong learning algorithm. JMLR Workshop Conf Proc 28(1):507-515
37. Blundell C, Cornebise J, Kavukcuoglu K, Wierstra D (2015) Weight uncertainty in neural networks. JMLR Workshop Conf Proc 37:1613-1622
38. Amit D (1989) Modeling Brain Function (Cambridge Univ Press, Cambridge, UK)
39. Aitchison L, Latham PE (2015) Synaptic sampling: A connection between PSP variability and uncertainty explains neurophysiological observations. arXiv:1505.04544
40. Pfister JP, Dayan P, Lengyel M (2010) Synapses with short-Term plasticity are optimal estimators of presynaptic membrane potentials. Nat Neurosci 13(10):1271-1275
41. Clopath C, Ziegler L, Vasilaki E, Bü sing L, GerstnerW(2008) Tag-Trigger-consolidation: A model of early and late long-Term-potentiation and depression. PLoS Comput Biol 4(12):e1000248
42. van Hasselt H, Guez A, Silver D (2016) Deep reinforcement learning with double q-learning. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, eds Schuurmans D, Wellman M (AAAI Press, Palo Alto, CA), pp 2094-2100
43. Veness J, Ng KS, Hutter M, Bowling M (2012) Context tree switching. 2012 Data Compression Conference, ed Malvar H (IEEE, New York), pp 327-336
44. Dowson D, Landau B (1982) The fré chet distance between multivariate normal distributions J Multivar Anal 12(3):450-455