FearNet: Brain-inspired model for incremental learning

6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings

Volumn , Issue , 2018, Pages

  Kemker, Ronald ^a   Kanan, Christopher ^a  

^a ROCHESTER INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AUDIO ACOUSTICS; BENCHMARKING; BRAIN; DEEP NEURAL NETWORKS; MAMMALS;

AUDIO CLASSIFICATION; CATASTROPHIC FORGETTING; HIPPOCAMPAL COMPLEX; INCREMENTAL LEARNING; LONG-TERM STORAGE; MEMORY CONSOLIDATION; PREFRONTAL CORTEX; STATE-OF-THE-ART PERFORMANCE;

SLEEP RESEARCH;

EID: 85083951383     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (42)

1. Wickliffe C Abraham and Anthony Robins. Memory retention–the synaptic stability versus plasticity dilemma. Trends in Neurosciences, 28(2):73–78, 2005.
2. Sami Abu-El-Haija, Nisarg Kothari, Joonseok Lee, et al. Youtube-8m: A large-scale video classification benchmark. arXiv:1609.08675, 2016.
3. Joseph Altman. Autoradiographic investigation of cell proliferation in the brains of rats and cats. The Anatomical Record, 145(4):573–591, 1963.
4. Bernard Ans and Stphane Rousset. Avoiding catastrophic forgetting by coupling two reverberating neural networks. Comptes Rendus de l’Acadmie des Sciences - Series III - Sciences de la Vie, 320 (12):989 – 997, 1997. ISSN 0764-4469.
5. Bruno Bontempi, Catherine Laurent-Demir, Claude Destrade, and Robert Jaffard. Time-dependent reorganization of brain circuitry underlying long-term memory storage. Nature, 400(6745):671–675, 1999.
6. Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and accurate deep network learning by exponential linear units (elus). In ICLR, 2016.
7. Robert Coop, Aaron Mishtal, and Itamar Arel. Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE Trans. on Neural Networks and Learning Systems, 24(10):1623–1634, 2013.
8. Wei Deng, James B Aimone, and Fred H Gage. New neurons and new memories: how does adult hippocampal neurogenesis affect learning and memory? Nature Reviews Neuroscience, 11(5): 339–350, 2010.
9. Timothy J Draelos, Nadine E Miner, Christopher C Lamb, Jonathan A Cox, Craig M Vineyard, Kristofor D Carlson, William M Severa, Conrad D James, and James B Aimone. Neurogenesis deep learning: Extending deep networks to accommodate new classes. In International Joint Conference on Neural Networks, pp. 526–533. IEEE, 2017.
10. Peter S Eriksson, Ekaterina Perfilieva, Thomas Björk-Eriksson, Ann-Marie Alborn, Claes Nordborg, Daniel A Peterson, and Fred H Gage. Neurogenesis in the adult human hippocampus. Nature medicine, 4(11):1313–1317, 1998.
11. David R Euston, Aaron J Gruber, and Bruce L McNaughton. The role of medial prefrontal cortex in memory and decision making. Neuron, 76(6):1057–1070, 2012.
12. Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A Rusu, Alexander Pritzel, and Daan Wierstra. Pathnet: Evolution channels gradient descent in super neural networks. arXiv:1701.08734, 2017.
13. Paul W Frankland, Bruno Bontempi, Lynn E Talton, Leszek Kaczmarek, and Alcino J Silva. The involvement of the anterior cingulate cortex in remote contextual fear memory. Science, 304 (5672):881–883, 2004.
14. Robert M French. Pseudo-recurrent connectionist networks: An approach to the ‘sensitivity-stability’ dilemma. Connection Science, 9(4):353–380, 1997.
15. Robert M French. Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences, 3(4):128–135, 1999.
16. Steffen Gais, Geneviève Albouy, Mélanie Boly, Thien Thanh Dang-Vu, Annabelle Darsaud, Martin Desseilles, Géraldine Rauchs, Manuel Schabus, Virginie Sterpenich, Gilles Vandewalle, et al. Sleep transforms the cerebral trace of declarative memories. Proceedings of the National Academy of Sciences, 104(47):18778–18783, 2007.
17. Jort F. Gemmeke, Daniel P. W. Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R. Channing Moore, Manoj Plakal, and Marvin Ritter. Audio set: An ontology and human-labeled dataset for audio events. In ICASSP, New Orleans, LA, 2017.
18. Alexander Gepperth and Cem Karaoguz. A bio-inspired incremental learning architecture for applied perceptual problems. Cognitive Computation, 8(5):924–934, 2016.
19. AJ Gillies. The Stability/Plasticity Dilemma in Self-organising Neural Networks. PhD thesis, MSc Thesis, Computer Science Department, University of Otago, New Zealand, 1991.
20. Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 249–256, 2010.
21. Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv:1312.6211, 2013.
22. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CPVR, pp. 770–778, 2016.
23. P Hetherington and Mark S Seidenberg. Is there catastrophic interference in connectionist networks. In Proceedings of the 11th annual conference of the cognitive science society, volume 26, pp. 33. Erlbaum Hillsdale, NJ, 1989.
24. Ronald Kemker, Marc McClure, Angelina Abitino, Tyler Hayes, and Christopher Kanan. Measuring catastrophic forgetting in neural networks. In AAAI, 2018.
25. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, et al. Overcoming catastrophic forgetting in neural networks. Proc. of the National Academy of Sciences, pp. 201611835, 2017.
26. Takashi Kitamura, Sachie K Ogawa, Dheeraj S Roy, Teruhiro Okuyama, Mark D Morrissey, Lillian M Smith, Roger L Redondo, and Susumu Tonegawa. Engrams and circuits crucial for systems consolidation of a memory. Science, 356(6333):73–78, 2017.
27. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25, pp. 1097–1105, 2012.
28. Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. In Data Mining, 2008. ICDM’08. Eighth IEEE International Conference on, pp. 413–422. IEEE, 2008.
29. James L McClelland, Bruce L McNaughton, and Randall C O’reilly. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological review, 102(3):419, 1995.
30. Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of learning and motivation, 24:109–165, 1989.
31. Gina R Poe. Sleep is for forgetting. Journal of Neuroscience, 37(3):464–473, 2017.
32. Björn Rasch and Jan Born. About sleeps role in memory. Physiological reviews, 93(2):681–766, 2013.
33. Antti Rasmus, Mathias Berglund, Mikko Honkala, Harri Valpola, and Tapani Raiko. Semi-supervised learning with ladder networks. In NIPS, pp. 3546–3554, 2015.
34. Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, and Christoph H Lampert. iCaRL: Incremental classifier and representation learning. In CVPR, 2017.
35. Anthony Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science, 7(2): 123–146, 1995.
36. Peter J Rousseeuw and Katrien Van Driessen. A fast algorithm for the minimum covariance determinant estimator. Technometrics, 41(3):212–223, 1999.
37. Bernhard Schölkopf, John C. Platt, John C. Shawe-Taylor, Alex J. Smola, and Robert C. Williamson. Estimating the support of a high-dimensional distribution. Neural Comput., 13(7):1443–1471, July 2001.
38. Donald F Specht. Probabilistic neural networks. Neural networks, 3(1):109–118, 1990.
39. A Takashima, Karl Magnus Petersson, F Rutters, I Tendolkar, O Jensen, MJ Zwarts, BL McNaughton, and G Fernandez. Declarative memory consolidation in humans: a prospective functional magnetic resonance imaging study. Proceedings of the National Academy of Sciences of the United States of America, 103(3):756–761, 2006.
40. Philippe Taupin and Fred H Gage. Adult neurogenesis and neural stem cells of the central nervous system in mammals. Journal of neuroscience research, 69(6):745–749, 2002.
41. P. Welinder, S. Branson, T. Mita, C. Wah, F. Schroff, S. Belongie, and P. Perona. Caltech-UCSD Birds 200. Technical Report CNS-TR-2010-001, California Institute of Technology, 2010.
42. Seung-Schik Yoo, Peter T Hu, Ninad Gujar, Ferenc A Jolesz, and Matthew P Walker. A deficit in the ability to form new human memories without sleep. Nature Neuroscience, 10(3):385–392, 2007.