Neurogenesis deep learning: Extending deep networks to accommodate new classes

Proceedings of the International Joint Conference on Neural Networks

Volumn 2017-May, Issue , 2017, Pages 526-533

  Draelos, Timothy J ^a   Miner, Nadine E ^a   Lamb, Christopher C ^a   Cox, Jonathan A ^a,b   Vineyard, Craig M ^a   Carlson, Kristofor D ^a   Severa, William M ^a   James, Conrad D ^a   Aimone, James B ^a  

^a SANDIA NATIONAL LABORATORIES   (United States)

^b QUALCOMM INC   (United States)

Author keywords

Autoencoder; Class conditional sampling; Deep learning; Deep neural networks; Hippocampus; Replay

Indexed keywords

ARTIFICIAL INTELLIGENCE; DATA HANDLING; DEEP LEARNING; LEARNING ALGORITHMS; LEARNING SYSTEMS; NEURAL NETWORKS;

ADAPTIVE MACHINE LEARNING; AUTO ENCODERS; BIOLOGICAL NEURAL SYSTEMS; CONDITIONAL SAMPLING; HIPPOCAMPUS; MACHINE LEARNING METHODS; REPLAY; STABILITY-PLASTICITY DILEMMA;

DEEP NEURAL NETWORKS;

EID: 85031007550     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2017.7965898     Document Type: Conference Paper

References (30)

1. Y. LeCun, Y. Bengio, and G. E. Hinton, "Deep learning, " Nature 521, 436-444 (2015).
2. J. Schmidhuber, "Deep learning in neural networks: An overview, " Neural Networks 61, 85-117 (2015).
3. D. C. Van Essen, C. Anderson, D. Felleman, "Information processing in the primate visual system: An integrated systems perspective, " Science, 1992. Jan 24;255 (5043): pp. 419-423.
4. Q. V. Le, in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on. 8595-8598 (IEEE).
5. D. C. Cirean, U. Meier, and J. Schmidhuber, "Transfer learning for Latin and Chinese characters with Deep Neural Networks, " The 2012 Intl. Joint Conference on Neural Networks (IJCNN). 1-6 (IEEE).
6. J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, "How transferable are features in deep neural networks, " in Advances in Neural Information Processing Systems 2014: pp. 3320-3328.
7. J. B. Aimone, et al. "Regulation and function of adult neurogenesis: from genes to cognition, " Physiological reviews 94, 991-1026, doi: 10. 1152/physrev. 00004. 2014 (2014).
8. J. B. Aimone, W. Deng, and J. H. Gage, "Resolving new memories: A critical look at the dentate gyrus, adult neurogenesis, and pattern separation, " Neuron 70, 589-596, 2011.
9. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition, " Proceedings of the IEEE, 86 (11): pp. 2278-2324, 1998.
10. P. J. Grother, "NIST special database 19-Handprinted forms and characters database, " Technical report, National Institute of Standards and Thechnology (NIST), 1995.
11. P. A. Appleby, G. Kempermann, and L. Wiskott, "The role of additive neurogenesis and synaptic plasticity in a hippocampal memory model with grid-cell like input, " PLoS comput. biology 7, e1001063 (2011).
12. P. A. Appleby and L. Wiskott, "Additive neurogenesis as a strategy for avoiding interference in a sparsely-coding dentate gyrus, " Network: Computation in Neural Systems 20, 137-161 (2009).
13. G. A. Carpenter and S. Grossberg, "The ART of adaptive pattern recognition by a self-organizing neural network, " Computer 21, 77-88 (1988).
14. R. A. Chambers and S. K. Conroy, "Network modeling of adult neurogenesis: Shifting rates of neuronal turnover optimally gears network learning according to novelty gradient, " Journal of cognitive neuroscience 19, 1-12 (2007).
15. R. A. Chambers, M. N. Potenza, R. E. Hoffman, and W. Miranker, "Simulated apoptosis/neurogenesis regulates learning and memory capabilities of adaptive neural networks, " Neuropsychopharmacology: official publication of the American College of Neuropsychopharmacology 29, 747-758 (2004).
16. C. Crick and W. Miranker, "Apoptosis, neurogenesis, and information content in Hebbian networks, " Biol. cybernetics 94, 9-19 (2006).
17. L. Wiskott, M. J. Rasch, Mand G. Kempermann, "A functional hypothesis for adult hippocampal neurogenesis: Avoidance of catastrophic interference in the dentate gyrus, " Hippocampus 16, 329-343, doi: 10. 1002/hipo. 20167 (2006).
18. J. B. Aimone, and F. H. Gage, "Modeling new neuron function: A history of using computational neuroscience to study adult neurogenesis, " The European journal of neuroscience 33, 1160-1169, doi: 10. 1111/j. 1460-9568. 2011. 07615. x (2011).
19. R. Calandra, R. Tapani, M. P. Deisenroth, and M. F. Pouzols, "Learning deep belief networks from non-stationary streams, " Artif. Neural Networks and Machine Learning-ICANN 2012: pp. 379-386.
20. C Kandaswamy, L. M. Silva, L. A. Alexandre, J. M. Santos, and J. M. de Sá, "Improving deep neural network performance by reusing features trained with transductive transference, " International Conference on Artificial Neural Networks, pp. 265-272, 2014.
21. M. F. Carr, S. P. Jadhav, and L. M. Frank, "Hippocampal replay in the awake state: A potential substrate for memory consolidation and retrieval, " Nature neuroscience 14 (2): pp. 147-153, 2011.
22. K. Louie and M. A. Wilson, "Temporally structured replay of awake hippocampal ensemble activity during rapid eye movement sleep, " Neuron, 2001. 29 (1): pp. 145-156.
23. R. Stickgold, "Sleep-dependent memory consolidation, " Nature, 2005. 437 (7063): pp. 1272-1278.
24. D. J. Felleman and D. C. Van Essen, "Distributed hierarchical processing in the primate cerebral cortex, " Cerebral cortex, 1991. 1 (1): pp. 1-47.
25. G. E. Hinton, P. Dayan, B. J. Frey, and R. Neal, "The "Wake-Sleep" algorithm for unsupervised neural networks. Science, " 1995 May26; 268 (5214): pp. 1158-1161.
26. R. Salakhutdinov, "Learning deep generative models, " PhD thesis, University of Toronto, 2009.
27. K. Gregor, I. Danihelka, A. Graves, D. J. Rezende, and D. Wierstra, "DRAW: A recurrent neural network for image generation, " ICML, 2015.
28. J. Rudy and G. Taylor, "Generative class-conditional autoencoders, " ArXiv e-prints 1412. 7009 (2015).
29. A. Krishnamoorthy and D. Menon, "Matrix inversion using Cholesky decomposition, " 2013 Signal Processing: Algorithms, Architectures, Arrangements, and Applications (SPA), Poznan, 2013, pp. 70-72.
30. G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks, " Science, 2006. Jul 28;313 (5786): pp. 504-507.