Neuroscience-Inspired Artificial Intelligence

Neuron

Volumn 95, Issue 2, 2017, Pages 245-258

  Hassabis, Demis ^a,b   Kumaran, Dharshan ^a,c   Summerfield, Christopher ^a,d   Botvinick, Matthew ^a,b  

^a DEEPMIND   (United Kingdom)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

^c UNIVERSITY COLLEGE LONDON   (United Kingdom)

^d UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

artificial intelligence; brain; cognition; learning; neural network

Indexed keywords

ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; ATTENTION; BRAIN; EPISODIC MEMORY; HISTORY; HUMAN; IMAGINATION; MACHINE; MACHINE LEARNING; NEUROSCIENCE; NONHUMAN; PLANNING; PRIORITY JOURNAL; REINFORCEMENT; RESEARCH; REVIEW; WORKING MEMORY; ANIMAL; INTELLIGENCE; LEARNING; PHYSIOLOGY;

ANIMALS; ARTIFICIAL INTELLIGENCE; BRAIN; HUMANS; INTELLIGENCE; LEARNING; NEURAL NETWORKS (COMPUTER); NEUROSCIENCES;

EID: 85029563607     PISSN: 08966273     EISSN: 10974199     Source Type: Journal    
DOI: 10.1016/j.neuron.2017.06.011     Document Type: Review

References (178)

1. Adolph, K., Learning to learn in the development of action. Lockman, J., Reiser, J., Nelson, C.A., (eds.) Action as an organizer of perception and cognition during learning and development, 2005, Erlbaum Press, 91–122.
2. Anderson, J.R., Bothell, D., Byrne, M.D., Douglass, S., Lebiere, C., Qin, Y., An integrated theory of the mind. Psychol. Rev. 111 (2004), 1036–1060.
3. Ba, J.L., Mnih, V., and Kavukcuoglu, K. (2015). Multiple object recognition with visual attention. arXiv, arXiv:14127755.
4. Baddeley, A., Working memory: theories, models, and controversies. Annu. Rev. Psychol. 63 (2012), 1–29.
5. Bahdanau, D., Cho, K., and Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv, arXiv:14090473.
6. Balaguer, J., Spiers, H., Hassabis, D., Summerfield, C., Neural Mechanisms of Hierarchical Planning in a Virtual Subway Network. Neuron 90 (2016), 893–903.
7. Barlow, H., Sensory mechanisms, the reduction of redundancy, and intelligence. The Mechanisation of Thought Processes, 1959, National Physical Laboratory, UK: H.M. Stationery Office, 535–539.
8. Barnett, S.M., Ceci, S.J., When and where do we apply what we learn? A taxonomy for far transfer. Psychol. Bull. 128 (2002), 612–637.
9. Bastos, A.M., Usrey, W.M., Adams, R.A., Mangun, G.R., Fries, P., Friston, K.J., Canonical microcircuits for predictive coding. Neuron 76 (2012), 695–711.
10. Battaglia, P.W., Hamrick, J.B., Tenenbaum, J.B., Simulation as an engine of physical scene understanding. Proc. Natl. Acad. Sci. USA 110 (2013), 18327–18332.
11. Battaglia, P., Pascanu, R., Lai, M., Rezende, D., and Kavukcuoglu, K. (2016). Interaction networks for learning about objects, relations and physics. arXiv, arXiv:161200222.
12. Bengio, Y., Lee, D.H., Bornschein, J., Mesnard, T., and Lin, Z. (2015). Towards biologically plausible deep learning. arXiv, arXiv:150204156.
13. Bi, G.Q., Poo, M.M., Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 18 (1998), 10464–10472.
14. Blundell, C., Uria, B., Pritzel, A., Yazhe, L., Ruderman, A., Leibo, J.Z., Rae, J., Wierstra, D., and Hassabis, D. (2016). Model-free episodic control. arXiv, arXiv:160604460.
15. Botvinick, M.M., Plaut, D.C., Short-term memory for serial order: a recurrent neural network model. Psychol. Rev. 113 (2006), 201–233.
16. Brooks, R., Hassabis, D., Bray, D., Shashua, A., Turing centenary: is the brain a good model for machine intelligence?. Nature 482 (2012), 462–463.
17. Browne, C., Powley, E., Whitehouse, D., Lucas, S.M., Cowling, P.I., Rohlfshagen, P., Tavener, S., Perez, D., Samothrakis, S., Colton, S., A survey of Monte-Carlo tree search methods. IEEE Trans. Comput. Intell. AI Games 4 (2012), 1–43.
18. Chang, M.B., Ullman, T., Torralba, A., and Tenenbaum, J.B. (2016). A compositional object-based approach to learning physical dynamics. arXiv, arXiv:161200341.
19. nd International Conference on Machine Learning, pp. 1–61.
20. Churchland, P.S., Sejnowski, T.J., Perspectives on cognitive neuroscience. Science 242 (1988), 741–745.
21. Cichon, J., Gan, W.B., Branch-specific dendritic Ca(2+) spikes cause persistent synaptic plasticity. Nature 520 (2015), 180–185.
22. Cichy, R.M., Pantazis, D., Oliva, A., Resolving human object recognition in space and time. Nat. Neurosci. 17 (2014), 455–462.
23. Collins, A., Koechlin, E., Reasoning, learning, and creativity: frontal lobe function and human decision-making. PLoS Biol., 10, 2012, e1001293.
24. Constantinescu, A.O., O'Reilly, J.X., Behrens, T.E., Organizing conceptual knowledge in humans with a gridlike code. Science 352 (2016), 1464–1468.
25. Craik, K., The Nature of Explanation. 1943, Cambridge University Press.
26. Çukur, T., Nishimoto, S., Huth, A.G., Gallant, J.L., Attention during natural vision warps semantic representation across the human brain. Nat. Neurosci. 16 (2013), 763–770.
27. Daw, N.D., Niv, Y., Dayan, P., Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8 (2005), 1704–1711.
28. Deisseroth, K., Schnitzer, M.J., Engineering approaches to illuminating brain structure and dynamics. Neuron 80 (2013), 568–577.
29. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009). Imagenet: a large-scale hierarchical image database. In Computer Vision and Pattern Recognition, pp. 1–8.
30. Denil, M., Agrawal, P., Kulkarni, T.D., Erez, T., Battaglia, P., and de Freitas, N. (2016). Learning to perform physics experiments via deep reinforcement learning. arXiv, arXiv:161101843.
31. Dolan, R.J., Dayan, P., Goals and habits in the brain. Neuron 80 (2013), 312–325.
32. Doll, B.B., Duncan, K.D., Simon, D.A., Shohamy, D., Daw, N.D., Model-based choices involve prospective neural activity. Nat. Neurosci. 18 (2015), 767–772.
33. Donoso, M., Collins, A.G., Koechlin, E., Human cognition. Foundations of human reasoning in the prefrontal cortex. Science 344 (2014), 1481–1486.
34. Doumas, L.A., Hummel, J.E., Sandhofer, C.M., A theory of the discovery and predication of relational concepts. Psychol. Rev. 115 (2008), 1–43.
35. Duan, Y., Schulman, J., Chen, X., Bartlett, P.L., Sutskever, I., and Abbeel, P. (2016). RLˆ2: fast reinforcement learning via slow reinforcement learning. arXiv, arXiv:1611.02779.
36. Durstewitz, D., Seamans, J.K., Sejnowski, T.J., Neurocomputational models of working memory. Nat. Neurosci. 3:Suppl (2000), 1184–1191.
37. Elman, J.L., Finding structure in time. Cogn. Sci. 14 (1990), 179–211.
38. Eslami, A., Heess, N., Weber, T.Y.T., Szepesvari, D., Kavukcuoglu, K., and Hinton, G. (2016). Attend, infer, repeat: fast scene understanding with generative models. arXiv, arXiv:160308575.
39. Esser, S.K., Merolla, P.A., Arthur, J.V., Cassidy, A.S., Appuswamy, R., Andreopoulos, A., Berg, D.J., McKinstry, J.L., Melano, T., Barch, D.R., et al. Convolutional networks for fast, energy-efficient neuromorphic computing. Proc. Natl. Acad. Sci. USA 113 (2016), 11441–11446.
40. Fodor, J.A., Pylyshyn, Z.W., Connectionism and cognitive architecture: a critical analysis. Cognition 28 (1988), 3–71.
41. French, R.M., Catastrophic forgetting in connectionist networks. Trends Cogn. Sci. 3 (1999), 128–135.
42. Fukushima, K., Neocognitron: a self organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern. 36 (1980), 193–202.
43. Fusi, S., Drew, P.J., Abbott, L.F., Cascade models of synaptically stored memories. Neuron 45 (2005), 599–611.
44. Gallistel, C., King, A.P., Memory and the Computational Brain: Why Cognitive Science will Transform Neuroscience. 2009, Wiley-Blackwell.
45. Gatys, L.A., Ecker, A.S., and Bethge, M. (2015). A neural algorithm of artistic style. arXiv, arXiv:1508.06576.
46. Gemici, M., Hung, C., Santoro, A., Wayne, G., Mohamed, S., Rezende, D., Amos, D., and Lillicrap, T. (2017). Generative temporal models with memory. arXiv, arXiv:170204649.
47. Gershman, S.J., Daw, N.D., Reinforcement learning and episodic memory in humans and animals: an integrative framework. Annu. Rev. Psychol. 68 (2017), 101–128.
48. Gilmore, C.K., McCarthy, S.E., Spelke, E.S., Symbolic arithmetic knowledge without instruction. Nature 447 (2007), 589–591.
49. Glasser, M.F., Smith, S.M., Marcus, D.S., Andersson, J.L., Auerbach, E.J., Behrens, T.E., Coalson, T.S., Harms, M.P., Jenkinson, M., Moeller, S., et al. The Human Connectome Project's neuroimaging approach. Nat. Neurosci. 19 (2016), 1175–1187.
50. Goldman-Rakic, P.S., Cellular and circuit basis of working memory in prefrontal cortex of nonhuman primates. Prog. Brain Res. 85 (1990), 325–335.
51. Gopnik, A., Schulz, L., Mechanisms of theory formation in young children. Trends Cogn. Sci. 8 (2004), 371–377.
52. Graves, A., Wayne, G., and Danihelka, I. (2014). Neural turing machines. arXiv, arXiv:1410.5401.
53. Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S.G., Grefenstette, E., Ramalho, T., Agapiou, J., et al. Hybrid computing using a neural network with dynamic external memory. Nature 538 (2016), 471–476.
54. Gregor, K., Danihelka, I., Graves, A., Renzende, D., and Wierstra, D. (2015). DRAW: a recurrent neural network for image generation. arXiv, arXiv:150204623.
55. Hafner, R., Riedmiller, M., Reinforcement learning in feedback control. Mach. Learn. 84 (2011), 137–169.
56. th International Conference on Learning Representations (ICLR 2017), pp. 1–21.
57. Harlow, H.F., The formation of learning sets. Psychol. Rev. 56 (1949), 51–65.
58. Hassabis, D., Maguire, E.A., Deconstructing episodic memory with construction. Trends Cogn. Sci. 11 (2007), 299–306.
59. Hassabis, D., Maguire, E.A., The construction system of the brain. Philos. Trans. R. Soc. Lond. B Biol. Sci. 364 (2009), 1263–1271.
60. Haugeland, J., Artificial Intelligence: The Very Idea. 1985, MIT Press.
61. Hayashi-Takagi, A., Yagishita, S., Nakamura, M., Shirai, F., Wu, Y.I., Loshbaugh, A.L., Kuhlman, B., Hahn, K.M., Kasai, H., Labelling and optical erasure of synaptic memory traces in the motor cortex. Nature 525 (2015), 333–338.
62. Hebb, D.O., The Organization of Behavior. 1949, John Wiley & Sons.
63. Higgins, I., Matthey, L., Glorot, X., Pal, A., Uria, B., Blundell, C., Mohamed, S., and Lerchner, A. (2016). Early visual concept learning with unsupervised deep learning. arXiv, arXiv:160605579.
64. Hinton, G.E., McClelland, J.L., Rumelhart, D.E., Distributed Representations. Explorations in the Microstructure of Cognition, 1986, MIT Press, 77–109.
65. Hinton, G.E., Osindero, S., Teh, Y.W., A fast learning algorithm for deep belief nets. Neural Comput. 18 (2006), 1527–1554.
66. Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., and Salakhutdinov, R.R. (2012). Improving neural networks by preventing co-adaptation of feature detectors. arXiv, arXiv:12070580.
67. Hochreiter, S., Schmidhuber, J., Long short-term memory. Neural Comput. 9 (1997), 1735–1780.
68. Holyoak, K.J., Thagard, P., The analogical mind. Am. Psychol. 52 (1997), 35–44.
69. Hong, S., Oh, J., Bohyung, H., and Lee, H. (2015). Learning transferrable knowledge for semantic segmentation with deep convolutional neural network. arXiv, arXiv:151207928.
70. Hong, H., Yamins, D.L., Majaj, N.J., DiCarlo, J.J., Explicit information for category-orthogonal object properties increases along the ventral stream. Nat. Neurosci. 19 (2016), 613–622.
71. Hopfield, J.J., Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79 (1982), 2554–2558.
72. Hopfield, J.J., Tank, D.W., Computing with neural circuits: a model. Science 233 (1986), 625–633.
73. Hubel, D.H., Wiesel, T.N., Receptive fields of single neurones in the cat's striate cortex. J. Physiol. 148 (1959), 574–591.
74. Huys, Q.J., Eshel, N., O'Nions, E., Sheridan, L., Dayan, P., Roiser, J.P., Bonsai trees in your head: how the pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Comput. Biol., 8, 2012, e1002410.
75. Johnson, A., Redish, A.D., Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J. Neurosci. 27 (2007), 12176–12189.
76. Jonas, E., Kording, K.P., Could a neuroscientist understand a microprocessor?. PLoS Comput. Biol., 13, 2017, e1005268.
77. Jordan, M.I., Serial order: a parallel distributed processing approach. Adv. Psychol. 121 (1997), 471–495.
78. Kemp, C., Goodman, N.D., Tenenbaum, J.B., Learning to learn causal models. Cogn. Sci. 34 (2010), 1185–1243.
79. Khaligh-Razavi, S.M., Kriegeskorte, N., Deep supervised, but not unsupervised, models may explain IT cortical representation. PLoS Comput. Biol., 10, 2014, e1003915.
80. Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. Overcoming catastrophic forgetting in neural networks. Proc. Natl. Acad. Sci. USA 114 (2017), 3521–3526.
81. Koch, C., Ullman, S., Shifts in selective visual attention: towards the underlying neural circuitry. Hum. Neurobiol. 4 (1985), 219–227.
82. Krakauer, J.W., Ghazanfar, A.A., Gomez-Marin, A., MacIver, M.A., Poeppel, D., Neuroscience needs behavior: correcting a reductionist bias. Neuron 93 (2017), 480–490.
83. Kriegeskorte, N., Kievit, R.A., Representational geometry: integrating cognition, computation, and the brain. Trends Cogn. Sci. 17 (2013), 401–412.
84. Krizhevsky, A., Sutskever, I., and Hinton, G. (2012). ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25, pp. 1097–1105.
85. Kumaran, D., McClelland, J.L., Generalization through the recurrent interaction of episodic memories: a model of the hippocampal system. Psychol. Rev. 119 (2012), 573–616.
86. Kumaran, D., Hassabis, D., McClelland, J.L., What learning systems do intelligent agents need? Complementary learning systems theory updated. Trends Cogn. Sci. 20 (2016), 512–534.
87. Kurth-Nelson, Z., Economides, M., Dolan, R.J., Dayan, P., Fast sequences of non-spatial state representations in humans. Neuron 91 (2016), 194–204.
88. Lake, B.M., Salakhutdinov, R., Tenenbaum, J.B., Human-level concept learning through probabilistic program induction. Science 350 (2015), 1332–1338.
89. Lake, B.M., Ullman, T.D., Tenenbaum, J.B., and Gershman, S.J. (2016). Building machines that learn and think like people. arXiv, arXiv:1604.00289.
90. Larochelle, H., and Hinton, G. (2010). Learning to combine foveal glimpses with a third-order Boltzmann machine. NIPS’10 Proceedings of the International Conference on Neural Information Processing Systems, pp. 1243–1251.
91. LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D., Backpropagation applied to handwritten zip code recognition. Neural Comput. 1 (1989), 541–551.
92. LeCun, Y., Bengio, Y., Hinton, G., Deep learning. Nature 521 (2015), 436–444.
93. Leibo, J.Z., Liao, Q., Anselmi, F., Freiwald, W.A., Poggio, T., View-tolerant face recognition and Hebbian Learning imply mirror-symmetric neural tuning to head orientation. Curr. Biol. 27 (2017), 62–67.
94. Legg, S., Hutter, M., A collection of definitions of intelligence. Goertzel, B., Wang, P., (eds.) Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms—Proceedings of the AGI Workshop, 2007, Amsterdam IOS, 17–24.
95. Lengyel, M., and Dayan, P. (2007). Hippocampal contributions to control: the third way. In Advances in Neural Information Processing Systems 20, pp. 889–896.
96. Liao, Q., Leibo, J.Z., and Poggio, T. (2015). How important is weight symmetry in backpropagation? arXiv, arXiv:151005067.
97. Lillicrap, T.P., Cownden, D., Tweed, D.B., Akerman, C.J., Random synaptic feedback weights support error backpropagation for deep learning. Nat. Commun., 7, 2016, 13276.
98. Lloyd, K., Becker, N., Jones, M.W., Bogacz, R., Learning to use working memory: a reinforcement learning gating model of rule acquisition in rats. Front. Comput. Neurosci., 6, 2012, 87.
99. Marblestone, A.H., Wayne, G., Kording, K.P., Toward an integration of deep learning and neuroscience. Front. Comput. Neurosci., 10, 2016, 94.
100. Marcus, G.F., Rethinking eliminative connectionism. Cognit. Psychol. 37 (1998), 243–282.
101. Markram, H., The blue brain project. Nat. Rev. Neurosci. 7 (2006), 153–160.
102. Marr, D., Poggio, T., From understanding computation to understanding neural circuitry. A.I. Memo 357 (1976), 1–22.
103. McClelland, J.L., Rogers, T.T., The parallel distributed processing approach to semantic cognition. Nat. Rev. Neurosci. 4 (2003), 310–322.
104. McClelland, J.L., McNaughton, B.L., O'Reilly, R.C., Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol. Rev. 102 (1995), 419–457.
105. McCulloch, W., Pitts, W., A logical calculus of ideas immanent in nervous activity. Bull. Math. Biophys. 5 (1943), 115–133.
106. Mnih, V., Heess, N., Graves, A., and Kavukcuoglu, K. (2014). Recurrent models of visual attention. arXiv, arXiv:14066247.
107. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., et al. Human-level control through deep reinforcement learning. Nature 518 (2015), 529–533.
108. Moore, T., Zirnsak, M., Neural mechanisms of selective visual attention. Annu. Rev. Psychol. 68 (2017), 47–72.
109. Moravčík, M., Schmid, M., Burch, N., Lisý V., Morrill, D., Bard, N., Davis, T., Waugh, K., Johanson, M., Bowling, M., DeepStack: expert-level artificial intelligence in heads-up no-limit poker. Science 356 (2017), 508–513.
110. Nguyen, A., Dosovitskiy, A., Yosinski, J., Borx, T., and Clune, J. (2016). Synthesizing the preferred inputs for neurons in neural networks via deep generator networks. arXiv, arXiv:160509304.
111. Nishiyama, J., Yasuda, R., Biochemical computation for spine structural plasticity. Neuron 87 (2015), 63–75.
112. O'Doherty, J.P., Dayan, P., Friston, K., Critchley, H., Dolan, R.J., Temporal difference models and reward-related learning in the human brain. Neuron 38 (2003), 329–337.
113. O'Neill, J., Pleydell-Bouverie, B., Dupret, D., Csicsvari, J., Play it again: reactivation of waking experience and memory. Trends Neurosci. 33 (2010), 220–229.
114. O'Reilly, R.C., Frank, M.J., Making working memory work: a computational model of learning in the prefrontal cortex and basal ganglia. Neural Comput. 18 (2006), 283–328.
115. Oh, J., Guo, X., Lee, H., Lewis, R., and Singh, S. (2015). Action-conditional video prediction using deep networks in Atari games. arXiv, arXiv:150708750.
116. Ólafsdóttir, H.F., Barry, C., Saleem, A.B., Hassabis, D., Spiers, H.J., Hippocampal place cells construct reward related sequences through unexplored space. eLife, 4, 2015, e06063.
117. Olshausen, B.A., Anderson, C.H., Van Essen, D.C., A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information. J. Neurosci. 13 (1993), 4700–4719.
118. Pfeiffer, B.E., Foster, D.J., Hippocampal place-cell sequences depict future paths to remembered goals. Nature 497 (2013), 74–79.
119. Posner, M.I., Petersen, S.E., The attention system of the human brain. Annu. Rev. Neurosci. 13 (1990), 25–42.
120. Raby, C.R., Alexis, D.M., Dickinson, A., Clayton, N.S., Planning for the future by western scrub-jays. Nature 445 (2007), 919–921.
121. Redish, A.D., Vicarious trial and error. Nat. Rev. Neurosci. 17 (2016), 147–159.
122. th International Conference on Neural Information Processing Systems, pp. 1252–1260.
123. Reed, S., Akata, Z., Mohan, S., Tenka, S., Schiele, B., and Lee, H. (2016). Learning what and where to draw. arXiv, arXiv:161002454.
124. Rezende, D., Eslami, A., Mohamed, S., Battaglia, P., Jaderberg, M., and Heess, N. (2016a). Unsupervised learning of 3D structure from images. arXiv, arXiv:160700662.
125. Rezende, D., Mohamed, S., Danihelka, I., Gregor, K., and Wierstra, D. (2016b). One-shot generalization in deep generative models. arXiv, arXiv:160305106.
126. Riesenhuber, M., Poggio, T., Hierarchical models of object recognition in cortex. Nat. Neurosci. 2 (1999), 1019–1025.
127. Rosenblatt, F., The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65 (1958), 386–408.
128. Rowland, D.C., Roudi, Y., Moser, M.B., Moser, E.I., Ten years of grid cells. Annu. Rev. Neurosci. 39 (2016), 19–40.
129. Rumelhart, D.E., Hinton, G., Williams, R.J., Learning internal representations by error propagation. Rumelhart, D.E., McClelland, J.L., Group, P.R., (eds.) Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1, 1985, MIT Press, 318–362.
130. Rumelhart, D.E., McClelland, J.L., Group, P.R., Parallel Distributed Processing: Explorations in the Microstructures of Cognition, Volume 1, 1986, MIT Press.
131. Rusu, A.A., Rabinowitz, N., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., and Hadsell, R. (2016a). Progressive neural networks. arXiv, arXiv:160604671.
132. Rusu, A.A., Vecerik, M., Rothorl, T., Heess, N., Pascanu, R., and Hadsell, R. (2016b). Sim-to-real robot learning from pixels with progressive nets. arXiv, arXiv:161004286.
133. Salinas, E., Abbott, L.F., Invariant visual responses from attentional gain fields. J. Neurophysiol. 77 (1997), 3267–3272.
134. Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., and Lillicrap, T. (2016). One-shot Learning with Memory-Augmented Neural Networks. arXiv, arXiv:160506065.
135. Santoro, A., Raposo, D., Barrett, D.G.T., Malinowski, M., Pascanu, R., Battaglia, P., and Lillicrap. T. (2017). A simple neural network module for relational reasoning. arXiv, arXiv:1706.01427, https://arxiv.org/abs/1706.01427.
136. Saxe, A.M., Ganguli, S., and McClelland, J.L. (2013). Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv, arXiv:13126120v3.
137. Scellier, B., and Bengio, Y. (2016). Equilibrium propagation: bridging the gap between energy-based models and backpropagation. arXiv, arXiv:160205179.
138. Schacter, D.L., Addis, D.R., Hassabis, D., Martin, V.C., Spreng, R.N., Szpunar, K.K., The future of memory: remembering, imagining, and the brain. Neuron 76 (2012), 677–694.
139. Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2015). Prioritized experience replay. bioRxiv, arXiv:1511.05952.
140. Schmidhuber, J., Deep learning in neural networks: an overview. arXiv, 2014, 14047828.
141. Schultz, W., Dayan, P., Montague, P.R., A neural substrate of prediction and reward. Science 275 (1997), 1593–1599.
142. Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., Poggio, T., Robust object recognition with cortex-like mechanisms. IEEE Trans. Pattern Anal. Mach. Intell. 29 (2007), 411–426.
143. Shallice, T., From Neuropsychology to Mental Structure. 1988, Cambridge University Press.
144. Silver, D., Huang, A., Maddison, C.J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. Mastering the game of Go with deep neural networks and tree search. Nature 529 (2016), 484–489.
145. Simonyan, K., Vedaldi, A., and Zisserman, A. (2013). Deep inside convolutional networks: visualising image classification models and saliency maps. arXiv, arXiv:13126034.
146. Singer, A.C., Frank, L.M., Rewarded outcomes enhance reactivation of experience in the hippocampus. Neuron 64 (2009), 910–921.
147. Skaggs, W.E., McNaughton, B.L., Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience. Science 271 (1996), 1870–1873.
148. Smith, L.B., Self-organizing processes in learning to learn words: development is not induction. Basic and Applied Perspectives on Learning, Cognition, and Development. The Minnesota Symposia on Child Psychology, 1995, Lawrence Erlbaum Associates, 1–32.
149. Solway, A., Diuk, C., Córdova, N., Yee, D., Barto, A.G., Niv, Y., Botvinick, M.M., Optimal behavioral hierarchy. PLoS Comput. Biol., 10, 2014, e1003779.
150. Spelke, E.S., Kinzler, K.D., Core knowledge. Dev. Sci. 10 (2007), 89–96.
151. Squire, L.R., Stark, C.E., Clark, R.E., The medial temporal lobe. Annu. Rev. Neurosci. 27 (2004), 279–306.
152. St. John, M.F., McClelland, J.L., Learning and applying contextual constraints in sentence comprehension. Artif. Intell. 46 (1990), 217–257.
153. Stachenfeld, K., Botvinick, M.M., and Gershman, S.J. (2014). Design principles of hippocampal cognitive maps. In Advances in Neural Information Processing Systems 27, pp. 2528–2536.
154. Sukhbaatar, S., Szlam, A., Weston, J., and Fergus, R. (2015). End-to-end memory networks. arXiv, arXiv:150308895.
155. Summerfield, J.J., Lepsien, J., Gitelman, D.R., Mesulam, M.M., Nobre, A.C., Orienting attention based on long-term memory experience. Neuron 49 (2006), 905–916.
156. Sutton, R.S., Dyna, an integrated architecture for learning, planning, and reacting. ACM SIGART Bull. 2 (1991), 160–163.
157. Sutton, R.S., Barto, A.G., Toward a modern theory of adaptive networks: expectation and prediction. Psychol. Rev. 88 (1981), 135–170.
158. Sutton, R., Barto, A., Reinforcement Learning. 1998, MIT Press.
159. Tesauro, G., Temporal difference learning and TD-Gammon. Commun. ACM 38 (1995), 58–68.
160. Thrun, S., Mitchell, T.M., Lifelong robot learning. Robot. Auton. Syst. 15 (1995), 25–46.
161. Tolman, E.C., Cognitive maps in rats and men. Psychol. Rev. 55 (1948), 189–208.
162. Tsutsui, K., Grabenhorst, F., Kobayashi, S., Schultz, W., A dynamic code for economic object valuation in prefrontal cortex neurons. Nat. Commun., 7, 2016, 12554.
163. Tulving, E., How many memory systems are there?. American Psychologist 40 (1985), 385–398.
164. Tulving, E., Episodic memory: from mind to brain. Annu. Rev. Psychol. 53 (2002), 1–25.
165. Turing, A.M., On computable numbers, with an application to the Entscheidungs problem. Proc. Lond. Math. Soc. 2 (1936), 230–265.
166. Turing, A., Computing machinery and intelligence. Mind 236 (1950), 433–460.
167. van den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., and Kavukcuoglu, K. (2016). WaveNet: a generative model for raw audio. arXiv, arXiv:160903499.
168. Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., and Wierstra, D. (2016). Matching networks for one shot learning. arXiv, arXiv:160604080.
169. Wang, J., Kurth-Nelson, Z., Tirumala, D., Soyer, H., Leibo, J.Z., Munos, R., Blundell, C., Kumaran, D., and Botvinick, M.M. (2016). Learning to reinforcement learn. arXiv, arXiv:161105763.
170. Werbos, P.J., Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. 1974, Harvard University.
171. Weston, J., Chopra, S., and Bordes, A. (2014). Memory networks. arXiv, arXiv:14103916.
172. Whittington, J.C.R., Bogacz, R., An approximation of the error backpropagation algorithm in a predictive coding network with local Hebbian synaptic plasticity. Neural Comput. 29 (2017), 1229–1262.
173. Wu, Y., Schuster, M., Chen, Z., Le, Q.V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., et al. (2016). Google's neural machine translation system: bridging the gap between human and machine translation. arXiv, arXiv:160908144.
174. Xu, K., Kiros, J., Courville, A., Salakhutdinov, R., and Bengio, Y. (2015). Show, attend and tell: neural image caption generation with visual attention. arXiv, arXiv:150203044.
175. Yamins, D.L., DiCarlo, J.J., Using goal-driven deep learning models to understand sensory cortex. Nat. Neurosci. 19 (2016), 356–365.
176. Yang, G., Pan, F., Gan, W.B., Stably maintained dendritic spines are associated with lifelong memories. Nature 462 (2009), 920–924.
177. Zahavy, T., Zrihem, N.B., and Mannor, S. (2016). Graying the black box: understanding DQNs. arXiv, arXiv:160202658.
178. Zaremba, W., and Sutskever, I. (2014). Learning to execute. arXiv, arXiv:1410.4615.