Does computational neuroscience need new synaptic learning paradigms?

Current Opinion in Behavioral Sciences

Volumn 11, Issue , 2016, Pages 61-66

  Brea, Johanni ^a   Gerstner, Wulfram ^a  

^a EPFL   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

ASSOCIATIVE MEMORY; EPISODIC MEMORY; HUMAN; LEARNING ALGORITHM; MEMBRANE POTENTIAL; NERVE CELL PLASTICITY; NEUROSCIENCE; REINFORCEMENT; REVIEW;

EID: 84971251166     PISSN: None     EISSN: 23521546     Source Type: Journal    
DOI: 10.1016/j.cobeha.2016.05.012     Document Type: Review

References (133)

1. Kuhn T.S. The Structure of Scientific Revolutions 1970, University of Chicago Press.
2. Hopfield J.J. Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci U S A 1982, 79:2554-2558.
3. Bienenstock E.L., Cooper L.N., Munroe P.W. Theory of the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. J Neurosci 1982, 2:32-48.
4. Schultz W., Dayan P., Montague R.R. A neural substrate for prediction and reward. Science 1997, 275:1593-1599.
5. Hertz J., Krogh A., Palmer R.G. Introduction to the Theory of Neural Computation 1991, Addison-Wesley, Redwood City, CA.
6. Haykin S. Neural Networks 1994, Prentice Hall, Upper Saddle River, NJ.
7. Bishop C.M. Neural Networks for Pattern Recognition 1995, Clarendon Press, Oxford.
8. Sutton R., Barto A. Reinforcement Learning 1998, MIT Press, Cambridge.
9. Schölkopf B., Smola A. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond 2002, MIT Press, Cambridge.
10. Rasmussen C.E., Williams C.K.I. Gaussian Processes for Machine Learning 2006, The MIT Press.
11. Krizhevsky A., Sutskever I., Hinton G.E. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25 2012, 1097-1105. Curran Associates, Inc. F. Pereira, C.J.C. Burges, L. Bottou, K.Q. Weinberger (Eds.).
12. Friedman J. Exploratory projection pursuit. J Am Stat Assoc 1987, 82:249-266.
13. Bell C., Han V., Sugawara Y., Grant K. Synaptic plasticity in a cerebellum-like structure depends on temporal order. Nature 1997, 387:278-281.
14. Hyvärinen A., Karhunen J., Oja E. Independent Component Analysis 2001, Wiley.
15. Olshausen B.A., Field D.J. Natural image statistics and efficient coding. Network: Comput Neural Syst 1996, 7:333-339.
16. Dayan P. The convergence of TD(λ) for general λ. Mach Learning 1992, 8:341-362.
17. Watkins C.J.C.H., Dayan P. Q-learning. Mach Learning 1992, 8:279-292.
18. Rummery G.A., Niranjan M. Online Q-Learning Using Connectionist Systems 1994, Cambridge University.
19. Williams R. Simple statistical gradient-following methods for connectionist reinforcement learning. Mach Learning 1992, 8:229-256.
20. Baxter J., Bartlett P., Weaver L. Experiments with infinite-horizon, policy-gradient estimation. J Artif Intell Res 2001, 15:351-381.
21. Morris R.G.M., Anderson E., Lynch G.S., Baudry M. Selective impairment of learning and blockade of long-term potentiation by an n-methyl-d-aspartate receptor antagonist, ap5. Nature 1986, 319:774-776.
22. Martin S., Grimwood P., Morris R. Synaptic plasticity and memory: an evaluation of the hypothesis. Annu Rev Neurosci 2000, 23:649-711.
23. Bliss T., Gardner-Medwin A. Long-lasting potentiation of synaptic transmission in the dendate area of unanaesthetized rabbit following stimulation of the perforant path. J Physiol 1973, 232:357-374.
24. Malenka R.C., Nicoll R.A. Long-term potentiation - a decade of progress?. Science 1999, 285:1870-1874.
25. Lisman J. Long-term potentiation: outstanding questions and attempted synthesis. Phil Trans R Soc Lond B: Biol Sci 2003, 358:829-842.
26. Lynch G., Dunwiddie T., Gribkoff V. Heterosynaptic depression: a postsynaptic correlate of long-term potentiation. Nature 1977, 266:737-739.
27. Levy W.B., Stewart D. Temporal contiguity requirements for long-term associative potentiation/depression in hippocampus. Neuroscience 1983, 8:791-797.
28. Markram H., Lübke J., Frotscher M., Sakmann B. Regulation of synaptic efficacy by coincidence of postsynaptic AP and EPSP. Science 1997, 275:213-215.
29. Bi G., Poo M. Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J Neurosci 1998, 18:10464-10472.
30. Sjöström P., Turrigiano G., Nelson S. Rate, timing, and cooperativity jointly determine cortical synaptic plasticity. Neuron 2001, 32:1149-1164.
31. Hebb D.O. The Organization of Behavior 1949, New York, Wiley.
32. Willshaw D.J., von der C., Malsburg How patterned neuronal connections can be set up by self-organization. Proc R Soc Lond Ser B 1976, 194:431-445.
33. Kohonen T. Self-Organization and Associative Memory 1989, Springer-Verlag, Berlin, Heidelberg, New York. 3rd ed.
34. Linsker R. From basic network principles to neural architecture: emergence of spatial-opponent cells. Proc Natl Acad Sci U S A 1986, 83:7508-7512.
35. Miller K., Keller J.B., Stryker M.P. Ocular dominance column development: analysis and simulation. Science 1989, 245:605-615.
36. Erwin E., Obermayer K., Schulten K. Models of orientation and ocular dominance columns in the visual cortex: a critical comparison. Neural Comput 1995, 7:425-468.
37. Wiskott L., Sejnowski T. Constraint optimization for neural map formation: a unifying framework for weight growth and normalization. Neural Comput 1998, 10:671-716.
38. Kempter R., Gerstner W., van Hemmen J.L. Hebbian learning and spiking neurons. Phys Rev E 1999, 59:4498-4514.
39. Song S., Miller K., Abbott L. Competitive Hebbian learning through spike-time-dependent synaptic plasticity. Nat Neurosci 2000, 3:919-926.
40. Senn W., Markram H., Tsodyks M. An algorithm for modifying neurotransmitter release probability based on pre- and postsynaptic spike timing. Neural Comput 2000, 13:35-67.
41. Shouval H.Z., Bear M.F., Cooper L.N. A unified model of NMDA receptor-dependent bidirectional synaptic plasticity. Proc Natl Acad Sci U S A 2002, 99:10831-10836.
42. Pfister J.-P., Gerstner W. Triplets of spikes in a model of spike timing-dependent plasticity. J Neurosci 2006, 26:9673-9682.
43. Clopath C., Busing L., Vasilaki E., Gerstner W. Connectivity reflects coding: a model of voltage-based spike-timing-dependent-plasticity with homeostasis. Nat Neurosci 2010, 13:344-352.
44. Brito C.S. Nonlinear Hebbian learning as a unifying principle in receptive field formation 2016, arxiv:http://arxiv.org/abs/1601.00701.
45. Brown T.H., Zador A.M., Mainen Z.F., Claiborne B.J. Hebbian modifications in hippocampal neurons. Long-Term Potentiation 1991, 357-389. MIT Press, Cambridge, London. M. Baudry, J. Davis (Eds.).
46. Morrison A., Diesmann M., Gerstner W. Phenomenological models of synaptic plasticity based on spike timing. Biol Cybern 2008, 98:459-478.
47. Gerstner W., Kistler W., Naud R., Paninski L. Neuronal Dynamics. From Single Neurons to Networks and Cognition 2014, Cambridge University Press.
48. Oja E. A simplified neuron model as a principal component analyzer. J Math Biol 1982, 15:267-273.
49. Hyvarinen A., Oja E. Independent component analysis by general nonlinear Hebbian-like learning rules. Signal Process 1998, 64:301-313.
50. Fyfe C., Baddeley R. Non-linear data structure extraction using simple Hebbian networks. Biol Cybern 1995, 72:533-541.
51. Kohonen T. Physiological interpretation of the self-organizing map algorithm. Neural Netw 1993, 6:895-905.
52. Carpenter G. Distributed learning, recognition and prediction by ART and ARTMAP neural networks. Neural Netw 1997, 10:1473-1494.
53. Pfister J.-P., Toyoizumi T., Barber D., Gerstner W. Optimal spike-timing dependent plasticity for precise action potential firing in supervised learning. Neural Comput 2006, 18:1318-1348.
54. Urbanczik R., Senn W. Learning by the dendritic prediction of somatic spiking. Neuron 2014, 81:521-528.
55. Gütig R. Spiking neurons can discover predictive features by aggregate-label learning. Science 2016, 351:aab4113-1.
56. Harris K.D. Stability of the fittest: organizing learning through retroaxonal signals. Trends Neurosci 2008, 31:130-136.
57. Sussillo D., Abbott L.F. Generating coherent patterns of activity from chaotic neural networks. Neuron 2009, 63:544-557.
58. Hoerzer G.M., Legenstein R., Maass W. Emergence of complex computational structures from chaotic neural networks through reward-modulated Hebbian learning. Cereb Cortex 2014, 24:677-690.
59. Schiess M., Urbanczik R., Senn W. Somato-dendritic synaptic plasticity and error-backpropagation in active dendrites. PLoS Comput Biol 2016, 12:e1004638.
60. Bengio Y., Lee D.-H., Bornschein J., Lin Z. Towards Biologically Plausible Deep Learning 2015, arXiv: http://arxiv.org/pdf/1502.04156.pdf.
61. Scellier B., Bengio Y. Towards a Biologically Plausible Backprop 2016, arXiv: http://arxiv.org/abs/1602.05179.
62. Reynolds J., Wickens J. Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw 2002, 15:507-521.
63. Seol G., Ziburkus J., Huang S., Song L., Kim I., Takamiya K., Huganir R., Lee H.-K., Kirkwood A. Neuromodulators control the polarity of spike-timing-dependent synaptic plasticity. Neuron 2007, 55:919-929.
64. Zhang J., Lau P., Bi G. Gain in sensitivity and loss in temporal contrast of STDP by dopaminergic modulation at hippocampal synapses. Proc Natl Acad Sci U S A 2009, 106:13028-13033.
65. Pawlak V., Kerr J. Dopamine receptor activation is required for corticostriatal spike-timing-dependent plasticity. J Neurosci 2008, 28:2435-2446.
66. Pawlak V., Wickens J., Kirkwood A., Kerr J. Timing is not everything: neuromodulation opens the STDP gate. Front Synap Neurosci 2010, 2:146.
67. Frey U., Morris R. Synaptic tagging and long-term potentiation. Nature 1997, 385:533-536.
68. Redondo R.L., Morris R.G.M. Making memories last: the synaptic tagging and capture hypothesis. Nat Rev Neurosci 2011, 12:17-30.
69. Lisman J., Grace A.A., Duzel E. A neoHebbian framework for episodic memory; role of dopamine-dependent late LTP. Trends Neurosci 2011, 34:536-547.
70. Arleo A., Gerstner W. Spatial cognition and neuro-mimetic navigation: a model of hippocampal place cell activity. Biol Cybernet 2000, 83:287-299.
71. Foster D., Morris R., Dayan P. Models of hippocampally dependent navigation using the temporal difference learning rule. Hippocampus 2000, 10:1-16.
72. Xie X., Seung S. Learning in neural networks by reinforcement of irregular spiking. Phys Rev E 2004, 69:41909.
73. Florian R.V. Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Comput 2007, 19:1468-1502.
74. Izhikevich E. Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb Cortex 2007, 17:2443-2452.
75. Legenstein R., Pecevski D., Maass W. Theoretical analysis of learning with reward-modulated spike-timing-dependent plasticity. Advances in Neural Information Processing Systems 20 2008, MIT Press, Cambridge, MA. J. Platt, D. Koller, Y. Singer, S. Roweis (Eds.).
76. Frémaux N., Sprekeler H., Gerstner W. Functional requirements for reward-modulated spike-timing-dependent plasticity. J Neurosci 2010, 40:13326-13337.
77. Frémaux N., Sprekeler H., Gerstner W. Reinforcement learning using a continuous time actor-critic framework with spiking neurons. PLoS Comput Biol 2013, 9:e1003024.
78. Frémaux N., Gerstner W. Neuromodulated spike-timing-dependent plasticity and theory of three-factor learning rules. Front Neural Circuits 2016, 9:85.
79. Schultz W. Getting formal with dopamine and reward. Neuron 2002, 36:241-263.
80. Willshaw D.J., Bunemann O.P., Longuet-Higgins H.C. Non-holographic associative memory. Nature 1969, 222:960-962.
81. Little W.A., Shaw G.L. Analytical study of the memory storage capacity of a neural network. Math Biosci 1978, 39:281-290.
82. Hopfield J.J. Neurons with graded response have computational properties like those of two-state neurons. Proc Natl Acad Sci U S A 1984, 81:3088-3092.
83. Amit D.J., Gutfreund H., Sompolinsky H. Storing infinite number of patterns in a spin-glass model of neural networks. Phys Rev Lett 1985, 55:1530-1533.
84. Amit D.J., Gutfreund H., Sompolinsky H. Information storage in neural networks with low levels of activity. Phys Rev A 1987, 35:2293-2303.
85. Tsodyks M., Feigelman M. The enhanced storage capacity in neural networks with low activity level. Europhys Lett 1988, 6:101-105.
86. Herz A.V.M., Sulzer B., Kühn R., van Hemmen J.L. Hebbian learning reconsidered: representation of static and dynamic objects in associative neural nets. Biol Cybern 1989, 60:457-467.
87. Amit D.J., Tsodyks M.V. Quantitative study of attractor neural networks retrieving at low spike rates. I. Substrate - spikes, rates, and neuronal gain. Network 1991, 2:259-273.
88. Gerstner W., van Hemmen J.L. Associative memory in a network of 'spiking' neurons. Network 1992, 3:139-164.
89. Amit D.J., Brunel N. A model of spontaneous activity and local delay activity during delay periods in the cerebral cortex. Cereb Cortex 1997, 7:237-252.
90. Brunel N. Dynamics of sparsely connected networks of excitatory and inhibitory neurons. Comput Neurosci 2000, 8:183-208.
91. Hasselmo M. The role of acetylcholine in learning and memory. Curr Opin Neurobiol 2006, 16:710-715.
92. Gu Q. Neuromodulatory transmitter systems in the cortex and their role in cortical plasticity. Neuroscience 2002, 111:815-835.
93. Moncada D., Viola H. Induction of long-term memory by exposure to novelty requires protein synthesis: evidence for a behavioral tagging. J Neurosci 2007, 27:7476-7481.
94. Fusi S. Hebbian spike-driven synaptic plasticity for learning patterns of mean firing rates. Biol Cybern 2002, 87:459-470.
95. Mongillo G., Curti E., Romani S., Amit D. Learning in realistic networks of spiking neurons and spike-driven plastic synapses. Eur J Neurosci 2005, 21:3143-3160.
96. Fusi S., Abbott L. Limits on the memory storage capacity of bounded synapses. Nat Neurosci 2007, 10:485-493.
97. Litwin-Kumar A., Doiron B. Formation and maintenance of neuronal assemblies through synaptic plasticity. Nat Commun 2014, 5:5319.
98. Zenke F., Agnes E., Gerstner W. Diverse synaptic plasticity mechanisms orchestrated to form and retrieve memories in spiking neural networks. Nature Commun 2015, 6:6922.
99. Brea J., Senn W., Pfister J.-P. Matching recall and storage in sequence learning with spiking neural networks. J Neurosci 2013, 33:9565-9575.
100. Rezende D., Gerstner W. Stochastic variational learning in recurrent spiking networks. Front Comput Neurosci 2014, 8:38.
101. Ko H., Cossell L., Baragli C., Antolik J., Clopath C., Hofer S.B., Mrsic-Flogel T.D. The emergence of functional microcircuits in visual cortex. Nature 2013, 496:96-100.
102. Kesner R.P., Rolls E.T. A computational theory of hippocampal function, and tests of the theory: new developments. Neurosci Biobehav Rev 2015, 48:92-147.
103. Norman K., Greg D., Polyn S.M. Computational models of episodic memory. The Cambridge Handbook of Computational Cognitive Modeling 2008, 189-224. Cambridge University Press.
104. Rolls E.T., Stringer S.M., Trappenberg T.P. A unified model of spatial and episodic memory. Proc R Soc B: Biol Sci 2002, 269:1087-1093.
105. Nadal J., Toulouse G., Changeux J., Dehaene S. Networks of formal neurons and memory palimpsests. Europhys Lett 1986, 1:535-542.
106. Amit D.J., Fusi S. Learning in neural networks with material synapses. Neural Comput 1994, 6:957-982.
107. Fusi S., Drew P.J., Abbott L.F. Cascade models of synaptically stored memories. Neuron 2005, 45:599-611.
108. Senn W., Fusi S. Convergence of stochastic learning in perceptrons with binary synapses. Phys Rev E 2005, 71:061907.
109. Päpper M., Kempter R., Leibold C. Synaptic tagging, evaluation of memories, and the distal reward problem. Learn Mem 2011, 18:58-70.
110. Amit Y., Huang Y. Precise capacity analysis in binary networks with multiple coding level inputs. Neural Comput 2010, 22:660-688.
111. Elliott T. Discrete states of synaptic strength in a stochastic model of spike-timing-dependent plasticity. Neural Comput 2010, 22:244-272.
112. Sheynikhovich D., Chavarriaga R., Strosslin T., Arleo A., Gerstner W. Is there a geometric module for spatial orientation? Insights from a rodent navigation model. Psychol Rev 2009, 116:540-566.
113. Franzius M., Sprekeler H., Wiskott L. Slowness and sparseness lead to place, head-direction, and spatial-view cells. PLoS Comput Biol 2007, 3:e166.
114. O'Keefe J., Dostrovsky J. The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat. Brain Res 1971, 34:171-175.
115. Lake B.M., Salakhutdinov R., Tenenbaum J.B. Human-level concept learning through probabilistic program induction. Science 2015, 350:1332-1338.
116. Guez A., Silver D., Dayan P. Scalable and efficient Bayes-adaptive reinforcement learning based on Monte-Carlo tree search. J Artif Intell Res 2013, 48:841-883.
117. Kemp C., Tenenbaum J.B. The discovery of structural form. Proc Natl Acad Sci U S A 2008, 105:10687-10692.
118. Muggleton S., de Raedt L. Inductive logic programming: theory and methods. J Logic Program 1994, 19:629-679.
119. Blum K., Abbott L. A model of spatial map formation in the hippocampus of the rat. Neural Comput 1996, 8:85-93.
120. Gerstner W., Abbott L.F. Learning navigational maps through potentiation and modulation of hippocampal place cells. J Comput Neurosci 1997, 4:79-94.
121. Stachenfeld K.L., Botvinick M.M., Gershman S.J. Design principles of the hippocampal cognitive map. Adv Neural Inform Process Syst 2014, 27:1-9.
122. Corneil D.S., Gerstner W. Attractor network dynamics enable preplay and rapid path planning in maze-like environments. Adv Neural Inform Process Syst 2015, 28:1675-1683.
123. Milford M.J., Wyeth G.F., Prasser D. RatSLAM: a hippocampal model for simultaneous localization and mapping. Proceedings of the 2004 IEEE international Conference on Robotics & Automation 2004, 403-408.
124. Penny W.D., Zeidman P., Burgess N. Forward and backward inference in spatial cognition. PLoS Comput Biol 2013, 9:e1003383.
125. Friedrich J., Lengyel M. Goal-directed decision making with spiking neurons. J Neurosci 2016, 36:1529-1546.
126. Vander Wall S.B. Food Hoarding in Animals 1990, University of Chicago Press, Chicago.
127. Macdonald D.W. Food caching by red foxes and some other carnivores. Zeit Tierpsychol 1976, 42:170-185.
128. Clayton N.S., Krebs J.R. Memory for spatial and object-specific cues in food-storing and non-storing birds. J Comp Physiol A 1994, 174:371-379.
129. Clayton N.S., Emery N.J. Avian models for human cognitive neuroscience: a proposal. Neuron 2015, 86:1330-1342.
130. Clayton N.S., Yu K.S., Dickinson A. Interacting Cache memories: evidence for flexible memory use by Western Scrub-Jays (Aphelocoma californica). J Exp Psychol Anim Behav Process 2003, 29:14-22.
131. Wilson B., Mackintosh N.J., Boakes R.A. Transfer of relational rules in matching and oddity learning by pigeons and corvids. Q J Exp Psychol Sect B 1985, 37:313-332.
132. Paz-y Mi no C.G., Bond A.B., Kamil A.C., Balda R.P. Pinyon jays use transitive inference to predict social dominance. Nature 2004, 430:778-781.
133. Dally J.M., Emery N.J., Clayton N.S. Food-caching western scrub-jays keep track of who was watching when. Science 2006, 312:1662-1665.