Gradient estimation in dendritic reinforcement learning

Journal of Mathematical Neuroscience

Volumn 2, Issue 1, 2012, Pages

  Schiess, Mathieu ^a   Urbanczik, Robert ^a   Senn, Walter ^a  

^a UNIVERSITY OF BERN   (Switzerland)

Author keywords

Dendritic computation; Reinforcement learning; Spiking neuron

Indexed keywords

N METHYL DEXTRO ASPARTIC ACID;

ACTION POTENTIAL; ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; DENDRITIC CELL; LEARNING; NERVE CELL; NERVE CELL NETWORK; NERVE CELL PLASTICITY; PRIORITY JOURNAL; REINFORCEMENT; SIMULATION; STOCHASTIC MODEL; SYNAPSE;

EID: 84859496500     PISSN: None     EISSN: 21908567     Source Type: Journal    
DOI: 10.1186/2190-8567-2-2     Document Type: Article

References (26)

1. A. Polsky, B. W. Mel, and J. Schiller. Computational subunits in thin dendrites of pyramidal cells. Nat. Neurosci., 7:621-627, Jun 2004.
2. W. Maass. Computation with spiking neurons. In M. A. Arbib, editor, The Handbook of Brain Theory and Neural Networks, pages 1080-1083. MIT Press (Cambridge), 2nd edition, 2003.
3. P. Poirazi, T. Brannon, and B. W. Mel. Pyramidal neuron as two-layer neural network. Neuron, 37:989-999, Mar 2003.
4. T. Nevian, M. E. Larkum, A. Polsky, and J. Schiller. Properties of basal dendrites of layer 5 pyramidal neurons: a direct patch-clamp recording study. Nat. Neurosci., 10:206-214, Feb 2007.
5. W. L. Zhou, P. Yan, J. P. Wuskell, L. M. Loew, and S. D. Antic. Dynamics of action potential backpropagation in basal dendrites of prefrontal cortical pyramidal neurons. Eur. J. Neurosci., 27:923-936, Feb 2008.
6. J. Schiller, G. Major, H. J. Koester, and Y. Schiller. NMDA spikes in basal dendrites of cortical pyramidal neurons. Nature, 404:285-289, Mar 2000.
7. J. Schiller and Y. Schiller. NMDA receptor-mediated dendritic spikes and coincident signal amplification. Curr. Opin. Neurobiol., 11:343-348, Jun 2001.
8. G. Major, A. Polsky, W. Denk, J. Schiller, and D. W. Tank. Spatiotemporally graded NMDA spike/plateau potentials in basal dendrites of neocortical pyramidal neurons. J. Neurophysiol., 99:2584-2601, May 2008.
9. M. E. Larkum, J. J. Zhu, and B. Sakmann. A new cellular mechanism for coupling inputs arriving at different cortical layers. Nature, 398:338-341, Mar 1999.
10. M. E. Larkum, T. Nevian, M. Sandler, A. Polsky, and J. Schiller. Synaptic integration in tuft dendrites of layer 5 pyramidal neurons: a new unifying principle. Science, 325:756-760, Aug 2009.
11. H. Seung. Learning in spiking neural networks by reinforcement of stochastic synaptic transmission. Neuron, 40:1063-1073, 2003.
12. N. Fremaux, H. Sprekeler, and W. Gerstner. Functional requirements for reward-modulated spike-timing-dependent plasticity. J. Neurosci., 30:13326-13337, Oct 2010.
13. R. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8:229-256, 1992.
14. Y. Matsuda, A. Marzo, and S. Otani. The presence of background dopamine signal converts long-term synaptic depression to potentiation in rat prefrontal cortex. J. Neurosci., 26:4803-4810, 2006.
15. G. Seol, J. Ziburkus, S Huang, L. Song, I. Kim, K. Takamiya, R. Huganir, H. Lee, and A Kirkwood. Neuromodulators control the polarity of spike-timing-dependent synaptic plasticity. Neuron, 55:919-929, 2007. Erratum in: Neuron. 56:754.
16. V. Pawlak and J. N. Kerr. Dopamine receptor activation is required for corticostriatal spike-timing-dependent plasticity. J. Neurosci., 28:2435-2446, Mar 2008.
17. J. Werfel, X. Xie, and H. S. Seung. Learning curves for stochastic gradient descent in linear feedforward networks. Neural Comput, 17:2699-2718, 2005.
18. R. Urbanczik and W. Senn. Reinforcement learning in populations of spiking neurons. Nature Neurosci., 12:250-252, 2009.
19. P. Dayan and L. Abbott. Theoretical Neuroscience. The MIT Press, 2001.
20. J. Pfister, T. Toyoizumi, D. Barber, and W. Gerstner. Optimal spike-timing-dependent plasticity for precise action potential firing in supervised learning. Neural Computation, 18:1318-1348, 2006.
21. D.P. Bertsekas and J. N. Tsitsiklis. Parallel and Distributed Computation: Numerical Methods. PrenticeHall, Englewood Cliffs, 1989.
22. J. Baxter and P. Bartlett. Infinite-horizon policy-gradient estimation. Journal of Artificial Intelligence Research, 15:319-350, 2001.
23. J. Baxter, P. Bartlett, and L. Weaver. Experiments with infinite-horizon, policy-gradient estimation. Journal of Artificial Intelligence Research, 15:351-381, 2001.
24. J. Friedrich, R. Urbanczik, and W. Senn. Spatio-temporal credit assignment in neuronal population learning. PLoS Comput. Biol., 7:e1002092, Jun 2011.
25. C. Clopath, L. Büsing, E. Vasilaki, and W. Gerstner. Connectivity reflects coding: a model of voltage-based STDP with homeostasis. Nat. Neurosci., 13:344-352, Mar 2010.
26. J. Friedrich, R. Urbanczik, and W. Senn. Learning spike-based population codes by reward and population feedback. Neural Comput., 22:1698-1717, 2010.