Code-specific learning rules improve action selection by populations of spiking neurons

International Journal of Neural Systems

Volumn 24, Issue 5, 2014, Pages

  Friedrich, Johannes ^a,b   Urbanczik, Robert ^a   Senn, Walter ^a  

^a UNIVERSITY OF BERN   (Switzerland)

^b UNIVERSITY OF CAMBRIDGE   (United Kingdom)

Author keywords

Action perturbation; Code specificity; Decision making; Policy gradient; Population coding; Reinforcement learning; Spike timing dependent synaptic plasticity; Spiking neural network

Indexed keywords

DECISION MAKING; NEURAL NETWORKS; NEURONS; POPULATION STATISTICS; REINFORCEMENT LEARNING;

ACTION PERTURBATION; CODE SPECIFICITY; POLICY GRADIENT; POPULATION CODING; SPIKE-TIMING-DEPENDENT SYNAPTIC PLASTICITIES; SPIKING NEURAL NETWORKS;

CODES (SYMBOLS);

ACTION POTENTIAL; ANIMAL; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; DECISION MAKING; HUMAN; LEARNING; NERVE CELL; PHYSIOLOGY;

ACTION POTENTIALS; ANIMALS; DECISION MAKING; HUMANS; LEARNING; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER); NEURONS;

EID: 84901857481     PISSN: 01290657     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0129065714500026     Document Type: Article

References (29)

1. H. Markram, J. Lübke, M. Frotscher and B. Sakmann, Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs, Science 275 (1997) 213-215. (Pubitemid 27034762
2. G. Bi andM. Poo, Synapticmodifications in cultured hippocampal neurons: Dependence on spike timing, synaptic strength and postsynaptic cell type, J. Neurosci. 18(24) (1998) 10464-10472
3. M. N. Shadlen and W. T. Newsome, Noise, neural codes and cortical organization, Curr. Opin. Neurobiol. 4 (1994) 569-579
4. M. J. Tovee and E. T. Rolls, Information encoding in short firing rate epochs by single neurons in the primate temporal visual cortex, Vis. Cognit. 2(1) (1995) 35-58
5. M. Badoual, Q. Zou, A. P. Davison, M. Rudolph, T. Bal, Y. Fregnac and A. Destexhe, Biophysical and phenomenological models of multiple spike interactions in spike-Timing dependent plasticity, Int. J. Neural Syst. 16(2) (2006) 79-97
6. S. Ghosh-Dastidar and H. Adeli, Spiking neural networks, Int. J. Neural Syst. 19(4) (2009) 295-308
7. A. Pouget, R. S. Zemel and P. Dayan, Information processing with population codes, Nat. Rev. Neurosci. 1(2) (2000) 125-132
8. B. Averbeck, P. E. Latham and A. Pouget, Neural correlations, population coding and computation, Nat. Rev. Neurosci. 7 (2006) 358-366
9. R. Urbanczik and W. Senn, Reinforcement learning in populations of spiking neurons, Nat. Neurosci. 12(3) (2009) 250-252
10. J. Friedrich, R. Urbanczik and W. Senn, Learning spike-based population codes by reward and population feedback, Neural Comput. 22(7) (2010) 1698-1717
11. J. Friedrich and W. Senn, Spike-based decision learning of Nash equilibria in two-player games, PLoS. Comput. Biol. 8(9) (2012) e1002691
12. S. Schliebs, N. Kasabov and M. Defoin-Platel, On the probabilistic optimization of spiking neural networks, Int. J. Neural. Syst. 20(6) (2010) 481-500
13. H. Sprekeler, G. Hennequin and W. Gerstner, Codespecific policy gradient rules for spiking neurons, in Advances in Neural Information Processing Systems 22, eds. Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams and A. Culotta (2009), pp. 1741-1749
14. R. J. Williams, Simple statistical gradient-following algorithms for connectionist reinforcement learning, Mach. Learn. 8 (1992) 229-256
15. J. Pfister, T. Toyoizumi, D. Barber and W. Gerstner, Optimal spike-Timing-dependent plasticity for precise action potential firing in supervised learning, Neural Comput. 18(6) (2006) 1318-1348. (Pubitemid 43765446
16. R. V. Florian, Reinforcement learning through modulation of spike-Timing-dependent synaptic plasticity, Neural Comput. 19(6) (2007) 1468-1502
17. I. R. Fiete and H. S. Seung, Gradient learning in spiking neural networks by dynamic perturbation of conductances, Phys. Rev. Lett. 97(4) (2006) 048104
18. D. Di Castro, D. Volkinshtein and R. Meir, Temporal difference based actor critic learning-convergence and neural implementation, in Advances in Neural Information Processing Systems 21, eds. D. Koller, D. Schuurmans, Y. Bengio and L. Bottou (2008), pp. 385-392
19. M. Schiess, R. Urbanczik and W. Senn, Gradient estimation in dendritic reinforcement learning, J. Math. Neurosci. 2(1) (2012) 2
20. X. Xie and H. S. Seung, Learning in neural networks by reinforcement of irregular spiking, Phys. Rev. E 69(4) (2004) 041909
21. A. Mohemmed, S. Schliebs, S. Matsuda and N. Kasabov, SPAN: Spike pattern association neuron for learning spatio-Temporal spike patterns, Int. J. Neural Syst. 22(4) (2012) 1250012
22. H. S. Seung, Learning in spiking neural networks by reinforcement of stochastic synaptic transmission, Neuron 40(6) (2003) 1063-1073
23. D. Sridharan, P. S. Prashanth and V. S. Chakravarthy, The role of the basal ganglia in exploration in a neural model based on reinforcement learning, Int. J. Neural Syst. 16(2) (2006) 111-124. (Pubitemid 44099468
24. R. G. Morris, P. Garrud, J. N. Rawlins and J. O'Keefe, Place navigation impaired in rats with hippocampal lesions, Nature 297(5868) (1982) 681-683. (Pubitemid 12096189
25. E. Vasilaki, N. Fr'emaux, R. Urbanczik, W. Senn and W. Gerstner, Spike-based reinforcement learning in continuous state and action space: When policy gradient methods fail, PLoS Comput. Biol. 5(12) (2009) e1000586
26. J. Friedrich, R. Urbanczik and W. Senn, Spatiotemporal credit assignment in neuronal population learning, PLoS Comput. Biol. 7(6) (2011) e1002092
27. N. Fr'emaux, H. Sprekeler and W. Gerstner, Reinforcement learning using a continuous time actorcritic framework with spiking neurons, PLoS Comput. Biol. 9(4) (2013) e1003024
28. J. Werfel, X. Xie and H. S. Seung, Learning curves for stochastic gradient descent in linear feedforward networks, Neural Comput. 17(12) (2005) 2699-2718
29. C. Clopath, L. Büsing, E. Vasilaki and W. Gerstner, Connectivity reflects coding: A model of voltagebased STDP with homeostasis, Nat. Neurosci. 13(3) (2010) 344-352.