RM-SORN: A reward-modulated self-organizing recurrent neural network

Frontiers in Computational Neuroscience

Volumn 9, Issue MAR, 2015, Pages

  Aswolinskiy, Witali ^a   Pipa, Gordon ^a  

^a UNIVERSITY OF OSNABRÜCK   (Germany)

Author keywords

Hebbian learning; Intrinsic plasticity; Plasticity; Recurrent neural networks; Reward modulated STDP; Self organization

Indexed keywords

BRAIN; NEURONS; PATTERN RECOGNITION; PLASTICITY;

HEBBIAN LEARNING; INTRINSIC PLASTICITY; NEURAL NETWORK MODEL; NON LINEAR COMPUTATIONS; REWARD-MODULATED STDP; SELF ORGANIZATIONS; SEQUENCE GENERATION; SPIKE TIMING DEPENDENT PLASTICITIES;

RECURRENT NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; CONTROLLED STUDY; COUNTING TASK; INTRINSIC PLASTICITY; MARKOV 85 TASK; MEASUREMENT ACCURACY; MEMORY CAPACITY TASK; MOTION GENERATION TASK; MOTION PREDICTION TASK; NERVE CELL PLASTICITY; NEUROPSYCHOLOGICAL TEST; OCCLUDER TASK; PARITY TASK; PATTERN RECOGNITION TASK; PERFORMANCE MEASUREMENT SYSTEM; REWARD; REWARD MODULATED SELF ORGANIZING RECURRENT NEURAL NETWORK; SPIKE TIMING DEPENDENT PLASTICITY;

EID: 84927610540     PISSN: None     EISSN: 16625188     Source Type: Journal    
DOI: 10.3389/fncom.2015.00036     Document Type: Article

References (26)

1. Ahissar, E., Vaadia, E., Ahissar, M., Bergman, H., Arieli, A., and Abeles, M. (1992). Dependence of cortical plasticity on correlated activity of single neurons and on behavioral context. Science 257, 1412-1415. doi: 10.1126/science.1529342
2. Bourjaily, M. A., and Miller, P. (2011). Synaptic plasticity and connectivity requirements to produce stimulus-pair specific responses in recurrent networks of spiking neurons. PLoS Comput. Biol. 7:e1001091. doi: 10.1371/journal.pcbi.1001091
3. Dasgupta, S., Wörgötter, F., and Manoonpong, P. (2014). Neuromodulatory adaptive combination of correlation-based learning in cerebellum and reward-based learning in basal ganglia for goal-directed behavior control. Front. Neural Circuits 8:126. doi: 10.3389/fncir.2014.00126
4. Desai, N. S., Rutherford, L. C., and Turrigiano, G. G. (1999). Plasticity in the intrinsic excitability of cortical pyramidal neurons. Nat. Neurosci. 2, 515-520. doi: 10.1038/9165
5. Duarte, R., Seriès, P., and Morrison, A. (2014). "Self-organized artificial grammar learning in spiking neural networks," in 36th Annual Conference of the Cognitive Science Society. (Freiburg).
6. Fetz, E. E. (1969). Operant conditioning of cortical unit activity. Science 163, 955-958. doi: 10.1126/science.163.3870.955
7. Franz, A. (2010). Neural Network Models of Cognitive Development in Infancy. Frankfurt: Frankfurt Institute for Advanced Studies (FIAS).
8. Frémaux, N., Sprekeler, H., and Gerstner, W. (2013). Reinforcement learning using a continuous time actor-critic framework with spiking neurons. PLoS Comput. Biol. 9:e1003024. doi: 10.1371/journal.pcbi.1003024
9. Hoerzer, G. M., Legenstein, R., and Maass, W. (2014). Emergence of complex computational structures from chaotic neural networks through reward-modulated Hebbian learning. Cereb. Cortex 24, 677-690. doi: 10.1093/cercor/bhs348
10. Izhikevich, E. M. (2007). Solving the distal reward problem through linkage of STDP and dopamine signaling.Cereb. Cortex 17, 2443-2452. doi: 10.1093/cercor/bhl152
11. Jaeger, H. (2001). The "echo state" approach to analysing and training recurrent neural networks.
12. Lawson, C. L., and Hanson, R. J. (1995). Solving Least Squares Problems. Englewood Cliffs, NJ: SIAM. doi: 10.1137/1.9781611971217
13. Lazar, A., Pipa, G., and Triesch, J. (2009). SORN: a self-organizing recurrent neural network. Front. Comput. Neurosci. 3:23. doi: 10.3389/neuro.10.023.2009
14. Legenstein, R., Pecevski, D., and Maass, W. (2008). A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback. PLoS Comput. Biol. 4:e1000180. doi: 10.1371/journal.pcbi.1000180
15. Loewenstein, Y., and Seung, H. S. (2006). Operant matching is a generic outcome of synaptic plasticity based on the covariance between reward and neural activity. Proc. Natl. Acad. Sci. U.S.A. 103, 15224-15229. doi: 10.1073/pnas.0505220103
16. Pawlak, V., Wickens, J. R., Kirkwood, A., and Kerr, J. N. D. (2010). Timing is not Everything: Neuromodulation Opens the STDP Gate. Front. Synaptic Neurosci. 2:146. doi: 10.3389/fnsyn.2010.00146
17. Savin, C., and Triesch, J. (2014). Emergence of task-dependent representations in working memory circuits. Front. Comput. Neurosci. 8:57. doi: 10.3389/fncom.2014.00057
18. Senn, W., and Pfister, J.-P. (2014). "Reinforcement learning in cortical networks," in Encyclopedia of Computational Neuroscience, eds D. Jaeger and R. Jung (New York, NY: Springer), 1-6.
19. Sigala, N., and Logothetis, N. K. (2002). Visual categorization shapes feature selectivity in the primate temporal cortex. Nature 415, 318-320. doi: 10.1038/415318a
20. Soltani, A., and Wang, X.-J. (2010). Synaptic computation underlying probabilistic inference. Nat. Neurosci. 13, 112-119. doi: 10.1038/nn.2450
21. Soltoggio, A., Lemme, A., Reinhart, F., and Steil, J. J. (2013). Rare neural correlations implement robotic conditioning with delayed rewards and disturbances. Front. Neurorobot. 7:6. doi: 10.3389/fnbot.2013.00006
22. Soltoggio, A., and Steil, J. J. (2013). Solving the distal reward problem with rare correlations. Neural Comput. 25, 940-978. doi: 10.1162/NECO_a_00419
23. Sussillo, D., and Abbott, L. F. (2009). Generating coherent patterns of activity from chaotic neural networks.Neuron 63, 544-557. doi: 10.1016/j.neuron.2009.07.018
24. Toutounji, H., and Pipa, G. (2014). Spatiotemporal computations of an excitable and plastic brain: neuronal plasticity leads to noise-robust and noise-constructive computations. PLoS Comput. Biol. 10:e1003512. doi: 10.1371/journal.pcbi.1003512
25. Turrigiano, G. G., Leslie, K. R., Desai, N. S., Rutherford, L. C., and Nelson, S. B. (1998). Activity-dependent scaling of quantal amplitude in neocortical neurons. Nature 391, 892-896. doi: 10.1038/36103
26. Zheng, P., Dimitrakakis, C., and Triesch, J. (2013). Network self-organization explains the statistics and dynamics of synaptic connection strengths in cortex. PLoS Comput. Biol. 9:e1002848. doi: 10.1371/journal.pcbi.1002848