Emergence of complex computational structures from chaotic neural networks through reward-modulated hebbian learning

Cerebral Cortex

Volumn 24, Issue 3, 2014, Pages 677-690

  Hoerzer, Gregor M ^a   Legenstein, Robert ^a   Maass, Wolfgang ^a  

^a GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

Author keywords

cortical microcircuit model; cortical plasticity; pattern generation; working memory

Indexed keywords

CORTICAL MICROCIRCUIT MODEL; CORTICAL PLASTICITY; PATTERN GENERATION; WORKING MEMORY;

ACTION POTENTIALS; COMPUTER SIMULATION; HUMANS; LEARNING; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER); NEURONS; REWARD;

EID: 84894276345     PISSN: 10473211     EISSN: 14602199     Source Type: Journal    
DOI: 10.1093/cercor/bhs348     Document Type: Article

References (60)

1. Akam T, Kullmann DM. 2010. Oscillations and filtering networks support flexible routing of information. Neuron. 67:308-320.
2. Amari S-I. 1972. Characteristics of random nets of analog neuron-like elements. IEEE Trans Syst Man Cybernetics. 2:643-657.
3. Anderson CH, Van Essen DC. 1987. Shifter circuits: a computational strategy for dynamic aspects of visual processing. Proc Natl Acad Sci USA. 84:6297-6301.
4. Bailey CH, Giustetto M, Huang Y-Y, Hawkins RD, Kandel ER. 2000. Is heterosynaptic modulation essential for stabilizing Hebbian plasticity and memory? Nat Rev Neurosci. 1:11-20.
5. Bernacchia A, Seo H, Lee D, Wang X-J. 2011. A reservoir of time constants for memory traces in cortical neurons. Nat Neurosci. 14:366-372.
6. Buonomano D, Maass W. 2009. State-dependent computations: spatiotemporal processing in cortical networks. Nat Rev Neurosci. 10:113-125.
7. Crick F, Koch C. 1990. Some reflections on visual awareness. Cold Spring Harb Symp Quant Biol. 55:953-962.
8. Dominey PF, Hoen M, Inui T. 2006. A neurolinguistic model of grammatical construction processing. J Cogn Neurosci. 18:2088-2107.
9. Doyon J. 2008. Motor sequence learning and movement disorders. Curr Opin Neurol. 21:478-483.
10. Durstewitz D, Seamans JK. 2002. The computational role of dopamine D1 receptors in working memory. Neural Netw. 15:561-572.
11. Durstewitz D, Vittoz NM, Floresco SB, Seamans JK. 2010. Abrupt transitions between prefrontal neural ensemble states accompany behavioral transitions during rule learning. Neuron. 66:438-448.
12. Fetz EE, Baker MA. 1973. Operantly conditioned patterns on precentral unit activity and correlated responses in adjacent cells and contralateral muscles. J Neurophysiol. 36:179-204.
13. Fiete I, Seung HS. 2006. Gradient learning in spiking neural networks by dynamic perturbation of conductances. Phys Rev Lett. 97:48104.
14. Fremaux N, Sprekeler H, Gerstner W. 2010. Functional requirements for reward-modulated spike-timing-dependent plasticity. J Neurosci. 30:13326-13337.
15. Fries P. 2005. A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. Trends Cogn Sci. 9:474-480.
16. Fuster JM, Alexander GE. 1971. Neuron activity related to short-term memory. Science. 173:652-654.
17. Goldman-Rakic PS. 1995. Cellular basis of working memory. Neuron. 14:477-485.
18. Haeusler S, Maass W. 2007. A statistical analysis of information processing properties of lamina-specific cortical microcircuit models. Cereb Cortex. 17:149-162.
19. Haeusler S, Schuch K, Maass W. 2008. Motif distribution and computational performance of two data-based cortical microcircuit templates. 38th Annual Conference of the Society for Neuroscience, Program 2209.
20. Hasegawa RP, Blitz AM, Geller NL, Goldberg ME. 2000. Neurons in monkey prefrontal cortex that track past of predict future performance. Science. 290:1786-1789.
21. Haykin S. 1999. Neural networks: a comprehensive foundation. New Jersey: Prentice Hall.
22. Hopfield JJ. 1984. Neurons with graded response have collective computational properties like those of two-state neurons. PNAS. 81:3088-3092.
23. Hosp JA, Pekanovic A, Rioult-Pedotti MS, Luft AR. 2011. Dopaminergic projections from midbrain to primary motor cortex mediate motor skill learning. J Neurosci. 31:2481-2487.
24. Jaeger H, Haas H. 2004. Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science. 304:78-80.
25. Jaeger H. 2003. Adaptive nonlinear system identification with echo state networks. In: Becker S, Thrun K. Obermayer, editors. Advances in Neural Information Processing Systems, Vol. 15. Cambridge (MA): MIT Press. p. 593-600.
26. Jaeger H, Haas H. 2004. Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science. 304:78-80.
27. Jarosiewicz B, Chase SM, Fraser GW, Velliste M, Kass RE, Schwartz AB. 2008. Functional network reorganization during learning in a brain-computer interface paradigm. Proc Natl Acad Sci USA 105:19486-19491.
28. Kandel ER, Tauc L. 1965a. Heterosynaptic facilitation in neurones of the abdominal ganglion of Aplysia depilans. J Physiol. 181:1-27.
29. Kandel ER, Tauc L. 1965b. Mechanisms of heterosynaptic facilitation in the giant cellof the abdominal ganglion of Aplysia depilans. J Physiol. 181:28-47.
30. Klingberg T. 2010. Training and plasticity of working memory. Trends Cogn Sci. 14:317-324.
31. Klingberg T, Forssberg H, Westerberg H. 2002. Training of working memory in children with ADHD. J Clin Exp Neuropsychol. 24:781-791.
32. Legenstein R, Chase SM, Schwartz AB, Maass W. 2010. A rewardmodulated Hebbian learning rule can explain experimentally observed network reorganization in a brain control task. J Neurosci. 30:8400-8410.
33. Legenstein R, Maass W. 2007. Edge of chaos and prediction of computational performance for neural circuit models. Neural Netw. 20:323-334.
34. Legenstein R, Pecevski D, Maass W. 2008. A learning theory for reward-modulated spike-timing-dependent plasticity with application to biofeedback. PLoS Comput Biol. 4:1-27.
35. Maass W, Joshi P, Sontag ED. 2007. Computational aspects of feedback in neural circuits. PLoS Comput Biol. 3(1):e165.
36. Maass W, Natschlaeger T, Markram H. 2002. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14:2531-2560.
37. Marder E, Goaillard J-M. 2006. Variability, compensation and homeostasis in neuron and network function. Nat Rev Neurosci. 7:563-574.
38. Miller EK, Erickson CA, Desimone R. 1996. Neural mechanisms of visual working memory in prefrontal cortex of the macaque. J Neurosci. 16:5154-5167.
39. Molina-Luna K, Pekanovic A, Röhrich S, Hertler B, Schubring-Giese M, Rioult-Pedotti M-S, Luft AR. 2009. Dopamine in motor cortex is necessary for skill learning and synaptic plasticity. PLoS One. 4:e7082.
40. Olesen PJ, Westerberg H, Klingberg T. 2003. Increased prefrontal and parietal brain activity after training of working memory. Nat Neurosci. 17:75-79.
41. Olshausen BA, Anderson CH, Essen DCV. 1995. A multiscale dynamic routing circuit for forming size-and position-invariant object representations. J Comput Neurosci. 2:45-62.
42. Olshausen BA, Anderson CH, Essen DCV. 1993. A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information. J Neurosci. 13:4700-4719.
43. Pawlak V, Wickens JR, Kirkwood A, Kerr JN. 2010. Timing is not everything: neuromodulation opens the STDP gate. Front Synaptic Neurosci. 2:146.
44. Rainer G, Miller EK. 2000. Effects of visual experience on the representation of objects in the prefrontal cortex. Neuron. 27:179-189.
45. Reynolds JN, Hyland BI, Wickens JR. 2001. A cellular mechanism of reward-related learning. Nature. 413:67-70.
46. Reynolds JN, Wickens JR. 2002. Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw. 15:507-521.
47. Rich EL, Shapiro M. 2009. Rat prefrontal cortical neurons selectively code strategy switches. J Neurosci. 29:7208-7219.
48. Ridderinkhof KR, Ullsperger M, Crone EA, Nieuwenhuis S. 2004. The role of the medial frontal cortex in cognitive control. Science. 306:443-447.
49. Rodriguez JS, Paule MG. 2009. Working Memory Delayed Response Tasks in Monkeys. In: Buccafusco JJ, editor. Methods of Behavior Analysis in Neuroscience. 2nd ed. Boca Raton (FL): CRC Press: Chapter 12. Available from: http://www.ncbi.nlm.nih.gov/books/NBK5227/.
50. Salinas E, Sejnowski TJ. 2001. Correlated neuronal activity and the flow of neural information. Nat Rev Neurosci. 2:539-550.
51. Schultz W. 2007. Behavioral dopamine signals. Trends Neurosci. 30:203-210.
52. Schultz W, Dayan P, Montague PR. 1997. A neural substrate of prediction and reward. Science. 275:1593-1599.
53. Sjöström PJ, Häusser M. 2006. A cooperative switch determines the sign of synaptic plasticity in distal dendrites of neocortical pyramidal neurons. Neuron. 51:227-238.
54. Sussillo D, Abbott LF. 2009. Generating coherent patterns of activity from chaotic neural networks. Neuron. 63:544-557.
55. Vogels TP, Abbott LF. 2009. Gating multiple signals through detailed balance of excitation and inhibition in spiking networks. Nat Neurosci. 12:483-491.
56. von der Malsburg C. 2003. Dynamic link architecture. In:Arbib MA, editor. The Handbook of Brain Theory and Neural Networks, 2nd ed. Cambridge (MA): MIT Press. p. 365-368.
57. von Melchner L, Pallas SL, Sur M. 2000. Visual behaviour mediated by retinal projection directed to the auditory pathway. Nature. 404:871-876.
58. Waters J, Helmchen F. 2004. Boosting of action potential backpropagation by neocortical network activity in vivo. J Neurosci. 24:11127-11136.
59. Yao D, Yamamura K, Narita N, Martin RE, Murray GM, Sessle BJ. 2002. Neuronal activity patterns in primate motor cortex related to trained or semiautomatic jaw and tongue movements. J Neurophysiol. 87:2531-2541.
60. Zylberberg A, Fernandez Slezak D, Roelfsema PR, Dehaene S, Sigman M. 2010. The brain's router: a cortical network model of serial processing in the primate brain. PLoS Comput Biol. 6: e1000765.