Dopamine-signaled reward predictions generated by competitive excitation and inhibition in a spiking neural network model

Frontiers in Computational Neuroscience

Volumn 5, Issue , 2011, Pages

  Chorley, Paul ^a   Seth, Anil K ^a  

^a UNIVERSITY OF SUSSEX   (United Kingdom)

Author keywords

Basal ganglia; Dopamine; Neuronal excitability; Prefrontal cortex; Reinforcement learning; STDP

Indexed keywords

AMINES; FORECASTING; MAMMALS; NEURAL NETWORKS; NEUROPHYSIOLOGY; REINFORCEMENT LEARNING;

BASAL GANGLIA; DOPAMINE; NEURONAL EXCITABILITY; PREFRONTAL CORTEX; STDP;

NEURONS;

DOPAMINE;

ARTICLE; CONTROLLED STUDY; DOPAMINERGIC TRANSMISSION; LATENT PERIOD; MATHEMATICAL ANALYSIS; MATHEMATICAL MODEL; NERVE CELL INHIBITION; NERVE CELL NETWORK; NERVE CELL PLASTICITY; NERVE CELL STIMULATION; NERVE CONDUCTION; NERVE EXCITABILITY; NEUROMODULATION; POSTSYNAPTIC INHIBITION; PREFRONTAL CORTEX; REWARD; STRIATE CORTEX;

EID: 84959515284     PISSN: None     EISSN: 16625188     Source Type: Journal    
DOI: 10.3389/fncom.2011.00021     Document Type: Article

References (49)

1. Abeles, M. (1982). Local Cortical Circuits: An Electrophysiological Study. Berlin: Springer.
2. Albin, R. L., Young, A. B., and Penney, J. B. (1989). The functional anatomy of basal ganglia disorders. Trends Neurosci. 12, 366-375.
3. Brown, J., Bullock, D., and Grossberg, S. (1999). How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues. J. Neurosci. 19, 10502-10511.
4. Dan, Y., and Poo, M. (2004). Spike timingdependent plasticity of neural circuits. Neuron 44, 23-30.
5. Di Filippo, M., Picconi, B., Tantucci, M., Ghiglierib, V., Bagettab, V., Sgobiob, C., Tozzia, A., Parnetti, L., Calabresia, P. (2009). Short-term and long-term plasticity at corticostriatal synapses: implications for learning and memory. Behav. Brain Res. 199, 108-118.
6. Ding, M., Chen, Y., and Bressler, S. (2006). Granger Causality: Basic Theory and Application to Neuroscience. Handbook of Time Series Analysis. Wienheim: Wiley, 438-460.
7. Durstewitz, D., and Gabriel, T. (2007). Dynamical basis of irregular spiking in NMDA-driven prefrontal cortex neurons. Cereb. Cortex 17, 894-908
8. Durstewitz, D., Seamans, J. K., and Sejnowski, T. J. (2000). Dopaminemediated stabilization of delayperiod activity in a network model of prefrontal cortex. J. Neurophysiol. 83, 1733-1750.
9. Fino, E., Glowinski, J., and Venance, L. (2005). Bidirectional activitydependent plasticity at corticostriatal synapses. J. Neurosci. 25, 11279-11287.
10. Friston, K. (2010). The free-energy principle: a unified brain theory? Nat. Rev. Neurosci. 11, 127-138.
11. Funahashi, S., and Goldman-Rakic, P. S. (1989). Mnemonic coding of visual space in the monkey’s dorsolateral prefrontal cortex. J. Neurophysiol. 61, 331-349.
12. Fuster, J. M. (1972). Cortex and memory: emergence of a new paradigm. J. Cogn. Neurosci. 21, 2047-2072.
13. Goldman-Rakic, P. S. (1996). Regional and cellular fractionation of working memory. Proc. Natl. Acad. Sci. U.S.A. 93, 473-480.
14. Gonon, F. (1997). Prolonged and extrasynaptic excitatory action of dopamine mediated by D1 receptors in the rat striatum in vivo. J. Neurosci. 17, 5972-5978.
15. Harris, K. D. (2005). Neural signatures of cell assembly organization. Nat. Rev. Neurosci. 6, 399-407.
16. Hazy, T. E., Frank, M. J., and O’Reilly, R. C. (2010). Neural mechanisms of acquired phasic dopamine responses in learning. Neurosci. Biobehav. Rev. 34, 701-720.
17. Hebb, D. (1949). The Organization of Behavior. New York: Wiley and Sons
18. Humphries, M. D., Lepora, N., Wood, R., and Gurney, K. (2009). Capturing dopaminergic modulation and bimodal membrane behaviour of striatal medium spiny neurons in accurate, reduced models. Front. Comput. Neurosci. 3:26. doi: 10.3389/neuro.10.026.2009
19. Hyland, B. I., Reynolds, N. J., Hay, J., Perk, C. G., and Miller, R. (2002). Firing modes of midbrain dopamine cells in the freely moving rat. Neuroscience 114, 475-492.
20. Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Trans. Neural Netw. 14, 1569-1572.
21. Izhikevich, E. M. (2006). Polychronization: computation with spikes. Neural Comput. 18, 245-282.
22. Izhikevich, E. M. (2007). Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb. Cortex 17, 2443-2452.
23. Ljungberg, T., Apicella, P., and Schultz, W. (1991). Responses of monkey dopamine neurons during delayed alternation performance. Brain Res. 567, 337-341.
24. Ljungberg, T., Apicella, P., and Schultz, W. (1992). Responses of monkey dopamine neurons during learning of behavioural reactions. J. Neurophysiol. 67, 145-163.
25. Lustig, C., Matell, M. S., and Meck, W. H. (2005). Not ‘just’; a coincidence: frontal-striatal interactions in working memory and interval timing. Memory 13, 441-448.
26. Maass, W., Natschlager, T., and Markram, H. (2002). Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531-2560.
27. McHaffie, J. G., Stanford, T. R., Stein, B. E., Coizet, V., and Redgrave, P. (2005). Subcortical loops through the basal ganglia. Trends Neurosci. 28, 401-407
28. Nambu, A., Tokuno, H., and Takada, M. (2002). Functional significance of the cortico-subthalamo-pallidal ‘hyperdirect’ pathway. Neurosci. Res. 43, 111-117.
29. Nicola, S. M., Surmeier, J., and Malenka, R. C. (2000). Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annu. Rev. Neurosci. 23, 185-215.
30. Pan, W., Schmidt, R., Wickens, J. R., and Hyland, B. I. (2005). Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the rewardlearning network. J. Neurosci. 26, 6242-6235.
31. Pan, W., Schmidt, R., Wickens, J. R., and Hyland, B. I. (2008). Tripartite mechanism of extinction suggested by dopamine neuron activity and temporal difference model. J. Neurosci. 28, 9619-9631.
32. Pascual-Leone, A., and Hallett, M. (1994). Induction of errors in a delayed response task by transcranial magnetic stimulation of the dorsolateral prefrontal cortex. Neuroreport 5, 2517-2520.
33. Pawlak, V., and Kerr, J. N. (2008). Dopamine receptor activation is required for corticostriatal spike-timing-dependent plasticity. J. Neurosci. 28, 2435-2446.
34. Redgrave, P., and Gurney, K. (2006). The short-latency dopamine signal: a role in discovering novel actions? Nat. Rev. Neurosci. 7, 967-975.
35. Redgrave, P., Gurney, K., and Reynolds, J. (2008). What is reinforced by phasic dopamine signals? Brain Res. Rev. 58, 322-339.
36. Schultz, W. (1992). Neuronal activity in monkey ventral striatum related to the expectation of reward. J. Neurosci. 12, 4595-4610.
37. Schultz, W. (1998). Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1-27.
38. Schultz, W. (2003). Changes in behavior-related neuronal activity in the striatum during learning. Trends Neurosci. 26, 321-328.
39. Schultz, W. (2007). Multiple dopamine functions at different time courses. Annu. Rev. Neurosci. 30, 259-288
40. Schultz, W., and Romo, R. (1990). Dopamine neurons of the monkey midbrain contingencies of responses to stimuli eliciting immediate behavioral reaction. J. Neurophysiol. 63, 607-624
41. Shen, W., Flajolet, M., Greengard, P., and Surmeier, D. J. (2008). Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321, 848-851.
42. Softky, W. R., and Koch, C. (1993). The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. J. Neurosci. 13, 334-350.
43. Stassinoploulos, D., and Bak, P. (1995). Democratic reinforcement: a principle for brain function. Phys. Rev. E Stat. Phys. Plasmas Fluids Relat. Interdiscip. Topics. 51, 5033-5039
44. Sutton, R. S., and Barto, A. G. (1998). Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press.
45. Szatmary, B., and Izhikevich, E. M. (2010). Spike-timing theory of working memory. PLoS Comput. Biol. 6, 20808877. doi: 10.1371/journal.pcbi.1000879
46. Tan, C. O., and Bullock, D. (2008). A local circuit model of learned striatal and dopamine cell responses under probabilistic schedules of reward. J. Neurosci. 28, 10062-10074.
47. Voorn, P., Vanderschuren, L. J. M. J., Groenewegen, H. J., Robbins, T. W., and Pennartz, C. M. A. (2004). Putting a spin on the dorsal-ventral divide of the striatum. Trends Neurosci. 27, 468-474.
48. Williams, G. V., and Castner, S. A. (2006). Under the curve: critical issues for elucidating D1 receptor function in working memory. Neuroscience 139, 263-276.
49. Zheng, T., and Wilson, C. J. (2002). Corticostriatal combinatorics: the implications of cortico-striatal axonal arborizations. J. Neurophysiol. 87, 1007-1017