Neural control of dopamine neurotransmission: Implications for reinforcement learning

European Journal of Neuroscience

Volumn 35, Issue 7, 2012, Pages 1115-1123

  Aggarwal, Mayank ^a   Hyland, Brian I ^b   Wickens, Jeffery R ^a  

^a OKINAWA INSTITUTE OF SCIENCE AND TECHNOLOGY GRADUATE UNIVERSITY   (Japan)

^b Medical School   (New Zealand)

Author keywords

Basal ganglia; GABA; Striatum; Temporal difference

Indexed keywords

DOPAMINE;

ARTICLE; ASSOCIATION; BASAL GANGLION; CORPUS STRIATUM; DOPAMINE RELEASE; DOPAMINERGIC NERVE CELL; DOPAMINERGIC TRANSMISSION; GLOBUS PALLIDUS; NERVE CELL INHIBITION; NERVE CELL STIMULATION; NERVE PROJECTION; NEUROMODULATION; NONHUMAN; PREDICTION; PRIORITY JOURNAL; REINFORCEMENT; REWARD; STIMULUS RESPONSE; SUBSTANTIA NIGRA; SYNAPSE;

ANIMALS; DOPAMINE; DOPAMINERGIC NEURONS; HUMANS; LEARNING; NEURAL INHIBITION; NEURONS; REINFORCEMENT (PSYCHOLOGY); SYNAPTIC TRANSMISSION;

EID: 84859369427     PISSN: 0953816X     EISSN: 14609568     Source Type: Journal    
DOI: 10.1111/j.1460-9568.2012.08055.x     Document Type: Article

References (72)

1. Adamantidis, A.R., Tsai, H.C., Boutrel, B., Zhang, F., Stuber, G.D., Budygin, E.A., Tourino, C., Bonci, A., Deisseroth, K. & de Lecea, L. (2011) Optogenetic interrogation of dopaminergic modulation of the multiple phases of reward-seeking behavior. J. Neurosci., 31, 10829-10835.
2. Arbuthnott, G.W. & Wickens, J. (2007) Space, time and dopamine. Trends Neurosci., 30, 62-69.
3. Barto, A.G., Sutton, R.S. & Anderson, C.W. (1983) Neuronlike elements that can solve difficult learning control problems. IEEE Trans. Syst. Man Cyber., 15, 835-846.
4. Bassareo, V., Musio, P. & Di Chiara, G. (2011) Reciprocal responsiveness of nucleus accumbens shell and core dopamine to food- and drug-conditioned stimuli. Psychopharmacology, 214, 687-697.
5. Beckstead, R.M., Domesick, V.B. & Nauta, W.J.H. (1979) Efferent connections of the substantia nigra and ventral tegmental area in the rat. Brain Res., 175, 191-217.
6. Brog, J.S., Salyapongse, A., Deutch, A.Y. & Zahm, D.S. (1993) The patterns of afferent innervation of the core and shell in the 'accumbens' part of the rat ventral striatum: immunohistochemical detection of retrogradely transported fluoro-gold. J. Comp. Neurol., 338, 255-278.
7. Brown, H.D., McCutcheon, J.E., Cone, J.J., Ragozzino, M.E. & Roitman, M.F. (2011) Primary food reward and reward-predictive stimuli evoke different patterns of phasic dopamine signaling throughout the striatum. Eur. J. Neurosci., 34, 1997-2006.
8. Carrillo-Reid, L., Tecuapetla, F., Tapia, D., Hernandez-Cruz, A., Galarraga, E., Drucker-Colin, R. & Bargas, J. (2008) Encoding network states by striatal cell assemblies. J. Neurophysiol., 99, 1435-1450.
9. Christoph, G.R., Leonzio, R.J. & Wilcox, K.S. (1986) Stimulation of the lateral habenula inhibits dopamine-containing neurons in the substantia nigra and ventral tegmental area of the rat. J. Neurosci., 6, 613-619.
10. Deniau, J.M., Kitai, S.T., Donoghue, J.P. & Grofova, I. (1982) Neuronal interactions in the substantia nigra pars reticulata through axon collaterals of the projection neurons. An electrophysiological and morphological study. Exp. Brain Res., 47, 105-113.
11. Deniau, J.M., Menetrey, A. & Charpier, S. (1996) The lamellar organization of the rat substantia nigra pars reticulata: segregated patterns of striatal afferents and relationship to the topography of corticostriatal projections. Neuroscience, 73, 761-781.
12. DeVito, J.L. & Anderson, M.E. (1982) An autoradiographic study of efferent connections of the globus pallidus in Macaca mulatta. Exp. Brain Res., 46, 107-117.
13. DeVito, J.L., Anderson, M.E. & Walsh, K.E. (1980) A horseradish peroxidase study of afferent connections of the globus pallidus in Macaca mulatta. Exp. Brain Res., 38, 65-73.
14. Doucet, G., Descarries, L. & Garcia, S. (1986) Quantification of the dopamine innervation in adult rat neostriatum. Neuroscience, 19, 427-445.
15. Engberg, G., Kling-Petersen, T. & Nissbrandt, H. (1993) GABAB-receptor activation alters the firing pattern of dopamine neurons in the rat substantia nigra. Synapse, 15, 229-238.
16. Ferguson, S.M., Eskenazi, D., Ishikawa, M., Wanat, M.J., Phillips, P.E., Dong, Y., Roth, B.L. & Neumaier, J.F. (2011) Transient neuronal inhibition reveals opposing roles of indirect and direct pathways in sensitization. Nat. Neurosci., 14, 22-24.
17. Fiorillo, C.D., Newsome, W.T. & Schultz, W. (2008) The temporal precision of reward prediction in dopamine neurons. Nat. Neurosci., 11, 966-973.
18. Floresco, S.B., West, A.R., Ash, B., Moore, H. & Grace, A.A. (2003) Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission. Nat. Neurosci., 6, 968-973.
19. Garris, P.A. & Wightman, R.M. (1994) Different kinetics govern dopaminergic neurotransmission in the amygdala, prefrontal cortex, and striatum: an in vivo voltammetric study. J. Neurosci., 14, 442-450.
20. Gauthier, J., Parent, M., Levesque, M. & Parent, A. (1999) The axonal arborization of single nigrostriatal neurons in rats. Brain Res., 834, 228-232.
21. Glimcher, P.W. (2011) Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proc. Natl Acad. Sci. USA, 108(Suppl. 3), 15647-15654.
22. Grace, A.A. & Bunney, B.S. (1983) Intracellular and extracellular electrophysiology of nigral dopaminergic neurons-1. Identification and characterization. Neuroscience, 10, 301-315.
23. Grace, A.A. & Bunney, B.S. (1984) The control of firing pattern in nigral dopamine neurons: burst firing. J. Neurosci., 4, 2877-2890.
24. Grace, A.A. & Onn, S.P. (1989) Morphology and electrophysiological properties of immunocytochemically identified rat dopamine neurons recorded in vitro. J. Neurosci., 9, 3463-3481.
25. Grillner, P. & Mercuri, N.B. (2002) Intrinsic membrane properties and synaptic inputs regulating the firing activity of the dopamine neurons. Behav. Brain Res., 130, 149-169.
26. Groenewegen, H.J., Wright, C.I., Beijer, A.V. & Voorn, P. (1999) Convergence and segregation of ventral striatal inputs and outputs. Ann. N. Y. Acad. Sci., 877, 49-63.
27. Grossberg, S. & Schmajuk, N. (1989) Neural dynamics of adaptive timing and temporal discrimination during associative learning. Neural Netw., 2, 79-102.
28. Hara, E., Kubikova, L., Hessler, N.A. & Jarvis, E.D. (2007) Role of the midbrain dopaminergic system in modulation of vocal brain activation by social context. Eur. J. Neurosci., 25, 3406-3416.
29. Hassani, O.K., Francois, C., Yelnik, J. & Feger, J. (1997) Evidence for a dopaminergic innervation of the subthalamic nucleus in the rat. Brain Res., 749, 88-94.
30. Hazy, T.E., Frank, M.J. & O'Reilly, R.C. (2010) Neural mechanisms of acquired phasic dopamine responses in learning. Neurosci. Biobehav. Rev., 34, 701-720.
31. Heimer, L., Zahm, D.S., Churchill, L., Kalivas, P.W. & Wohltmann, C. (1991) Specificity in the projection patterns of accumbal core and shell in the rat. Neuroscience, 41, 89-125.
32. Herkenham, M. & Nauta, W.J. (1977) Afferent connections of the habenular nuclei in the rat. A horseradish peroxidase study, with a note on the fiber-of-passage problem. J. Comp. Neurol., 173, 123-146.
33. Hyland, B.I., Reynolds, J.N., Hay, J., Perk, C.G. & Miller, R. (2002) Firing modes of midbrain dopamine cells in the freely moving rat. Neuroscience, 114, 475-492.
34. Ito, R., Robbins, T.W. & Everitt, B.J. (2004) Differential control over cocaine-seeking behavior by nucleus accumbens core and shell. Nat. Neurosci., 7, 389-397.
35. Izhikevich, E.M. (2007) Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb. Cortex, 17, 2443-2452.
36. Jimenez-Castellanos, J. & Graybiel, A.M. (1987) Subdivisions of the dopamine-containing A8-A9-A10 complex identified by their differential mesostriatal innervation of striosomes and extrastriosomal matrix. Neuroscience, 23, 223-242.
37. Johnson, S.W., Seutin, V. & North, R.A. (1992) Burst firing in dopamine neurons induced by N-methyl-D-aspartate: role of electrogenic sodium pump. Science, 258, 665-667.
38. Joshua, M., Adler, A., Prut, Y., Vaadia, E., Wickens, J.R. & Bergman, H. (2009) Synchronization of midbrain dopaminergic neurons is enhanced by rewarding events. Neuron, 62, 695-704.
39. Kawaguchi, Y., Wilson, C.J. & Emson, P.C. (1990) Projection subtypes of rat neostriatal matrix cells revealed by intracellular injection of biocytin. J. Neurosci., 10, 3421-3438.
40. Korotkova, T.M., Sergeeva, O.A., Eriksson, K.S., Haas, H.L. & Brown, R.E. (2003) Excitation of ventral tegmental area dopaminergic and nondopaminergic neurons by orexins/hypocretins. J. Neurosci., 23, 7-11.
41. Korotkova, T.M., Ponomarenko, A.A., Brown, R.E. & Haas, H.L. (2004) Functional diversity of ventral midbrain dopamine and GABAergic neurons. Mol. Neurobiol., 29, 243-259.
42. Kravitz, A.V., Freeze, B.S., Parker, P.R., Kay, K., Thwin, M.T., Deisseroth, K. & Kreitzer, A.C. (2010) Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry. Nature, 466, 622-626.
43. Lacey, M.G., Mercuri, N.B. & North, R.A. (1989) Two cell types in rat substantia nigra zona compacta distinguished by membrane properties and the actions of dopamine and opioids. J. Neurosci., 9, 1233-1241.
44. Levesque, M. & Parent, A. (2005) The striatofugal fiber system in primates: a reevaluation of its organization based on single-axon tracing studies. Proc. Natl Acad. Sci. USA, 102, 11888-11893.
45. Lodge, D.J. & Grace, A.A. (2006) The hippocampus modulates dopamine neuron responsivity by regulating the intensity of phasic neuron activation. Neurospsychopharmacology, 31, 1356-1361.
46. Ludvig, E.A., Sutton, R.S. & Kehoe, E.J. (2008) Stimulus representation and the timing of reward-prediction errors in models of the dopamine system. Neural Comput., 20, 3034-3054.
47. Luo, A.H., Tahsili-Fahadan, P., Wise, R.A., Lupica, C.R. & Aston-Jones, G. (2011) Linking context with reward: a functional circuit from hippocampal CA3 to ventral tegmental area. Science, 333, 353-357.
48. Montague, P.R., Dayan, P. & Sejnowski, T.J. (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci., 16, 1936-1947.
49. Owesson-White, C.A., Cheer, J.F., Beyene, M., Carelli, R.M. & Wightman, R.M. (2008) Dynamic changes in accumbens dopamine correlate with learning during intracranial self-stimulation. Proc. Natl Acad. Sci. USA, 105, 11957-11962.
50. Paladini, C.A. & Tepper, J.M. (1999) GABA(A) and GABA(B) antagonists differentially affect the firing pattern of substantia nigra dopaminergic neurons in vivo. Synapse, 32, 165-176.
51. Pan, W.X., Schmidt, R., Wickens, J.R. & Hyland, B.I. (2005) Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci., 25, 6235-6242.
52. Pan, W.X., Schmidt, R., Wickens, J.R. & Hyland, B.I. (2008) Tripartite mechanism of extinction suggested by dopamine neuron activity and temporal difference model. J. Neurosci., 28, 9619-9631.
53. Parent, A., Smith, Y., Filion, M. & Dumas, J. (1989) Distinct afferents to internal and external pallidal segments in the squirrel-monkey. Neurosci. Lett., 96, 140-144.
54. Ponzi, A. & Wickens, J. (2010) Sequentially switching cell assemblies in random inhibitory networks of spiking neurons in the striatum. J. Neurosci., 30, 5894-5911.
55. Reynolds, J.N. & Wickens, J.R. (2002) Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw., 15, 507-521.
56. Reynolds, J.N.J., Hyland, B.I. & Wickens, J.R. (2001) A cellular mechanism of reward-related learning. Nature, 413, 67-70.
57. Schultz, W., Dayan, P. & Montague, P.R. (1997) A neural substrate of prediction and reward. Science, 275, 1593-1599.
58. Shepard, P.D., Holcomb, H.H. & Gold, J.M. (2006) The presence of absence: habenular regulation of dopamine neurons and the encoding of negative outcomes. Schizophr. Bull., 32, 417-421.
59. Smith, K.S., Tindell, A.J., Aldridge, J.W. & Berridge, K.C. (2009) Ventral pallidum roles in reward and motivation. Behav. Brain Res., 196, 155-167.
60. Steffensen, S.C., Svingos, A.L., Pickel, V.M. & Henriksen, S.J. (1998) Electrophysiological characterization of GABAergic neurons in the ventral tegmental area. J. Neurosci., 18, 8003-8015.
61. Tepper, J.M. & Lee, C.R. (2007) GABAergic control of substantia nigra dopaminergic neurons. Prog. Brain Res., 160, 189-208.
62. Tsai, H.C., Zhang, F., Adamantidis, A., Stuber, G.D., Bonci, A., de Lecea, L. & Deisseroth, K. (2009) Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science, 324, 1080-1084.
63. Usuda, I., Tanaka, K. & Chiba, T. (1998) Efferent projections of the nucleus accumbens in the rat with special reference to subdivision of the nucleus: biotinylated dextran amine study. Brain Res., 797, 73-93.
64. Voorn, P., Jorritsmabyham, B., Vandijk, C. & Buijs, R.M. (1986) The dopaminergic innervation of the ventral striatum in the rat - a light-microscopic and electron-microscopic study with antibodies against dopamine. J. Comp. Neurol., 251, 84-99.
65. Wickens, J.R. (1993) A Theory of the Striatum. Pergamon Press, Oxford.
66. Wickens, J.R. (2009) Synaptic plasticity in the basal ganglia. Behav. Brain Res., 199, 119-128.
67. Wickens, J.R., Begg, A.J. & Arbuthnott, G.W. (1996) Dopamine reverses the depression of rat cortico-striatal synapses which normally follows high frequency stimulation of cortex in vitro. Neuroscience, 70, 1-5.
68. Wilson, C.J. & Callaway, J.C. (2000) Coupled oscillator model of the dopaminergic neuron of the substantia nigra. J. Neurophysiol., 83, 3084-3100.
69. Wu, Y., Richard, S. & Parent, A. (2000) The organization of the striatal output system: a single-cell juxtacellular labeling study in the rat. Neurosci. Res., 38, 49-62.
70. Yanagihara, S. & Hessler, N.A. (2006) Modulation of singing-related activity in the songbird ventral tegmental area by social context. Eur. J. Neurosci., 24, 3619-3627.
71. Yin, R. & French, E.D. (2000) A comparison of the effects of nicotine on dopamine and non-dopamine neurons in the rat ventral tegmental area: an in vitro electrophysiological study. Brain Res. Bull., 51, 507-514.
72. Zhang, L., Doyon, W.M., Clark, J.J., Phillips, P.E. & Dani, J.A. (2009) Controls of tonic and phasic dopamine transmission in the dorsal and ventral striatum. Mol. Pharmacol., 76, 396-404.