Dopamine Prediction Errors in Reward Learning and Addiction: From Theory to Neural Circuitry

Neuron

Volumn 88, Issue 2, 2015, Pages 247-263

  Keiflin, Ronald ^a   Janak, Patricia H ^a,b  

^a JOHNS HOPKINS UNIVERSITY   (United States)

^b JOHNS HOPKINS UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DOPAMINE;

ADAPTATION; ADDICTION; BRAIN FUNCTION; DOPAMINERGIC NERVE CELL; DORSOMEDIAL STRIATUM; DRUG DEPENDENCE; ELECTROENCEPHALOGRAM; EXCITATORY POSTSYNAPTIC POTENTIAL; FOOD ADDICTION; HUMAN; LEARNING; NERVE CELL NETWORK; NERVE CELL PLASTICITY; NERVE PROJECTION; NERVOUS SYSTEM PARAMETERS; NONHUMAN; PHENOTYPE; PRIORITY JOURNAL; REVIEW; REWARD; REWARD PREDICTION ERROR; SIGNAL TRANSDUCTION; STIMULUS RESPONSE; SUBSTANTIA NIGRA; SYNAPSE; VENTRAL TEGMENTUM; ANIMAL; FORECASTING; METABOLISM; PHYSIOLOGY; THEORETICAL MODEL;

ANIMALS; BEHAVIOR, ADDICTIVE; DOPAMINE; DOPAMINERGIC NEURONS; FORECASTING; HUMANS; LEARNING; MODELS, THEORETICAL; NERVE NET; REWARD;

EID: 84944881652     PISSN: 08966273     EISSN: 10974199     Source Type: Journal    
DOI: 10.1016/j.neuron.2015.08.037     Document Type: Review

References (195)

1. Addy N.A., Daberkow D.P., Ford J.N., Garris P.A., Wightman R.M. Sensitization of rapid dopamine signaling in the nucleus accumbens core and shell after repeated cocaine in rats. J. Neurophysiol. 2010, 104:922-931.
2. Aggarwal M., Hyland B.I., Wickens J.R. Neural control of dopamine neurotransmission: implications for reinforcement learning. Eur. J. Neurosci. 2012, 35:1115-1123.
3. Bayer H.M., Glimcher P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 2005, 47:129-141.
4. Beckstead R.M., Domesick V.B., Nauta W.J. Efferent connections of the substantia nigra and ventral tegmental area in the rat. Brain Res. 1979, 175:191-217.
5. Belin D., Everitt B.J. Cocaine seeking habits depend upon dopamine-dependent serial connectivity linking the ventral with the dorsal striatum. Neuron 2008, 57:432-441.
6. Belin D., Jonkman S., Dickinson A., Robbins T.W., Everitt B.J. Parallel and interactive learning processes within the basal ganglia: relevance for the understanding of addiction. Behav. Brain Res. 2009, 199:89-102.
7. Berke J.D., Hyman S.E. Addiction, dopamine, and the molecular mechanisms of memory. Neuron 2000, 25:515-532.
8. Berridge K.C. From prediction error to incentive salience: Mesolimbic computation of reward motivation. Eur. J. Neurosci. 2012, 35:1124-1143.
9. Berridge K.C., Robinson T.E. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience?. Brain Res. Brain Res. Rev. 1998, 28:309-369.
10. Berridge K.C., Robinson T.E. Parsing reward. Trends Neurosci. 2003, 26:507-513.
11. Bocklisch C., Pascoli V., Wong J.C., House D.R., Yvon C., Roo M., Tan K.R., Luscher C. Cocaine disinhibits dopamine neurons by potentiation of GABA transmission in the ventral tegmental area. Science 2013, 341:1521-1525.
12. Botvinick M.M., Niv Y., Barto A.C. Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective. Cognition 2009, 113:262-280.
13. Brischoux F., Chakraborty S., Brierley D.I., Ungless M.A. Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc. Natl. Acad. Sci. USA 2009, 106:4894-4899.
14. Bromberg-Martin E.S., Matsumoto M., Hikosaka O. Dopamine in motivational control: Rewarding, aversive, and alerting. Neuron 2010, 68:815-834.
15. Brown J., Bullock D., Grossberg S. How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues. J. Neurosci. 1999, 19:10502-10511.
16. Brown H.D., McCutcheon J.E., Cone J.J., Ragozzino M.E., Roitman M.F. Primary food reward and reward-predictive stimuli evoke different patterns of phasic dopamine signaling throughout the striatum. Eur. J. Neurosci. 2011, 34:1997-2006.
17. Cheer J.F., Wassum K.M., Sombers L.A., Heien M.L., Ariansen J.L., Aragona B.J., Phillips P.E., Wightman R.M. Phasic dopamine release evoked by abused substances requires cannabinoid receptor activation. J. Neurosci. 2007, 27:791-795.
18. Chen B.T., Bowers M.S., Martin M., Hopf F.W., Guillory A.M., Carelli R.M., Chou J.K., Bonci A. Cocaine but not natural reward self-administration nor passive cocaine infusion produces persistent LTP in the VTA. Neuron 2008, 59:288-297.
19. Chergui K., Charléty P.J., Akaoka H., Saunier C.F., Brunet J.L., Buda M., Svensson T.H., Chouvet G. Tonic activation of NMDA receptors causes spontaneous burst discharge of rat midbrain dopamine neurons in vivo. Eur. J. Neurosci. 1993, 5:137-144.
20. Chuhma N., Tanaka K.F., Hen R., Rayport S. Functional connectome of the striatal medium spiny neuron. J. Neurosci. 2011, 31:1183-1192.
21. Clark J.J., Collins A.L., Sanford C.A., Phillips P.E. Dopamine encoding of pavlovian incentive stimuli diminishes with extended training. J. Neurosci. 2013, 33:3526-3532.
22. Cohen J.Y., Haesler S., Vong L., Lowell B.B., Uchida N. Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 2012, 482:85-88.
23. Corbit L.H., Nie H., Janak P.H. Habitual alcohol seeking: time course and the contribution of subregions of the dorsal striatum. Biol. Psychiatry 2012, 72:389-395.
24. Corbit L.H., Chieng B.C., Balleine B.W. Effects of repeated cocaine exposure on habit learning and reversal by n-acetylcysteine. Neuropsychopharmacology 2014, 39:1893-1901.
25. Corbit L.H., Nie H., Janak P.H. Habitual responding for alcohol depends upon both AMPA and D2 receptor signaling in the dorsolateral striatum. Front. Behav. Neurosci. 2014, 8:301.
26. Costa R.M. Plastic corticostriatal circuits for action learning: what's dopamine got to do with it?. Ann. N Y Acad. Sci. 2007, 1104:172-191.
27. Covey D.P., Roitman M.F., Garris P.A. Illicit dopamine transients: reconciling actions of abused drugs. Trends Neurosci. 2014, 37:200-210.
28. Daberkow D.P., Brown H.D., Bunner K.D., Kraniotis S.A., Doellman M.A., Ragozzino M.E., Garris P.A., Roitman M.F. Amphetamine paradoxically augments exocytotic dopamine release and phasic dopamine signals. J. Neurosci. 2013, 33:452-463.
29. Day J.J., Roitman M.F., Wightman R.M., Carelli R.M. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat. Neurosci. 2007, 10:1020-1028.
30. Deroche-Gamonet V., Belin D., Piazza P.V. Evidence for addiction-like behavior in the rat. Science 2004, 305:1014-1017.
31. Di Chiara G. Drug addiction as dopamine-dependent associative learning disorder. Eur. J. Pharmacol. 1999, 375:13-30.
32. Di Chiara G., Imperato A. Drugs abused by humans preferentially increase synaptic dopamine concentrations in the mesolimbic system of freely moving rats. Proc. Natl. Acad. Sci. USA 1988, 85:5274-5278.
33. Everitt B.J., Robbins T.W. Neural systems of reinforcement for drug addiction: From actions to habits to compulsion. Nat. Neurosci. 2005, 8:1481-1489.
34. Faure A., Haberland U., Conde F., El Massioui N. Lesion to the nigrostriatal dopamine system disrupts stimulus-response habit formation. J. Neurosci. 2005, 25:2771-2780.
35. Ferrario C.R., Robinson T.E. Amphetamine pretreatment accelerates the subsequent escalation of cocaine self-administration behavior. Eur. Neuropsychopharmacol. 2007, 17:352-357.
36. Fields H.L., Hjelmstad G.O., Margolis E.B., Nicola S.M. Ventral tegmental area neurons in learned appetitive behavior and positive reinforcement. Annu. Rev. Neurosci. 2007, 30:289-316.
37. Fiorino D.F., Phillips A.G. Facilitation of sexual behavior and enhanced dopamine efflux in the nucleus accumbens of male rats after d-amphetamine-induced behavioral sensitization. J. Neurosci. 1999, 19:456-463.
38. Flagel S.B., Clark J.J., Robinson T.E., Mayo L., Czuj A., Willuhn I., Akers C.A., Clinton S.M., Phillips P.E., Akil H. A selective role for dopamine in stimulus-reward learning. Nature 2011, 469:53-57.
39. Garavan H., Pankiewicz J., Bloom A., Cho J.K., Sperry L., Ross T.J., Salmeron B.J., Risinger R., Kelley D., Stein E.A. Cue-induced cocaine craving: Neuroanatomical specificity for drug users and drug stimuli. Am. J. Psychiatry 2000, 157:1789-1798.
40. Garris P.A., Wightman R.M. Different kinetics govern dopaminergic transmission in the amygdala, prefrontal cortex, and striatum: an in vivo voltammetric study. J. Neurosci. 1994, 14:442-450.
41. Gerfen C.R., Surmeier D.J. Modulation of striatal projection systems by dopamine. Annu. Rev. Neurosci. 2011, 34:441-466.
42. Gerfen C.R., Herkenham M., Thibault J. The neostriatal mosaic: II. Patch- and matrix-directed mesostriatal dopaminergic and non-dopaminergic systems. J. Neurosci. 1987, 7:3915-3934.
43. Gipson C.D., Kupchik Y.M., Shen H., Reissner K.J., Thomas C.A., Kalivas P.W. Relapse induced by cues predicting cocaine depends on rapid, transient synaptic potentiation. Neuron 2013, 77:867-872.
44. Glimcher P.W. Understanding dopamine and reinforcement learning: the dopamine reward prediction error hypothesis. Proc. Natl. Acad. Sci. USA 2011, 108:15647-15654.
45. Grace A.A., Floresco S.B., Goto Y., Lodge D.J. Regulation of firing of dopaminergic neurons and control of goal-directed behaviors. Trends Neurosci. 2007, 30:220-227.
46. Groenewegen H.J., Berendse H.W. Connections of the subthalamic nucleus with ventral striatopallidal parts of the basal ganglia in the rat. J. Comp. Neurol. 1990, 294:607-622.
47. Haber S.N., Fudge J.L., McFarland N.R. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J. Neurosci. 2000, 20:2369-2382.
48. Hart A.S., Rutledge R.B., Glimcher P.W., Phillips P.E. Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error term. J. Neurosci. 2014, 34:698-704.
49. Heather N. A conceptual framework for explaining drug addiction. J. Psychopharmacol. 1998, 12:3-7.
50. Heien M.L., Khan A.S., Ariansen J.L., Cheer J.F., Phillips P.E., Wassum K.M., Wightman R.M. Real-time measurement of dopamine fluctuations after cocaine in the brain of behaving rats. Proc. Natl. Acad. Sci. USA 2005, 102:10023-10028.
51. Henny P., Brown M.T., Northrop A., Faunes M., Ungless M.A., Magill P.J., Bolam J.P. Structural correlates of heterogeneous in vivo activity of midbrain dopaminergic neurons. Nat. Neurosci. 2012, 15:613-619.
52. Holland P.C. Overshadowing and blocking as acquisition deficits: no recovery after extinction of overshadowing or blocking cues. Q. J. Exp. Psychol. B 1999, 52:307-333.
53. Hong S., Hikosaka O. Pedunculopontine tegmental nucleus neurons provide reward, sensorimotor, and alerting signals to midbrain dopamine neurons. Neuroscience 2014, 282C:139-155.
54. Hong S., Jhou T.C., Smith M., Saleem K.S., Hikosaka O. Negative reward signals from the lateral habenula to dopamine neurons are mediated by rostromedial tegmental nucleus in primates. J. Neurosci. 2011, 31:11457-11471.
55. Horger B.A., Shelton K., Schenk S. Preexposure sensitizes rats to the rewarding effects of cocaine. Pharmacol. Biochem. Behav. 1990, 37:707-711.
56. Horger B.A., Giles M.K., Schenk S. Preexposure to amphetamine and nicotine predisposes rats to self-administer a low dose of cocaine. Psychopharmacology (Berl.) 1992, 107:271-276.
57. Humphries M.D., Prescott T.J. The ventral basal ganglia, a selection mechanism at the crossroads of space, strategy, and reward. Prog. Neurobiol. 2010, 90:385-417.
58. Hyman S.E., Malenka R.C., Nestler E.J. Neural mechanisms of addiction: the role of reward-related learning and memory. Annu. Rev. Neurosci. 2006, 29:565-598.
59. Ikemoto S. Dopamine reward circuitry: two projection systems from the ventral midbrain to the nucleus accumbens-olfactory tubercle complex. Brain Res. Brain Res. Rev. 2007, 56:27-78.
60. Ilango A., Kesner A.J., Keller K.L., Stuber G.D., Bonci A., Ikemoto S. Similar roles of substantia nigra and ventral tegmental dopamine neurons in reward and aversion. J. Neurosci. 2014, 34:817-822.
61. Ito R., Dalley J.W., Howes S.R., Robbins T.W., Everitt B.J. Dissociation in conditioned dopamine release in the nucleus accumbens core and shell in response to cocaine cues and during cocaine-seeking behavior in rats. J. Neurosci. 2000, 20:7489-7495.
62. Ito R., Dalley J.W., Robbins T.W., Everitt B.J. Dopamine release in the dorsal striatum during cocaine-seeking behavior under the control of a drug-associated cue. J. Neurosci. 2002, 22:6247-6253.
63. Jennings J.H., Ung R.L., Resendez S.L., Stamatakis A.M., Taylor J.G., Huang J., Veleta K., Kantak P.A., Aita M., Shilling-Scrivo K., et al. Visualizing hypothalamic network dynamics for appetitive and consummatory behaviors. Cell 2015, 160:516-527.
64. Jhou T.C., Fields H.L., Baxter M.G., Saper C.B., Holland P.C. The rostromedial tegmental nucleus (rMTG), a GABAergic afferent to midbrain dopamine neurons, encodes aversive stimuli and inhibits motor responses. Neuron 2009, 61:786-800.
65. Kalivas P.W. The glutamate homeostasis hypothesis of addiction. Nat. Rev. Neurosci. 2009, 10:561-572.
66. Kalivas P.W., Duffy P. Effect of acute and daily cocaine treatment on extracellular dopamine in the nucleus accumbens. Synapse 1990, 5:48-58.
67. Kamin L.J. Predictability, surprise, attention, and conditioning. Punishment and Aversive Behavior 1969, 279-296. Appleton-Century-Crofts, New York. B.A. Cambell, R.M. Church (Eds.).
68. Kempadoo K.A., Tourino C., Cho S.L., Magnani F., Leinninger G.M., Stuber G.D., Zhang F., Myers M.G., Deisseroth K., Lecea L., Bonci A. Hypothalamic neurotensin projections promote reward by enhancing glutamate transmission in the VTA. J. Neurosci. 2013, 33:7618-7626.
69. Keramati M., Gutkin B. Imbalanced decision hierarchy in addicts emerging from drug-hijacked dopamine spiraling circuit. PLoS ONE 2013, 8:e61489.
70. Kitai S.T., Shepard P.D., Callaway J.C., Scroggs R. Afferent modulation of dopamine neuron firing patterns. Curr. Opin. Neurobiol. 1999, 9:690-697.
71. Kobayashi Y., Okada K. Reward prediction error computation in the pedunculopontine tegmental nucleus neurons. Ann. N Y Acad. Sci. 2007, 1104:310-323.
72. Kobayashi S., Schultz W. Reward contexts extend dopamine signals to unrewarded stimuli. Curr. Biol. 2014, 24:56-62.
73. Koob G.F., Le Moal M. Neurobiological mechanisms for opponent motivational processes in addiction. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2008, 363:3113-3123.
74. Koob G.F., Volkow N.D. Neurocircuitry of addiction. Neuropsychopharmacology 2010, 35:217-238.
75. Koulchitsky S., De Backer B., Quertemont E., Charlier C., Seutin V. Differential effects of cocaine on dopamine neuron firing in awake and anesthetized rats. Neuropsychopharmacology 2012, 37:1559-1571.
76. Kravitz A.V., Tye L.D., Kreitzer A.C. Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nat. Neurosci. 2012, 15:816-818.
77. Kreitzer A.C., Malenka R.C. Striatal plasticity and basal ganglia circuit function. Neuron 2008, 60:543-554.
78. Lammel S., Ion D.I., Roeper J., Malenka R.C. Projection-specific modulation of dopamine neuron synapses by aversive and rewarding stimuli. Neuron 2011, 70:855-862.
79. Lammel S., Lim B.K., Ran C., Huang K.W., Betley M.J., Tye K.M., Deisseroth K., Malenka R.C. Input-specific control of reward and aversion in the ventral tegmental area. Nature 2012, 491:212-217.
80. Lashley K.S. The problem of serial order in behavior. Cerebral Mechanisms in Behavior 1951, 112-131. Wiley, New York. L.A. Jeffress (Ed.).
81. LeBlanc K.H., Maidment N.T., Ostlund S.B. Repeated cocaine exposure facilitates the expression of incentive motivation and induces habitual control in rats. PLoS ONE 2013, 8:e61355.
82. Lee B.R., Ma Y.Y., Huang Y.H., Wang X., Otaka M., Ishikawa M., Neumann P.A., Graziane N.M., Brown T.E., Suska A., et al. Maturation of silent synapses in amygdala-accumbens projection contributes to incubation of cocaine craving. Nat. Neurosci. 2013, 16:1644-1651.
83. Lee T., Cai L.X., Lelyveld V.S., Hai A., Jasanoff A. Molecular-level functional magnetic resonance imaging of dopaminergic signaling. Science 2014, 344:533-535.
84. Lenoir M., Serre F., Cantin L., Ahmed S.H. Intense sweetness surpasses cocaine reward. PLoS ONE 2007, 2:e698.
85. Liu Q.S., Pu L., Poo M.M. Repeated cocaine exposure in vivo facilitates LTP induction in midbrain dopamine neurons. Nature 2005, 437:1027-1031.
86. Ljungberg T., Apicella P., Schultz W. Responses of monkey dopamine neurons during learning of behavioral reactions. J. Neurophysiol. 1992, 67:145-163.
87. Lovinger D.M. Neurotransmitter roles in synaptic modulation, plasticity and learning in the dorsal striatum. Neuropharmacology 2010, 58:951-961.
88. Luscher C. Drug-evoked synaptic plasticity causing addictive behavior. J. Neurosci. 2013, 33:17641-17646.
89. Ma Y.Y., Lee B.R., Wang X., Guo C., Liu L., Cui R., Lan Y., Balcita-Pedicino J.J., Wolf M.E., Sesack S.R., et al. Bidirectional modulation of incubation of cocaine craving by silent synapse-based remodeling of prefrontal cortex to accumbens projections. Neuron 2014, 83:1453-1467.
90. Mahler S.V., Vazey E.M., Beckley J.T., Keistler C.R., McGlinchey E.M., Kaufling J., Wilson S.P., Deisseroth K., Woodward J.J., Aston-Jones G. Designer receptors show role for ventral pallidum input to ventral tegmental area in cocaine seeking. Nat. Neurosci. 2014, 17:577-585.
91. Mameli M., Luscher C. Synaptic plasticity and addiction: learning mechanisms gone awry. Neuropharmacology 2011, 61:1052-1059.
92. Mameli M., Halbout B., Creton C., Engblom D., Parkitna J.R., Spanagel R., Luscher C. Cocaine-evoked synaptic plasticity: persistence in the VTA triggers adaptations in the NAC. Nat. Neurosci. 2009, 12:1036-1041.
93. Matsumoto M., Hikosaka O. Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 2007, 447:1111-1115.
94. Matsumoto M., Hikosaka O. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 2009, 459:837-841.
95. McClure S.M., Daw N.D., Montague P.R. A computational substrate for incentive salience. Trends Neurosci. 2003, 26:423-428.
96. McCutcheon J.E., Cone J.J., Sinon C.G., Fortin S.M., Kantak P.A., Witten I.B., Deisseroth K., Stuber G.D., Roitman M.F. Optical suppression of drug-evoked phasic dopamine release. Front. Neural Circuits 2014, 8:114.
97. McNally G.P., Johansen J.P., Blair H.T. Placing prediction into the fear circuit. Trends Neurosci. 2011, 34:283-292.
98. Miller R.R., Barnet R.C., Grahame N.J. Assessment of the Rescorla-Wagner model. Psychol. Bull. 1995, 117:363-386.
99. Montague P.R., Dayan P., Sejnowski T.J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 1996, 16:1936-1947.
100. Morita K., Morishima M., Sakai K., Kawaguchi Y. Reinforcement learning: computing the temporal difference of values via distinct corticostriatal pathways. Trends Neurosci. 2012, 35:457-467.
101. Morita K., Morishima M., Sakai K., Kawaguchi Y. Dopaminergic control of motivation and reinforcement learning: a closed-circuit account for reward-oriented behavior. J. Neurosci. 2013, 33:8866-8890.
102. Muscat R., Iravani M.M., Kruk Z.L. The role of ventral tegmental dopamine neurons in locomotor sensitization following quinpirole or (+)-amphetamine: ex vivo voltammetric evidence. Neuroscience 1996, 75:1175-1184.
103. Nair-Roberts R.G., Chatelain-Badie S.D., Benson E., White-Cooper H., Bolam J.P., Ungless M.A. Stereological estimates of dopaminergic, GABAergic and glutamatergic neurons in the ventral tegmental area, substantia nigra and retrorubral field in the rat. Neuroscience 2008, 152:1024-1031.
104. Nakamura K., Ono T. Lateral hypothalamus neuron involvement in integration of natural and artificial rewards and cue signals. J. Neurophysiol. 1986, 55:163-181.
105. Nelson A., Killcross S. Amphetamine exposure enhances habit formation. J. Neurosci. 2006, 26:3805-3812.
106. Nestler E.J. Is there a common molecular pathway for addiction?. Nat. Neurosci. 2005, 8:1445-1449.
107. Ng J.P., Hubert G.W., Justice J.B. Increased stimulated release and uptake of dopamine in nucleus accumbens after repeated cocaine administration as measured by in vivo voltammetry. J. Neurochem. 1991, 56:1485-1492.
108. Nicola S.M., Surmeier J., Malenka R.C. Dopaminergic modulation of neuronal excitability in the striatum and nucleus accumbens. Annu. Rev. Neurosci. 2000, 23:185-215.
109. Nicola S.M., Yun I.A., Wakabayashi K.T., Fields H.L. Cue-evoked firing of nucleus accumbens neurons encodes motivational significance during a discriminative stimulus task. J. Neurophysiol. 2004, 91:1840-1865.
110. Nieh E.H., Matthews G.A., Allsop S.A., Presbrey K.N., Leppla C.A., Wichmann R., Neve R., Wildes C.P., Tye K.M. Decoding neural circuits that control compulsive sucrose seeking. Cell 2015, 160:528-541.
111. Niv Y., Daw N.D., Joel D., Dayan P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl.) 2007, 191:507-520.
112. O'Doherty J., Dayan P., Schultz J., Deichmann R., Friston K., Dolan R.J. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 2004, 304:452-454.
113. Olausson P., Jentsch J.D., Tronson N., Neve R.L., Nestler E.J., Taylor J.R. Deltafosb in the nucleus accumbens regulates food-reinforced instrumental behavior and motivation. J. Neurosci. 2006, 26:9196-9204.
114. Pan W.X., Hyland B.I. Pedunculopontine tegmental nucleus controls conditioned responses of midbrain dopamine neurons in behaving rats. J. Neurosci. 2005, 25:4725-4732.
115. Pan W.X., Schmidt R., Wickens J.R., Hyland B.I. Dopamine cells respond to predicted events during classical conditioning: Evidence for eligibility traces in the reward-learning network. J. Neurosci. 2005, 25:6235-6242.
116. Pan B., Hillard C.J., Liu Q.S. Endocannabinoid signaling mediates cocaine-induced inhibitory synaptic plasticity in midbrain dopamine neurons. J. Neurosci. 2008, 28:1385-1397.
117. Pascoli V., Terrier J., Espallergues J., Valjent E., O'Connor E.C., Luscher C. Contrasting forms of cocaine-evoked plasticity control components of relapse. Nature 2014, 509:459-464.
118. Perry A.N., Westenbroek C., Becker J.B. The development of a preference for cocaine over food identifies individual rats with addiction-like behaviors. PLoS ONE 2013, 8:e79465.
119. Phillips P.E., Stuber G.D., Heien M.L., Wightman R.M., Carelli R.M. Subsecond dopamine release promotes cocaine seeking. Nature 2003, 422:614-618.
120. Piazza P.V., Deminiere J.M., Le Moal M., Simon H. Factors that predict individual vulnerability to amphetamine self-administration. Science 1989, 245:1511-1513.
121. Pierce R.C., Kumaresan V. The mesolimbic dopamine system: the final common pathway for the reinforcing effect of drugs of abuse?. Neurosci. Biobehav. Rev. 2006, 30:215-238.
122. Pignatelli M., Bonci A. Role of dopamine neurons in reward and aversion: a synaptic plasticity perspective. Neuron 2015, 86:1145-1157.
123. Porrino L., Lyons D., Smith H., Daunais J., Nader M. Cocaine self-administration produces a progressive involvement of limbic, association, and sensorimotor striatal domains. J. Neurosci. 2004, 24:3554-3562.
124. Redish A.D. Addiction as a computational process gone awry. Science 2004, 306:1944-1947.
125. Rescorla R.A., Wagner A.R. A theory of pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. Classical Conditioning II: Current Research and Theory 1972, 64-99. Appleton Century Crofts, New York. A.H. Black, W.F. Prokasy (Eds.).
126. Robbins T.W., Everitt B.J. Functions of dopamine in the dorsal and ventral striatum. Semin. Neurosci. 1992, 4:119-127.
127. Robbins T.W., Everitt B.J. A role for mesencephalic dopamine in activation: Commentary on berridge (2006). Psychopharmacology (Berl.) 2007, 191:433-437.
128. Robinson T.E., Becker J.B. Enduring changes in brain and behavior produced by chronic amphetamine administration: A review and evaluation of animal models of amphetamine psychosis. Brain Res. 1986, 396:157-198.
129. Robinson T.E., Berridge K.C. Incentive-sensitization and addiction. Addiction 2001, 96:103-114.
130. Roesch M.R., Singh T., Brown P.L., Mullins S.E., Schoenbaum G. Ventral striatal neurons encode the value of the chosen action in rats deciding between differently delayed or sized rewards. J. Neurosci. 2009, 29:13365-13376.
131. Roitman M.F., Stuber G.D., Phillips P.E., Wightman R.M., Carelli R.M. Dopamine operates as a subsecond modulator of food seeking. J. Neurosci. 2004, 24:1265-1271.
132. Rolls E.T., Burton M.J., Mora F. Hypothalamic neuronal responses associated with the sight of food. Brain Res. 1976, 111:53-66.
133. Saal D., Dong Y., Bonci A., Malenka R.C. Drugs of abuse and stress trigger a common synaptic adaptation in dopamine neurons. Neuron 2003, 37:577-582.
134. Salamone J., Correa M. The mysterious motivational functions of mesolimbic dopamine. Neuron 2012, 76:470-485.
135. Schilstrom B., Yaka R., Argilli E., Suvarna N., Schumann J., Chen B.T., Carman M., Singh V., Mailliard W.S., Ron D., Bonci A. Cocaine enhances NMDA receptor-mediated currents in ventral tegmental area cells via dopamine D5 receptor-dependent redistribution of NMDA receptors. J. Neurosci. 2006, 26:8549-8558.
136. Schoenbaum G., Setlow B. Cocaine makes actions insensitive to outcomes but not extinction: implications for altered orbitofrontal-amygdalar function. Cereb. Cortex 2005, 15:1162-1169.
137. Schultz W. Predictive reward signal of dopamine neurons. J. Neurophysiol. 1998, 80:1-27.
138. Schultz W. Behavioral dopamine signals. Trends Neurosci. 2007, 30:203-210.
139. Schultz W. Updating dopamine reward signals. Curr. Opin. Neurobiol. 2013, 23:229-238.
140. Schultz W., Dickinson A. Neuronal coding of prediction errors. Annu. Rev. Neurosci. 2000, 23:473-500.
141. Schultz W., Apicella P., Ljungberg T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci. 1993, 13:900-913.
142. Schultz W., Dayan P., Montague P.R. A neural substrate of prediction and reward. Science 1997, 275:1593-1599.
143. Sesack S.R., Grace A.A. Cortico-basal ganglia reward network: Microcircuitry. Neuropsychopharmacology 2010, 35:27-47.
144. Shen H.W., Gipson C.D., Huits M., Kalivas P.W. Prelimbic cortex and ventral tegmental area modulate synaptic plasticity differentially in nucleus accumbens during cocaine-reinstated drug seeking. Neuropsychopharmacology 2014, 39:1169-1177.
145. Skog O.-J. Addiction: Definition and mechanisms. Choice, Behavioural Economics and Addiction 2003, 157-175. Elsevier, Oxford. N. Heather, R.E. Vuchinich (Eds.).
146. Sombers L.A., Beyene M., Carelli R.M., Wightman R.M. Synaptic overflow of dopamine in the nucleus accumbens arises from neuronal activity in the ventral tegmental area. J. Neurosci. 2009, 29:1735-1742.
147. Stacy A.W., Wiers R.W. Implicit cognition and addiction: A tool for explaining paradoxical behavior. Annu. Rev. Clin. Psychol. 2010, 6:551-575.
148. Stalnaker T.A., Calhoon G.G., Ogawa M., Roesch M.R., Schoenbaum G. Neural correlates of stimulus-response and response-outcome associations in dorsolateral versus dorsomedial striatum. Front. Integr. Neurosci. 2010, 4:12.
149. Steinberg E.E., Keiflin R., Boivin J.R., Witten I.B., Deisseroth K., Janak P.H. A causal link between prediction errors, dopamine neurons and learning. Nat. Neurosci. 2013, 16:966-973.
150. Steinberg E.E., Boivin J.R., Saunders B.T., Witten I.B., Deisseroth K., Janak P.H. Positive reinforcement mediated by midbrain dopamine neurons requires D1 and D2 receptor activation in the nucleus accumbens. PLoS ONE 2014, 9:e94771.
151. Stuber G.D., Roitman M.F., Phillips P.E., Carelli R.M., Wightman R.M. Rapid dopamine signaling in the nucleus accumbens during contingent and noncontingent cocaine administration. Neuropsychopharmacology 2005, 30:853-863.
152. Stuber G.D., Wightman R.M., Carelli R.M. Extinction of cocaine self-administration reveals functionally and temporally distinct dopaminergic signals in the nucleus accumbens. Neuron 2005, 46:661-669.
153. Sulzer D. How addictive drugs disrupt presynaptic dopamine neurotransmission. Neuron 2011, 69:628-649.
154. Sutton R., Barto A. Reinforcement Learning: An Introduction 1998, MIT Press, Cambridge, Mass.
155. Swanson L.W. The projections of the ventral tegmental area and adjacent regions: a combined fluorescent retrograde tracer and immunofluorescence study in the rat. Brain Res. Bull. 1982, 9:321-353.
156. Takahashi Y., Geoffrey Schoenbaum G., Niv Y. Silencing the critics: understanding the effects of cocaine sensitization on dorsolateral and ventral striatum in the context of an actor/critic model. Front. Neurosci. 2008, 2:86-99.
157. Takahashi Y.K., Roesch M.R., Wilson R.C., Toreson K., O'Donnell P., Niv Y., Schoenbaum G. Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex. Nat. Neurosci. 2011, 14:1590-1597.
158. Tan K., Yvon C., Turiault M., Mirzabekov J., Doehner J., Labouèbe G., Deisseroth K., Tye K., Lüscher C. GABA neurons of the VTA drive conditioned place aversion. Neuron 2012, 73:1173-1183.
159. Taylor J.R., Jentsch J.D. Repeated intermittent administration of psychomotor stimulant drugs alters the acquisition of pavlovian approach behavior in rats: differential effects of cocaine, d-amphetamine and 3,4- methylenedioxymethamphetamine ("ecstasy"). Biol. Psychiatry 2001, 50:137-143.
160. Tepper J.M., Lee C.R. GABAergic control of substantia nigra dopaminergic neurons. Prog. Brain Res. 2007, 160:189-208.
161. Threlfell S., Cragg S.J. Dopamine signaling in dorsal versus ventral striatum: the dynamic role of cholinergic interneurons. Front. Syst. Neurosci. 2011, 5:11.
162. Tian J., Uchida N. Habenula lesions reveal that multiple mechanisms underlie dopamine prediction errors. Neuron 2015, 87:1304-1316.
163. Timberlake W. Behavior systems, associationism, and pavlovian conditioning. Psychon. Bull. Rev. 1994, 1:405-420.
164. Tsai H.C., Zhang F., Adamantidis A., Stuber G.D., Bonci A., de Lecea L., Deisseroth K. Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science 2009, 324:1080-1084.
165. Ungless M.A., Whistler J.L., Malenka R.C., Bonci A. Single cocaine exposure in vivo induces long-term potentiation in dopamine neurons. Nature 2001, 411:583-587.
166. van Zessen R., Phillips J.L., Budygin E.A., Stuber G.D. Activation of VTA GABA neurons disrupts reward consumption. Neuron 2012, 73:1184-1194.
167. Vander Weele C.M., Porter-Stransky K.A., Mabrouk O.S., Lovic V., Singer B.F., Kennedy R.T., Aragona B.J. Rapid dopamine transmission within the nucleus accumbens: dramatic difference between morphine and oxycodone delivery. Eur. J. Neurosci. 2014, 40:3041-3054.
168. Vanderschuren L.J., Everitt B.J. Drug seeking becomes compulsive after prolonged cocaine self-administration. Science 2004, 305:1017-1019.
169. Vanderschuren L.J., Kalivas P.W. Alterations in dopaminergic and glutamatergic transmission in the induction and expression of behavioral sensitization: A critical review of preclinical studies. Psychopharmacology (Berl.) 2000, 151:99-120.
170. Vezina P., Pierre P.J., Lorrain D.S. The effect of previous exposure to amphetamine on drug-induced locomotion and self-administration of a low dose of the drug. Psychopharmacology (Berl.) 1999, 147:125-134.
171. Volkow N.D., Wang G.J., Telang F., Fowler J.S., Logan J., Childress A.R., Jayne M., Ma Y., Wong C. Cocaine cues and dopamine in dorsal striatum: Mechanism of craving in cocaine addiction. J. Neurosci. 2006, 26:6583-6588.
172. Vollstadt-Klein S., Wichert S., Rabinstein J., Buhler M., Klein O., Ende G., Hermann D., Mann K. Initial, habitual and compulsive alcohol use is characterized by a shift of cue processing from ventral to dorsal striatum. Addiction 2010, 105:1741-1749.
173. Waelti P., Dickinson A., Schultz W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 2001, 412:43-48.
174. Walker D.M., Cates H.M., Heller E.A., Nestler E.J. Regulation of chromatin states by drugs of abuse. Curr. Opin. Neurobiol. 2015, 30:112-121.
175. Wanat M.J., Willuhn I., Clark J.J., Phillips P.E. Phasic dopamine release in appetitive behaviors and drug addiction. Curr. Drug Abuse Rev. 2009, 2:195-213.
176. Wang L.P., Li F., Wang D., Xie K., Wang D., Shen X., Tsien J.Z. NMDA receptors in dopaminergic neurons are crucial for habit learning. Neuron 2011, 72:1055-1066.
177. Wassum K.M., Ostlund S.B., Maidment N.T. Phasic mesolimbic dopamine signaling precedes and predicts performance of a self-initiated action sequence task. Biol. Psychiatry 2012, 71:846-854.
178. Watabe-Uchida M., Zhu L., Ogawa S., Vamanrao A., Uchida N. Whole-brain mapping of direct inputs to midbrain dopamine neurons. Neuron 2012, 74:858-873.
179. White F.J., Hu X.T., Zhang X.F., Wolf M.E. Repeated administration of cocaine or amphetamine alters neuronal responses to glutamate in the mesoaccumbens dopamine system. J. Pharmacol. Exp. Ther. 1995, 273:445-454.
180. Wickens J.R. Synaptic plasticity in the basal ganglia. Behav. Brain Res. 2009, 199:119-128.
181. Williams J.E., Wieczorek W., Willner P., Kruk Z.L. Parametric analysis of the effects of cocaine and cocaine pretreatment on dopamine release in the nucleus accumbens measured by fast cyclic voltammetry. Brain Res. 1995, 678:225-232.
182. Willuhn I., Burgeno L.M., Everitt B.J., Phillips P.E. Hierarchical recruitment of phasic dopamine signaling in the striatum during the progression of cocaine use. Proc. Natl. Acad. Sci. USA 2012, 109:20703-20708.
183. Willuhn I., Burgeno L.M., Groblewski P.A., Phillips P.E. Excessive cocaine use results from decreased phasic dopamine signaling in the striatum. Nat. Neurosci. 2014, 17:704-709.
184. Wilson R.C., Takahashi Y.K., Schoenbaum G., Niv Y. Orbitofrontal cortex as a cognitive map of task space. Neuron 2014, 81:267-279.
185. Wise R.A. Dopamine, learning and motivation. Nat. Rev. Neurosci. 2004, 5:483-494.
186. Witten I.B., Steinberg E.E., Lee S.Y., Davidson T.J., Zalocusky K.A., Brodsky M., Yizhar O., Cho S.L., Gong S., Ramakrishnan C., et al. Recombinase-driver rat lines: Tools, techniques, and optogenetic application to dopamine-mediated reinforcement. Neuron 2011, 72:721-733.
187. Wong D.F., Kuwabara H., Schretlen D.J., Bonson K.R., Zhou Y., Nandi A., Brasic J.R., Kimes A.S., Maris M.A., Kumar A., et al. Increased occupancy of dopamine receptors in human striatum during cue-elicited cocaine craving. Neuropsychopharmacology 2006, 31:2716-2727.
188. Xia Y., Driscoll J.R., Wilbrecht L., Margolis E.B., Fields H.L., Hjelmstad G.O. Nucleus accumbens medium spiny neurons target non-dopaminergic neurons in the ventral tegmental area. J. Neurosci. 2011, 31:7811-7816.
189. Yamaguchi T., Wang H., Morales M. Glutamate neurons in the substantia nigra compacta and retrorubral field. Eur. J. Neurosci. 2013, 38:3602-3610.
190. Yamaguchi T., Qi J., Wang H.L., Zhang S., Morales M. Glutamatergic and dopaminergic neurons in the mouse ventral tegmental area. Eur. J. Neurosci. 2015, 41:760-772.
191. Yin H.H., Knowlton B.J. The role of the basal ganglia in habit formation. Nat. Rev. Neurosci. 2006, 7:464-476.
192. Yin H.H., Knowlton B.J., Balleine B.W. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 2004, 19:181-189.
193. Zapata A., Minney V.L., Shippenberg T.S. Shift from goal-directed to habitual cocaine seeking after prolonged experience in rats. J. Neurosci. 2010, 30:15457-15463.
194. Zernig G., Ahmed S.H., Cardinal R.N., Morgan D., Acquas E., Foltin R.W., Vezina P., Negus S.S., Crespo J.A., Stockl P., et al. Explaining the escalation of drug use in substance dependence: models and appropriate animal laboratory tests. Pharmacology 2007, 80:65-119.
195. Zweifel L.S., Parker J.G., Lobb C.J., Rainwater A., Wall Z.V., Fadok J.P., Darvas M., Kim M.J., Mizumori S.J., Paladini C.A., et al. Disruption of NMDAR-dependent burst firing by dopamine neurons provides selective assessment of phasic dopamine-dependent behavior. Proc. Natl. Acad. Sci. USA 2009, 106:7281-7288.