Differential modulation of reinforcement learning by D2 dopamine and NMDA glutamate receptor antagonism

Journal of Neuroscience

Volumn 34, Issue 39, 2014, Pages 13151-13162

  Jocham, Gerhard ^a,b   Klein, Tilmann A ^c,d   Ullsperger, Markus ^a,e  

^a OTTO VON GUERICKE UNIVERSITY   (Germany)

^b MAX PLANCK INSTITUTE FOR NEUROLOGICAL RESEARCH   (Germany)

^c MAX PLANCK INSTITUTE FOR HUMAN COGNITIVE AND BRAIN SCIENCES   (Germany)

^d UNIVERSITY HOSPITAL LEIPZIG   (Germany)

^e RADBOUD UNIVERSITY   (Netherlands)

Author keywords

Dopamine; Glutamate; NMDA; Prediction error; Reinforcement learning; Striatum

Indexed keywords

AMISULPRIDE; DOPAMINE 2 RECEPTOR; MEMANTINE; N METHYL DEXTRO ASPARTIC ACID RECEPTOR; PLACEBO; DOPAMINE 2 RECEPTOR BLOCKING AGENT; DOPAMINE RECEPTOR BLOCKING AGENT; SULPIRIDE; SULTOPRIDE;

ADULT; ARTICLE; AVOIDANCE BEHAVIOR; BLOOD PRESSURE; BOLD SIGNAL; CAUDATE NUCLEUS; CLINICAL PHARMACOLOGY; CLINICAL TRIAL; DOPAMINERGIC NERVE CELL; FORCED CHOICE METHOD; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HEART RATE; HUMAN; HUMAN EXPERIMENT; MALE; NEUROMODULATION; NORMAL HUMAN; PSYCHOLOGIC TEST; PUTAMEN; REINFORCEMENT; REWARD; TASK PERFORMANCE; VISUAL ANALOG SCALE; ANALOGS AND DERIVATIVES; ANTAGONISTS AND INHIBITORS; INSTRUMENTAL CONDITIONING; METABOLISM;

ADULT; AVOIDANCE LEARNING; CONDITIONING, OPERANT; DOPAMINE ANTAGONISTS; DOPAMINE D2 RECEPTOR ANTAGONISTS; HUMANS; MALE; MEMANTINE; RECEPTORS, DOPAMINE D2; RECEPTORS, N-METHYL-D-ASPARTATE; REWARD; SULPIRIDE;

EID: 84907291631     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.0757-14.2014     Document Type: Article

References (66)

1. Bai JS, Perron P (1998) Estimating and testing linear models with multiple structural changes. Econometrica 66:47-78. CrossRef
2. Bayer HM, Glimcher PW (2005) Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47:129 -141. CrossRef Medline
3. Becker JB, Cha JH (1989) Estrous cycle-dependent variation in amphetamineinduced behaviors and striatal dopamine release assessed with microdialysis. Behav Brain Res 35:117-125. CrossRef Medline
4. Becker JB, Robinson TE, Lorenz KA (1982) Sex differences and estrous cycle variations in amphetamine-elicited rotational behavior. Eur J Pharmacol 80:65-72. CrossRef Medline
5. Bond A, Lader M (1974) The use of analogue scales in rating subjective feelings. Br J Psychol 47:211-218. CrossRef
6. Bressan RA, Erlandsson K, Spencer EP, Ell PJ, Pilowsky LS (2004) Prolactinemia is uncoupled from central D2/D3 dopamine receptor occupancy in amisulpride treated patients. Psychopharmacology 175:367-373. CrossRef Medline
7. Bromberg-Martin ES, Matsumoto M, Hong S, Hikosaka O (2010) A pallidus-habenula-dopamine pathway signals inferred stimulus values. J Neurophysiol 104:1068 -1076. CrossRef Medline
8. Caplin A, Dean M (2008) Axiomatic methods, dopamine and reward prediction error. Curr Opin Neurobiol 18:197-202. CrossRef Medline
9. Chéramy A, L'hirondel M, Godeheu G, Artaud F, Glowinski J (1998) Direct and indirect presynaptic control of dopamine release by excitatory amino acids. Amino Acids 14:63- 68. CrossRef Medline
10. Chiu PH, Lohrenz TM, Montague PR (2008) Smokers' brains compute, but ignore, a fictive error signal in a sequential investment task. Nat Neurosci 11:514 -520. CrossRef Medline
11. Creutz LM, Kritzer MF (2004) Mesostriatal and mesolimbic projections of midbrain neurons immunoreactive for estrogen receptor beta or androgen receptors in rats. J Comp Neurol 476:348 -362. CrossRef Medline
12. D'Ardenne K, McClure SM, Nystrom LE, Cohen JD (2008) BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science 319:1264 -1267. CrossRef Medline
13. Dreher JC, Schmidt PJ, Kohn P, Furman D, Rubinow D, Berman KF (2007) Menstrual cycle phase modulates reward-related neural function in women. Proc Natl Acad Sci U S A 104:2465-2470. CrossRef Medline
14. Eisenegger C, Naef M, Linssen A, Clark L, Gandamaneni PK, Müller U, Robbins TW (2014) Role of dopamine D2 receptors in human reinforcement learning. Neuropsychopharmacology 39:2366 -2375. CrossRef Medline
15. Fischer AG, Ullsperger M (2013) Real and fictive outcomes are processed differently but converge on a common adaptive mechanism. Neuron 79: 1243-1255. CrossRef Medline
16. Frank MJ (2005) Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J Cogn Neurosci 17:51-72. CrossRef Medline
17. Frank MJ, O'Reilly RC (2006) A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. Behav Neurosci 120:497-517. CrossRef Medline
18. Frank MJ, Seeberger LC, O'Reilly RC (2004) By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306:1940 -1943. CrossRef Medline
19. Frank MJ, Samanta J, Moustafa AA, Sherman SJ (2007) Hold your horses: impulsivity, deep brain stimulation, and medication in parkinsonism. Science 318:1309 -1312. CrossRef Medline
20. Hollerman JR, SchultzW (1998) Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1:304 -309. CrossRef Medline
21. Iravani MM, Kruk ZL (1996) Real-time effects of N-methyl-D-aspartic acid on dopamine release in slices of rat caudate putamen: a study using fast cyclic voltammetry. J Neurochem 66:1076 -1085. Medline
22. JenkinsonM (2003) Fast, automated, N-dimensional phase-unwrapping algorithm. Magn Reson Med 49:193-197. CrossRef Medline
23. Jenkinson M, Smith S (2001) A global optimisation method for robust affine registration of brain images. Med Image Anal 5:143-156. CrossRef Medline
24. Jenkinson M, Bannister P, Brady M, Smith S (2002) Improved optimization for the robust and accurate linear registration and motion correction of brain images. Neuroimage 17:825- 841. CrossRef Medline
25. Jocham G, Neumann J, Klein TA, Danielmeier C, Ullsperger M (2009) Adaptive coding of action values in the human rostral cingulate zone. J Neurosci 29:7489 -7496. CrossRef Medline
26. Jocham G, Klein TA, Ullsperger M (2011) Dopamine-mediated reinforcement learning signals in the striatum and ventromedial prefrontal cortex underlie value-based choices. J Neurosci 31:1606 -1613. CrossRef Medline
27. Kelley AE, Smith-Roe SL, Holahan MR (1997) Response-reinforcement learning is dependent on N-methyl-D-aspartate receptor activation in the nucleus accumbens core. Proc Natl Acad Sci U S A 94:12174 -12179. CrossRef Medline
28. Klein-Flügge MC, Hunt LT, Bach DR, Dolan RJ, Behrens TE (2011) Dissociable reward and timing signals in human midbrain and ventral striatum. Neuron 72:654-664. CrossRef Medline
29. Konradi C, Leveque JC, Hyman SE (1996) Amphetamine and dopamineinduced immediate early gene expression in striatal neurons depends on postsynaptic NMDA receptors and calcium. J Neurosci 16:4231- 4239. Medline
30. Krebs MO, Desce JM, Kemel ML, Gauchy C, Godeheu G, Cheramy A, Glowinski J (1991) Glutamatergic control of dopamine release in the rat striatum: evidence for presynaptic N-methyl-D-aspartate receptors on dopaminergic nerve terminals. J Neurochem 56:81- 85. CrossRef Medline
31. la Fougère C, Meisenzahl E, Schmitt G, Stauss J, Frodl T, Tatsch K, Hahn K, Moller HJ, Dresel S (2005) D2 receptor occupancy during high- and low-dose therapy with the atypical antipsychotic amisulpride: a 123Iiodobenzamide SPECT study. J Nucl Med 46:1028 -1033. Medline
32. Lohrenz T, McCabe K, Camerer CF, Montague PR (2007) Neural signature of fictive learning signals in a sequential investment task. Proc Natl Acad Sci U S A 104:9493-9498. CrossRef Medline
33. Martinot JL, Paillère-Martinot ML, Poirier MF, Dao-CastellanaMH,Loc'h C, Mazière B (1996) In vivo characteristics of dopamine D2 receptor occupancy by amisulpride in schizophrenia. Psychopharmacology 124:154- 158. CrossRef Medline
34. Matsumoto M, Hikosaka O (2007) Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447:1111-1115. CrossRef Medline
35. Matsumoto M, HikosakaO (2009) Representation of negative motivational value in the primate lateral habenula. Nat Neurosci 12:77- 84. CrossRef Medline
36. Meisenzahl EM, Schmitt G, Gründer G, Dresel S, Frodl T, la Fougère C, Scheuerecker J, Schwarz M, Boerner R, Stauss J, Hahn K, Möller HJ (2008) Striatal D2/D3 receptor occupancy, clinical response and side effects with amisulpride: an iodine-123-iodobenzamide SPET study. Pharmacopsychiatry 41:169 -175. CrossRef Medline
37. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304:452- 454. CrossRef Medline
38. Ohta K, Araki N, Shibata M, Komatsumoto S, Shimazu K, Fukuuchi Y (1994) Presynaptic ionotropic glutamate receptors modulate in vivo release and metabolism of striatal dopamine, noradrenaline, and 5-hydroxytryptamine: involvement of both NMDA and AMPA/kainate subtypes. Neurosci Res 21:83- 89. CrossRef Medline
39. Overton P, Clark D (1992) Iontophoretically administered drugs acting at the N-methyl-D-aspartate receptor modulate burst firing in A9 dopamine neurons in the rat. Synapse 10:131-140. CrossRef Medline
40. Palminteri S, Lebreton M, Worbe Y, Grabli D, Hartmann A, Pessiglione M (2009) Pharmacological modulation of subliminal learning in Parkinson's and Tourette's syndromes. Proc Natl Acad Sci U S A 106:19179- 19184. CrossRef Medline
41. Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD (2006) Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442:1042-1045. CrossRef Medline
42. Pessiglione M, Petrovic P, Daunizeau J, Palminteri S, Dolan RJ, Frith CD (2008) Subliminal instrumental conditioning demonstrated in the human brain. Neuron 59:561-567. CrossRef Medline
43. Pizzagalli DA, Evins AE, Schetter EC, Frank MJ, Pajtas PE, Santesso DL, CulhaneM (2008) Single dose of a dopamine agonist impairs reinforcement learning in humans: behavioral evidence from a laboratory-based measure of reward responsiveness. Psychopharmacology 196:221-232. CrossRef Medline
44. Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In: Classical conditioning II: current research and theory (Black AH, Prokasy WF, eds), pp 64-99. New York: Appleton Century Crofts.
45. Robbins TW, Murphy ER (2006) Behavioural pharmacology: 40+ years of progress, with a focus on glutamate receptors and cognition. Trends Pharmacol Sci 27:141-148. CrossRef Medline
46. Roesch MR, Calu DJ, SchoenbaumG (2007) Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat Neurosci 10:1615-1624. CrossRef Medline
47. Rutledge RB, Dean M, Caplin A, Glimcher PW (2010) Testing the reward prediction error hypothesis with an axiomatic model. J Neurosci 30: 13525-13536. CrossRef Medline
48. Salamone JD (2002) Functional significance of nucleus accumbens dopamine: behavior, pharmacology and neurochemistry. Behav Brain Res 137:1. CrossRef Medline
49. Salamone JD, Correa M, Mingote SM, WeberSM (2005) Beyond the reward hypothesis: alternative functions of nucleus accumbens dopamine. Curr Opin Pharmacol 5:34-41. CrossRef Medline
50. Satoh T, Nakai S, Sato T, KimuraM (2003) Correlated coding of motivation and outcome of decision by dopamine neurons. J Neurosci 23:9913-9923. Medline
51. Schönberg T, Daw ND, Joel D, O'Doherty JP (2007) Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J Neurosci 27:12860 -12867. CrossRef Medline
52. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593-1599. CrossRef Medline
53. Shen W, Flajolet M, Greengard P, Surmeier DJ (2008) Dichotomous dopaminergic control of striatal synaptic plasticity. Science 321:848-851. CrossRef Medline
54. Shiner T, Seymour B, Wunderlich K, Hill C, Bhatia KP, Dayan P, Dolan RJ (2012) Dopamine and performance in a reinforcement learning task: evidence from Parkinson's disease. Brain 135:1871-1883. CrossRef Medline
55. Smith SM, Jenkinson M, Woolrich MW, Beckmann CF, Behrens TE, Johansen-Berg H, Bannister PR, De Luca M, Drobnjak I, Flitney DE, Niazy RK, Saunders J, Vickers J, Zhang Y, De Stefano N, Brady JM, MatthewsPM (2004) Advances in functional and structuralMRimage analysis and implementation as FSL. Neuroimage 23 [Suppl 1]:S208 -S219. Medline
56. Smith-Roe SL, Kelley AE (2000) Coincident activation of NMDA and dopamine D1 receptors within the nucleus accumbens core is required for appetitive instrumental learning. J Neurosci 20:7737-7742. Medline
57. Steinberg EE, Keiflin R, Boivin JR, Witten IB, Deisseroth K, Janak PH (2013) A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci 16:966 -973. CrossRef Medline
58. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. Cambridge, MA: MIT.
59. Takahashi YK, Roesch MR, Stalnaker TA, Haney RZ, Calu DJ, Taylor AR, Burke KA, Schoenbaum G (2009) The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes. Neuron 62:269 -280. CrossRef Medline
60. Ullsperger M, von Cramon DY (2003) Error monitoring using external feedback: specific roles of the habenular complex, the reward system, and the cingulate motor area revealed by functional magnetic resonance imaging. J Neurosci 23:4308-4314. Medline
61. Ullsperger M, Danielmeier C, Jocham G (2014) Neurophysiology of performance monitoring and adaptive behavior. Physiol Rev 94:35-79. CrossRef Medline
62. Valentin VV, O'Doherty JP (2009) Overlapping prediction errors in dorsal striatum during instrumental learning with juice and money reward in the human brain. J Neurophysiol 102:3384 -3391. CrossRef Medline
63. Wang LP, Li F, Wang D, Xie K, Wang D, Shen X, Tsien JZ (2011) NMDA receptors in dopaminergic neurons are crucial for habit learning. Neuron 72:1055-1066. CrossRef Medline
64. Wise RA (2004) Dopamine, learning and motivation. Nat Rev Neurosci 5:483- 494. CrossRef Medline
65. Woolrich MW, Ripley BD, Brady M, Smith SM (2001) Temporal autocorrelation in univariate linear modeling of FMRI data. Neuroimage 14: 1370-1386. CrossRef Medline
66. Zaghloul KA, Blanco JA, Weidemann CT, McGill K, Jaggi JL, Baltuch GH, Kahana MJ (2009) Human substantia nigra neurons encode unexpected financial rewards. Science 323:1496 -1499. CrossRef Medline