Differential, but not opponent, effects of l-DOPA and citalopram on action learning with reward and punishment

Psychopharmacology

Volumn 231, Issue 5, 2014, Pages 955-966

  Guitart Masip, Marc ^a,b   Economides, Marcos ^a   Huys, Quentin J M ^a,c,d,h   Frank, Michael J ^e   Chowdhury, Rumana ^a,f   Duzel, Emrah ^f,g   Dayan, Peter ^d   Dolan, Raymond J ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b KAROLINSKA INSTITUTE   (Sweden)

^c UNIVERSITY OF ZURICH   (Switzerland)

^d UNIVERSITY COLLEGE LONDON   (United Kingdom)

^e BROWN UNIVERSITY   (United States)

^f UNIVERSITY COLLEGE LONDON   (United Kingdom)

^g OTTO VON GUERICKE UNIVERSITY   (Germany)

^h UNIVERSITY HOSPITAL OF PSYCHIATRY   (Switzerland)

Author keywords

Control; Dopamine; Instrumental; Pavlovian; Reinforcement learning; Serotonin

Indexed keywords

CITALOPRAM; DOPAMINE; LEVODOPA; SEROTONIN; DOPAMINE RECEPTOR STIMULATING AGENT; SEROTONIN UPTAKE INHIBITOR;

ADULT; ARTICLE; BEHAVIOR CONTROL; CONTROLLED STUDY; DECISION MAKING; FEMALE; HUMAN; HUMAN EXPERIMENT; INSTRUMENTAL LEARNING; LEARNING; MALE; NEUROMODULATION; NORMAL HUMAN; PRIORITY JOURNAL; REINFORCEMENT; REWARD; TASK PERFORMANCE; AVOIDANCE BEHAVIOR; DOUBLE BLIND PROCEDURE; DRUG EFFECTS; METABOLISM; PUNISHMENT; RANDOMIZED CONTROLLED TRIAL; YOUNG ADULT;

ADULT; AVOIDANCE LEARNING; CITALOPRAM; DECISION MAKING; DOPAMINE; DOPAMINE AGENTS; DOUBLE-BLIND METHOD; FEMALE; HUMANS; LEARNING; LEVODOPA; MALE; PUNISHMENT; REWARD; SEROTONIN; SEROTONIN UPTAKE INHIBITORS; YOUNG ADULT;

EID: 84896735229     PISSN: 00333158     EISSN: 14322072     Source Type: Journal    
DOI: 10.1007/s00213-013-3313-4     Document Type: Article

References (78)

1. Aron AR, Poldrack RA (2006) Cortical and subcortical contributions to Stop signal response inhibition: role of the subthalamic nucleus. J Neurosci Off J Soc Neurosci 26:2424-2433
2. Artigas F, Romero L, de Montigny C, Blier P (1996) Acceleration of the effect of selected antidepressant drugs in major depression by 5-HT1A antagonists. Trends Neurosci 19:378-383
3. Bayer HM, Glimcher PW (2005) Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47:129-141
4. Bayer HM, Lau B, Glimcher PW (2007) Statistics of midbrain dopamine neuron spike trains in the awake primate. J Neurophys 98:1428-1439
5. Beierholm U, Guitart-Masip M, Economides M, Chowdhury R, Duzel E, Dolan R, Dayan P (2013) Dopamine modulates reward-related vigor. Neuropsychopharmacology 38:1495-1503
6. Berridge KC, Robinson TE (1998) What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Brain Res Rev 28:309-369
7. Boureau YL, Dayan P (2011) Opponency revisited: competition and cooperation between dopamine and serotonin. Neuropsychopharmacol Off Publ Am Coll Neuropsychopharmacol 36:74-97
8. Breland K, Breland M (1961) The misbehavior of organisms. Am Psychol 16:681-684
9. Carli M, Samanin R (2000) The 5-HT(1A) receptor agonist 8-OH-DPAT reduces rats' accuracy of attentional performance and enhances impulsive responding in a five-choice serial reaction time task: role of presynaptic 5-HT(1A) receptors. Psychopharmacology 149:259-268
10. Cavanagh JF, Eisenberg E, Guitart-Masip M, Huys Q, Frank MJ (2013) Frontal Theta Overrides Pavlovian Learning Biases. J Neurosci Off J Soc Neurosci 33:8541-8548
11. Cavanagh JF, Wiecki TV, Cohen MX, Figueroa CM, Samanta J, Sherman SJ, Frank MJ (2011) Subthalamic nucleus stimulation reverses mediofrontal influence over decision threshold. Nat Neurosci 14:1462-1467
12. Chowdhury R, Guitart-Masip M, Lambert C, Dolan R, Duzel E (2013) Structural integrity of the substantia nigra and subthalamic nucleus determines the flexibility of instrumental learning in old age. Neurobiol Aging 34:2261-2270
13. Clatworthy PL, Lewis SJ, Brichard L, Hong YT, Izquierdo D, Clark L, Cools R, Aigbirhio FI, Baron JC, Fryer TD, Robbins TW (2009) Dopamine release in dissociable striatal subregions predicts the different effects of oral methylphenidate on reversal learning and spatial working memory. J Neurosci Off J Soc Neurosci 29:4690-4696
14. Cools R, D'Esposito M (2011) Inverted-U-shaped dopamine actions on human working memory and cognitive control. Biol Psychiatr 69:e113-e125
15. Cools R, Nakamura K, Daw ND (2011) Serotonin and dopamine: unifying affective, activational, and decision functions. Neuropsychopharmacol Off Publ Am Coll Neuropsychopharmacol 36:98-113
16. Crockett MJ, Clark L, Apergis-Schoute AM, Morein-Zamir S, Robbins TW (2012) Serotonin modulates the effects of Pavlovian aversive predictions on response vigor. Neuropsychopharmacol Off Publ Am Coll Neuropsychopharmacol 37:2244-2252
17. Crockett MJ, Clark L, Robbins TW (2009) Reconciling the role of serotonin in behavioral inhibition and aversion: acute tryptophan depletion abolishes punishment-induced inhibition in humans. J Neurosci Off J Soc Neurosci 29:11993-11999
18. Daw ND, Doya K (2006) The computational neurobiology of learning and reward. Curr Opin in Neurobiol 16:199-204
19. Daw ND, Gershman SJ, Seymour B, Dayan P, Dolan RJ (2011) Model-based influences on humans' choices and striatal prediction errors. Neuron 69:1204-1215
20. Daw ND, Niv Y, Dayan P (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 8:1704-1711
21. Dayan P (2012) Twenty-five lessons from computational neuromodulation. Neuron 76:240-256
22. Dayan P, Huys QJ (2009) Serotonin in affective control. Annu Rev Neurosci 32:95-126
23. Dayan P, Niv Y, Seymour B, Daw ND (2006) The misbehavior of value and the discipline of the will. Neural Netw 19:1153-1160
24. de Wit S, Standing HR, Devito EE, Robinson OJ, Ridderinkhof KR, Robbins TW, Sahakian BJ (2012) Reliance on habits at the expense of goal-directed control following dopamine precursor depletion. Psychopharmacology 219:621-631
25. Dickinson A, Balleine B (2002) The role of learning in motivation, 3rd edn. Wiley, New York
26. Doll BB, Hutchison KE, Frank MJ (2011) Dopaminergic genes predict individual differences in susceptibility to confirmation bias. J Neurosci Off J Soc Neurosci 31:6188-6198
27. Drueke B, Boecker M, Schlaegel S, Moeller O, Hiemke C, Grunder G, Gauggel S (2010) Serotonergic modulation of response inhibition and re-engagement? Results of a study in healthy human volunteers. Hum Psychopharmacol 25:472-480
28. Flagel SB, Clark JJ, Robinson TE, Mayo L, Czuj A, Willuhn I, Akers CA, Clinton SM, Phillips PE, Akil H (2011) A selective role for dopamine in stimulus-reward learning. Nature 469:53-57
29. Fleming SM, Thomas CL, Dolan RJ (2010) Overcoming status quo bias in the human brain. Proc Natl Acad Sci U S A 107:6005-6009
30. Frank MJ (2006) Hold your horses: a dynamic computational role for the subthalamic nucleus in decision making. Neural Netw 19:1120-1136
31. Frank MJ, Seeberger LC, O'Reilly RC (2004) By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306:1940-1943
32. Frank MJ, Moustafa AA, Haughey HM, Curran T, Hutchison KE (2007) Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning. Proc Natl Acad Sci U S A 104:16311-16316
33. Gray JA, McNaughton M (2000) The neuropsychology of anxiety: an inquiry into the function of the septohippocampal system, 2nd edn. Oxford University Press, Oxford
34. Guitart-Masip M, Fuentemilla L, Bach DR, Huys QJ, Dayan P, Dolan RJ, Duzel E (2011) Action dominates valence in anticipatory representations in the human striatum and dopaminergic midbrain. J Neurosci Off J Soc Neurosci 31:7867-7875
35. Guitart-Masip M, Chowdhury R, Sharot T, Dayan P, Duzel E, Dolan RJ (2012a) Action controls dopaminergic enhancement of reward representations. Proc Natl Acad Sci U S A 109:7511-7516
36. Guitart-Masip M, Huys QJ, Fuentemilla L, Dayan P, Duzel E, Dolan RJ (2012b) Go and no-go learning in reward and punishment: interactions between affect and effect. NeuroImage 62:154-166
37. Hajos M, Gartside SE, Sharp T (1995) Inhibition of median and dorsal raphe neurones following administration of the selective serotonin reuptake inhibitor paroxetine. Naunyn Schmiedeberg's Arch Pharmacol 351:624-629
38. Harrison AA, Everitt BJ, Robbins TW (1997) Doubly dissociable effects of median- and dorsal-raphe lesions on the performance of the five-choice serial reaction time test of attention in rats. Behav Brain Res 89:135-149
39. Harrison AA, Everitt BJ, Robbins TW (1999) Central serotonin depletion impairs both the acquisition and performance of a symmetrically reinforced go/no-go conditional visual discrimination. Behav Brain Res 100:99-112
40. Hikida T, Kimura K, Wada N, Funabiki K, Nakanishi S (2010) Distinct roles of synaptic transmission in direct and indirect striatal pathways to reward and aversive behavior. Neuron 66:896-907
41. Hitchcott PK, Quinn JJ, Taylor JR (2007) Bidirectional modulation of goal-directed actions by prefrontal cortical dopamine. Cereb Cortex 17:2820-2827
42. Huys Q, Cools R, Golzer M, Friedel E, Heinz A, Dolan R, Dayan P (2011) Disentangling the roles of approach, activation and valence in instrumental and pavlovian responding. PLoS Comput Biol 7:e1002028
43. Kass R, Raftery A (1995) Bayes factors. Journal of the American Statistical Association 106:1291-1303
44. Koller WC, Rueda MG (1998) Mechanism of action of dopaminergic agents in Parkinson's disease. Neurology 50: S11-4; discussion S44-8
45. Lau B, Glimcher PW (2005) Dynamic response-by-response models of matching behavior in rhesus monkeys. J Exp Anal Behav 84:555-579
46. Lex A, Hauber W (2008) Dopamine D1 and D2 receptors in the nucleus accumbens core and shell mediate Pavlovian-instrumental transfer. Learn Mem 15:483-491
47. Mehta MA, Gumaste D, Montgomery AJ, McTavish SF, Grasby PM (2005) The effects of acute tyrosine and phenylalanine depletion on spatial working memory and planning in healthy volunteers are predicted by changes in striatal dopamine levels. Psychopharmacology 180:654-663
48. McClure SM, Berns GS, Montague PR (2003) Temporal prediction errors in a passive learning task activate human striatum. Neuron 38:339-46
49. Miyazaki KW, Miyazaki K, Doya K (2012) Activation of dorsal raphe serotonin neurons is necessary for waiting for delayed rewards. J Neurosci Off J Soc Neurosci 32:10451-10457
50. Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci Off J Soc Neurosci 16:1936-1947
51. Morris G, Nevet A, Arkadir D, Vaadia E, Bergman H (2006) Midbrain dopamine neurons encode decisions for future action. Nat Neurosci 9:1057-1063
52. Niv Y, Daw ND, Joel D, Dayan P (2007) Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology 191:507-520
53. O'Doherty JP, Dayan P, Friston K, Critchley H, Dolan RJ (2003) Temporal difference models and reward-related learning in the human brain. Neuron 38:329-37
54. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304:452-454
55. Otto RA, Gershman SJ, Markman AB, Daw N (2013) The curse of planning: dissecting multiple reinforcement learning systems by taxing the central executive. Psychol Sci 24:751-761
56. Palmiter RD (2008) Dopamine signaling in the dorsal striatum is essential for motivated behaviors: lessons from dopamine-deficient mice. Ann N Y Acad Sci 1129:35-46
57. Parkinson JA, Dalley JW, Cardinal RN, Bamford A, Fehnert B, Lachenal G, Rudarakanchana N, Halkerston KM, Robbins TW, Everitt BJ (2002) Nucleus accumbens dopamine depletion impairs both acquisition and performance of appetitive Pavlovian approach behaviour: implications for mesoaccumbens dopamine function. Behav Brain Res 137:149-163
58. Parkinson JA, Olmstead MC, Burns LH, Robbins TW, Everitt BJ (1999) Dissociation in effects of lesions of the nucleus accumbens core and shell on appetitive pavlovian approach behavior and the potentiation of conditioned reinforcement and locomotor activity by d-amphetamine. J Neurosci Off J Soc Neurosci 19:2401-2411
59. Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD (2006) Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442:1042-1045
60. Rutledge RB, Lazzaro SC, Lau B, Myers CE, Gluck MA, Glimcher PW (2009) Dopaminergic drugs modulate learning rates and perseveration in Parkinson's patients in a dynamic foraging task. J Neurosci 29:15104-15114
61. Salamone JD, Correa M, Farrar A, Mingote SM (2007) Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology 191:461-482
62. Salamone JD, Correa M, Mingote SM, Weber SM (2005) Beyond the reward hypothesis: alternative functions of nucleus accumbens dopamine. Curr Opin Pharmacol 5:34-41
63. Salamone JD, Cousins MS, McCullough LD, Carriero DL, Berkowitz RJ (1994) Nucleus accumbens dopamine release increases during instrumental lever pressing for food but not free food consumption. Pharmacol Biochem Behav 49:25-31
64. Satoh T, Nakai S, Sato T, Kimura M (2003) Correlated coding of motivation and outcome of decision by dopamine neurons. J Neurosci Off J Soc Neurosci 23:9913-9923
65. Schonberg T, Daw ND, Joel D, O'Doherty JP (2007) Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making. J Neurosci 27:12860-12867
66. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593-1599
67. Selvaraj S, Faulkner P, Mouchlianitis E, Roiser JP, Howes O (2012) Effects of citalopram on serotonin neurotransmission. Mol Psychiatry 17:1143
68. Seymour B, Daw ND, Roiser JP, Dayan P, Dolan R (2012) Serotonin selectively modulates reward value in human decision-making. J Neurosci Off J Soc Neurosci 32:5833-5842
69. Soubrie P (1986) Reconciling the role of central serotonin neurons in human and animal behavior. Behav Brain Sci 9:319-364
70. Spinks D, Spinks G (2002) Serotonin reuptake inhibition: an update on current research strategies. Curr Med Chem 9:799-810
71. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. The MIT Press, Cambridge, Massachusetts
72. Taylor JR, Robbins TW (1984) Enhanced behavioural control by conditioned reinforcers following microinjections of d-amphetamine into the nucleus accumbens. Psychopharmacology 84:405-412
73. Taylor JR, Robbins TW (1986) 6-Hydroxydopamine lesions of the nucleus accumbens, but not of the caudate nucleus, attenuate enhanced responding with reward-related stimuli produced by intra-accumbens d-amphetamine. Psychopharmacology 90:390-397
74. Ungerstedt U (1971) Adipsia and aphagia after 6-hydroxydopamine induced degeneration of the nigro-striatal dopamine system. Acta Physiol Scand Suppl 367:95-122
75. Wickens JR, Budd CS, Hyland BI, Arbuthnott GW (2007) Striatal contributions to reward and decision making: making sense of regional variations in a reiterated processing matrix. Ann N Y Acad Sci 1104:192-212
76. Williams GV, Goldman-Rakic PS (1995) Modulation of memory fields by dopamine D1 receptors in prefrontal cortex. Nature 376:572-575
77. Wunderlich K, Smittenaar P, Dolan RJ (2012) Dopamine enhances model-based over model-free choice behavior. Neuron 75:418-424
78. Zaghloul KA, Weidemann CT, Lega BC, Jaggi JL, Baltuch GH, Kahana MJ (2012) Neuronal activity in the human subthalamic nucleus encodes decision conflict during action selection. J Neurosci Off J Soc Neurosci 32:2453-2460