Dopamine, salience, and response set shifting in prefrontal cortex

Cerebral Cortex

Volumn 25, Issue 10, 2015, Pages 3629-3639

  Shiner, T ^a   Symmonds, M ^a,b   Guitart Masip, M ^a,c   Fleming, S M ^a,d,e   Friston, K J ^a   Dolan, R J ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF OXFORD   (United Kingdom)

^c KAROLINSKA INSTITUTE   (Sweden)

^d NEW YORK UNIVERSITY   (United States)

^e UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

Dopamine; fMRI; Prefrontal cortex; Reversal learning; Set shifting

Indexed keywords

DOPA; DOPAMINE; PLACEBO; DOPAMINE RECEPTOR STIMULATING AGENT; LEVODOPA;

ADULT; ARTICLE; ASSOCIATION; CONTROLLED STUDY; CORPUS STRIATUM; EXECUTIVE FUNCTION; FUNCTIONAL NEUROIMAGING; HUMAN; HUMAN EXPERIMENT; IMPULSE CONTROL DISORDER; INSULA; LEARNING; MALE; NORMAL HUMAN; NUCLEAR MAGNETIC RESONANCE IMAGING; PARKINSON DISEASE; PERSEVERATION; PRIORITY JOURNAL; PSYCHOPHARMACOLOGY; RANDOMIZED CONTROLLED TRIAL; REACTION TIME; SALIENCE NETWORK; THALAMUS; VENTROMEDIAL PREFRONTAL CORTEX; ATTENTION; BRAIN MAPPING; DRUG EFFECTS; PHYSIOLOGY; PREFRONTAL CORTEX; YOUNG ADULT;

ADULT; ATTENTION; BRAIN MAPPING; CORPUS STRIATUM; CUES; DOPAMINE; DOPAMINE AGENTS; HUMANS; LEVODOPA; MAGNETIC RESONANCE IMAGING; MALE; PREFRONTAL CORTEX; REVERSAL LEARNING; THALAMUS; YOUNG ADULT;

EID: 84948650069     PISSN: 10473211     EISSN: 14602199     Source Type: Journal    
DOI: 10.1093/cercor/bhu210     Document Type: Article

References (97)

1. Adams RA, Shipp S, Friston KJ. 2013. Predictions not commands: active inference in the motor system. Brain Struct Funct. 218(3): 611-643.
2. Asaad WF, Rainer G, Miller EK. 2000. Task-specific neural activity in the primate prefrontal cortex. J Neurophysiol. 84:451-459.
3. Baeg EH, Kim YB, Huh K, Mook-Jung I, Kim HT, Jung MW. 2003. Dynamics of population code for working memory in the prefrontal cortex. Neuron. 40:177-188.
4. Bamford NS, Zhang H, Schmitz Y, Wu NP, Cepeda C, Levine MS, Schmauss C, Zakharenko SS, Zablow L, Sulzer D. 2004. Heterosy-naptic dopamine neurotransmission selects sets of corticostriatal terminals. Neuron. 42:653-663.
5. Becker JB, Cha JH. 1989. Estrous cycle-dependent variation in amphetamine-induced behaviors and striatal dopamine release assessed with microdialysis. Behav Brain Res. 35:117-125.
6. Berke JD, Hyman SE. 2000. Addiction, dopamine, and the molecular mechanisms of memory. Neuron. 25:515-532.
7. Braver TS, Reynolds JR, Donaldson DI. 2003. Neural mechanisms of transient and sustained cognitive control during task switching. Neuron. 39:713-726.
8. Cauda F, Costa T, Torta DM, Sacco K, D'Agata F, Duca S, Geminiani G, Fox PT, Vercelli A. 2012. Meta-analytic clustering of the insular cortex: characterizing the meta-analytic connectivity of the insula when involved in active tasks. Neuroimage. 62:343-355.
9. Chevalier G, Deniau JM. 1990. Disinhibition as a basic process in the expression of striatal functions. Trends Neurosci. 13:277-280.
10. Clarke HF, Hill GJ, Robbins TW, Roberts AC. 2011. Dopamine, but not serotonin, regulates reversal learning in the marmoset caudate nucleus. J Neurosci. 31:4290-4297.
11. Cohen JD, Braver TS, Brown JW. 2002. Computational perspectives on dopamine function in prefrontal cortex. Curr Opin Neurobiol. 12: 223-229.
12. Cools R, Barker RA, Sahakian BJ, Robbins TW. 2003. L-Dopa medication remediates cognitive inflexibility, but increases impulsivity in patients with Parkinson's disease. Neuropsychologia. 41:1431-1441.
13. Cools R, Clark L, Owen AM, Robbins TW. 2002. Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J Neurosci. 22:4563-4567.
14. Cools R, D'Esposito M. 2011. Inverted-u-shaped dopamine actions on human working memory and cognitive control. Biol Psychiatry. 69:e113-e125.
15. Cools R, Frank MJ, Gibbs SE, Miyakawa A, Jagust W, D'Esposito M. 2009. Striatal dopamine predicts outcome-specific reversal learning and its sensitivity to dopaminergic drug administration. J Neurosci. 29:1538-1543.
16. Cools R, Lewis SJG, Clark L, Barker RA, Robbins TW. 2006. L-Dopa disrupts activity in the nucleus accumbens during reversal learning in Parkinson's disease. Neuropsychopharmacology. 32:180-189.
17. Craig AD. 2003. Interoception: the sense of the physiological condition of the body. Curr Opin Neurobiol. 13:500-505.
18. Criaud M, Boulinguez P. 2012. Have we been asking the right questions when assessing response inhibition in go/no-go tasks with fMRI? A meta-analysis and critical review. Neurosci Biobehav Rev. 7:11-23.
19. Critchley HD, Wiens S, Rotshtein P, Öhman A, Dolan RJ. 2004. Neural systems supporting interoceptive awareness. Nat Neurosci. 7:189-195.
20. Crone EA, Wendelken C, Donohue SE, Bunge SA. 2006. Neural evidence for dissociable components of task-switching. Cereb Cortex. 16:475-486.
21. Dagher A, Robbins TW. 2009. Personality, addiction, dopamine: insights from Parkinson's disease. Neuron. 61:502-510.
22. Daw ND, Doya K. 2006. The computational neurobiology of learning and reward. Curr Opin Neurobiol. 16:199-204.
23. Deco G, Rolls ET. 2005. Synaptic and spiking dynamics underlying reward reversal in the orbitofrontal cortex. Cereb Cortex. 15:15-30.
24. Derrfuss J, Brass M, von Cramon DY. 2004. Cognitive control in the posterior frontolateral cortex: evidence from common activations in task coordination, interference control, and working memory. Neuroimage. 23:604.
25. Dias R, Robbins T, Roberts A. 1996. Primate analogue of the Wisconsin Card Sorting Test: effects of excitotoxic lesions of the prefrontal cortex in the marmoset. Behav Neurosci. 110:872.
26. Diekhof EK, Falkai P, Gruber O. 2008. Functional neuroimaging of reward processing and decision-making: a review of aberrant motivational and affective processing in addiction and mood disorders. Brain Res Rev. 59:164-184.
27. Dreher JC, Schmidt PJ, Kohn P, Furman D, Rubinow D, Berman KF. 2007. Menstrual cycle phase modulates reward-related neural function in women. Proc Natl Acad Sci USA. 104:2465-2470.
28. Durstewitz D, Seamans JK, Sejnowski TJ. 2000. Dopamine-mediated stabilization of delay-period activity in a network model of pre-frontal cortex. J Neurophysiol. 83:1733-1750.
29. Evans AH, Katzenschlager R, Paviour D, O'Sullivan JD, Appel S, Lawrence AD, Lees AJ. 2004. Punding in Parkinson's disease: its relation to the dopamine dysregulation syndrome. Mov Disord. 19: 397-405.
30. Fallon SJ, Williams-Gray CH, Barker RA, Owen AM, Hampshire A. 2013. Prefrontal dopamine levels determine the balance between cognitive stability and flexibility. Cereb Cortex. 23:361-369.
31. Fellows LK, Farah MJ. 2003. Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. Brain. 126:1830-1837.
32. Ferron A, Thierry AM, Le Douarin C, Glowinski J. 1984. Inhibitory influence of the mesocortical dopaminergic system on spontaneous activity or excitatory response induced from the thalamic mediodorsal nucleus in the rat medial prefrontal cortex. Brain Res. 302:257-265.
33. Floresco SB, Magyar O, Ghods-Sharifi S, Vexelman C, Tse MTL. 2005. Multiple dopamine receptor subtypes in the medial prefrontal cortex of the rat regulate set-shifting. Neuropsychopharmacology. 31:297-309.
34. Floresco SB, Phillips AG. 2001. Delay-dependent modulation of memory retrieval by infusion of a dopamine D1 agonist into the rat medial prefrontal cortex. Behav Neurosci. 115:934-939.
35. Flynn FG. 1999. Anatomy of the insula functional and clinical correlates. Aphasiology. 13:55-78.
36. Frank MJ, Claus ED. 2006. Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. Psychol Rev. 113:300-326.
37. Frank MJ, Loughry B, O'Reilly RC. 2001. Interactions between frontal cortex and basal ganglia in working memory: a computational model. Cogn Affect Behav Neurosci. 1:137-160.
38. Frank MJ, Seeberger LC, O'Reilly RC. 2004. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science. 306: 1940-1943.
39. Friston KJ. 1997. Transients, metastability, and neuronal dynamics. Neuroimage. 5:164-171.
40. Friston KJ, Shiner T, FitzGerald T, Galea JM, Adams R, Brown H, Dolan RJ, Moran R, Stephan KE, Bestmann S. 2012. Dopamine, affordance and active inference. PLoS Comput Biol. 8:e1002327.
41. Fuster JM. 2001. The prefrontal cortex-an update: time is of the essence. Neuron. 30:319-333.
42. Galea JM, Bestmann S, Beigi M, Jahanshahi M, Rothwell JC. 2012. Action reprogramming in Parkinson's disease: response to prediction error is modulated by levels of dopamine. J Neurosci. 32:542-550.
43. Gilbert DL, Christian BT, Gelfand MJ, Shi B, Mantil J, Sallee FR. 2006. Altered mesolimbocortical and thalamic dopamine in Tourette syndrome. Neurology. 67:1695-1697.
44. Gottfried JA, O'Doherty J, Dolan RJ. 2003. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science. 301:1104.
45. Greening SG, Finger EC, Mitchell DGV. 2011. Parsing decision making processes in prefrontal cortex: response inhibition, overcoming learned avoidance, and reversal learning. Neuroimage. 54:1432-1441.
46. Gu X, Hof PR, Friston KJ, Fan J. 2013. Anterior insular cortex and emotional awareness. J Comp Neurol. 521:3371-3388.
47. Guitart-Masip M, Chowdhury R, Sharot T, Dayan P, Duzel E, Dolan RJ. 2012a. Action controls dopaminergic enhancement of reward representations. Proc Natl Acad Sci USA. 109:7511-7516.
48. Guitart-Masip M, Huys QJM, Fuentemilla L, Dayan P, Duzel E, Dolan RJ. 2012b. Go and no-go learning in reward and punishment: interactions between affect and effect. Neuroimage. 62:154-166.
49. Haber SN. 2003. The primate basal ganglia: parallel and integrative networks. J Chem Neuroanat. 26:317-330.
50. Haber SN, Knutson B. 2009. The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology. 35:4-26.
51. Hampshire A, Chaudhry AM, Owen AM, Roberts AC. 2012. Dissociable roles for lateral orbitofrontal cortex and lateral prefrontal cortex during preference driven reversal learning. Neuroimage. 59: 4102-4112.
52. Izquierdo A, Suda RK, Murray EA. 2004. Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J Neurosci. 24:7540-7548.
53. Kasanetz F, Riquelme LA, Della-Maggiore V, O'Donnell P, Murer MG. 2008. Functional integration across a gradient of corticostriatal channels controls UP state transitions in the dorsal striatum. Proc Natl Acad Sci USA. 105:8124-8129.
54. Kelly C, de Zubicaray G, Di Martino A, Copland DA, Reiss PT, Klein DF, Castellanos X, Millham MP, McMahon K. 2009. L-Dopa modulates functional connectivity in striatal cognitive and motor networks: a double-blind placebo-controlled study. J Neurosci. 29:7364-7378.
55. Khor SP, Hsu A. 2007. The pharmacokinetics and pharmacodynamics of levodopa in the treatment of Parkinson's disease. Curr Clin Pharmacol. 2:234-243.
56. Kurniawan IT, Seymour B, Talmi D, Yoshida W, Chater N, Dolan RJ. 2010. Choosing to make an effort: the role of striatum in signaling physical effort of a chosen action. J Neurophysiol. 104:313-321.
57. Lee B, Groman S, London ED, Jentsch JD. 2007. Dopamine D2//D3 receptors play a specific role in the reversal of a learned visual discrimination in monkeys. Neuropsychopharmacology. 32:2125-2134.
58. McNab F, Klingberg T. 2008. Prefrontal cortex and basal ganglia control access toworking memory. Nat Neurosci. 11:103-107.
59. Miller EK, Cohen JD. 2001. An integrative theory of prefrontal cortex function. Annu Rev Neurosci. 24:167-202.
60. Niv Y, Schoenbaum G. 2008. Dialogues on prediction errors. Trends Cogn Sci. 12:265-272.
61. O'Doherty J, Critchley H, Deichmann R, Dolan RJ. 2003. Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J Neurosci. 23:7931-7939.
62. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ. 2004. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science. 304:452-454.
63. O'Doherty J, Kringelbach ML, Rolls ET, Hornak J, Andrews C. 2001. Abstract reward and punishment representations in the human orbitofrontal cortex. Nat Neurosci. 4:95-102.
64. O'Doherty JP. 2004. Reward representations and reward-related learning in the human brain: insights from neuroimaging. Curr Opin Neurobiol. 14:769-776.
65. O'Reilly RC, Noelle DC, Braver TS, Cohen JD. 2002. Prefrontal cortex and dynamic categorization tasks: representational organization and neuromodulatory control. Cereb Cortex. 12:246-257.
66. Ostlund SB, Balleine BW. 2007. Orbitofrontal cortex mediates outcome encoding in Pavlovian but not instrumental conditioning. J Neurosci. 27:4819-4825.
67. Oullier O, Kelso JA. 2006. Neuroeconomics and the metastable brain. Trends Cogn Sci. 10:353-354.
68. Padoa-Schioppa C, Assad JA. 2006. Neurons in the orbitofrontal cortex encode economic value. Nature. 441:223-226.
69. Pagnoni G, Zink CF, Montague PR, Berns GF. 2002. Activity in human ventral striatum locked to errors of reward prediction. Nat Neurosci. 5:97-98.
70. Phillips AG, Ahn S, Floresco SB. 2004. Magnitude of dopamine release in medial prefrontal cortex predicts accuracy of memory on a delayed response task. J Neurosci. 24:547-553.
71. Preuschoff K, Quartz SR, Bossaerts P. 2008. Human insula activation reflects risk prediction errors as well as risk. J Neurosci. 28:2745-2752.
72. Pycock CJ, Kerwin RW, Carter CJ. 1980. Effect of lesion of cortical dopamine terminals on subcortical dopamine receptors in rats. Nature. 286:74-76.
73. Rabinovich MI, Huerta R, Varona P, Afraimovich VS. 2008. Transient cognitive dynamics, metastability, and decision making. PLoS Comput Biol. 4:e1000072.
74. Ramnani N, Elliott R, Athwal BS, Passingham RE. 2004. Prediction error for free monetary reward in the human prefrontal cortex. Neuroimage. 23:777-786.
75. Robinson OJ, Frank MJ, Sahakian BJ, Cools R. 2010. Dissociable responses to punishment in distinct striatal regions during reversal learning. Neuroimage. 51:1459-1467.
76. Rubia K, Russell T, Overmeyer S, Brammer MJ, Bullmore ET, Sharma T, Simmons A, Williams SCR, Giampietro V, Andrew CM et al. 2001. Mapping motor inhibition: conjunctive brain activations across different versions of go/no-go and stop tasks. Neuroimage. 13:250-261.
77. Rushworth M, Behrens T, Rudebeck P, Walton M. 2007. Contrasting roles for cingulate and orbitofrontal cortex in decisions and social behaviour. Trends Cogn Sci. 11:168-176.
78. Rygula R, Walker SC, Clarke HF, Robbins TW, Roberts AC. 2010. Differential contributions of the primate ventrolateral prefrontal and orbitofrontal cortex to serial reversal learning. J Neurosci. 30:14552-14559.
79. Sakurai Y, Sugimoto S. 1986. Multiple unit activity of prefrontal cortex and dorsomedial thalamus during delayed go/no-go alternation in the rat. Behav Brain Res. 20:295-301.
80. Salamone JD, Correa M, Mingote S, Weber SM. 2003. Nucleus accumbens dopamine and the regulation of effort in food-seeking behavior: implications for studies of natural motivation, psychiatry, and drug abuse. J Pharmacol Exp Ther. 305:1-8.
81. Sawaguchi T, Goldman-Rakic PS. 1991. D1 dopamine receptors in prefrontal cortex: involvement in working memory. Science. 251: 947-950.
82. Schoenbaum G, Nugent SL, Saddoris MP, Setlow B. 2002. Orbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations. Neuroreport. 13:885-890.
83. Schoenbaum G, Saddoris MP, Stalnaker TA. 2007. Reconciling the roles of orbitofrontal cortex in reversal learning and the encoding of outcome expectancies. Ann N Y Acad Sci. 1121:320-335.
84. Schultz W, Dayan P, Montague PR. 1997. A neural substrate of prediction and reward. Science. 275:1593-1599.
85. Seamans JK, Yang CR. 2004. The principal features and mechanisms of dopamine modulation in the prefrontal cortex. Prog Neurobiol. 74:1.
86. Seymour B, O'Doherty JP, Koltzenburg M, Wiech K, Frackowiak R, Friston K, Dolan R. 2005. Opponent appetitive-aversive neural processes underlie predictive learning of pain relief. Nat Neurosci. 8:1234-1240.
87. Shiner T, Seymour B, Wunderlich K, Hill C, Bhatia KP, Dayan P, Dolan RJ. 2012. Dopamine and performance in a reinforcement learning task: evidence from Parkinson's disease. Brain. 135: 1871-1883.
88. Shohamy D, Myers CE, Geghman KD, Sage J, Gluck MA. 2006. L-Dopa impairs learning, but spares generalization, in Parkinson's disease. Neuropsychologia. 44:774-784.
89. Shohamy D, Myers CE, Grossman S, Sage J, Gluck MA. 2005. The role of dopamine in cognitive sequence learning: evidence from Parkinson's disease. Behav Brain Res. 156:191-200.
90. Stuss DT, Knight RT, editors. 2013. Principles of Frontal Lobe function. 2nd ed. New York: Oxford University Press.
91. Symmonds M, Bossaerts P, Dolan RJ. 2010. A behavioral and neural evaluation of prospective decision-making under risk. J Neurosci. 30:14380-14389.
92. Tsuchida A, Doll BB, Fellows LK. 2010. Beyond reversal: a critical role for human orbitofrontal cortex in flexible learning from probabilistic feedback. J Neurosci. 30:16868-16875.
93. van der Schaaf ME, van Schouwenburg MR, Geurts DE, Schellekens AF, Buitelaar JK, Verkes RJ, Cools R. 2014. Establishing the dopa-mine dependency of human striatal signals during reward and punishment reversal learning. Cereb Cortex. 24:633-642.
94. van Schouwenburg M, Aarts E, Cools R. 2010. Dopaminergic modulation of cognitive control: distinct roles for the prefrontal cortex and the basal ganglia. Curr Pharm Des. 16:2026-2032.
95. Vertes RP, Hoover WB, Rodriguez JJ. 2012. Projections of the central medial nucleus of the thalamus in the rat: node in cortical, striatal and limbic forebrain circuitry. Neuroscience. 219:120-136.
96. Winter S, Dieckmann M, Schwabe K. 2009. Dopamine in the prefrontal cortex regulates rats behavioral flexibility to changing reward value. Behav Brain Res. 198:206-213.
97. Wunderlich K, Dayan P, Dolan RJ. 2012. Mapping value based planning and extensively trained choice in the human brain. Nat Neurosci. 15:786-791.