Action dominates valence in anticipatory representations in the human striatum and dopaminergic midbrain

Journal of Neuroscience

Volumn 31, Issue 21, 2011, Pages 7867-7875

  Guitart Masip, Marc ^a,b   Fuentemilla, Lluis ^a   Bach, Dominik R ^b   Huys, Quentin J M ^b,c   Dayan, Peter ^c   Dolan, Raymond J ^b   Duzel, Emrah ^a,d  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

^c UNIVERSITY COLLEGE LONDON   (United Kingdom)

^d OTTO VON GUERICKE UNIVERSITY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ARTICLE; CONTROLLED STUDY; CORPUS STRIATUM; DOPAMINERGIC SYSTEM; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; MALE; MESENCEPHALON; NORMAL HUMAN; PRIORITY JOURNAL; PUNISHMENT; REWARD; SUBSTANTIA NIGRA; VENTRAL TEGMENTUM; VOLUNTEER;

ADULT; ANTICIPATION, PSYCHOLOGICAL; CORPUS STRIATUM; DOPAMINE; FEMALE; HUMANS; MAGNETIC RESONANCE IMAGING; MALE; MESENCEPHALON; PHOTIC STIMULATION; REACTION TIME; YOUNG ADULT;

EID: 79957667916     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.6376-10.2011     Document Type: Article

References (68)

1. Aron AR, Poldrack RA (2006) Cortical and subcortical contributions to Stop signal response inhibition: role of the subthalamic nucleus. J Neurosci 26:2424 -2433.
2. Aron AR, Schlaghecken F, Fletcher PC, Bullmore ET, Eimer M, Barker R, Sahakian BJ, Robbins TW (2003) Inhibition of subliminally primed responses is mediated by the caudate and thalamus: evidence from functional MRI and Huntington's disease. Brain 126:713-723.
3. Bayer HM, Glimcher PW (2005) Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47:129-141.
4. Berridge KC, Robinson TE (1998) What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Brain Res Rev 28:309-369.
5. Boureau YL, Dayan P (2011) Opponency revisited: competition and cooperation between dopamine and serotonin. Neuropsychopharmacology 36:74-97.
6. Brett M, Anton J-L, Valabregue R, Poline J-B (2002) Region of interest analysis using an SPM toolbox. Presented at the Eighth International Conference on Functional Mapping of the Human Brain, Sendai, Japan, June.
7. Brischoux F, Chakraborty S, Brierley DI, Ungless MA (2009) Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc Natl Acad Sci U S A 106:4894-4899.
8. Bunzeck N, Düzel E (2006) Absolute coding of stimulus novelty in the human substantia nigra/VTA. Neuron 51:369-379.
9. Cools R, Nakamura K, Daw ND (2011) Serotonin and dopamine: unifying affective, activational, and decision functions. Neuropsychopharmacology 36:98-113.
10. Cooper JC, Knutson B (2008) Valence and salience contribute to nucleus accumbens activation. Neuroimage 39:538-547.
11. Crockett MJ, Clark L, Robbins TW (2009) Reconciling the role of seroton in in behavioral inhibition and aversion: acute tryptophan depletion abolishes punishment-induced inhibition in humans. J Neurosci 29:11993-11999.
12. D'Ardenne K, McClure SM, Nystrom LE, Cohen JD (2008) BOLD responses reflecting dopaminergic signals in the human ventral tegmental area. Science 319:1264-1267.
13. Darvas M, Fadok JP, Palmiter RD (2011) Requirement of dopamine signaling in the amygdala and striatum for learning and maintenance of a conditioned avoidance response. Learn Mem 18:136-143.
14. Daw ND, Kakade S, Dayan P (2002) Opponent interactions between serotonin and dopamine. Neural Netw 15:603-616.
15. Daw ND, O'Doherty JP, Dayan P, Seymour B, Dolan RJ (2006) Cortical substrates for exploratory decisions in humans. Nature 441:876-879.
16. Dayan P (2002) Motivated reinforcement learning. In: Advances in neural information processing systems 14 (Dietterich TG, Becker S, Ghahramani Z, eds). Cambridge, MA: MIT.
17. Dayan P, Huys QJ (2009) Serotonin in affective control. Annu Rev Neurosci 32:95-126.
18. Deakin JFW, Graeff FG (1991) 5-HT and mechanisms of defence. J Psychopharmacol 5:305-315.
19. Delgado MR, Nystrom LE, Fissell C, Noll DC, Fiez JA (2000) Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 84:3072-3077.
20. Delgado MR, Locke HM, Stenger VA, Fiez JA (2003) Dorsal striatum responses to reward and punishment: effects of valence and magnitude manipulations. Cogn Affect Behav Neurosci 3:27-38.
21. Delgado MR, Stenger VA, Fiez JA (2004) Motivation-dependent responses in the human caudate nucleus. Cereb Cortex 14:1022-1030.
22. Delgado MR, Li J, Schiller D, Phelps EA (2008) The role of the striatum in aversive learning and aversive prediction errors. Philos Trans R Soc Lond B Biol Sci 363:3787-3800.
23. Domjan M (2005) Pavlovian conditioning: a functional perspective. Annu Rev Psychol 56:179-206.
24. Düzel E, Bunzeck N, Guitart-Masip M, Wittmann B, Schott BH, Tobler PN (2009) Functional imaging of the human dopaminergic midbrain. Trends Neurosci 32:321-328.
25. Elliott R, Newman JL, Longe OA, William Deakin JF (2004) Instrumental responding for rewards is associated with enhanced neuronal response in subcortical reward systems. Neuroimage 21:984-990.
26. Faure A, Reynolds SM, Richard JM, Berridge KC (2008) Mesolimbic dopamine in desire and dread: enabling motivation to be generated by localized glutamate disruptions in nucleus accumbens. J Neurosci 28:7184-7192.
27. Frank MJ, Seeberger LC, O'Reilly RC (2004) By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306:1940-1943.
28. Gerfen CR (1992) The neostriatal mosaic: multiple levels of comportamental organization. Trends Neurosci 15:133-139.
29. Gray JA, McNaughton M (2000) The neuropsychology of anxiety: an inquiry into the function of the septohippocampal system, Ed 2. Oxford: Oxford UP.
30. Grossberg S (1972) A neural theory of punishment and avoidance. Math Biosci 15:39-67.
31. Guitart-Masip M, Bunzeck N, Stephan KE, Dolan RJ, Düzel E (2010) Contextual novelty changes reward representations in the striatum. J Neurosci 30:1721-1726.
32. Haber SN, Knutson B (2010) The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology 35:4-26.
33. Haber SN, Fudge JL, McFarland NR (2000) Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J Neurosci 20:2369-2382.
34. Johnson J, Li W, Li J, Klopf A (2002) A computational model of learned avoidance behavior in a one-way avoidance experiment. Adapt Behav 9:91-104.
35. Klein A, Andersson J, Ardekani BA, Ashburner J, Avants B, Chiang MC, Christensen GE, Collins DL, Gee J, Hellier P, Song JH, Jenkinson M, Lepage C, Rueckert D, Thompson P, Vercauteren T, Woods RP, Mann JJ, Parsey RV (2009) Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration. Neuroimage 46:786-802.
36. Knutson B, Adams CM, Fong GW, Hommer D (2001) Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J Neurosci 21:RC159(1-5).
37. Knutson B, Taylor J, Kaufman M, Peterson R, Glover G (2005) Distributed neural representation of expected value. J Neurosci 25:4806-4812.
38. Lammel S, Hetzel A, Häckel O, Jones I, Liss B, Roeper J (2008) Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system. Neuron 57:760-773.
39. Maia TV (2010) Two-factor theory, the actor-critic model, and conditioned avoidance. Learn Behav 38:50-67.
40. Matsumoto M, Hikosaka O (2009) Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature 459: 837-841.
41. McClure SM, Berns GS, Montague PR (2003) Temporal prediction errors in a passive learning task activate human striatum. Neuron 38:339-346.
42. McCullough LD, Sokolowski JD, Salamone JD (1993) A neurochemical and behavioral investigation of the involvement of nucleus accumbens dopamine in instrumental avoidance. Neuroscience 52:919-925.
43. Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16:1936-1947.
44. Moutoussis M, Bentall RP, Williams J, Dayan P (2008) A temporal difference account of avoidance learning. Network 19:137-160.
45. Mowrer OH (1947) On the dual nature of learning: a reinterpretation of conditioning and problem solving. Harv Educ Rev 17:102-148.
46. Niv Y, Daw ND, Joel D, Dayan P (2007) Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl) 191:507-520.
47. O'Doherty JP (2004) Reward representations and reward-related learning in the human brain: insights from neuroimaging. Curr Opin Neurobiol 14:769-776.
48. O'Doherty JP, Deichmann R, Critchley HD, Dolan RJ (2002) Neural responses during anticipation of a primary taste reward. Neuron 33:815-826.
49. O'Doherty JP, Dayan P, Friston K, Critchley H, Dolan RJ (2003) Temporal difference models and reward-related learning in the human brain. Neuron 38:329-337.
50. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304:452- 454.
51. Pezze MA, Heidbreder CA, Feldon J, Murphy CA (2001) Selective responding of nucleus accumbens core and shell dopamine to aversively conditioned contextual and discrete stimuli. Neuroscience 108:91-102.
52. Redgrave P, Prescott TJ, Gurney K (1999) Is the short-latency dopamine response too short to signal reward error? Trends Neurosci 22:146-151.
53. Robbins TW, Everitt BJ (2002) Limbic-striatal memory systems and drug addiction. Neurobiol Learn Mem 78:625-636.
54. Rubia K, Smith AB, Brammer MJ, Taylor E (2003) Right inferior prefrontal cortex mediates response inhibition while mesial prefrontal cortex is responsible for error detection. Neuroimage 20:351-358.
55. Salamone JD, Correa M (2002) Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behav Brain Res 137:3-25.
56. Salamone JD, Correa M, Mingote SM, Weber SM (2005) Beyond the reward hypothesis: alternative functions of nucleus accumbens dopamine. Curr Opin Pharmacol 5:34-41.
57. Schmajuk N, Zanutto B (1997) Escape avoidance and imitation: A neural network approach. Adapt Behav 6:63-129.
58. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593-1599.
59. Seymour B, O'Doherty JP, Dayan P, Koltzenburg M, Jones AK, Dolan RJ, Friston KJ, Frackowiak RS (2004) Temporal difference models describe higher-order learning in humans. Nature 429:664-667.
60. Seymour B, O'Doherty JP, Koltzenburg M, Wiech K, Frackowiak R, Friston K, Dolan R (2005) Opponent appetitive-aversive neural processes underlie predictive learning of pain relief. Nat Neurosci 8:1234-1240.
61. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. Cambridge, MA: Massachusetts Institute of Technology.
62. Tricomi EM, Delgado MR, Fiez JA (2004) Modulation of caudate activity by action contingency. Neuron 41:281-292.
63. Ungerstedt U (1971) Adipsia and aphagia after 6-hydroxydopamine induced degeneration of the nigro-striatal dopamine system. Acta Physiol Scand Suppl 367:95-122.
64. Weiskopf N, Hutton C, Josephs O, Deichmann R (2006) Optimal EPI parameters for reduction of susceptibility-induced BOLD sensitivity losses: a whole-brain analysis at 3 T and 1.5 T. Neuroimage 33:493-504.
65. Wrase J, Kahnt T, Schlagenhauf F, Beck A, Cohen MX, Knutson B, Heinz A (2007) Different neural systems adjust motor behavior in response to reward and punishment. Neuroimage 36:1253-1262.
66. Young AM (2004) Increased extracellular dopamine in nucleus accumbens in response to unconditioned and conditioned aversive stimuli: studies using 1 min microdialysis in rats. J Neurosci Methods 138:57-63.
67. Zink CF, Pagnoni G, Martin ME, Dhamala M, Berns GS (2003) Human striatal response to salient nonrewarding stimuli. J Neurosci 23:8092-8097.
68. Zink CF, Pagnoni G, Martin-Skurski ME, Chappelow JC, Berns GS (2004) Human striatal responses to monetary reward depend on saliency. Neuron 42:509-517.