Adaptive coding of reward prediction errors is gated by striatal coupling

Proceedings of the National Academy of Sciences of the United States of America

Volumn 109, Issue 11, 2012, Pages 4285-4289

  Park, Soyoung Q ^a,b   Kahnt, Thorsten ^b,c   Talmi, Deborah ^d   Rieskamp, Jörg ^e   Dolan, Raymond J ^d   Heekeren, Hauke R ^a,b,f  

^a FREIE UNIVERSITÄT BERLIN   (Germany)

^b HUMBOLDT UNIVERSITY   (Germany)

^c CHARITÉ UNIVERSITÄTSMEDIZIN BERLIN   (Germany)

^d UNIVERSITY COLLEGE LONDON   (United Kingdom)

^e UNIVERSITY OF BASEL   (Switzerland)

^f Dahlem Institue for the Neuroimaging of Emotion   (Germany)

Author keywords

Adjustment; Context invariance; Dopamine; Functional connectivity; Normalization

Indexed keywords

ARTICLE; ASSOCIATION; BRAIN REGION; CORPUS STRIATUM; ERROR; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; MESENCEPHALON; NORMAL HUMAN; PREDICTION; PREFRONTAL CORTEX; PRIORITY JOURNAL; REWARD; TASK PERFORMANCE;

ADAPTATION, PHYSIOLOGICAL; ADULT; FEMALE; HUMANS; MAGNETIC RESONANCE IMAGING; MALE; NEOSTRIATUM; REWARD; SOCIAL BEHAVIOR; TASK PERFORMANCE AND ANALYSIS;

EID: 84863269144     PISSN: 00278424     EISSN: 10916490     Source Type: Journal    
DOI: 10.1073/pnas.1119969109     Document Type: Article

References (46)

1. Tversky A, Kahneman D (1974) Judgment under uncertainty: Heuristics and biases. Science 185:1124-1131.
2. Kahneman D, Tversky A (1979) Prospect theory: Analysis of decision under risk. Econometrica 47(2):263-291.
3. Rescorla RA, Wagner AR (1972) A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. Classical Conditioning II: Current Research and Theory, eds Black AH, Prokasy WF (Appleton-Century-Crofts, New York), pp 64-99.
4. Sutton R, Barto A (1998) Reinforcement Learning: An Introduction (MIT Press, Cambridge, MA).
5. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593-1599. (Pubitemid 27120526)
6. Bayer HM, Glimcher PW (2005) Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47(1):129-141. (Pubitemid 40922855)
7. Tobler PN, Fiorillo CD, Schultz W (2005) Adaptive coding of reward value by dopamine neurons. Science 307:1642-1645. (Pubitemid 40354749)
8. Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD (2006) Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442: 1042-1045. (Pubitemid 44330261)
9. Pagnoni G, Zink CF, Montague PR, Berns GS (2002) Activity in human ventral striatum locked to errors of reward prediction. Nat Neurosci 5(2):97-98. (Pubitemid 34121573)
10. McClure SM, Berns GS, Montague PR (2003) Temporal prediction errors in a passive learning task activate human striatum. Neuron 38:339-346. (Pubitemid 36555658)
11. O'Doherty JP, Dayan P, Friston K, Critchley H, Dolan RJ (2003) Temporal difference models and reward-related learning in the human brain. Neuron 38:329-337. (Pubitemid 36555657)
12. Park SQ, et al. (2010) Prefrontal cortex fails to learn from reward prediction errors in alcohol dependence. J Neurosci 30:7749-7753.
13. Kahnt T, et al. (2009) Dorsal striatal-midbrain connectivity in humans predicts how reinforcements are used to guide decisions. J Cogn Neurosci 21:1332-1345.
14. Breiter HC, Aharon I, Kahneman D, Dale A, Shizgal P (2001) Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30:619-639. (Pubitemid 32530601)
15. Tremblay L, Schultz W (1999) Relative reward preference in primate orbitofrontal cortex. Nature 398:704-708. (Pubitemid 29197648)
16. Padoa-Schioppa C (2009) Range-adapting representation of economic value in the orbitofrontal cortex. J Neurosci 29:14004-14014.
17. Kobayashi S, Pinto de Carvalho O, Schultz W (2010) Adaptation of reward sensitivity in orbitofrontal neurons. J Neurosci 30:534-544.
18. Haber SN, McFarland NR (1999) The concept of the ventral striatum in nonhuman primates. Ann N Y Acad Sci 877:33-48. (Pubitemid 29351849)
19. Kahnt T, Heinzle J, Park SQ, Haynes JD (2010) The neural code of reward anticipation in human orbitofrontal cortex. Proc Natl Acad Sci USA 107:6010-6015.
20. Philiastides MG, Biele G, Heekeren HR (2010) A mechanistic account of value computation in the human brain. Proc Natl Acad Sci USA 107:9430-9435.
21. Ferry AT, Ongür D, An X, Price JL (2000) Prefrontal cortical projections to the striatum in macaque monkeys: Evidence for an organization related to prefrontal networks. J Comp Neurol 425:447-470. (Pubitemid 30659209)
22. Ongür D, Price JL (2000) The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. Cereb Cortex 10(3):206-219. (Pubitemid 30159289)
23. Park SQ, Kahnt T, Rieskamp J, Heekeren HR (2011) Neurobiology of value integration: When value impacts valuation. J Neurosci 31:9307-9314.
24. Kahnt T, Heinzle J, Park SQ, Haynes JD (2011) Decoding different roles for vmPFC and dlPFC in multi-attribute decision making. Neuroimage 56:709-715.
25. Fairhall AL, Lewen GD, Bialek W, de Ruyter Van Steveninck RR (2001) Efficiency and ambiguity in an adaptive neural code. Nature 412:787-792. (Pubitemid 32801454)
26. Brenner N, Bialek W, de Ruyter van Steveninck R (2000) Adaptive rescaling maximizes information transmission. Neuron 26:695-702.
27. Dunn FA, Lankheet MJ, Rieke F (2007) Light adaptation in cone vision involves switching between receptor and post-receptor sites. Nature 449:603-606. (Pubitemid 47525145)
28. Preuschoff K, Bossaerts P (2007) Adding prediction risk to the theory of reward learning. Ann N Y Acad Sci 1104:135-146. (Pubitemid 47092731)
29. Louie K, Grattan LE, Glimcher PW (2011) Reward value-based gain control: Divisive normalization in parietal cortex. J Neurosci 31:10627-10639.
30. Elliott R, Agnew Z, Deakin JF (2008) Medial orbitofrontal cortex codes relative rather than absolute value of financial rewards in humans. Eur J Neurosci 27:2213-2218. (Pubitemid 351590911)
31. Nieuwenhuis S, et al. (2005) Activity in human reward-sensitive brain areas is strongly context dependent. Neuroimage 25:1302-1309.
32. Bunzeck N, Dayan P, Dolan RJ, Duzel E (2010) A common mechanism for adaptive scaling of reward and novelty. Hum Brain Mapp 31:1380-1394.
33. Lohrenz T, McCabe K, Camerer CF, Montague PR (2007) Neural signature of fictive learning signals in a sequential investment task. Proc Natl Acad Sci USA 104: 9493-9498. (Pubitemid 47185807)
34. Van Dijk KR, et al. (2010) Intrinsic functional connectivity as a tool for human connectomics: Theory, properties, and optimization. J Neurophysiol 103:297-321.
35. Murphy K, Birn RM, Handwerker DA, Jones TB, Bandettini PA (2009) The impact of global signal regression on resting state correlations: Are anti-correlated networks introduced? Neuroimage 44:893-905.
36. Fox MD, Zhang D, Snyder AZ, Raichle ME (2009) The global signal and observed anticorrelated resting state brain networks. J Neurophysiol 101:3270-3283.
37. Haber SN, Kunishio K, Mizobuchi M, Lynd-Balta E (1995) The orbital and medial prefrontal circuit through the primate basal ganglia. J Neurosci 15:4851-4867.
38. Haber SN, Fudge JL, McFarland NR (2000) Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J Neurosci 20: 2369-2382. (Pubitemid 30230089)
39. Haber SN, Knutson B (2010) The reward circuit: Linking primate anatomy and human imaging. Neuropsychopharmacology 35(1):4-26.
40. Ikemoto S (2007) Dopamine reward circuitry: Two projection systems from the ventral midbrain to the nucleus accumbens-olfactory tubercle complex. Brain Res Brain Res Rev 56(1):27-78.
41. Frankle WG, Laruelle M, Haber SN (2006) Prefrontal cortical projections to the midbrain in primates: Evidence for a sparse connection. Neuropsychopharmacology 31: 1627-1636. (Pubitemid 44106694)
42. Karreman M, Moghaddam B (1996) The prefrontal cortex regulates the basal release of dopamine in the limbic striatum: An effect mediated by ventral tegmental area. J Neurochem 66:589-598. (Pubitemid 26036693)
43. Yacubian J, et al. (2006) Dissociable systems for gain- and loss-related value predictions and errors of prediction in the human brain. J Neurosci 26:9530-9537. (Pubitemid 44396866)
44. Delgado MR, Locke HM, Stenger VA, Fiez JA (2003) Dorsal striatum responses to reward and punishment: Effects of valence and magnitude manipulations. Cogn Affect Behav Neurosci 3(1):27-38. (Pubitemid 37517033)
45. Büchel C, Holmes AP, Rees G, Friston KJ (1998) Characterizing stimulus-response functions using nonlinear regressors in parametric fMRI experiments. Neuroimage 8(2):140-148. (Pubitemid 28471359)
46. Friston KJ, et al. (1997) Psychophysiological and modulatory interactions in neuroimaging. Neuroimage 6(3):218-229. (Pubitemid 27488453)