Value and probability coding in a feedback-based learning task utilizing food rewards

Journal of Neurophysiology

Volumn 113, Issue 1, 2015, Pages 4-13

  Tricomi, Elizabeth ^a   Lempert, Karolina M ^a  

^a RUTGERS UNIVERSITY   (United States)

Author keywords

Caudate; FMRI; Nucleus accumbens; Reward processing; Striatum

Indexed keywords

ADULT; ANGULAR GYRUS; ARTICLE; CAUDATE NUCLEUS; CEREBELLUM; CINGULATE GYRUS; CORPUS STRIATUM; DECISION MAKING; FEEDING BEHAVIOR; FEMALE; FOOD; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; INFERIOR FRONTAL GYRUS; INFERIOR TEMPORAL GYRUS; LINGUAL GYRUS; MALE; MIDDLE FRONTAL GYRUS; MIDDLE TEMPORAL GYRUS; MOTIVATION; NEGATIVE FEEDBACK; NORMAL HUMAN; NUCLEUS ACCUMBENS; OCCIPITAL GYRUS; ORBITAL CORTEX; OUTCOME ASSESSMENT; PAIRED ASSOCIATE LEARNING; PLANNING; POSTERIOR CINGULATE; POSTERIOR INSULA; PRECUNEUS; PRIMARY MOTOR CORTEX; PRIORITY JOURNAL; PROBABILITY; PUTAMEN; REWARD; SATIETY; SUPERIOR FRONTAL GYRUS; SUPERIOR TEMPORAL GYRUS; TRAINING; ADAPTIVE BEHAVIOR; ANTICIPATION; ASSOCIATION; BRAIN MAPPING; FOOD PREFERENCE; NEUROPSYCHOLOGICAL TEST; NUCLEAR MAGNETIC RESONANCE IMAGING; PHYSIOLOGY; YOUNG ADULT;

ANTICIPATION, PSYCHOLOGICAL; BRAIN MAPPING; CORPUS STRIATUM; CUES; FEEDBACK, PSYCHOLOGICAL; FEMALE; FOOD; FOOD PREFERENCES; HUMANS; MAGNETIC RESONANCE IMAGING; MALE; NEUROPSYCHOLOGICAL TESTS; PROBABILITY; REWARD; YOUNG ADULT;

EID: 84920896747     PISSN: 00223077     EISSN: 15221598     Source Type: Journal    
DOI: 10.1152/jn.00086.2014     Document Type: Article

References (70)

1. Abler B, Walter H, Erk S, Kammerer H, Spitzer M. Prediction error as a linear function of reward probability is coded in human nucleus accumbens. Neuroimage 31: 790-795, 2006.
2. Amsel A. The role of frustrative nonreward in noncontinuous reward situations. Psychol Bull 55: 102-119, 1958.
3. Balleine BW, Dickinson A. The role of incentive learning in instrumental outcome revaluation by sensory-specific satiety. Anim Learn Behav 26: 46-59, 1998.
4. Balleine BW, Liljeholm M, Ostlund SB. The integrative function of the basal ganglia in instrumental conditioning. Behav Brain Res 199: 43-52, 2009.
5. Bennett CM, Wolford GL, Miller MB. The principled control of false positives in neuroimaging. Soc Cogn Affect Neurosci 4: 417-422, 2009.
6. Bischoff-Grethe A, Hazeltine E, Bergren L, Ivry RB, Grafton ST. The influence of feedback valence in associative learning. Neuroimage 44: 243-251, 2009.
7. Bjork JM, Hommer DW. Anticipating instrumentally obtained and passivelyreceived rewards: a factorial fMRI investigation. Behav Brain Res 177: 165-170, 2007.
8. Breiter HC, Aharon I, Kahneman D, Dale A, Shizgal P. Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30: 619-639, 2001.
9. Cannon DM, Klaver JM, Peck SA, Rallis-Voak D, Erickson K, Drevets WC. Dopamine type-1 receptor binding in major depressive disorder assessed using positron emission tomography and [11C]NNC-112. Neuropsychopharmacology 34: 1277-1287, 2009.
10. Cools R, Clark L, Owen AM, Robbins TW. Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J Neurosci 22: 4563-4567, 2002.
11. Coricelli G, Critchley HD, Joffily M, O’Doherty JP, Sirigu A, Dolan RJ. Regret and its avoidance: a neuroimaging study of choice behavior. Nat Neurosci 8: 1255-1262, 2005.
12. Croxson PL, Walton ME, O’Reilly JX, Behrens TE, Rushworth MF. Effort-based cost-benefit valuation and the human brain. J Neurosci 29: 4531-4541, 2009.
13. Daw ND, Doya K. The computational neurobiology of learning and reward. Curr Opin Neurobiol 16: 199-204, 2006.
14. de Borchgrave R, Rawlins JN, Dickinson A, Balleine BW. Effects of cytotoxic nucleus accumbens lesions on instrumental conditioning in rats. Exp Brain Res 144: 50-68, 2002.
15. de Wit S, Corlett PR, Aitken MR, Dickinson A, Fletcher PC. Differential engagement of the ventromedial prefrontal cortex by goal-directed and habitual behavior toward food pictures in humans. J Neurosci 29: 11330-11338, 2009.
16. Deichmann R, Gottfried JA, Hutton C, Turner R. Optimized EPI for fMRI studies of the orbitofrontal cortex. Neuroimage 19: 430-441, 2003.
17. Delgado MR, Locke HM, Stenger VA, Fiez JA. Dorsal striatum responses to reward and punishment: effects of valence and magnitude manipulations. Cogn Affect Behav Neurosci 3: 27-38, 2003.
18. Delgado MR, Nystrom LE, Fissell C, Noll DC, Fiez JA. Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 84: 3072-3077, 2000.
19. Delgado MR, Stenger VA, Fiez JA. Motivation-dependent responses in the human caudate nucleus. Cereb Cortex 14: 1022-1030, 2004.
20. Dickinson A. Transient effects of reward presentation and omission on subsequent operant responding. J Comp Physiol Psychol 88: 447-458, 1975.
21. Doya K. Modulators of decision making. Nat Neurosci 11: 410-416, 2008.
22. Dudley RT, Papini MR. Amsel’s frustration effect: a pavlovian replication with control for frequency and distribution of rewards. Physiol Behav 61: 627-629, 1997.
23. Elliott R, Newman JL, Longe OA, Deakin JF. Instrumental responding for rewards is associated with enhanced neuronal response in subcortical reward systems. Neuroimage 21: 984-990, 2004.
24. Ernst M, Nelson EE, McClure EB, Monk CS, Munson S, Eshel N, Zarahn E, Leibenluft E, Zametkin A, Towbin K, Blair J, Charney D, Pine DS. Choice selection and reward anticipation: an fMRI study. Neuropsychologia 42: 1585-1597, 2004.
25. Galvan A, Hare TA, Davidson M, Spicer J, Glover G, Casey BJ. The role of ventral frontostriatal circuitry in reward-based learning in humans. J Neurosci 25: 8650-8656, 2005.
26. Garner DM, Olmsted MP, Bohr Y, Garfinkel PE. The eating attitudes test: psychometric features and clinical correlates. Psychol Med 12: 871-878, 1982.
27. Gottfried JA, O’Doherty J, Dolan RJ. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 301: 1104-1107, 2003.
28. Grabenhorst F, Rolls ET. Value, pleasure and choice in the ventral prefrontal cortex. Trends Cogn Sci 15: 56-67, 2011.
29. Guitart-Masip M, Fuentemilla L, Bach DR, Huys QJ, Dayan P, Dolan RJ, Duzel E. Action dominates valence in anticipatory representations in the human striatum and dopaminergic midbrain. J Neurosci 31: 7867-7875, 2011.
30. Haber SN, Fudge JL, McFarland NR. Striatonigrostriatal pathways in primates form an ascending spiral from the shell to the dorsolateral striatum. J Neurosci 20: 2369-2382, 2000.
31. Hare TA, O’Doherty J, Camerer CF, Schultz W, Rangel A. Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. J Neurosci 28: 5623-5630, 2008.
32. Haruno M, Kawato M. Different neural correlates of reward expectation and reward expectation error in the putamen and caudate nucleus during stimulus-action-reward association learning. J Neurophysiol 95: 948-959, 2006.
33. Hsu M, Bhatt M, Adolphs R, Tranel D, Camerer CF. Neural systems responding to degrees of uncertainty in human decision-making. Science 310: 1680-1683, 2005.
34. Kable JW, Glimcher PW. The neural correlates of subjective value during intertemporal choice. Nat Neurosci 10: 1625-1633, 2007.
35. Knutson B, Adams CM, Fong GW, Hommer D. Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J Neurosci 21: 1-5, 2001.
36. Knutson B, Taylor J, Kaufman M, Peterson R, Glover G. Distributed neural representation of expected value. J Neurosci 25: 4806-4812, 2005.
37. Kriegeskorte N, Simmons WK, Bellgowan PS, Baker CI. Circular analysis in systems neuroscience: the dangers of double dipping. Nat Neurosci 12: 535-540, 2009.
38. Kringelbach ML, O’Doherty J, Rolls ET, Andrews C. Activation of the human orbitofrontal cortex to a liquid food stimulus is correlated with its subjective pleasantness. Cereb Cortex 13: 1064-1071, 2003.
39. Kurniawan IT, Guitart-Masip M, Dayan P, Dolan RJ. Effort and valuation in the brain: the effects of anticipation and execution. J Neurosci 33: 6160-6169, 2013.
40. Kurniawan IT, Seymour B, Talmi D, Yoshida W, Chater N, Dolan RJ. Choosing to make an effort: the role of striatum in signaling physical effort of a chosen action. J Neurophysiol 22: 222-223, 2010.
41. Levy I, Snell J, Nelson AJ, Rustichini A, Glimcher PW. Neural representation of subjective value under risk and ambiguity. J Neurophysiol 103: 1036-1047, 2010.
42. Martin-Soelch C, Szczepanik J, Nugent A, Barhaghi K, Rallis D, Herscovitch P, Carson RE, Drevets WC. Lateralization and gender differences in the dopaminergic response to unpredictable reward in the human ventral striatum. Eur J Neurosci 33: 1706-1715, 2011.
43. McClure SM, Berns GS, Montague PR. Temporal prediction errors in a passive learning task activate human striatum. Neuron 38: 339-346, 2003.
44. Montague PR, King-Casas B, Cohen JD. Imaging valuation models in human choice. Annu Rev Neurosci 29: 417-448, 2006.
45. Murphy FC, Nimmo-Smith I, Lawrence AD. Functional neuroanatomy of emotions: a meta-analysis. Cogn Affect Behav Neurosci 3: 207-233, 2003.
46. O’Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304: 452-454, 2004.
47. O’Doherty JP. Lights, Camembert, action! The role of human orbitofrontal cortex in encoding stimuli, rewards and choices. Ann NY Acad Sci 1121: 254-272, 2007.
48. O’Doherty JP, Dayan P, Friston K, Critchley H, Dolan RJ. Temporal difference models and reward-related learning in the human brain. Neuron 28: 329-337, 2003.
49. Orbitello B, Ciano R, Corsaro M, Rocco PL, Taboga C, Tonutti L, Armellini M, Balestrieri M. The EAT-26 as screening instrument for clinical nutrition unit attenders. Int J Obes (Lond) 30: 977-981, 2006.
50. Peters J, Buchel C. Overlapping and distinct neural systems code for subjective value during intertemporal and risky decision making. J Neurosci 29: 15727-15734, 2009.
51. Peters J, Buchel C. Neural representations of subjective reward value. Behav Brain Res 213: 135-141, 2010.
52. Rudenga KJ, Small DM. Ventromedial prefrontal cortex response to concentrated sucrose reflects liking rather than sweet quality coding. Chem Senses 38: 585-594, 2013.
53. Rushworth MF, Behrens TE. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat Neurosci 11: 389-397, 2008.
54. Samejima K, Ueda Y, Doya K, Kimura M. Representation of action-specific reward values in the striatum. Science 310: 1337-1340, 2005.
55. Smith BW, Mitchell DG, Hardin MG, Jazbec S, Fridberg D, Blair RJ, Ernst M. Neural substrates of reward magnitude, probability, and risk during a wheel of fortune decision-making task. Neuroimage 44: 600-609, 2009.
56. Stout SC, Boughner RL, Papini MR. Reexamining the frustration effect in rats: aftereffects of surprising reinforcement and nonreinforcement. Learn Motiv 34: 437-456, 2003.
57. Talairach J, Tournoux P. Co-Planar Stereotaxic Atlas of the Human Brain: an Approach to Medical Cerebral Imaging. Stuttgart, Germany: Thieme Medical, 1988.
58. Tanaka SC, Balleine BW, O’Doherty JP. Calculating consequences: brain systems that encode the causal effects of actions. J Neurosci 28: 6750-6755, 2008.
59. Tobler PN, O’Doherty JP, Dolan RJ, Schultz W. Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems. J Neurophysiol 97: 1621-1632, 2007.
60. Tomer R, Slagter HA, Christian BT, Fox AS, King CR, Murali D, Gluck MA, Davidson RJ. Love to win or hate to lose? Asymmetry of dopamine D2 receptor binding predicts sensitivity to reward versus punishment. J Cogn Neurosci 26: 1039-1048, 2014.
61. Treadway MT, Buckholtz JW, Cowan RL, Woodward ND, Li R, Ansari MS, Baldwin RM, Schwartzman AN, Kessler RM, Zald DH. Dopaminergic mechanisms of individual differences in human effort-based decision-making. J Neurosci 32: 6170-6176, 2012.
62. Tricomi E, Balleine BW, O’Doherty JP. A specific role for posterior dorsolateral striatum in human habit learning. Eur J Neurosci 29: 2225-2232, 2009.
63. Tricomi E, Rangel A, Camerer CF, O’Doherty JP. Neural evidence for inequality-averse social preferences. Nature 463: 1089-1091, 2010.
64. Tricomi EM, Delgado MR, Fiez JA. Modulation of caudate activity by action contingency. Neuron 41: 281-292, 2004.
65. Valentin VV, Dickinson A, O’Doherty JP. Determining the neural substrates of goal-directed learning in the human brain. J Neurosci 27: 4019-4026, 2007.
66. van Dyck CH, Seibyl JP, Malison RT, Laruelle M, Zoghbi SS, Baldwin RM, Innis RB. Age-related decline in dopamine transporters: analysis of striatal subregions, nonlinear effects, and hemispheric asymmetries. Am J Geriatr Psychiatry 10: 36-43, 2002.
67. Vernaleken I, Weibrich C, Siessmeier T, Buchholz HG, Rosch F, Heinz A, Cumming P, Stoeter P, Bartenstein P, Grunder G. Asymmetry in dopamine D2/3 receptors of caudate nucleus is lost with age. Neuroimage 34: 870-878, 2007.
68. Yacubian J, Glascher J, Schroeder K, Sommer T, Braus DF, Buchel C. Dissociable systems for gain-and loss-related value predictions and errors of prediction in the human brain. J Neurosci 26: 9530-9537, 2006.
69. Yin HH, Knowlton BJ, Balleine BW. Blockade of NMDA receptors in the dorsomedial striatum prevents action-outcome learning in instrumental conditioning. Eur J Neurosci 22: 505-512, 2005.
70. Zink CF, Pagnoni G, Martin-Skurski ME, Chappelow JC, Berns GS. Human striatal responses to monetary reward depend on saliency. Neuron 42: 509-517, 2004.