Neural correlates of the divergence of instrumental probability distributions

Journal of Neuroscience

Volumn 33, Issue 30, 2013, Pages 12519-12527

  Liljeholm, Mimi ^a   Wang, Shuo ^a   Zhang, June ^a   O'Doherty, John P ^a  

^a CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ARTICLE; COMPUTER MODEL; DECISION MAKING; ENTROPY; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; INFERIOR PARIETAL LOBULE; LEARNING; MALE; NERVE CELL; NORMAL HUMAN; PERCEPTIVE DISCRIMINATION; PRIORITY JOURNAL; PROBABILITY; REINFORCEMENT; REWARD; SENSORY STIMULATION; SUPPLEMENTARY MOTOR AREA; TASK PERFORMANCE;

EID: 84880601439     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.1353-13.2013     Document Type: Article

References (64)

1. Abe H, Lee D (2011) Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex. Neuron 70:731-741. CrossRef Medline
2. Abler B, Herrnberger B, Grön G, Spitzer M (2009) From uncertainty to reward: BOLD characteristics differentiate signaling pathways. BMC Neurosci 10:154. CrossRef Medline
3. Arcizet F, Mirpour K, Bisley JW (2011) A pure salience response in posterior parietal cortex. Cereb Cortex 21:2498-2506. CrossRef Medline
4. Balleine BW, Dickinson A (1998) Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37:407-419. CrossRef Medline
5. Becker GM, DeGroot MH, Marschak J (1964) Measuring utility by a singleresponse sequential method. Behav Sci 9:226-232. CrossRef Medline
6. Blodgett HC (1929) The effect of the introduction of reward upon the maze performance of rats. University of California Publications in Psychology 4:113-134.
7. Buschman TJ, Miller EK (2007) Top-down versus bottom-up control of attention in the prefrontal and posterior parietal cortices. Science 315: 1860-1862. CrossRef Medline
8. Chaminade T, Decety J (2002) Leader or follower? Involvement of the inferior parietal lobule in agency. Neuroreport 13:1975-1978. CrossRef Medline
9. Corbetta M, Shulman GL (2002) Control of goal-directed and stimulusdriven attention in the brain. Nat Rev Neurosci 3:201-215. Medline
10. Daw ND, Niv Y, Dayan P (2005) Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat Neurosci 8:1704-1711. CrossRef Medline
11. den Ouden HE, Friston KJ, Daw ND, McIntosh AR, Stephan KE (2009) A dual role for prediction error in associative learning. Cereb Cortex 19: 1175-1185. Medline
12. Doya K, Samejima K, Katagiri K, Kawato M (2002) Multiple model-based reinforcement learning. Neural Comput 14:1347-1369. CrossRef Medline
13. Ernst M, Nelson EE, McClure EB, Monk CS, Munson S, Eshel N, Zarahn E, Leibenluft E, Zametkin A, Towbin K, Blair J, Charney D, Pine DS (2004) Choice selection and reward anticipation: an fMRI study. Neuropsychologia 42:1585-1597. CrossRef Medline
14. Esterman M, Tamber-Rosenau BJ, Chiu YC, Yantis S (2010) Avoiding nonindependence in fMRI data analysis: leave one subject out. Neuroimage 50:572-576. CrossRef Medline
15. Farrer C, Frey SH, Van Horn JD, Tunik E, Turk D, Inati S, Grafton ST (2008) The angular gyrus computes action awareness representations. Cereb Cortex 18:254-261. Medline
16. Feinstein JS, Stein MB, Paulus MP (2006) Anterior insula reactivity during certain decisions is associated with neuroticism. Soc Cogn Affect Neurosci 1:136-142. CrossRef Medline
17. Fincham JM, Carter CS, van Veen V, Stenger VA, Anderson JR (2002) Neural mechanisms of planning: a computational analysis using event-related fMRI. Proc Natl Acad Sci U S A 99:3346-3351. CrossRef Medline
18. Fitzgerald TH, Seymour B, Bach DR, Dolan RJ (2010) Differentiable neural substrates for learned and described value and risk. Curr Biol 20:1823-1829. CrossRef Medline
19. Forman SD, Cohen JD, Fitzgerald M, Eddy WF, Mintun MA, Noll DC (1995) Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): use of a cluster-size threshold. Magn Reson Med 33:636-647. CrossRef Medline
20. Garner DM, Olmsted MP, Bohr Y, Garfinkel PE (1982) The eating attitudes test: psychometric features and clinical correlates. Psychol Med 12:871-878. CrossRef Medline
21. Gläscher J (2009) Visualization of group inference data in functional neuroimaging. Neuroinformatics 7:73-82. CrossRef Medline
22. Gläscher J, Hampton AN, O'Doherty JP (2009) Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. Cereb Cortex 19:483-495. Medline
23. Gläscher J, Daw N, Dayan P, O'Doherty JP (2010) States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66:585-595. CrossRef Medline
24. Gottlieb JP, Kusunoki M, Goldberg ME (1998) The representation of visual salience in monkey parietal cortex. Nature 391:481-484. CrossRef Medline
25. Hamilton AF, Grafton ST (2006) Goal representation in human anterior intraparietal sulcus. J Neurosci 26:1133-1137. CrossRef Medline
26. Hamilton AF, Grafton ST (2008) Action outcomes are represented in human inferior frontoparietal cortex. Cereb Cortex 18:1160-1168. Medline
27. Hammond LJ (1980) The effect of contingency upon the appetitive conditioning of free-operant behavior. J Exp Anal Behav 34:297-304. CrossRef Medline
28. Hampton AN, Bossaerts P, O'Doherty JP (2006) The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J Neurosci 26:8360-8367. CrossRef Medline
29. Hare TA, O'Doherty J, Camerer CF, Schultz W, Rangel A (2008) Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. J Neurosci 28:5623-5630. CrossRef Medline
30. Hertwig R (2012) The psychology and rationality of decisions from experience. Synthese 187:269-292. CrossRef
31. Husain M, Rorden C (2003) Non-spatially lateralized mechanisms in hemispatial neglect. Nat Rev Neurosci 4:26-36. CrossRef Medline
32. Kastner S, Pinsk MA, De Weerd P, Desimone R, Ungerleider LG (1999) Increased activity in human visual cortex during directed attention in the absence of visual stimulation. Neuron 22:751-761. CrossRef Medline
33. Klein-Flugge MC, Barron HC, Brodersen KH, Dolan RJ, Behrens TE (2013) Segregated encoding of reward-identity and stimulus-reward associations in human orbitofrontal cortex. J Neurosci 33:3202-3211. CrossRef Medline
34. Knoch D, Gianotti LR, Pascual-Leone A, Treyer V, Regard M, Hohmann M, Brugger P (2006) Disruption of right prefrontal cortex by lowfrequency repetitive transcranial magnetic stimulation induces risktaking behavior. J Neurosci 26:6469-6472. CrossRef Medline
35. Leathers ML, Olson CR (2012) In monkeys making value-based decisions, LIP neurons encode cue salience and not action value. Science 338:132-135. CrossRef Medline
36. Liljeholm M, Tricomi E, O'Doherty JP, Balleine BW (2011) Neural correlates of instrumental contingency learning: differential effects of actionreward conjunction and disjunction. J Neurosci 31:2474-2480. CrossRef Medline
37. Liljeholm M, Molloy CJ, O'Doherty JP (2012) Dissociable brain systems mediate vicarious learning of stimulus-response and action-outcome contingencies. J Neurosci 32:9878-9886. CrossRef Medline
38. Lin J (1991) Divergence measures based on the Shannon entropy. IEEE Trans Inform Theory 37:145-151.
39. Luhmann CC, Chun MM, Yi DJ, Lee D, Wang XJ (2008) Neural dissociation of delay and uncertainty in intertemporal choice. J Neurosci 28: 14459-14466. CrossRef Medline
40. Mars RB, Jbabdi S, Sallet J, O'Reilly JX, Croxson PL, Olivier E, Noonan MP, Bergmann C, Mitchell AS, Baxter MG, Behrens TE, Johansen-Berg H, Tomassini V, Miller KL, Rushworth MF (2011) Diffusion-weighted imaging tractography-based parcellation of the human parietal cortex and comparison with human and macaque resting-state functional connectivity. J Neurosci 31:4087-4100. CrossRef Medline
41. Mevorach C, Humphreys GW, Shalev L (2006) Opposite biases in saliencebased selection for the left and right posterior parietal cortex. Nat Neurosci 9:740-742. CrossRef Medline
42. Muri RM, Iba-Zizen MT, Derosier C, Cabanis EA, Pierrot-Deseilligny C (1996) Location of the human posterior eye field with functional magnetic resonance imaging. J Neurol Neurosurg Psychiatry 60:445-448. CrossRef Medline
43. O'Doherty JP, Deichmann R, Critchley HD, Dolan RJ (2002) Neural responses during anticipation of a primary taste reward. Neuron 33:815-826. CrossRef Medline
44. O'Doherty JP, Dayan P, Friston K, Critchley H, Dolan RJ (2003) Temporal difference models and reward-related learning in the human brain. Neuron 38:329-337. CrossRef Medline
45. O'Doherty J, Rolls ET, Francis S, Bowtell R, McGlone F (2001) Representation of pleasant and aversive taste in the human brain. J Neurophysiol 85:1315-1321. Medline
46. O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304:452-454. CrossRef Medline
47. Paulus MP, Hozack N, Frank L, Brown GG (2002) Error rate and outcome predictability affect neural activation in prefrontal cortex and anterior cingulate during decision-making. Neuroimage 15:836-846. CrossRef Medline
48. Pelchat ML, Johnson A, Chan R, Valdez J, Ragland JD (2004) Images of desire: food-craving activation during fMRI. Neuroimage 23:1486-1493. CrossRef Medline
49. Penny WD, Trujillo-Barreto NJ, Friston KJ (2005) Bayesian fMRI time series analysis with spatial priors. Neuroimage 24:350-362. CrossRef Medline
50. Plassman H, O'Doherty JP, Rangel A (2010) Appetitive and aversive goal values are encoded in the medial orbitofrontal cortex at the time of decision making. J Neurosci 30:10799-10808.
51. Rosa MJ, Bestmann S, Harrison L, Penny W (2010) Bayesian model selection maps for group studies. Neuroimage 49:217-224. CrossRef Medline
52. Schmidt L, Lebreton M, Cléy-Melin ML, Daunizeau J, Pessiglione M (2012) Neural mechanisms underlying motivation of mental versus physical effort. PLoS Biol 10:e1001266. CrossRef Medline
53. Shanks DR, Dickinson A (1991) Instrumental judgment and performance under variations in action-outcome contingency and contiguity. Mem Cognit 19:353-360. CrossRef Medline
54. Simon DA, Daw ND (2011) Neural correlates of forward planning in a spatial decision task in humans. J Neurosci 31:5526-5539.
55. Smith BW, Mitchell DG, Hardin MG, Jazbec S, Fridberg D, Blair RJ, Ernst M (2009) Neural substrates of reward magnitude, probability, and risk during a wheel of fortune decision-making task. Neuroimage 44:600-609. CrossRef Medline
56. Sperduti M, Delaveau P, Fossati P, Nadel J (2011) Different brain structures related to selfand external-agency attribution: a brief review and metaanalysis. Brain Struct Funct 216:151-157. CrossRef Medline
57. Stalnaker TA, Calhoon GG, Ogawa M, Roesch MR, Schoenbaum G (2010) Neural correlates of stimulus-response and response-outcome associations in dorsolateral versus dorsomedial striatum. Front Integr Neurosci 4:12. Medline
58. Symmonds M, Wright ND, Bach DR, Dolan RJ (2011) Deconstructing risk: separable encoding of variance and skewness in the brain. Neuroimage 58:1139-1149. CrossRef Medline
59. Tanaka SC, Balleine BW, O'Doherty JP (2008) Calculating consequences: brain systems that encode the causal effects of actions. J Neurosci 28: 6750-6755. CrossRef Medline
60. Thorndike EL (1933) A proof of the Law of Effect. Science 77:173-175. CrossRef Medline
61. Tolman EC (1948) Cognitive maps in rats and men. Psychol Rev 55:189-208. CrossRef Medline
62. Tolman EC, Honzik CH (1930) "Insight" in rats. University of California, Publications in Psychology 4:215-232.
63. Wang GJ, Volkow ND, Telang F, Jayne M, Ma J, Rao M, Zhu W, Wong CT, Pappas NR, Geliebter A, Fowler JS (2004) Exposure to appetitive food stimuli markedly activates the human brain. Neuroimage 21: 1790-1797. CrossRef Medline
64. Weber BJ, Huettel SA (2008) The neural substrates of probabilistic and intertemporal decision making. Brain Res 1234:104-115. CrossRef Medline