Separate encoding of model-based and model-free valuations in the human brain

NeuroImage

Volumn 58, Issue 3, 2011, Pages 955-962

  Beierholm, Ulrik R ^a   Anen, Cedric ^b   Quartz, Steven ^b,c   Bossaerts, Peter ^b,c  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b EPFL   (Switzerland)

^c CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Bayesian inference; Decision making; Dual systems; Reinforcement learning; Striatum; VmPFC

Indexed keywords

ADULT; ANTERIOR CINGULATE; ARTICLE; BAYES THEOREM; BRAIN FUNCTION; CONFLICT; DECISION MAKING; FUNCTIONAL MAGNETIC RESONANCE IMAGING; GLOBUS PALLIDUS; HUMAN; HUMAN EXPERIMENT; INTUITION; LEARNING; MATHEMATICAL ANALYSIS; NEUROBIOLOGY; NORMAL HUMAN; PREDICTION; PREFRONTAL CORTEX; PRIORITY JOURNAL; PROBLEM SOLVING; REINFORCEMENT; SIGNAL PROCESSING; STRIATE CORTEX; TASK PERFORMANCE; THINKING; UNCERTAINTY;

BAYES THEOREM; BRAIN; BRAIN MAPPING; DECISION MAKING; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; PROBLEM SOLVING;

EID: 80052143971     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2011.06.071     Document Type: Article

References (34)

1. Bossaerts P., Preuschoff K., Hsu M. The neurobiological foundations of valuation in human decision-making under uncertainty. Neuroeconomics: Decision Making and the Brain 2008, Academic Press, London. P.W. Glimcher, C.F. Camerer, E. Fehr, R.A. Poldrack (Eds.).
2. Brainard D. The psychophysics toolbox. Spat. Vis. 1997, 10:433-436.
3. Coricelli G., Critchley H.D., Joffily M., O'Doherty J.P., Sirigu A., Dolan R.J. Regret and its avoidance: a neuroimaging study of choice behavior. Nature Neuroscience 2005, 8(9):1255-1262.
4. Daw N.D., Niv Y., Dayan P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 2005, 8:1704-1711.
5. Daw N.D., O'Doherty J.P., Dayan P., Seymour B., Dolan R.J. Cortical substrates for exploratory decisions in humans. Nature 2006, 441:876-879.
6. De Martino B., Kumaran D., Seymour B., Dolan R.J. Frames, biases, and rational decision-making in the human brain. Science 2006, 313:684-687.
7. De Neys W., Glumicic T. Conflict monitoring in dual process theories of thinking. Cognition 2008, 106:1248-1299.
8. Evans J.S.B.T. Dual-processing accounts of reasoning, judgment, and social cognition. Annu. Rev. Psychol. 2008, 59:255-278.
9. Fermin A., Yoshida T., Ito M., Yoshimoto J., Doya K. Evidence for model-based action planning in a sequential finger movement task. J. Mot. Behav. 2010, 42:371-379.
10. Gläscher J., Daw N., Dayan P., O'Doherty J.P. States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 2010, 66:585-595.
11. Haber S.N., Knutson B. The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology 2009, 35:4-26.
12. Hampton A.N., Bossaerts P., O'Doherty J.P. The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J. Neurosci. 2006, 26:8360-8367.
13. Hsu M., Bhatt M., Adolphs R., Tranel D., Camerer C.F. Neural systems responding to degrees of uncertainty in human decision making. Science 2005, 310:1680-1683.
14. Kable J.W., Glimcher P.W. The neural correlates of subjective value during intertemporal choice. Nat. Neurosci. 2007, 10:1625-1633.
15. Kahneman D., Frederick S. Representativeness revisited: attribute substitution in intuitive judgment. Heuristics and Biases: The Psychology of Intuitive Judgment 2002, 49-81. Cambridge University Press. T. Gilovich, D.W. Griffin, D. Kahneman (Eds.).
16. Kahneman D., Frederick S. Frames and brains: elicitation and control of response tendencies. Trends in Cognitive Sciences 2007, 11:45-46.
17. Kerns J.G., Cohen J.D., MacDonald A.W., Cho R.Y., Stenger V.A., Carter C.S. Anterior cingulate conflict monitoring and adjustments in control. Science 2004, 303:1023-1026.
18. Levy I., Snell J., Nelson A.J., Rustichini A., Glimcher P.W. Neural representation of subjective value under risk and ambiguity. J. Neurophysiol. 2010, 103:1036-1047.
19. McClure S.M., Berns G.S., Montague P.R. Temporal prediction errors in a passive learning task activate human striatum. Neuron 2003, 38:339-346.
20. McClure S.M., Laibson D.I., Loewenstein G., Cohen J.D. Separate neural systems value immediate and delayed monetary rewards. Science 2004, 306:503-507.
21. Monti M.M., Osherson D.N., Martinez M.J., Parsons L.M. Functional neuroanatomy of deductive inference: a language-independent distributed network. NeuroImage 2007, 37:1005-1016.
22. O'Doherty J.P., Dayan P., Friston K., Critchley H., Dolan R.J. Temporal difference models and reward-related learning in the human brain. Neuron 2003, 38:329-337.
23. Padoa-Schioppa C., Assad J.A. Neurons in the orbitofrontal cortex encode economic value. Nature 2006, 441:223-226.
24. Plassmann H., O'Doherty J., Rangel A. Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. J. Neurosci. 2007, 27:9984-9988.
25. Platt M.L., Glimcher P.W. Neural correlates of decision variables in parietal cortex. Nature 1999, 400:233-238.
26. Rescorla R.A., Wagner A.R. A theory of Pavlovian conditioning: variations on the effectiveness of reinforcement and non-reinforcement. Classical Conditioning II: Current Research and Theory 1972, 64-99. Appleton-Century-Crofts, New York. A.H. Black, W.F. Prokasy (Eds.).
27. Sloman S.A. The empirical case for two systems of reasoning. Psychol. Bull. 1996, 119:3-22.
28. Sutton R.S., Barto A.G. Reinforcement Learning: An Introduction 1998, MIT Press, Cambridge, MA.
29. Tobler P.N., O'Doherty J.P., Dolan R.J., Schultz W. Reward value coding distinct from risk attitude-related uncertainty coding in human reward systems. J. Neurophysiol. 2007, 97:1621-1632.
30. Tremblay L., Schultz W. Relative reward preference in primate orbitofrontal cortex. Nature 1999, 398:704-708.
31. Tversky A., Kahneman D. Judgment under uncertainty: heuristics and biases. Science 1974, 185:1124-1131.
32. Tzieropoulos H.n., Grave De Peralta R., Bossaerts P., Gonzalez Andino S.L. The impact of disappointment in decision making: inter-individual differences and electrical neuroimaging. Front. Hum. Neurosci. 2011, 4:235.
33. Valdes-Sosa, P.A., Roebroeck, A., Daunizeau, J., Friston, K., in press. Effective connectivity: Influence, causality and biophysical modeling. Neuroimage. http://dx.doi.org/10.1016/j.neuroimage.2011.03.058.
34. Yoshida A., Tanaka M. Enhanced modulation of neuronal activity during antisaccades in the primate Globus pallidus. Cereb. Cortex 2009, 19:206-217.