Differentiable neural substrates for learned and described value and risk

Current Biology

Volumn 20, Issue 20, 2010, Pages 1823-1829

  Fitzgerald, Thomas H B ^a,b   Seymour, Ben ^a,c   Bach, Dominik R ^a   Dolan, Raymond J ^a  

^a WELLCOME TRUST CENTRE FOR NEUROIMAGING   (United Kingdom)

^b KING'S COLLEGE LONDON   (United Kingdom)

^c KING'S COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; BASAL GANGLION; BRAIN MAPPING; DECISION MAKING; HUMAN; LEARNING; METABOLISM; NERVE CELL; NUCLEAR MAGNETIC RESONANCE IMAGING; PARIETAL LOBE; PHYSIOLOGY; PREFRONTAL CORTEX; PROBABILITY; RISK; STATISTICAL MODEL;

BASAL GANGLIA; BRAIN MAPPING; CHOICE BEHAVIOR; DECISION MAKING; HUMANS; LEARNING; LOGISTIC MODELS; MAGNETIC RESONANCE IMAGING; NEURONS; PARIETAL LOBE; PREFRONTAL CORTEX; PROBABILITY; RISK;

EID: 78049278720     PISSN: 09609822     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.cub.2010.08.048     Document Type: Article

References (41)

1. R.S. Sutton, and A.G. Barto Reinforcement Learning: An Introduction 1998 MIT Press Cambridge, MA
2. D.W. Stephens, and J.R. Krebs Foraging Theory 1986 Princeton University Press Princeton, NJ
3. N.J. Mackintosh, and A. Dickinson Instrumental (type II) conditioning A. Dickinson, R.A. Boakes, Mechanisms of Learning and Motivation: A Memorial Volume to Jerzy Konorski 1979 Lawrence Erlbaum Associates Hillsdale, NJ 143 169
4. R. Hertwig, G. Barron, E.U. Weber, and I. Erev Decisions from experience and the effect of rare events in risky choice Psychol. Sci. 15 2004 534 539
5. R.K. Jessup, A.J. Bishara, and J.R. Busemeyer Feedback produces divergence from prospect theory in descriptive choice Psychol. Sci. 19 2008 1015 1022
6. C. Ungemach, N. Chater, and N. Stewart Are probabilities overweighted or underweighted when rare outcomes are experienced (rarely)? Psychol. Sci. 20 2009 473 479
7. R. Hertwig, and I. Erev The description-experience gap in risky choice Trends Cogn. Sci. 13 2009 517 523
8. S.W. Wu, M.R. Delgado, and L.T. Maloney Economic decision-making compared with an equivalent motor task Proc. Natl. Acad. Sci. USA 106 2009 6088 6093
9. C. Camerer, and T.H. Ho Experience-weighted attraction learning in normal form games Econometrica 67 1999 827 874
10. B. Lau, and P.W. Glimcher Dynamic response-by-response models of matching behavior in rhesus monkeys J. Exp. Anal. Behav. 84 2005 555 579
11. N.D. Daw, J.P. O'Doherty, P. Dayan, B. Seymour, and R.J. Dolan Cortical substrates for exploratory decisions in humans Nature 441 2006 876 879
12. J.W. Kable, and P.W. Glimcher The neural correlates of subjective value during intertemporal choice Nat. Neurosci. 10 2007 1625 1633
13. B. Knutson, J. Taylor, M. Kaufman, R. Peterson, and G. Glover Distributed neural representation of expected value J. Neurosci. 25 2005 4806 4812
14. S.M. McClure, D.I. Laibson, G. Loewenstein, and J.D. Cohen Separate neural systems value immediate and delayed monetary rewards Science 306 2004 503 507
15. J. O'Doherty, P. Dayan, J. Schultz, R. Deichmann, K. Friston, and R.J. Dolan Dissociable roles of ventral and dorsal striatum in instrumental conditioning Science 304 2004 452 454
16. T. Schönberg, N.D. Daw, D. Joel, and J.P. O'Doherty Reinforcement learning signals in the human striatum distinguish learners from nonlearners during reward-based decision making J. Neurosci. 27 2007 12860 12867
17. Y. Niv, N.D. Daw, D. Joel, and P. Dayan Tonic dopamine: Opportunity costs and the control of response vigor Psychopharmacology (Berl.) 191 2007 507 520
18. P.R. Montague, and G.S. Berns Neural economics and the biological substrates of valuation Neuron 36 2002 265 284
19. B. Seymour, J.P. O'Doherty, P. Dayan, M. Koltzenburg, A.K. Jones, R.J. Dolan, K.J. Friston, and R.S. Frackowiak Temporal difference models describe higher-order learning in humans Nature 429 2004 664 667
20. S.A. Huettel, A.W. Song, and G. McCarthy Decisions under uncertainty: Probabilistic context influences activation of prefrontal and parietal cortices J. Neurosci. 25 2005 3304 3311
21. K. Preuschoff, S.R. Quartz, and P. Bossaerts Human insula activation reflects risk prediction errors as well as risk J. Neurosci. 28 2008 2745 2752
22. T.E.J. Behrens, M.W. Woolrich, M.E. Walton, and M.F.S. Rushworth Learning the value of information in an uncertain world Nat. Neurosci. 10 2007 1214 1221
23. H.D. Critchley, C.J. Mathias, and R.J. Dolan Neural activity in the human brain relating to uncertainty and arousal during anticipation Neuron 29 2001 537 545
24. J.W. Brown, and T.S. Braver Learned predictions of error likelihood in the anterior cingulate cortex Science 307 2005 1118 1121
25. J.W. Brown, and T.S. Braver Risk prediction and aversion by anterior cingulate cortex Cogn. Affect. Behav. Neurosci. 7 2007 266 277
26. M. d'Acremont, Z.L. Lu, X. Li, M. Van der Linden, and A. Bechara Neural correlates of risk prediction error during reinforcement learning in humans Neuroimage 47 2009 1929 1939
27. D. Kahneman, and A. Tversky Prospect theory: An analysis of decision under risk Econometrica 47 1979 263 292
28. B. De Martino, D. Kumaran, B. Seymour, and R.J. Dolan Frames, biases, and rational decision-making in the human brain Science 313 2006 684 687
29. S.M. Tom, C.R. Fox, C. Trepel, and R.A. Poldrack The neural basis of loss aversion in decision-making under risk Science 315 2007 515 518
30. T.A. Hare, J. O'Doherty, C.F. Camerer, W. Schultz, and A. Rangel Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors J. Neurosci. 28 2008 5623 5630
31. C. Padoa-Schioppa, and J.A. Assad Neurons in the orbitofrontal cortex encode economic value Nature 441 2006 223 226
32. H. Plassmann, J. O'Doherty, and A. Rangel Orbitofrontal cortex encodes willingness to pay in everyday economic transactions J. Neurosci. 27 2007 9984 9988
33. G. Schoenbaum, and M. Roesch Orbitofrontal cortex, associative learning, and expectancies Neuron 47 2005 633 636
34. L. Tremblay, and W. Schultz Relative reward preference in primate orbitofrontal cortex Nature 398 1999 704 708
35. V.S. Chib, A. Rangel, S. Shimojo, and J.P. O'Doherty Evidence for a common representation of decision values for dissimilar goods in human ventromedial prefrontal cortex J. Neurosci. 29 2009 12315 12320
36. T.H.B. FitzGerald, B. Seymour, and R.J. Dolan The role of human orbitofrontal cortex in value comparison for incommensurable objects J. Neurosci. 29 2009 8388 8395
37. M. Gallagher, R.W. McMahan, and G. Schoenbaum Orbitofrontal cortex and representation of incentive value in associative learning J. Neurosci. 19 1999 6610 6614
38. A. Izquierdo, R.K. Suda, and E.A. Murray Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency J. Neurosci. 24 2004 7540 7548
39. J.A. Gottfried, J. O'Doherty, and R.J. Dolan Encoding predictive reward value in human amygdala and orbitofrontal cortex Science 301 2003 1104 1107
40. V.V. Valentin, A. Dickinson, and J.P. O'Doherty Determining the neural substrates of goal-directed learning in the human brain J. Neurosci. 27 2007 4019 4026
41. W. Schultz, P. Dayan, and P.R. Montague A neural substrate of prediction and reward Science 275 1997 1593 1599