Neurobiology of economic choice: A good-based model

Annual Review of Neuroscience

Volumn 34, Issue , 2011, Pages 333-359

  Padoa Schioppa, Camillo ^a  

^a WASHINGTON UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

Abstract representation; Action value; Adaptation; Neuroeconomics; Orbitofrontal cortex; Subjective value; Transitivity

Indexed keywords

ARTICLE; COMPARATIVE STUDY; COMPUTER MODEL; CONTROLLED STUDY; ECONOMICS; MOTOR CONTROL; NEUROBIOLOGY; NEUROPHYSIOLOGY; ORBITAL CORTEX; PRIORITY JOURNAL; SENSORIMOTOR FUNCTION; VENTROMEDIAL PREFRONTAL CORTEX;

ADAPTATION, PHYSIOLOGICAL; ANIMALS; CHOICE BEHAVIOR; HUMANS; MODELS, ECONOMIC; NEUROBIOLOGY;

EID: 79959892907     PISSN: 0147006X     EISSN: 15454126     Source Type: Book Series    
DOI: 10.1146/annurev-neuro-061010-113648     Document Type: Article

References (122)

1. Adams CD. 1982. Variations in the sensitivity of instrumental responding to reinforcer devaluation. Q. J. Exp. Psych. 34B:77-98
2. Adams CD, Dickinson A. 1981. Instrumental responding following reinforcer devaluation. Q. J. Exp. Psych. 33B:109-21
3. Amador N, Schlag-Rey M, Schlag J. 2000. Reward-predicting and reward-detecting neuronal activity in the primate supplementary eye field. J. Neurophysiol. 84:2166-70
4. Balleine BW, Dickinson A. 1998. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 37:407-19 (Pubitemid 28306930)
5. Balleine BW, Ostlund SB. 2007. Still at the choice-point: action selection and initiation in instrumental conditioning. Ann. N.Y. Acad. Sci. 1104:147-71 (Pubitemid 47092732)
6. Baxter MG, Gaffan D, Kyriazis DA, Mitchell AS. 2009. Ventrolateral prefrontal cortex is required for performance of a strategy implementation task but not reinforcer devaluation effects in rhesus monkeys. Eur. J. Neurosci. 29:2049-59
7. Bennur S, Gold JI. 2011. Distinct representations of a perceptual decision and the associated oculomotor plan in the monkey lateral intraparietal area. J. Neurosci. 31:913-21
8. Berlin HA, Rolls ET, Kischka U. 2004. Impulsivity, time perception, emotion and reinforcement sensitivity in patients with orbitofrontal cortex lesions. Brain 127:1108-26 (Pubitemid 38623046)
9. Brooks AM, Pammi VS, Noussair C, Capra CM, Engelmann JB, Berns GS. 2010. From bad to worse: striatal coding of the relative value of painful decisions. Front. Neurosci. 4:176
10. Bundesen C. 1990. A theory of visual attention. Psychol. Rev. 97:523-47
11. Cai X, Kim S, Lee D. 2011. Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice. Neuron 69:170-82
12. Cai X, Padoa-Schioppa C. 2010. Dissociating economic choice from action selection. Soc. Neurosci. Meet. Plann. Abstr. 813.1
13. Camerer C. 2003. Behavioral Game Theory: Experiments in Strategic Interaction. Princeton, NJ: Russell Sage Found., Princeton Univ. Press
14. Christopoulos GI, Tobler PN, Bossaerts P, Dolan RJ, Schultz W. 2009. Neural correlates of value, risk, and risk aversion contributing to decision making under risk. J. Neurosci. 29:12574-83
15. Cisek P, Kalaska JF. 2002. Simultaneous encoding of multiple potential reach directions in dorsal premotor cortex. J. Neurophysiol. 87:1149-54 (Pubitemid 34670880)
16. Colwill RM, Rescorla RA. 1986. Associative structures in instrumental conditioning. In The Psychology of Learning and Motivation, ed. GH Bower, pp. 55-104. Orlando, FL: Academic
17. Critchley HD. 2005. Neural mechanisms of autonomic, affective, and cognitive integration. J. Comp. Neurol. 493:154-66 (Pubitemid 41636198)
18. Damasio AR. 1994. Descartes' Error: Emotion, Reason, and the Human Brain. New York: Putnam. xix, 312 pp.
19. De Martino B, Kumaran D, Holt B, Dolan RJ. 2009. The neurobiology of reference-dependent value computation. J. Neurosci. 29:3833-42
20. Dorris MC, Glimcher PW. 2004. Activity in posterior parietal cortex is correlated with the relative subjective desirability of action. Neuron 44:365-78 (Pubitemid 39346258)
21. Feierstein CE, Quirk MC, Uchida N, Sosulski DL, Mainen ZF. 2006. Representation of spatial goals in rat orbitofrontal cortex. Neuron 51:495-507 (Pubitemid 44189800)
22. Fellows LK, Farah MJ. 2007. The role of ventromedial prefrontal cortex in decision making: judgment under uncertainty or judgment per se? Cereb. Cortex 17:2669-74 (Pubitemid 47537827)
23. FitzGerald TH, Seymour B, Dolan RJ. 2009. The role of human orbitofrontal cortex in value comparison for incommensurable objects. J. Neurosci. 29:8388-95
24. Frederick S, Loewenstein G, O'Donogue T. 2002. Time discounting and time preference: a critical review. J. Econ. Lit. 40:351-401
25. Fredrikson M, Furmark T, Olsson MT, Fischer H, Andersson J, Langstrom B. 1998. Functional neuroanatom-ical correlates of electrodermal activity: a positron emission tomographic study. Psychophysiology 35:179-85 (Pubitemid 28113702)
26. Gallagher M, McMahan RW, Schoenbaum G. 1999. Orbitofrontal cortex and representation of incentive value in associative learning. J. Neurosci. 19:6610-14 (Pubitemid 29347459)
27. Glimcher PW. 2008. Choice: towards a standard back-pocket model. See Glimcher et al. 2008, pp. 501-19
28. Glimcher PW, Camerer CF, Fehr E, Poldrack RA. 2008. Neuroeconomics: Decision Making and the Brain. Amsterdam: Elsevier. 538 pp.
29. Glimcher PW, Dorris MC, Bayer HM. 2005. Physiological utility theory and the neuroeconomics of choice. Games Econ. Behav. 52: 213-56 (Pubitemid 41051723)
30. Gold JI, Shadlen MN. 2003. The influence of behavioral context on the representation of a perceptual decision in developing oculomotor commands. J. Neurosci. 23:632-51 (Pubitemid 36139562)
31. Grace RC. 1993. Violations of transitivity: implications for a theory of contextual choice. J. Exp. Anal. Behav. 60:185-201
32. Grattan LE, Glimcher PW. 2010. No evidence for spatial tuning in orbitofrontal cortex. Soc. Neurosci. Meet. Plann. Abstr. 102.23
33. Gregorios-Pippas L, Tobler PN, Schultz W. 2009. Short-term temporal discounting of reward value in human ventral striatum. J. Neurophysiol. 101:1507-23
34. Horwitz GD, Batista AP, Newsome WT. 2004. Representation of an abstract perceptual decision in macaque superior colliculus. J. Neurophysiol. 91:2281-96 (Pubitemid 38500943)
35. Hsu M, Bhatt M, Adolphs R, Tranel D, Camerer CF. 2005. Neural systems responding to degrees of uncertainty in human decision-making. Science 310:1680-83 (Pubitemid 41780790)
36. Hsu M, Krajbich I, Zhao C, Camerer CF. 2009. Neural response to reward anticipation under risk is nonlinear in probabilities. J. Neurosci. 29:2231-37
37. Ikeda T, Hikosaka O. 2003. Reward-dependent gain and bias of visual responses in primate superior colliculus. Neuron 39:693-700 (Pubitemid 36993585)
38. Izquierdo A, Suda RK, Murray EA. 2004. Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J. Neurosci. 24:7540-48 (Pubitemid 39166745)
39. Kable JW, Glimcher PW. 2007. The neural correlates of subjective value during intertemporal choice. Nat. Neurosci. 10:1625-33 (Pubitemid 350175876)
40. Kable JW, Glimcher PW. 2009. The neurobiology of decision: consensus and controversy. Neuron 63:733-45
41. Kahneman D, Tversky A. 1979. Prospect theory: an analysis of decision under risk. Econometrica 47:263-91
42. Kahneman D, Tversky A, eds. 2000. Choices, Values and Frames. Cambridge, UK/New York: Russell Sage Found./Cambridge Univ. Press. xx, 840 pp.
43. Kalenscher T, Tobler PN, Huijbers W, Daselaar SM, Pennartz CM. 2010. Neural signatures of intransitive preferences. Front. Hum. Neurosci. 4:49
44. Kawagoe R, Takikawa Y, Hikosaka O. 1998. Expectation of reward modulates cognitive signals in the basal ganglia. Nat. Neurosci. 1:411-16
45. Kazama A, Bachevalier J. 2009. Selective aspiration or neurotoxic lesions of orbital frontal areas 11 and 13 spared monkeys' performance on the object discrimination reversal task. J. Neurosci. 29:2794-804
46. Kennerley SW, Dahmubed AF, Lara AH, Wallis JD. 2009. Neurons in the frontal lobe encode the value of multiple decision variables. J. Cogn. Neurosci. 21:1162-78
47. Kennerley SW, Wallis JD. 2009. Encoding of reward and space during a working memory task in the or-bitofrontal cortex and anterior cingulate sulcus. J. Neurophysiol. 102:3352-64
48. Kennerley SW, Walton ME, Behrens TE, Buckley MJ, Rushworth MF. 2006. Optimal decision making and the anterior cingulate cortex. Nat. Neurosci. 9:940-47 (Pubitemid 43980509)
49. Kim H, Sul JH, Huh N, Lee D, Jung MW. 2009. Role of striatum in updating values of chosen actions. J. Neurosci. 29:14701-12
50. Kim S, Hwang J, Lee D. 2008. Prefrontal coding of temporally discounted values during intertemporal choice. Neuron 59:161-72 (Pubitemid 351905851)
51. Kimmel DL, Rangel A, Newsome WT. 2010. Value representations in the primate orbitofrontal cortex during cost-benefit decision making. Soc. Neurosci. Meet. Plann. Abstr. 129.14
52. Klein JT, Deaner RO, Platt ML. 2008. Neural correlates of social target value in macaque parietal cortex. Curr. Biol. 18:419-24
53. Kobayashi S, Pinto de Carvalho O, Schultz W. 2010. Adaptation of reward sensitivity in orbitofrontal neurons. J. Neurosci. 30:534-44
54. Kobayashi S, Schultz W. 2008. Influence of reward delays on responses of dopamine neurons. J. Neurosci. 28:7837-46
55. Koenigs M, Tranel D. 2007. Irrational economic decision-making after ventromedial prefrontal damage: evidence from the Ultimatum Game. J. Neurosci. 27:951-56 (Pubitemid 46174516)
56. Kreps DM. 1990. A Course in Microeconomic Theory. Princeton, NJ: Princeton Univ. Press. 850 pp.
57. Lau B, Glimcher PW. 2008. Value representations in the primate striatum during matching behavior. Neuron 58:451-63 (Pubitemid 351615183)
58. Leon MI, Shadlen MN. 1999. Effect of expected reward magnitude on the response of neurons in the dorso-lateral prefrontal cortex of the macaque. Neuron 24:415-25 (Pubitemid 29508414)
59. Levy I, Snell J, Nelson AJ, Rustichini A, Glimcher PW. 2010. Neural representation of subjective value under risk and ambiguity. J. Neurophysiol. 103:1036-47
60. Louie K, Glimcher PW. 2010. Separating value from choice: delay discounting activity in the lateral intra-parietal area. J. Neurosci. 30:5498-507
61. Machado CJ, Bachevalier J. 2007a. The effects of selective amygdala, orbital frontal cortex or hippocampal formation lesions on reward assessment in nonhuman primates. Eur. J. Neurosci. 25:2885-904 (Pubitemid 46888840)
62. Machado CJ, Bachevalier J. 2007b. Measuring reward assessment in a semi-naturalistic context: the effects of selective amygdala, orbital frontal or hippocampal lesions. Neuroscience 148:599-611 (Pubitemid 47418381)
63. Matsumoto K, Suzuki W, Tanaka K. 2003. Neuronal correlates of goal-based motor selection in the prefrontal cortex. Science 301:229-32 (Pubitemid 36858478)
64. Maunsell JH. 2004. Neuronal representations of cognitive state: reward or attention? Trends Cogn. Sci. 8:261-65 (Pubitemid 38692516)
65. McClure SM, Laibson DI, Loewenstein G, Cohen JD. 2004. Separate neural systems value immediate and delayed monetary rewards. Science 306:503-7 (Pubitemid 39372453)
66. McCoy AN, Crowley JC, Haghighian G, Dean HL, Platt ML. 2003. Saccade reward signals in posterior cingulate cortex. Neuron 40:1031-40 (Pubitemid 37522060)
67. Minamimoto T, Hori Y, Kimura M. 2005. Complementary process to response bias in the centromedian nucleus of the thalamus. Science 308:1798-801 (Pubitemid 40848010)
68. Mobini S, Body S, Ho MY, Bradshaw CM, Szabadi E, et al. 2002. Effects of lesions of the orbitofrontal cortex on sensitivity to delayed and probabilistic reinforcement. Psychopharmacology (Berl.) 160:290-98 (Pubitemid 34230681)
69. Montague PR, King-Casas B, Cohen JD. 2006. Imaging valuation models in human choice. Annu. Rev. Neurosci. 29:417-48 (Pubitemid 44476852)
70. Morrison SE, Salzman CD. 2009. The convergence of information about rewarding and aversive stimuli in single neurons. J. Neurosci. 29:11471-83
71. O'Neill M, Schultz W. 2010. Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value. Neuron 68:789-800
72. Ongur D, Price JL. 2000. The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. Cereb. Cortex 10:206-19 (Pubitemid 30159289)
73. Padoa-Schioppa C. 2007. Orbitofrontal cortex and the computation of economic value. Ann. N.Y. Acad. Sci. 1121:232-53
74. Padoa-Schioppa C. 2009. Range-adapting representation of economic value in the orbitofrontal cortex. J. Neurosci. 29:14004-14
75. Padoa-Schioppa C, Assad JA. 2006. Neurons in orbitofrontal cortex encode economic value. Nature 441:223-26
76. Padoa-Schioppa C, Assad JA. 2008. The representation of economic value in the orbitofrontal cortex is invariant for changes of menu. Nat. Neurosci. 11:95-102
77. Peters J, Buchel C. 2009. Overlapping and distinct neural systems code for subjective value during intertem-poral and risky decision making. J. Neurosci. 29:15727-34
78. Picard N, Strick PL. 2001. Imaging the premotor areas. Curr. Opin. Neurobiol. 11:663-72 (Pubitemid 34014913)
79. Pickens CL, Saddoris MP, Setlow B, Gallagher M, Holland PC, Schoenbaum G. 2003. Different roles for orbitofrontal cortex and basolateral amygdala in a reinforcer devaluation task. J. Neurosci. 23:11078-84 (Pubitemid 37548942)
80. Pine A, Seymour B, Roiser JP, Bossaerts P, Friston KJ, et al. 2009. Encoding of marginal utility across time in the human brain. J. Neurosci. 29:9575-81
81. Plassmann H, O'Doherty J, Rangel A. 2007. Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. J. Neurosci. 27:9984-88 (Pubitemid 47404004)
82. Popper K. 1963. Science: conjectures and refutations. In Conjectures and Refutations, pp. 43-86. London/New York: Routledge Classics
83. Price JL. 1999. Prefrontal cortical networks related to visceral function and mood. Ann. N.Y. Acad. Sci. 877:383-96 (Pubitemid 29351868)
84. Price JL, Drevets WC. 2010. Neurocircuitry of mood disorders. Neuropsychopharmacology 35:192-216
85. Rahman S, Sahakian BJ, Hodges JR, Rogers RD, Robbins TW. 1999. Specific cognitive deficits in mild frontal variant of frontotemporal dementia. Brain 122(Pt. 8):1469-93 (Pubitemid 29374124)
86. Rangel A, Camerer C, Montague PR. 2008. A framework for studying the neurobiology of value-based decision making. Nat. Rev. Neurosci. 9:545-56 (Pubitemid 351873461)
87. Rangel A, Hare T. 2010. Neural computations associated with goal-directed choice. Curr. Opin. Neurobiol. 20:262-70
88. Roesch MR, Olson CR. 2003. Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. J. Neurophysiol. 90:1766-89 (Pubitemid 37129385)
89. Roesch MR, Olson CR. 2005. Neuronal activity in primate orbitofrontal cortex reflects the value of time. J. Neurophysiol. 94:2457-71 (Pubitemid 41324670)
90. Roesch MR, Taylor AR, Schoenbaum G. 2006. Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation. Neuron 51:509-20 (Pubitemid 44189799)
91. Rudebeck PH, Behrens TE, Kennerley SW, Baxter MG, Buckley MJ, et al. 2008. Frontal cortex subregions play distinct roles in choices between actions and stimuli. J. Neurosci. 28:13775-85
92. Rudebeck PH, Walton ME, Smyth AN, Bannerman DM, Rushworth MF. 2006. Separate neural pathways process different decision costs. Nat. Neurosci. 9:1161-68 (Pubitemid 44306310)
93. Rushworth MF, Mars RB, Summerfield C. 2009. General mechanisms for making decisions? Curr. Opin. Neurobiol. 19:75-83
94. Samejima K, Ueda Y, Doya K, Kimura M. 2005. Representation of action-specific reward values in the striatum. Science 310:1337-40 (Pubitemid 41698867)
95. Schoenbaum G, Roesch MR, Stalnaker TA, Takahashi YK. 2009. A new perspective on the role of the orbitofrontal cortex in adaptive behaviour. Nat. Rev. Neurosci. 10:885-92
96. Seo H, Lee D. 2007. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 27:8366-77 (Pubitemid 47219624)
97. Seymour B, McClure SM. 2008. Anchors, scales and the relative coding of value in the brain. Curr. Opin. Neurobiol. 18:173-78
98. Shafir S. 2002. Context-dependent violations of rational choice in honeybees (Apis mellifera) and gray jays (Perisoreus canadensis). Behav. Ecol. Sociobiol. 51:180-87 (Pubitemid 36059139)
99. Shenhav A, Greene JD. 2010. Moral judgments recruit domain-general valuation mechanisms to integrate representations of probability and magnitude. Neuron 67:667-77
100. Shidara M, Richmond BJ. 2002. Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 296:1709-11 (Pubitemid 34579174)
101. Skinner BF. 1953. Science and Human Behavior. New York: Macmillan. 461 pp.
102. Sloan J, Kennerley SW, Wallis JD. 2010. Neural encoding of cost-based decisions: effort versus delay. Soc. Neurosci. Meet. Plann. Abstr. 805.16
103. Sugrue LP, Corrado GS, Newsome WT. 2004. Matching behavior and the representation of value in the parietal cortex. Science 304:1782-87 (Pubitemid 38787891)
104. Sutton RS, Barto AG. 1998. Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press. xviii, 322 pp.
105. Thevarajah D, Webb R, Ferrall C, Dorris MC. 2010. Modeling the value of strategic actions in the superior colliculus. Front. Behav. Neurosci. 3:57
106. Tversky A. 1969. The intransitivity of preferences. Psychol. Rev. 76:31-48
107. Tversky A, Shafir E. 2004. Preference, Belief, and Similarity: Selected Writings. Cambridge, MA: MIT Press. xvi, 1023 pp.
108. Tversky A, Simonson I. 1993. Context-dependent preferences. Manag. Sci. 39:117-85
109. van Duuren E, Escamez FA, Joosten RN, Visser R, Mulder AB, Pennartz CM. 2007. Neural coding of reward magnitude in the orbitofrontal cortex of the rat during a five-odor olfactory discrimination task. Learn. Mem. 14:446-56
110. Walton ME, Bannerman DM, Rushworth MF. 2002. The role of rat medial frontal cortex in effort-based decision making. J. Neurosci. 22:10996-1003 (Pubitemid 36005176)
111. Watson KK, Platt ML. 2008. Orbitofrontal neurons encode both social and nonsocial rewards. Soc. Neurosci. Meet. Plann. Abstr. 691.11
112. Williams ZM, Eskandar EN. 2006. Selective enhancement of associative learning by microstimulation of the anterior caudate. Nat. Neurosci. 9:562-68
113. Winstanley CA, Theobald DE, Cardinal RN, Robbins TW. 2004. Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. J. Neurosci. 24:4718-22 (Pubitemid 38668195)
114. Wise SP. 2008. Forward frontal fields: phylogeny and fundamental function. Trends Neurosci. 31:599-608
115. Wunderlich K, Rangel A, O'Doherty JP. 2010. Economic choices can be made using only stimulus values. Proc. Natl. Acad. Sci. USA 107:15005-10
116. Zald DH. 2006. The rodent orbitofrontal cortex gets time and direction. Neuron 51:395-97 (Pubitemid 44189806)
117. Ziegler G, Dahnke R, Yeragani VK, Bar KJ. 2009. The relation of ventromedial prefrontal cortex activity and heart rate fluctuations at rest. Eur. J. Neurosci. 30:2205-10
118. Heyman GM. 2009. Addiction: A Disorder of Choice. Cambridge, MA: Harvard Univ. Press. 200 pp.
119. Kagel JH, Battalio RC, Green L. 1995. Economic Choice Theory: An Experimental Analysis of Animal Behavior. Cambridge, UK: Cambridge Univ. Press. 230 pp.
120. Niehans J. 1990. A History of Economic Theory: Classic Contributions, 1720-1980. Baltimore, MD: Johns Hopkins Univ. Press. 578 pp.
121. Padoa-Schioppa C. 2008. The syllogism of neuro-economics. Econ. Philos. 24:449-57
122. Ross D. 2005. Economic Theory and Cognitive Science: Microexplanation. Cambridge, MA: MIT Press. 444 pp.