Neural basis of quasi-rational decision making

Current Opinion in Neurobiology

Volumn 16, Issue 2, 2006, Pages 191-198

  Lee, Daeyeol ^a  

^a UNIVERSITY OF ROCHESTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BRAIN; DECISION MAKING; ECONOMICS; EMOTION; EXPERIENCE; HUMAN; LEARNING; MENTAL HEALTH; NEUROSCIENCE; PRIORITY JOURNAL; PSYCHOLOGY; REVIEW;

ANIMALS; BRAIN; COGNITION; DECISION MAKING; ECONOMICS; EMOTIONS; HUMANS; LEARNING; MODELS, NEUROLOGICAL; NERVE NET; REWARD; SOCIAL BEHAVIOR;

EID: 33645775037     PISSN: 09594388     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.conb.2006.02.001     Document Type: Review

References (72)

1. Glimcher P.W., and Rustichini A. Neuroeconomics: the consilience of brain and decision. Science 306 (2004) 447-452
2. Camerer C., Loewenstein G., and Prelec D. Neuroeconomics: how neuroscience can inform economics. J Econ Lit 43 (2005) 9-64
3. Zak P.J. Neuroeconomics. Philos Trans R Soc Lond B Biol Sci 359 (2004) 1737-1748
4. Thaler R.H. Quasi Rational Economics (1991), Russell Sage Foundation
5. Gigerenzer G., Todd P.M., and the ABC Research Group. Simple Heuristics That Make Us Smart (1999), Oxford University Press
6. Kahneman D. New challenges to the rationality assumption. J Inst Theor Econ 150 (1994) 18-36
7. Sutton R.S., and Barto A.G. Reinforcement Learning: an Introduction (1998), MIT Press
8. Hollerman J.R., Tremblay L., and Schultz W. Influence of reward expectation on behavior-related neuronal activity in primate striatum. J Neurophysiol 80 (1998) 947-963
9. Kawagoe R., Kakikawa Y., and Hikosaka O. Expectation of reward modulates cognitive signals in the basal ganglia. Nat Neurosci 1 (1998) 411-416
10. Cromwell H.C., and Schultz W. Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J Neurophysiol 89 (2003) 2823-2838
11. Knutson B., Adams C.M., Fong G.W., and Hommer D. Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J Neurosci 21 (2001) RC159 (1-5)
12. Knutson B., Taylor J., Kaufman M., Peterson R., and Glover G. Distributed neural representation of expected value. J Neurosci 25 (2005) 4806-4812. By manipulating the valence, magnitude and probability of the reward delivered during a so-called probabilistic monetary incentive delay task, the authors found that different components of expected value are represented in distinct brain regions. For example, anterior cingulate cortex and ventral striatum respond predominantly to the incentive valence. The authors also identified several cortical and subcortical areas that displayed a pattern of activation correlated with the outcome magnitude and high-order interactions among various factors. Interestingly, the 3-way interaction (valence × magnitude × probability) was identified only in anterior cingulate cortex and the right tail of the caudate nucleus.
13. Leon M.I., and Shadlen M.N. Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque. Neuron 24 (1999) 415-425
14. O'Doherty J.P., Deichmann R., Critchley H.D., and Dolan R.J. Neural responses during anticipation of a primary taste reward. Neuron 33 (2002) 815-826
15. Shidara M., and Richmond B.J. Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science 296 (2002) 1709-1711
16. McCoy A.N., Crowley J.C., Haghighian G., Dean H.L., and Platt M.L. Saccade reward signals in posterior cingulate cortex. Neuron 40 (2003) 1031-1040
17. Platt M.L., and Glimcher P.W. Neural correlates of decision variables in parietal cortex. Nature 400 (1999) 233-238
18. Barraclough D.J., Conroy M.L., and Lee D. Prefrontal cortex and decision making in a mixed-strategy game. Nat Neurosci 7 (2004) 404-410
19. Sugrue L.P., Corrado G.S., and Newsome W.T. Matching behavior and the representation of value in the parietal cortex. Science 304 (2004) 1782-1787. The authors examined the choice behavior of monkeys during a dynamic foraging task in which two alternative targets are baited at different rates. Consistent with Herrnstein's matching law, the probability of choosing a given target was tightly correlated with its fractional income. In addition, activity of neurons in the posterior parietal cortex encoded the fractional income of the target in their receptive fields.
20. ••].
21. McCoy A.N., and Platt M.L. Risk-sensitive neurons in macaque posterior cingulate cortex. Nat Neurosci 8 (2005) 1220-1227. People and animals often seek or avoid uncertainty in expected rewards even when the average amount of reward is equated, which implies non-linear utility functions. The authors found that monkeys displayed risk-seeking behavior during a binary decision making task. In addition, activity of neurons in the posterior cingulate cortex often modulated their activity according to the uncertainty of reward, suggesting that such activity might be more closely related to the animal's subjective preference for risky targets than their objective values.
22. Haruno M., Kuroda T., Doya K., Toyama K., Kumura M., Samejima K., Imamizu H., and Kawato M. A neural correlate of reward-based behavioral learning in caudate nucleus: A functional magnetic resonance imaging study of a stochastic decision task. J Neurosci 24 (2004) 1660-1665. Subjects in this study performed a series of stochastic decision making tasks in which the payoff in a given trial was determined by the subject's response and a related matrix of transition probabilities. Activity in the caudate nucleus was correlated with the rate of behavioral change during learning and the reward received in a single trial, demonstrating an important role for the caudate nucleus in reward-based learning. By contrast, activity of cortical areas implicated in procedural learning, such as the supplementary motor area and cerebellum, increased as the subject's behavioral strategy converged to the final pattern.
23. Tanaka S.C., Doya K., Okada G., Ueda K., Okamoto Y., and Yamawaki S. Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat Neurosci 7 (2004) 887-893. In this study subjects performed two different versions of Markov decision tasks. In one condition, the optimal strategy required the subjects to collect a small reward in each trial, whereas in the other condition, the subjects had to accept small losses to obtain a large reward later. By applying a temporal difference learning algorithm, authors found a regional specialization within the caudate nucleus and the insular cortex for the time scale used to compute the value function and reward prediction error, respectively.
24. Knutson B., Fong G.W., Adams C.M., Varner J.L., and Hommer D. Dissociation of reward anticipation and outcome with event-related fMRI. Neuroreport 12 (2001) 3683-3687
25. Amador N., Schlag-Rey M., and Schlag J. Reward-predicting and reward-detecting neuronal activity in the primate supplementary eye field. J Neurophysiol 84 (2000) 2166-2170
26. Stuphorn V., Taylor T.L., and Schall J.D. Performance monitoring by the supplementary eye field. Nature 408 (2000) 857-860
27. Ito S., Stuphorn V., Brown J.W., and Schall J.D. Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302 (2003) 120-122
28. Allais M. Le comportement de l'homme rationnel devant le risque: critique des postulats et axiomes de l'ecole americaine. Econometrica 21 (1953) 503-546. Translation of title: Behavior of rational human beings when facing risk: a criticism of the assumptions and axioms of the American school.
29. Kahneman D., and Tversky A. Prospect theory: an analysis of decision under risk. Econometrica 47 (1979) 263-291
30. Mellers B., Schwartz A., and Ritov I. Emotion-based choice. J Exp Psychol Gen 128 (1999) 332-345
31. Bechara A., and Damasio A.R. The somatic marker hypothesis: a neural theory of economic decision. Games Econ Beh 52 (2005) 336-372. This article provides an excellent overview of the neural basis of emotion-based decision making, in particular, from the perspective of the somatic marker hypothesis.
32. Bell D.E. Regret in decision making under uncertainty. Oper Res 30 (1982) 961-981
33. Loomes G., and Sugden R. Regret theory: an alternative theory of rational choice under uncertainty. Econ J 92 (1982) 805-824
34. Camille N., Coricelli G., Sallet J., Pradat-Diehl P., Duhamel J.-R., and Sirigu A. The involvement of the orbitofrontal cortex in the experience of regret. Science 304 (2004) 1167-1170. Regret refers to an emotional reaction resulting from counter-factual reasoning that the outcome of an unchosen action would have been better than the obtained outcome. This study found that regret was abolished in patients with lesions in the orbitofrontal cortex during a binary choice between two different gambles [30]. Because the analysis of foregone payoffs was beneficial in this task, this also reduced the earnings for the patients compared with those of normal subjects.
35. Coricelli G., Critchley H.D., Joffily M., O'Doherty J.P., Sirigu A., and Dolan R.J. Regret and its avoidance: a neuroimaging study of choice behavior. Nat Neurosci 8 (2005) 1255-1262. This study shows that choice behavior is influenced not only by the expected value but also by anticipated disappointment and regret. The outcome-related activity in the middle temporal gyrus and dorsal brainstem was correlated with disappointment, whereas activity in anterior cingulate cortex, putamen, inferior parietal cortex and orbitofrontal cortex was correlated with regret. In addition, activity in the dorsal anterior cingulate cortex and substantia nigra was enhanced when the subjects selected gambles that minimize regret.
36. Tinklepaugh O.L. An experimental study of representative factors in monkeys. J Comp Psychol 8 (1928) 197-236
37. Brosnan S.F., and de Waal F.B.M. Monkeys reject unequal pay. Nature 425 (2003) 297-299
38. Cromwell H.C., Hassani O.K., and Schultz W. Relative reward processing in primate striatum. Exp Brain Res 162 (2005) 520-525
39. Breiter H.C., Aharon I., Kahneman D., Dale A., and Shizgal P. Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30 (2001) 619-639
40. Tremblay L., and Schultz W. Relative reward preference in primate orbitofrontal cortex. Nature 398 (1999) 704-708
41. Watanabe M., Hikosaka K., Sakagami M., and Shirakawa S. Coding and monitoring of motivational context in the primate prefrontal cortex. J Neurosci 22 (2002) 2391-2400
42. Nash J.F. Equilibrium points in n-person games. Proc Natl Acad Sci USA 36 (1950) 48-49
43. Camerer C.F. Behavioral Game Theory (2003), Princeton Univ Press
44. King-Casas B., Tomlin D., Anen C., Camerer C.F., Quartz S.R., and Montague P.R. Getting to know you: reputation and trust in a two-person economic exchange. Science 308 (2005) 78-83. Subjects in this study played a multi-round trust game and the brains of two individuals playing this game were scanned simultaneously. Reciprocity was used as a measure of how the amount of money sent by a given player changes in response to the change in money previously sent by the other player. Activation in the trustee's caudate nucleus was related to the benevolent reciprocity of the investor, and provided the 'intention to trust' signal. Initially, this activation began after the investor's decision was revealed, but eventually occurred before the investor's decision, suggesting that the trustee gradually developed a model for the investor's behavior.
45. Gallagher H.L., Jack A.I., Roepstorff A., and Frith C.D. Imaging the intentional stance in a competitive game. Neuroimage 16 (2002) 814-821
46. Sanfey A.G., Rilling J.K., Aronson J.A., Nystrom L.E., and Cohen J.D. The neural basis of economic decision-making in the ultimatum game. Science 300 (2003) 1755-1758
47. Rilling J., Gutman D.A., Zeh T.R., Pagnoni G., Berns G.S., and Kilts C.D. A neural basis for social cooperation. Neuron 35 (2002) 395-405
48. Rilling J.K., Sanfey A.G., Aronson J.A., Nystrom L.E., and Cohen J.D. The neural correlates of theory of mind within interpersonal interactions. Neuroimage 22 (2004) 1694-1703. The authors investigated how the activity in various brain areas changes during the ultimatum game or the prisoner's dilemma game when the players think that they are interacting with other human players, rather than with computer partners. Interaction with an alleged human player led to stronger activation in a large number of areas, including superior temporal sulcus, anterior paracingulate cortex and posterior cingulate cortex.
49. de Quervain D.J.F., Fischbacher U., Treyer V., Schellhammer M., Schnyder U., Buck A., and Fehr E. The neural basis of altruistic punishment. Science 305 (2004) 1254-1258. In this study, subjects played a version of the trust game and the investor had the option of punishing the trustee who kept all the money. Activity in the caudate nucleus of the investor increased when the punishment was costly to the trustee, and the magnitude of activation was correlated with the punishment size. In addition, ventromedial prefrontal cortex and orbitofrontal cortex were activated when punishment was costly to the investor (punisher), suggesting that these regions might be involved in weighing the cost of punishment against its potential benefit.
50. Fehr E., and Rockenbach B. Human altruism: economic, neural, and evolutionary perspectives. Curr Opin Neurobiol 14 (2004) 784-790
51. Kosfeld M., Heinrichs M., Zak P.J., Fischbacher U., and Fehr E. Oxytocin increases trust in humans. Nature 435 (2005) 673-676. Oxytocin facilitates social attachment in animals. In this study, authors found that during a trust game, oxytocin increased the level of trust in the investors and hence the amount of money they transferred to the trustees, whereas the behavior of the trustees was not affected. This effect appears to be specific to social interaction, because oxytocin had no effect when the outcome of the investor's decision was determined randomly rather than by the trustee's decision.
52. Tobler P.N., Fiorillo C.D., and Schultz W. Adaptive coding of reward value by dopamine neurons. Science 307 (2005) 1642-1645. Midbrain dopamine neurons encode discrepancy between the predicted and the actual rewards, referred to as reward prediction error. Authors in this study found that the activity of dopamine neurons encodes the magnitude of reward prediction error in relative terms.
53. Bayer H.M., and Glimcher P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47 (2005) 129-141. In this study, the amount of reward was determined by the timing of the animal's eye movement relative to a particular time window. Authors found that dopamine neurons reliably code the magnitude of reward prediction error, but this was true only for the positive reward prediction error. Depression of dopamine activity was not related to the magnitude of negative prediction error.
54. O'Doherty J.P., Dayan P., Friston K., Critchley H., and Dolan R.J. Temporal difference models and reward-related learning in the human brain. Neuron 38 (2003) 329-337
55. McClure S.M., Berns G.S., and Montague P.R. Temporal prediction errors in a passive learning task activates human striatum. Neuron 38 (2003) 339-346
56. Fudenberg D., and Levine D.K. The Theory of Learning in Games (1998), MIT Press
57. Lee D., Conroy M.L., McGreevy B.P., and Barraclough D.J. Reinforcement learning and decision making in monkeys during a competitive game. Brain Res Cogn Brain Res 22 (2004) 45-58
58. Erev I., and Roth A.E. Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria. Am Econ Rev 88 (1998) 848-881
59. Feltovich N. Reinforcement-based vs. belief-based learning models in experimental asymmetric-information games. Econometrica 68 (2000) 605-641
60. Mookherjee D., and Sopher B. Learning and decision costs in experimental constant sum games. Games Econ Beh 19 (1997) 97-132
61. Lee D., McGreevy B.P., and Barraclough D.J. Learning and decision making in monkeys during a rock-paper-scissors game. Brain Res Cogn Brain Res 25 (2005) 416-430
62. Camerer C., and Ho T. Experience-weighted attraction learning in normal form games. Econometrica 67 (1999) 827-874
63. Bhatt M., and Camerer C.F. Self-referential thinking and equilibrium as states of mind in games: fMRI evidence. Games Econ Beh 52 (2005) 424-459. For a game-theoretic equilibrium, a player's choice, her expectation about another player's choice, and her expectation about the other player's expectation about her own choice, etc. must be consistent. This study examined how different brain regions might be involved in such various orders of belief. Subjects in this experiment performed three different tasks in which they simply selected their strategies, guessed what another player would do, or guessed what other players might think the subjects themselves would do. When choices and beliefs were in equilibrium, activity in ventral striatum increased when the subjects chose their strategy compared with when they simply guessed what the other player might do. By contrast, differential activity between choice and belief conditions was found in the paracingulate cortex for non-equilibrium trials, indicating a role of this region in mentalizing.
64. Povinelli D.J., and Vonk J. Chimpanzee minds: suspiciously human?. Trends Cogn Sci 7 (2003) 157-160
65. Tomasello M., Call J., and Hare B. Chimpanzees understand psychological states - the question is which ones and to what extent. Trends Cogn Sci 7 (2003) 153-156
66. Flombaum J.I., and Santos L.R. Rhesus monkeys attribute perceptions to others. Curr Biol 15 (2005) 447-452
67. Frith U., and Frith C.D. Development and neurophysiology of mentalizing. Philos Trans R Soc Lond B Biol Sci 358 (2003) 459-473
68. Gallagher H.L., and Frith C.D. Functional imaging of 'theory of mind'. Trends Cogn Sci 7 (2003) 77-83
69. McCabe K., Houser D., Ryan L., Smith V., and Trouard T. A functional imaging study of cooperation in two-person reciprocal exchange. Proc Natl Acad Sci USA 98 (2001) 11832-11835
70. Saxe R., and Wexler A. Making sense of another mind: the role of the right temporo parietal junction. Neuropsychologia 43 (2005) 1391-1399
71. Samejima K., Ueda Y., Doya K., and Kimura M. Representation of action-specific reward values in the striatum. Science 310 (2005) 1337-1340
72. Singer T., Seymour B., O'Doherty J.P., Stephan K.E., Dolan R.J., and Frith C.D. Empathic neural responses are modulated by the perceived fairness of others. Nature 439 (2006) 466-469