Dialogue on economic choice, learning theory, and neuronal representations

Current Opinion in Behavioral Sciences

Volumn 5, Issue , 2015, Pages 16-23

  Padoa Schioppa, Camillo ^a   Schoenbaum, Geoffrey ^b  

^a WASHINGTON UNIVERSITY   (United States)

^b NATIONAL INSTITUTE ON DRUG ABUSE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ASSOCIATION; ATTITUDE; BRAIN REGION; CONDITIONING; COST BENEFIT ANALYSIS; DECISION MAKING; ECONOMIC CHOICE; ECONOMICS; HUMAN; LATERAL PREFRONTAL CORTEX; LEARNING THEORY; NERVE CELL; NEUROBIOLOGY; NEUROSCIENCE; NONHUMAN; ORBITAL CORTEX; PUNISHMENT; REINFORCEMENT; REVIEW; STATE DEPENDENT LEARNING; TASK PERFORMANCE;

EID: 84955255201     PISSN: None     EISSN: 23521546     Source Type: Journal    
DOI: 10.1016/j.cobeha.2015.06.004     Document Type: Review

References (61)

1. Arana FS, Parkinson JA, Hinton E, Holland AJ, Owen AM, Roberts AC: Dissociable contributions of the human amygdala and orbitofrontal cortex to incentive motivation and goal selection. J Neurosci 2003, 23:9632-9638.
2. Roesch MR, Olson CR: Neuronal activity in primate orbitofrontal cortex reflects the value of time. J Neurophysiol 2005, 94:2457-2471.
3. Padoa-Schioppa C, Assad JA: Neurons in orbitofrontal cortex encode economic value. Nature 2006, 441:223-226.
4. Kable JW, Glimcher PW: The neural correlates of subjective value during intertemporal choice. Nat Neurosci 2007, 10:1625-1633.
5. Camille N, Griffiths CA, Vo K, Fellows LK, Kable JW: Ventromedial frontal lobe damage disrupts value maximization in humans. J Neurosci 2011, 31:7527-7532.
6. Gallagher M, McMahan RW, Schoenbaum G: Orbitofrontal cortex and representation of incentive value in associative learning. J Neurosci 1999, 19:6610-6614.
7. Gottfried JA, O'Doherty J, Dolan RJ: Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 2003, 301:1104-1107.
8. Rudebeck PH, Saunders RC, Prescott AT, Chau LS, Murray EA: Prefrontal mechanisms of behavioral flexibility, emotion regulation and value updating. Nat Neurosci 2013, 16:1140-1145.
9. Jones JL, Esber GR, McDannald MA, Gruber AJ, Hernandez G, Mirenzi A, Schoenbaum G: Orbitofrontal cortex supports behavior and learning using inferred but not cached values. Science 2012, 338:953-956.
10. McDannald MA, Lucantonio F, Burke KA, Niv Y, Schoenbaum G: Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. J Neurosci 2011, 31:2700-2705.
11. Padoa-Schioppa C: Neurobiology of economic choice: a goodbased model. Annu Rev Neurosci 2011, 34:333-359.
12. Kreps DM: A Course in Microeconomic Theory. Princeton, NJ: Princeton University Press; 1990.
13. Wallis JD: Cross-species studies of orbitofrontal cortex and value-based decision-making. Nat Neurosci 2012, 15:13-19.
14. Bartra O, McGuire JT, Kable JW: The valuation system: a coordinate-based meta-analysis of BOLD fMRI experiments examining neural correlates of subjective value. Neuroimage 2013, 76:412-427.
15. Clithero JA, Rangel A: Informatic parcellation of the network involved in the computation of subjective value. Soc Cogn Affect Neurosci 2013.
16. Morrison SE, Salzman CD: Representations of appetitive and aversive information in the primate orbitofrontal cortex. Ann N Y Acad Sci 2011, 1239:59-70.
17. Bendesky A, Tsunozaki M, Rockman MV, Kruglyak L, Bargmann CI: Catecholamine receptor polymorphisms affect decision-making in C. elegans. Nature 2011, 472:313-318.
18. Yang CH, Belawat P, Hafen E, Jan LY, Jan YN: Drosophila egglaying site selection as a system to study simple decisionmaking processes. Science 2008, 319:1679-1683.
19. Conradt L, Roper TJ: Consensus decision making in animals. Trends Ecol Evol 2005, 20:449-456.
20. Berns GS, Bell E, Capra CM, Prietula MJ, Moore S, Anderson B, Ginges J, Atran S: The price of your soul: neural evidence for the non-utilitarian representation of sacred values. Philos Trans R Soc Lond B Biol Sci 2012, 367:754-762.
21. Grattan LE, Glimcher PW: Absence of spatial tuning in the orbitofrontal cortex. PLOS ONE 2014, 9:e112750.
22. Platt ML, Glimcher PW: Neural correlates of decision variables in parietal cortex. Nature 1999, 400:233-238.
23. Louie K, Glimcher PW: Separating value from choice: delay discounting activity in the lateral intraparietal area. J Neurosci 2010, 30:5498-5507.
24. Roesch MR, Olson CR: Neuronal activity dependent on anticipated and elapsed delay in macaque prefrontal cortex, frontal and supplementary eye fields, and premotor cortex. J Neurophysiol 2005, 94:1469-1497.
25. Abe H, Lee D: Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex. Neuron 2011, 70:731-741.
26. Tsujimoto S, Genovesio A, Wise SP: Monkey orbitofrontal cortex encodes response choices near feedback time. J Neurosci 2009, 29:2569-2574.
27. Padoa-Schioppa C, Cai X: The orbitofrontal cortex and the computation of subjective value: consolidated concepts and new perspectives. Ann N Y Acad Sci 2011, 1239:130-137.
28. Padoa-Schioppa C: Neuronal origins of choice variability in economic decisions. Neuron 2013, 80:1322-1336.
29. Raghuraman AP, Padoa-Schioppa C: Integration of multiple determinants in the neuronal computation of economic values. J Neurosci 2014, 34:11583-11603.
30. Cai X, Padoa-Schioppa C: Neuronal encoding of subjective value in dorsal and ventral anterior cingulate cortex. J Neurosci 2012, 32:3791-3808.
31. Kim S, Hwang J, Lee D: Prefrontal coding of temporally discounted values during intertemporal choice. Neuron 2008, 59:161-172.
32. Cai X, Kim S, Lee D: Heterogeneous coding of temporally discounted values in the dorsal and ventral striatum during intertemporal choice. Neuron 2011, 69:170-182.
33. Pearce JM, Bouton ME: Theories of associative learning in animals. Annu Rev Psychol 2001, 52:111-139.
34. Rescorla RA, Wagner AR: A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In Classical Conditioning II: Current Research and Theory. Edited by Black AH, Prokasy WF. Appleton-Century-Crofts; 1972:64-99.
35. Mackintosh NJ: A theory of attention: variations in the associability of stimuli with reinforcement. Psychol Rev 1975, 82:276-298.
36. Pearce JM, Hall G: A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev 1980, 87:532-552.
37. Wagner AR: SOP: a model of automatic memory processing in animal behavior. In Information Processing in Animals: Memory Mechanisms. Edited by Spear NE, Miller RR. Erlbaum; 1981:5-47.
38. Wagner AR, Brandon SE: A componential theory of associative learning. In Contemporary Learning: Theory and Applications. Edited by Mowrer RR, Klein SB. Erlbaum; 2001.
39. McLaren IPL, Mackintosh NJ: Associative learning and elemental representation: II. Generalization and discrimination. Anim Learn Behav 2002, 30:177-200.
40. McLaren IPL, Mackintosh NJ: An elemental model of associative learning: I. Latent inhibition and perceptual learning. Anim Learn Behav 2000, 28:211-246.
41. Esber GR, Haselgrove M: Reconciling the influence of predictiveness and uncertainty on stimulus salience: a model of attention in associative learning. Proc R Soc B 2011, 278:2553-2561.
42. Sutton RS, Barto AG: Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press; 1998.
43. O'Doherty JP: The problem with value. Neurosci Biobehav Rev 2014, 43:259-268.
44. Dickinson A, Balleine BW: Motivational control of goal-directed action. Anim Learn Behav 1994, 22:1-18.
45. Rescorla RA: Pavlovian conditioning: it's not what you think it is. Am Psychol 1988, 43:151-160.
46. Holland PC, Straub JJ: Differential effects of two ways of devaluing the unconditioned stimulus after Pavlovian appetitive conditioning. J Exp Psychol Anim Behav Process 1979, 5:65-78.
47. Colwill RM: An associative analysis of instrumental learning. Curr Dir Psychol Sci 1993, 2:111-116.
48. Yin HH, Knowlton BJ, Balleine BW: Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur J Neurosci 2004, 19:181-189.
49. Yin HH, Ostlund SB, Knowlton BJ, Balleine BW: The role of the dorsomedial striatum in instrumental conditioning. Eur J Neurosci 2005, 22:513-523.
50. Bradfield LA, Bertran-Gonzalez J, Chieng B, Balleine BW: The thalamo-striatal pathway and cholinergic control of goal-directed action: interlacing new and existing actionoutcome associations in the striatum. Neuron 2013, 79:153-166.
51. Valentin VV, Dickinson A, O'doherty JP: Determining the neural substrates of goal-directed learning in the human brain. J Neurosci 2007, 27:4019-4026.
52. Levy I, Snell J, Nelson AJ, Rustichini A, Glimcher PW: Neural representation of subjective value under risk and ambiguity. J Neurophysiol 2010, 103:1036-1047.
53. Plassmann H, O'Doherty J, Rangel A: Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. J Neurosci 2007, 27:9984-9988.
54. Rolls ET, Sienkiewicz ZJ, Yaxley S: Hunger modulates the responses to gustatory stimuli of single neurons in the caudolateral orbitofrontal cortex of the macaque monkey. Eur J Neurosci 1989, 1:53-60.
55. Izquierdo A, Murray EA: Combined unilateral lesions of the amygdala and orbital prefrontal cortex impair affective processing in rhesus monkeys. J Neurophysiol 2004, 91:2023-2039.
56. Jones JL, Esber GR, McDannald MA, Gruber AJ, Hernandez A, Mirenzi A, Schoenbaum G: Orbitofrontal cortex supports behavior and learning using inferred but not cached values. Science 2012, 338:953-956.
57. Rhodes SE, Murray EA: Differential effects of amygdala, orbital prefrontal cortex, and prelimbic cortex lesions on goal-directed behavior in rhesus macaques. J Neurosci 2013, 33:3380-3389.
58. West EA, DesJardin JT, Gale K, Malkova L: Transient inactivation of orbitofrontal cortex blocks reinforcer devaluation in macaques. J Neurosci 2011, 31:15128-15135.
59. Gremel CM, Costa RM: Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions. Nat Commun 2013, 4:2264.
60. Rudebeck PH, Murray EA: Dissociable effects of subtotal lesions within the macaque orbital prefrontal cortex on reward-guided behavior. J Neurosci 2011, 31:10569-10578.
61. Rescorla RA: Pavlovian conditioning. It's not what you think it is. Am Psychol 1988, 43:151-160.