What the orbitofrontal cortex does not do

Nature Neuroscience

Volumn 18, Issue 5, 2015, Pages 620-627

  Stalnaker, Thomas A ^a   Cooch, Nisha K ^a   Schoenbaum, Geoffrey ^a  

^a NATIONAL INSTITUTES OF HEALTH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BRAIN FUNCTION; BRAIN MAPPING; EMOTION; HUMAN; NONHUMAN; ORBITAL CORTEX; PRIORITY JOURNAL; RESPONSE TIME; REVIEW; STIMULUS RESPONSE; ANIMAL; BIOLOGICAL MODEL; DECISION MAKING; INHIBITION (PSYCHOLOGY); LEARNING; PHYSIOLOGY; PREFRONTAL CORTEX; PRIMATE; PSYCHOLOGICAL MODEL; RAT; REWARD;

ANIMALS; BRAIN MAPPING; DECISION MAKING; EMOTIONS; HUMANS; INHIBITION (PSYCHOLOGY); JUDGMENT; LEARNING; MODELS, NEUROLOGICAL; MODELS, PSYCHOLOGICAL; PREFRONTAL CORTEX; PRIMATES; RATS; REWARD;

EID: 84928639374     PISSN: 10976256     EISSN: 15461726     Source Type: Journal    
DOI: 10.1038/nn.3982     Document Type: Review

References (150)

1. Popper, K. The Logic of Scientifc Discovery (Hutchinson & Co, 1959).
2. Noonan, M.P., Kolling, N., Walton, M.E. & Rushworth, M.F. Re-evaluating the role of the orbitofrontal cortex in reward and reinforcement. Eur. J. Neurosci. 35, 997-1010 (2012).
3. Wallis, J.D. Cross-species studies of orbitofrontal cortex and value-based decision-making. Nat. Neurosci. 15, 13-19 (2012).
4. Rudebeck, P.H. & Murray, E.A. Balkanizing the primate orbitofrontal cortex: distinct subregions for comparing and contrasting values. Ann. NY Acad. Sci. 1239, 1-13 (2011).
5. Ferrier, D. The Functions of the Brain (GP Putnam's Sons, New York, 1876).
6. Harlow, J.M. Recovery after passage of an iron bar through the head. Pub. Mass. Med. Soc. 2, 329-346 (1868).
7. Jones, B. & Mishkin, M. Limbic lesions and the problem of stimulus-reinforcement associations. Exp. Neurol. 36, 362-377 (1972).
8. Walton, M.E., Behrens, T.E.J., Buckley, M.J., Rudebeck, P.H. & Rushworth, M.F.S. Separable learning systems in teh macaque brain and the role of the orbitofrontal cortex in contingent learning. Neuron 65, 927-939 (2010).
9. Rolls, E.T., Hornak, J., Wade, D. & McGrath, J. Emotion-related learning in patients with social and emotional changes associated with frontal lobe damage. J. Neurol. Neurosurg. Psychiatry 57, 1518-1524 (1994).
10. Meunier, M., Bachevalier, J. & Mishkin, M. Effects of orbital frontal and anterior cingulate lesions on object and spatial memory in rhesus monkeys. Neuropsychologia 35, 999-1015 (1997).
11. Bechara, A., Damasio, H., Tranel, D. & Damasio, A.R. Deciding advantageously before knowing the advantageous strategy. Science 275, 1293-1295 (1997).
12. Izquierdo, A., Suda, R.K. & Murray, E.A. Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J. Neurosci. 24, 7540-7548 (2004).
13. Chudasama, Y. & Robbins, T.W. Dissociable contributions of the orbitofrontal and infralimbic cortex to pavlovian autoshaping and discrimination reversal learning: further evidence for the functional heterogeneity of the rodent frontal cortex. J. Neurosci. 23, 8771-8780 (2003).
14. Hornak, J. et al. Reward-related reversal learning after surgical excisions in orbito-frontal or dorsolateral prefrontal cortex in humans. J. Cogn. Neurosci. 16, 463-478 (2004).
15. Kim, J. & Ragozzino, K.E. The involvement of the orbitofrontal cortex in learning under changing task contingencies. Neurobiol. Learn. Mem. 83, 125-133 (2005).
16. Fellows, L.K. & Farah, M.J. Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. Brain 126, 1830-1837 (2003).
17. Tsuchida, A., Doll, B.B. & Fellows, L.K. Beyond reversal: a critical role for human orbitofrontal cortex in fexible learning from probabilistic feedback. J. Neurosci. 30, 16868-16875 (2010).
18. Riceberg, J.S. & Shapiro, M.L. Reward stability determines the contribution of orbitofrontal cortex to adaptive behavior. J. Neurosci. 32, 16402-16409 (2012).
19. Izquierdo, A. & Jentsch, J.D. Reversal learning as a measure of impulsive and compulsive behavior in addictions. Psychopharmacology (Berl.) 219, 607-620 (2012).
20. Kringelbach, M.L. & Rolls, E.T. The functional neuroanatomy of the human orbitofrontal cortex: evidence from neuroimaging and neuropsychology. Prog. Neurobiol. 72, 341-372 (2004).
21. Schoenbaum, G., Nugent, S., Saddoris, M.P. & Setlow, B. Orbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations. Neuroreport 13, 885-890 (2002).
22. Schoenbaum, G., Setlow, B., Nugent, S.L., Saddoris, M.P. & Gallagher, M. Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals. Learn. Mem. 10, 129-140 (2003).
23. Kazama, A. & Bachevalier, J. Selective aspiration or neurotoxic lesions of orbital frontal areas 11 and 13 spared monkeys′ performance on the object discrimination reversal task. J. Neurosci. 29, 2794-2804 (2009).
24. Rudebeck, P.H., Saunders, R.C., Prescott, A.T., Chau, L.S. & Murray, E.A. Prefrontal mechanisms of behavioral fexibility, emotion regulation and value updating. Nat. Neurosci. 16, 1140-1145 (2013).
25. Rudebeck, P.H. & Murray, E.A. Amygdala and orbitofrontal cortex lesions differentially infuence choices during object reversal learning. J. Neurosci. 28, 8338-8343 (2008).
26. Winstanley, C.A., Theobald, D.E.H., Cardinal, R.N. & Robbins, T.W. Contrasting roles of basolateral amygdala and orbitofrontal cortex in impulsive choice. J. Neurosci. 24, 4718-4722 (2004).
27. Chudasama, Y., Kralik, J.D. & Murray, E.A. Rhesus monkeys with orbital prefrontal cortex lesions can learn to inhibit prepotent responses in the reversed reward contingency task. Cereb. Cortex 17, 1154-1159 (2007).
28. Wallis, J.D. Orbitofrontal cortex and its contribution to decision-making. Annu. Rev. Neurosci. 30, 31-56 (2007).
29. Aron, A.R., Robbins, T.W. & Poldrack, R.A. Inibition and the right inferior frontal cortex: one decade on. Trends Cogn. Sci. 18, 177-185 (2014).
30. Rolls, E.T. The orbitofrontal cortex. Phil. Trans. R. Soc. Lond. B 351, 1433-1443 (1996).
31. Thorpe, S.J., Rolls, E.T. & Maddison, S. The orbitofrontal cortex: neuronal activity in the behaving monkey. Exp. Brain Res. 49, 93-115 (1983).
32. Critchley, H.D. & Rolls, E.T. Hunger and satiety modify the responses of olfactory and visual neurons in the primate orbitofrontal cortex. J. Neurophysiol. 75, 1673-1686 (1996).
33. Critchley, H.D. & Rolls, E.T. Olfactory neuronal responses in the primate orbitofrontal cortex: analysis in an olfactory discrimination task. J. Neurophysiol. 75, 1659-1672 (1996).
34. Rolls, E.T., Critchley, H.D., Mason, R. & Wakeman, E.A. Orbitofrontal cortex neurons: role in olfactory and visual association learning. J. Neurophysiol. 75, 1970-1981 (1996).
35. Cohen, J.Y., Haesler, S., Vong, L., Lowell, B.B. & Uchida, N. Neuron-type-specifc signals for reward and punishment in the ventral tegmental area. Nature (in the press) (2012).
36. Burke, K.A., Takahashi, Y.K., Correll, J., Brown, P.L. & Schoenbaum, G. Orbitofrontal inactivation impairs reversal of Pavlovian learning by interfering with 'disinhibition' of responding for previously unrewarded cues. Eur. J. Neurosci. 30, 1941-1946 (2009).
37. Takahashi, Y.K. et al. The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes. Neuron 62, 269-280 (2009).
38. Gallagher, M., McMahan, R.W. & Schoenbaum, G. Orbitofrontal cortex and representation of incentive value in associative learning. J. Neurosci. 19, 6610-6614 (1999).
39. Dias, R., Robbins, T.W. & Roberts, A.C. Dissociation in prefrontal cortex of affective and attentional shifts. Nature 380, 69-72 (1996).
40. McAlonan, K. & Brown, V.J. Orbital prefrontal cortex mediates reversal learning and not attentional set shifting in the rat. Behav. Brain Res. 146, 97-103 (2003).
41. Bissonette, G.B. et al. Double dissociation of the effects of medial and orbital prefrontal cortical lesions on attentional and affective shifts in mice. J. Neurosci. 28, 11124-11130 (2008).
42. Ostlund, S.B. & Balleine, B.W. Orbitofrontal cortex mediates outcome encoding in Pavlovian but not instrumental learning. J. Neurosci. 27, 4819-4825 (2007).
43. Izquierdo, A.D. & Murray, E.A. Bilateral orbital prefrontal cortex lesions disrupt reinforcer devaluation effects in rhesus monkeys. Soc. Neurosci. Abstr. 26, 978 (2000).
44. West, E.A., Forcelli, P.A., McCue, D.L. & Malkova, L. Differential effects of serotonin-specifc and excitotoxic lesions of OFC on conditioned reinforcer devaluation and extinction in rats. Behav. Brain Res. 246, 10-14 (2013).
45. Schoenbaum, G., Chiba, A.A. & Gallagher, M. Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. J. Neurosci. 19, 1876-1884 (1999).
46. Paton, J.J., Belova, M.A., Morrison, S.E. & Salzman, C.D. The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature 439, 865-870 (2006).
47. Morrison, S.E., Saez, A., Lau, B. & Salzman, C.D. Different time courses for learning-related changes in amygdala and orbitofrontal cortex. Neuron 71, 1127-1140 (2011).
48. Kennerley, S.W., Dahmubed, A.F., Lara, A.H. & Wallis, J.D. Neurons in the frontal lobe encode the value of multiple decision variables. J. Cogn. Neurosci. 21, 1162-1178 (2009).
49. Kennerley, S.W. & Wallis, J.D. Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables. Eur. J. Neurosci. 29, 2061-2073 (2009).
50. Kennerley, S.W., Behrens, T.E. & Wallis, J.D. Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat. Neurosci. 14, 181-1589 (2011).
51. Stalnaker, T.A., Roesch, M.R., Franz, T.M., Burke, K.A. & Schoenbaum, G. Abnormal associative encoding in orbitofrontal neurons in cocaine-experienced rats during decision-making. Eur. J. Neurosci. 24, 2643-2653 (2006).
52. Rescorla, R.A. Pavlovian conditioning: it's not what you think it is. Am. Psychol. 43, 151-160 (1988).
53. Lang, P.J. The varieties of emotional experience: a meditation on James-Lange theory. Psychol. Rev. 101, 211-221 (1994).
54. Maia, T.V. & McClelland, J.L. A reexamination of the evidence for the somatic marker hypothesis: what participants really know in the Iowa gambling task. Proc. Natl. Acad. Sci. USA 101, 16075-16080 (2004).
55. Bechara, A., Damasio, H., Damasio, A.R. & Lee, G.P. Different contributions of the human amygdala and ventromedial prefrontal cortex to decision-making. J. Neurosci. 19, 5473-5481 (1999).
56. Bechara, A., Tranel, D. & Damasio, H. Characterization of the decision-making defcit of patients with ventromedial prefrontal cortex lesions. Brain 123, 2189-2202 (2000).
57. Bechara, A. Neurobiology of decision-making: risk and reward. Semin. Clin. Neuropsychiatry 6, 205-216 (2001).
58. Bechara, A. et al. Decision-making defcits, linked to a dysfunctional ventromedial prefrontal cortex, revealed in alcohol and stimulant abusers. Neuropsychologia 39, 376-389 (2001).
59. Bechara, A. & Damasio, H. Decision-making and addiction (part I): impaired activation of somatic states in substance dependent individuals when pondering decisions with negative future consequences. Neuropsychologia 40, 1675-1689 (2002).
60. Pais-Vieira, M., Lima, D. & Galhardo, V. Orbitofrontal cortex lesions disrupt risk assessment in a novel serial decision-making task for rats. Neuroscience 145, 225-231 (2007).
61. Zeeb, F.D. & Winstanley, C.A. Lesions of the basolateral amygdala and the orbitofrontal cortex differentially affect acquisition and performance of a rodent gambling task. J. Neurosci. 31, 2197-2204 (2011).
62. Zeeb, F.D. & Winstanley, C.A. Functional disconnection of the orbitofrontal cortex and basolateral amygdala impairs acquisition of a rat gambling task and disrupts animals′ ability to alter decision-making behavior after reinforcer devaluation. J. Neurosci. 33, 6434-6443 (2013).
63. Fellows, L.K. & Farah, M.J. Different underlying impairments in decision-making following ventromedial and dorsolateral frontal lobe damage in humans. Cereb. Cortex 15, 58-63 (2005).
64. Fellows, L.K. The role of orbitofrontal cortex in decision making: a component process account. Ann. NY Acad. Sci. 1121, 421-430 (2007).
65. Daw, N.D., Niv, Y. & Dayan, P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704-1711 (2005).
66. Dayan, P., Niv, Y., Seymour, P. & Daw, N.D. The misbehavior of value and the discipline of the will. Neural Netw. 19, 1153-1160 (2006).
67. Huys, Q.J. et al. Bonsai trees in your head: how the Pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Comput. Biol. 8, e1002410 (2012).
68. McDannald, M.A. et al. Model-based learning and the contribution of the orbitofrontal cortex to the model-free world. Eur. J. Neurosci. 35, 991-996 (2012).
69. Bechara, A., Dolan, S. & Hindes, A. Decision-making and addiction (part II): myopia for the future or hypersensitivity to reward? Neuropsychologia 40, 1690-1705 (2002).
70. O'Doherty, J.P. The problem with value. Neurosci. Biobehav. Rev. 43, 259-268 (2014).
71. Pearson, J.M., Watson, K.K. & Platt, M.L. Decision making: the neuroethological turn. Neuron 82, 950-965 (2014).
72. Montague, P.R. & Berns, G.S. Neural economics and the biological substrates of valuation. Neuron 36, 265-284 (2002).
73. Padoa-Schioppa, C. Neurobiology of economic choice: a goods-based model. Annu. Rev. Neurosci. 34, 333-359 (2011).
74. Levy, D.J. & Glimcher, P.W. The root of all value: a neural common currency for choice. Curr. Opin. Neurobiol. 22, 1027-1038 (2012).
75. Padoa-Schioppa, C. & Assad, J.A. Neurons in orbitofrontal cortex encode economic value. Nature 441, 223-226 (2006).
76. Padoa-Schioppa, C. & Assad, J.A. The representation of economic value in the orbitofrontal cortex is invariant for changes in menu. Nat. Neurosci. 11, 95-102 (2008).
77. Padoa-Schioppa, C. Range-adapting representation of economic value in the orbitofrontal cortex. J. Neurosci. 29, 14004-14014 (2009).
78. Plassmann, H., O'Doherty, J. & Rangel, A. Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. J. Neurosci. 27, 9984-9988 (2007).
79. McNamee, D., Rangel, A. & O'Doherty, J.P. Category-dependent and category-independent goal-value codes in human ventromedial prefrontal cortex. Nat. Neurosci. 16, 479-485 (2013).
80. Levy, D.J. & Glimcher, P.W. Comparing apples and oranges: using reward-specifc and reward-general subjective value representation in the brain. J. Neurosci. 31, 14693-14707 (2011).
81. Lebreton, M., Jorge, S., Michel, V., Thirion, B. & Pessiglione, M. An automatic valuation system in the human brain: evidence from functional neuroimaging. Neuron 64, 431-439 (2009).
82. Padoa-Schioppa, C. Neuronal origins of choice variability in economic decisions. Neuron 80, 1322-1336 (2013).
83. McDannald, M.A. et al. Orbitofrontal neurons acquire responses to ′valueless′ Pavlovian cues during unblocking. Elife 3, e02653 (2014).
84. Stalnaker, T.A. et al. Orbitofrontal neurons infer the value and identity of predicted outcomes. Nat. Commun. 5, 3926 (2014).
85. Knutson, B. & Gibbs, S.E.B. Linking nucleus accumbens dopamine and blood oxygenation. Psychopharmacology (Berl.) 191, 813-822 (2007).
86. Maunsell, J.H. Neuronal representations of cognitive state: reward or attention? Trends Cogn. Sci. 8, 261-265 (2004).
87. Pearce, J.M., Kaye, H. & Hall, G. Predictive accuracy and stimulus associability: development of a model for Pavlovian learning. in Quantitative Analyses of Behavior (eds. M.L. Commons, R.J. Herrnstein & A.R. Wagner) 241-255 (Ballinger, 1982).
88. Esber, G.R. & Haselgrove, M. Reconciling the infuence of predictiveness and uncertainty on stimulus salience: a model of attention in associative learning. Proc. Biol. Sci. 278, 2553-2561 (2011).
89. Ogawa, M. et al. Risk-responsive orbitofrontal neurons track acquired salience. Neuron 77, 251-258 (2013).
90. O'Neill, M. & Schultz, W. Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value. Neuron 68, 789-800 (2010).
91. Kepecs, A., Uchida, N., Zariwala, H.A. & Mainen, Z.F. Neural correlates, computation and behavioural impact of decision confdence. Nature 455, 227-231 (2008).
92. Leathers, M.L. & Olson, C.R. In monkeys making value-based decisions, LIP neurons encode cue salience and not action value. Science 338, 132-135 (2012).
93. Camille, N., Griffths, C.A., Vo, K., Fellows, L.K. & Kable, J.W. Ventromedial frontal lobe damage disrupts value maximization in humans. J. Neurosci. 31, 7527-7532 (2011).
94. Fellows, L.K. & Farah, M.J. The role of ventromedial prefrontal cortex in decision making: judgment under uncertainty or judgment per se? Cereb. Cortex 17, 2669-2674 (2007).
95. Rudebeck, P.H. & Murray, E.A. Dissociable effects of subtotal lesions within the macaque orbital prefrontal cortex on reward-guided behavior. J. Neurosci. 31, 10569-10578 (2011).
96. von Neumann, J. & Morgenstern, O. Theory of Games and Economic Behavior (Princeton University Press, 1947).
97. Bunsey, M. & Eichenbaum, E. Conservation of hippocampal memory function in rats and humans. Nature 379, 255-257 (1996).
98. Burke, K.A., Franz, T.M., Miller, D.N. & Schoenbaum, G. The role of the orbitofrontal cortex in the pursuit of happiness and more specifc rewards. Nature 454, 340-344 (2008).
99. Platt, M.L. & Glimcher, P.W. Neural correlates of decision variables in parietal cortex. Nature 400, 233-238 (1999).
100. Louie, K., Grattan, L.E. & Glimcher, P.W. Reward value-based gain control: divisive normalization in parietal cortex. J. Neurosci. 31, 10627-10639 (2011).
101. Cai, X. & Padoa-Schioppa, C. Neuronal encoding of subjective value in dorsal and ventral anterior cingulate cortex. J. Neurosci. 32, 3791-3808 (2012).
102. Hosokawa, T., Kennerley, S.W., Sloan, J. & Wallis, J.D. Single-neuron mechanisms underlying cost-beneft analysis in frontal cortex. J. Neurosci. 33, 17385-17397 (2013).
103. Watson, K.K. & Platt, M.L. Social signals in primate orbitofrontal cortex. Curr. Biol. 22, 2268-2273 (2012).
104. Roesch, M.R., Taylor, A.R. & Schoenbaum, G. Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation. Neuron 51, 509-520 (2006).
105. Blanchard, T.C., Hayden, B.Y. & Bromberg-Martin, E.S. Orbitofrontal cortex uses distinct codes for different choice attributes in decisions motivated by curiousity. Neuron (in the press).
106. Schoenbaum, G., Takahashi, Y.K., Liu, T.L. & McDannald, M. Does the orbitofrontal cortex signal value? Ann. NY Acad. Sci. 1239, 87-99 (2011).
107. Gremel, C.M. & Costa, R.M. Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions. Nat. Commun. 4, 2264 (2013).
108. McDannald, M.A., Lucantonio, F., Burke, K.A., Niv, Y. & Schoenbaum, G. Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. J. Neurosci. 31, 2700-2705 (2011).
109. McDannald, M.A., Saddoris, M.P., Gallagher, M. & Holland, P.C. Lesions of orbitofrontal cortex impair rats' differential outcome expectancy learning but not conditioned stimulus-potentiated feeding. J. Neurosci. 25, 4626-4632 (2005).
110. Machado, C.J. & Bachevalier, J. The effects of selective amygdala, orbital frontal cortex or hippocampal formation lesions on reward assessment in nonhuman primates. Eur. J. Neurosci. 25, 2885-2904 (2007).
111. Gottfried, J.A., O'Doherty, J. & Dolan, R.J. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 301, 1104-1107 (2003).
112. O'Doherty, J. et al. Sensory-specifc satiety-related olfactory activation of the human orbitofrontal cortex. Neuroreport 11, 893-897 (2000).
113. Pickens, C.L., Saddoris, M.P., Gallagher, M. & Holland, P.C. Orbitofrontal lesions impair use of cue-outcome associations in a devaluation task. Behav. Neurosci. 119, 317-322 (2005).
114. West, E.A., DesJardin, J.T., Gale, K. & Malkova, L. Transient inactivation of orbitofrontal cortex blocks reinforcer devaluation in macaques. J. Neurosci. 31, 15128-15135 (2011).
115. Lak, A. et al. Orbitofrontal cortex is required for optimal waiting based on decision confdence. Neuron 84, 190-201 (2014).
116. Abe, H. & Lee, D. Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex. Neuron 70, 731-741 (2011).
117. Steiner, A.P. & Redish, A.D. Behavioral and neurophysiological correlates of regret in rat decision-making on a neuroeconomic task. Nat. Neurosci. 17, 995-1002 (2014).
118. Camille, N. et al. The involvement of the orbitofrontal cortex in the experience of regret. Science 304, 1167-1170 (2004).
119. Jones, J.L. et al. Orbitofrontal cortex supports behavior and learning using inferred but not cached values. Science 338, 953-956 (2012).
120. Rescorla, R.A. & Wagner, A.R. A theory of Pavlovian conditiong: variations in the effectiveness of reinforcement and nonreinforcement. in. Classical Conditioning II: Current Research and Theory (eds. A.H. Black & W.F. Prokesy) 64-99 (Appleton Century Crofts 1972).
121. Pearce, J.M. & Hall, G. A model for Pavlovian learning: variations in the effectiveness of conditioned, but not of unconditioned, stimuli. Psychol. Rev. 87, 532-552 (1980).
122. Sutton, R.S. Learning to predict by the method of temporal difference. Mach. Learn. 3, 9-44 (1988).
123. Mirenowicz, J. & Schultz, W. Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol. 72, 1024-1027 (1994).
124. Schultz, W., Dayan, P. & Montague, P.R. A neural substrate for prediction and reward. Science 275, 1593-1599 (1997).
125. Waelti, P., Dickinson, A. & Schultz, W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412, 43-48 (2001).
126. Pan, W.-X., Schmidt, R., Wickens, J.R. & Hyland, B.I. Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J. Neurosci. 25, 6235-6242 (2005).
127. Roesch, M.R., Calu, D.J. & Schoenbaum, G. Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat. Neurosci. 10, 1615-1624 (2007).
128. Bayer, H.M. & Glimcher, P. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 47, 129-141 (2005).
129. Morris, G., Nevet, A., Arkadir, D., Vaadia, E. & Bergman, H. Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci. 9, 1057-1063 (2006).
130. D'Ardenne, K., McClure, S.M., Nystrom, L.E. & Cohen, J.D. BOLD responses refecting dopaminergic signals in the human ventral tegmental area. Science 319, 1264-1267 (2008).
131. Schultz, W. & Dickinson, A. Neuronal coding of prediction errors. Annu. Rev. Neurosci. 23, 473-500 (2000).
132. Tobler, P.N., O'Doherty, J., Dolan, R.J. & Schultz, W. Human neural learning depends on reward prediction errors in the blocking paradigm. J. Neurophysiol. 95, 301-310 (2006).
133. Nobre, A.C., Coull, J.T., Frith, C.D. & Mesulam, M.M. Orbitofrontal cortex is activated during breaches of expectation in tasks of visual attention. Nat. Neurosci. 2, 11-12 (1999).
134. Sul, J.H., Kim, H., Huh, N., Lee, D. & Jung, M.W. Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making. Neuron 66, 449-460 (2010).
135. Wallis, J.D. & Rich, E.L. Challenges of interpreting frontal neurons during value-based decision-making. Front. Neurosci. 5, 124 (2011).
136. Takahashi, Y.K. et al. Neural estimates of imagined outcomes in the orbitofrontal cortex drive behavior and learning. Neuron 80, 507-518 (2013).
137. Schoenbaum, G., Roesch, M.R., Stalnaker, T.A. & Takahashi, Y.K. A new perspective on the role of the orbitofrontal cortex in adaptive behaviour. Nat. Rev. Neurosci. 10, 885-892 (2009).
138. Takahashi, Y.K. et al. Expectancy-related changes in fring of dopamine neurons depend on orbitofrontal cortex. Nat. Neurosci. 14, 1590-1597 (2011).
139. Wheeler, E.Z. & Fellows, L.K. The human ventromedial frontal lobe is critical for learning from negative feedback. Brain 131, 1323-1331 (2008).
140. Walton, M.E., Behrens, T.E., Noonan, M.P. & Rushworth, M.F. Giving credit where credit is due: orbitofrontal cortex and valuation in an uncertain world. Ann. NY Acad. Sci. 1239, 14-24 (2011).
141. Thorndike, E.L. A proof of the law of effect. Science 77, 173-175 (1933).
142. Tolman, E.C. There is more than one kind of learning. Psychol. Rev. 56, 144-155 (1949).
143. Wilson, R.C., Takahashi, Y.K., Schoenbaum, G. & Niv, Y. Orbitofrontal cortex as a cognitive map of task space. Neuron 81, 267-279 (2014).
144. Deacon, T.W., Eichenbaum, H., Rosenberg, P. & Eckmann, K.W. Afferent connections of the perirhinal cortex in the rat. J. Comp. Neurol. 220, 168-190 (1983).
145. Voorn, P., Vanderschuren, L.J.M.J., Groenewegen, H.J., Robbins, T.W. & Pennartz, C.M.A. Putting a spin on the dorsal-ventral divide of the striatum. Trends Neurosci. 27, 468-474 (2004).
146. Groenewegen, H.J., Berendse, H.W., Wolters, J.G. & Lohman, A.H.M. The anatomical relationship of the prefrontal cortex with the striatopallidal system, the thalamus and the amygdala: evidence for a parallel organization. Prog. Brain Res. 85, 95-116 (1990).
147. Klein-Flügge, M.C., Barron, H.C., Brodersen, K.H., Dolan, R.J. & Behrens, T.E. Segregated encoding of reward-identity and stimulus-reward associations in human orbitofrontal cortex. J. Neurosci. 33, 3202-3211 (2013).
148. Schoenbaum, G. & Eichenbaum, H. Information coding in the rodent prefrontal cortex. I. Single-neuron activity in orbitofrontal cortex compared with that in pyriform cortex. J. Neurophysiol. 74, 733-750 (1995).
149. Ongür, D. & Price, J.L. The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. Cereb. Cortex 10, 206-219 (2000).
150. Schoenbaum, G., Setlow, B. & Gallagher, M. Orbitofrontal cortex: modeling prefrontal function in rats. in The Neuropsychology of Memory (eds. L. Squire & D. Schacter) 463-477 (Guilford Press, 2002).