Orbitofrontal cortex as a cognitive map of task space

Neuron

Volumn 81, Issue 2, 2014, Pages 267-279

  Wilson, Robert C ^a   Takahashi, Yuji K ^b   Schoenbaum, Geoffrey ^b   Niv, Yael ^a  

^a PRINCETON UNIVERSITY   (United States)

^b UNIVERSITY OF MARYLAND SCHOOL OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; BRAIN FUNCTION; DECISION MAKING; DELAYED ALTERNATION; DEVALUATION; DOPAMINERGIC NERVE CELL; NERVOUS SYSTEM PARAMETERS; ORBITAL CORTEX; PRIORITY JOURNAL; REINFORCEMENT; STATE DEPENDENT LEARNING; TASK PERFORMANCE; VENTRAL TEGMENTUM;

ACTION POTENTIALS; ANIMALS; COGNITION; DECISION MAKING; DOPAMINERGIC NEURONS; HUMANS; LEARNING; MODELS, PSYCHOLOGICAL; PREFRONTAL CORTEX; REINFORCEMENT (PSYCHOLOGY); VENTRAL TEGMENTAL AREA;

EID: 84892714870     PISSN: 08966273     EISSN: 10974199     Source Type: Journal    
DOI: 10.1016/j.neuron.2013.11.005     Document Type: Article

References (99)

1. Balleine B.W., Dickinson A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 1998, 37:407-419.
2. Banich M.T., Milham M.P., Atchley R.A., Cohen N.J., Webb A., Wszalek T., Kramer A.F., Liang Z., Barad V., Gullett D., et al. Prefrontal regions play a predominant role in imposing an attentional 'set': evidence from fMRI. Brain Res. Cogn. Brain Res. 2000, 10:1-9.
3. Bohn I., Giertler C., Hauber W. Orbital prefrontal cortex and guidance of instrumental behaviour in rats under reversal conditions. Behav. Brain Res. 2003, 143:49-56.
4. Bouton M.E. Context and behavioral processes in extinction. Learn. Mem. 2004, 11:485-494.
5. Brass M., Derrfuss J., von Cramon Y. The Role of the Posterior Frontolateral Cortex in Task Related Control. Neuroscience of Rule-Guided Behavior 2008, Oxford University Press, New York. S.A. Bunge, J.D. Wallis (Eds.).
6. Brown T.I., Ross R.S., Keller J.B., Hasselmo M.E., Stern C.E. Which way was I goingα Contextual retrieval supports the disambiguation of well learned overlapping navigational routes. J. Neurosci. 2010, 30:7414-7422.
7. Buckley M.J., Mansouri F.A., Hoda H., Mahboubi M., Browning P.G.F., Kwok S.C., Phillips A., Tanaka K. Dissociable components of rule-guided behavior depend on distinct medial and prefrontal regions. Science 2009, 325:52-58.
8. Butter C.M. Perseveration in extinction and in discrimination reversal tasks following selective frontal ablations in macaca mulatta. Physiol. Behav. 1969, 4:163-171.
9. Butter C.M., Mishkin M., Rosvold H.E. Conditioning and extinction of a food-rewarded response after selective ablations of frontal cortex in rhesus monkeys. Exp. Neurol. 1963, 7:65-75.
10. Butters N., Butter C., Rosen J., Stein D. Behavioral effects of sequential and one-stage ablations of orbital prefrontal cortex in the monkey. Exp. Neurol. 1973, 39:204-214.
11. Carmichael S.T., Price J.L. Limbic connections of the orbital and medial prefrontal cortex in macaque monkeys. J. Comp. Neurol. 1995, 363:615-641.
12. Carmichael S.T., Price J.L. Sensory and premotor connections of the orbital and medial prefrontal cortex of macaque monkeys. J. Comp. Neurol. 1995, 363:642-664.
13. Carmichael S.T., Price J.L. Connectional networks within the orbital and medial prefrontal cortex of macaque monkeys. J. Comp. Neurol. 1996, 371:179-207.
14. Chudasama Y., Robbins T.W. Dissociable contributions of the orbitofrontal and infralimbic cortex to pavlovian autoshaping and discrimination reversal learning: further evidence for the functional heterogeneity of the rodent frontal cortex. J. Neurosci. 2003, 23:8771-8780.
15. Colwill R.M., Rescorla R.A. Postconditioning devaluation of a reinforcer affects instrumental responding. J. Exp. Psychol. 1985, 11:120-132.
16. Critchley H.D., Rolls E.T. Hunger and satiety modify the responses of olfactory and visual neurons in the primate orbitofrontal cortex. J. Neurophysiol. 1996, 75:1673-1686.
17. Critchley H.D., Rolls E.T. Olfactory neuronal responses in the primate orbitofrontal cortex: analysis in an olfactory discrimination task. J. Neurophysiol. 1996, 75:1659-1672.
18. Croxson P.L., Johansen-Berg H., Behrens T.E.J., Robson M.D., Pinsk M.A., Gross C.G., Richter W., Richter M.C., Kastner S., Rushworth M.F. Quantitative investigation of connections of the prefrontal cortex in the human and macaque using probabilistic diffusion tractography. J. Neurosci. 2005, 25:8854-8866.
19. Damasio A.R. Descartes' error: Emotion, reason, and the human brain 1994, Putnam, New York.
20. Daw N.D., Niv Y., Dayan P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 2005, 8:1704-1711.
21. de Araujo I.E., Gutierrez R., Oliveira-Maia A.J., Pereira A., Nicolelis M.A.L., Simon S.A. Neural ensemble coding of satiety states. Neuron 2006, 51:483-494.
22. Dias R., Robbins T.W., Roberts A.C. Dissociation in prefrontal cortex of affective and attentional shifts. Nature 1996, 380:69-72.
23. Feierstein C.E., Quirk M.C., Uchida N., Sosulski D.L., Mainen Z.F. Representation of spatial goals in rat orbitofrontal cortex. Neuron 2006, 51:495-507.
24. Ferrier D. The functions of the brain 1876, G. P. Putnam's Sons, New York.
25. Furuyashiki T., Holland P.C., Gallagher M. Rat orbitofrontal cortex separately encodes response and outcome information during performance of goal-directed behavior. J. Neurosci. 2008, 28:5127-5138.
26. Fuster J.M. The prefrontal cortex 1997, Lippin-Ravencott, New York.
27. Gallagher M., McMahan R.W., Schoenbaum G. Orbitofrontal cortex and representation of incentive value in associative learning. J. Neurosci. 1999, 19:6610-6614.
28. Gershman S.J., Niv Y. Learning latent structure: carving nature at its joints. Curr. Opin. Neurobiol. 2010, 20:251-256.
29. Gershman S.J., Niv Y. Perceptual estimation obeys Occam's razor. Front. Psychol. 2013, 4:623.
30. Gershman S.J., Blei D.M., Niv Y. Context, learning, and extinction. Psychol. Rev. 2010, 117:197-209.
31. Gottfried J.A., O'Doherty J., Dolan R.J. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 2003, 301:1104-1107.
32. Hampton A.N., Bossaerts P., O'Doherty J.P. The role of the ventromedial prefrontal cortex in abstract state-based inference during decision making in humans. J. Neurosci. 2006, 26:8360-8367.
33. Izquierdo A., Suda R.K., Murray E.A. Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J. Neurosci. 2004, 24:7540-7548.
34. Ji J., Maren S. Hippocampal involvement in contextual modulation of fear extinction. Hippocampus 2007, 17:749-758.
35. Jones B., Mishkin M. Limbic lesions and the problem of stimulus-reinforcement associations. Exp. Neurol. 1972, 36:362-377.
36. Jones J.L., Esber G.R., McDannald M.A., Gruber A.J., Hernandez A., Mirenzi A., Schoenbaum G. Orbitofrontal cortex supports behavior and learning using inferred but not cached values. Science 2012, 338:953-956.
37. Kaelbling L.P., Littman M.I., Cassandra A.R. Planning and acting in partially observable stochastic domains. Artif. Intell. 1998, 101:99-134.
38. Kahnt T., Chang L.J., Park S.Q., Heinzle J., Haynes J.-D. Connectivity-based parcellation of the human orbitofrontal cortex. J. Neurosci. 2012, 32:6240-6250.
39. Kepecs A., Uchida N., Zariwala H.A., Mainen Z.F. Neural correlates, computation and behavioural impact of decision confidence. Nature 2008, 455:227-231.
40. Kim J., Ragozzino M.E. The involvement of the orbitofrontal cortex in learning under changing task contingencies. Neurobiol. Learn. Mem. 2005, 83:125-133.
41. Lara A.H., Kennerley S.W., Wallis J.D. Encoding of gustatory working memory by orbitofrontal neurons. J. Neurosci. 2009, 29:765-774.
42. MacDonald A.W., Cohen J.D., Stenger V.A., Carter C.S. Dissociating the role of the dorsolateral prefrontal and anterior cingulate cortex in cognitive control. Science 2000, 288:1835-1838.
43. McAlonan K., Brown V.J. Orbital prefrontal cortex mediates reversal learning and not attentional set shifting in the rat. Behav. Brain Res. 2003, 146:97-103.
44. McDannald M.A., Lucantonio F., Burke K.A., Niv Y., Schoenbaum G. Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. J. Neurosci. 2011, 31:2700-2705.
45. McEnaney K.W., Butter C.M. Perseveration of responding and nonresponding in monkeys with orbital frontal ablations. J. Comp. Physiol. Psychol. 1969, 68:558-561.
46. Meunier M., Bachevalier J., Mishkin M. Effects of orbital frontal and anterior cingulate lesions on object and spatial memory in rhesus monkeys. Neuropsychologia 1997, 35:999-1015.
47. Miller M.H., Orbach J. Retention of spatial alternation following frontal lobe resections in stump-tailed macaques. Neuropsychologia 1972, 10:291-298.
48. Mishkin M., Manning F.J. Non-spatial memory after selective prefrontal lesions in monkeys. Brain Res. 1978, 143:313-323.
49. Mishkin M., Vest B., Waxler M., Rosvold H.E. A re-examination of the effects of frontal lesions on object alternation. Neuropsychologia 1969, 7:357-363.
50. Morris G., Arkadir D., Nevet A., Vaadia E., Bergman H. Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 2004, 43:133-143.
51. Morrison S.E., Salzman C.D. The convergence of information about rewarding and aversive stimuli in single neurons. J. Neurosci. 2009, 29:11471-11483.
52. Murray E.A., O'Doherty J.P., Schoenbaum G. What we know and do not know about the functions of the orbitofrontal cortex after 20 years of cross-species studies. J. Neurosci. 2007, 27:8166-8169.
53. Murray E.A., Wise S.P., Rhodes S.E.V. What Can Different Brains Do with Rewardα. Neurobiology of Sensation and Reward 2011, LCRC Press, Boca Raton. J.A. Gottfried (Ed.).
54. Napier R.M., Macrae M., Kehoe E.J. Rapid reacquisition in conditioning of the rabbits nictitating membrane response. J. Exp. Psychol. 1992, 18:182-192.
55. Nee D.E., Brown J.W. Rostral-caudal gradients of abstraction revealed by multi-variate pattern analysis of working memory. Neuroimage 2012, 63:1285-1294.
56. Noonan M.P., Walton M.E., Behrens T.E.J., Sallet J., Buckley M.J., Rushworth M.F.S. Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex. Proc. Natl. Acad. Sci. USA 2010, 107:20547-20552.
57. Noonan M.P., Mars R.B., Rushworth M.F.S. Distinct roles of three frontal cortical areas in reward-guided behavior. J. Neurosci. 2011, 31:14399-14412.
58. Noonan M.P., Kolling N., Walton M.E., Rushworth M.F.S. Re-evaluating the role of the orbitofrontal cortex in reward and reinforcement. Eur. J. Neurosci. 2012, 35:997-1010.
59. O'Doherty J.P., Deichmann R., Critchley H.D., Dolan R.J. Neural responses during anticipation of a primary taste reward. Neuron 2002, 33:815-826.
60. Ostlund S.B., Balleine B.W. Orbitofrontal cortex mediates outcome encoding in Pavlovian but not instrumental conditioning. J. Neurosci. 2007, 27:4819-4825.
61. Otto A.R., Gureckis T.M., Markman A.B., Love B.C. Navigating through abstract decision spaces: evaluating the role of state generalization in a dynamic decision-making task. Psychon. Bull. Rev. 2009, 16:957-963.
62. Padoa-Schioppa C. Neurobiology of economic choice: a good-based model. Annu. Rev. Neurosci. 2011, 34:333-359.
63. Padoa-Schioppa C., Assad J.A. Neurons in the orbitofrontal cortex encode economic value. Nature 2006, 441:223-226.
64. Padoa-Schioppa C., Assad J.A. The representation of economic value in the orbitofrontal cortex is invariant for changes of menu. Nat. Neurosci. 2008, 11:95-102.
65. Petrides M. Mapping prefrontal cortical systems for the control of cognition. Brain mapping: the systems 2000, 159-176. Academic Press, San Diego. A.W. Toga, J.C. Mazziotta (Eds.).
66. Pickens C.L., Saddoris M.P., Setlow B., Gallagher M., Holland P.C., Schoenbaum G. Different roles for orbitofrontal cortex and basolateral amygdala in a reinforcer devaluation task. J. Neurosci. 2003, 23:11078-11084.
67. Preuss T.M. Do rats have prefrontal cortexα The rose-woolsey-akert program reconsidered. J. Cogn. Neurosci. 1995, 7:1-24.
68. Price J.L. Definition of the orbital cortex in relation to specific connections with limbic and visceral structures and other cortical regions. Ann. N Y Acad. Sci. 2007, 1121:54-71.
69. Ramus S.J., Eichenbaum H. Neural correlates of olfactory recognition memory in the rat orbitofrontal cortex. J. Neurosci. 2000, 20:8199-8208.
70. Redish A.D., Jensen S., Johnson A., Kurth-Nelson Z. Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychol. Rev. 2007, 114:784-805.
71. Rescorla R.A., Wagner A.R. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. Classical conditioning II: Current research and theory 1972, 64-99. Appleton-Century-Crofts, New York. A.H. Black, W.F. Prokasy (Eds.).
72. Ricker S.T., Bouton M.E. Reacquisition following extinction in appetitive conditioning. Anim. Learn. Behav. 1996, 24:423-436.
73. Roesch M.R., Taylor A.R., Schoenbaum G. Encoding of time-discounted rewards in orbitofrontal cortex is independent of value representation. Neuron 2006, 51:509-520.
74. Rolls E.T., Hornak J., Wade D., McGrath J. Emotion-related learning in patients with social and emotional changes associated with frontal lobe damage. J. Neurol. Neurosurg. Psychiatry 1994, 57:1518-1524.
75. Rudebeck P.H., Murray E.A. Balkanizing the primate orbitofrontal cortex: distinct subregions for comparing and contrasting values. Ann. N Y Acad. Sci. 2011, 1239:1-13.
76. Rudebeck P.H., Murray E.A. Dissociable effects of subtotal lesions within the macaque orbital prefrontal cortex on reward-guided behavior. J. Neurosci. 2011, 31:10569-10578.
77. Rushworth M.F.S., Noonan M.P., Boorman E.D., Walton M.E., Behrens T.E. Frontal cortex and reward-guided learning and decision-making. Neuron 2011, 70:1054-1069.
78. Schoenbaum G., Eichenbaum H. Information coding in the rodent prefrontal cortex. I. Single-neuron activity in orbitofrontal cortex compared with that in pyriform cortex. J. Neurophysiol. 1995, 74:733-750.
79. Schoenbaum G., Chiba A.A., Gallagher M. Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. J. Neurosci. 1999, 19:1876-1884.
80. Schoenbaum G., Nugent S.L., Saddoris M.P., Setlow B. Orbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations. Neuroreport 2002, 13:885-890.
81. Schoenbaum G., Setlow B., Nugent S.L., Saddoris M.P., Gallagher M. Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals. Learn. Mem. 2003, 10:129-140.
82. Schoenbaum G., Setlow B., Ramus S.J. A systems approach to orbitofrontal cortex function: recordings in rat orbitofrontal cortex reveal interactions with different learning systems. Behav. Brain Res. 2003, 146:19-29.
83. Schultz W., Dayan P., Montague P.R. A neural substrate of prediction and reward. Science 1997, 275:1593-1599.
84. Seo H., Barraclough D.J., Lee D. Dynamic signals related to choices and outcomes in the dorsolateral prefrontal cortex. Cereb. Cortex 2007, 17(Suppl 1):i110-i117.
85. Simmons J.M., Richmond B.J. Dynamic changes in representations of preceding and upcoming reward in monkey orbitofrontal cortex. Cereb. Cortex 2008, 18:93-103.
86. Sul J.H., Kim H., Huh N., Lee D., Jung M.W. Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making. Neuron 2010, 66:449-460.
87. Sutton R.S., Barto A.G. Reinforcement learning: An introduction 1998, MIT Press, Cambridge.
88. Takahashi Y.K., Roesch M.R., Stalnaker T.A., Haney R.Z., Calu D.J., Taylor A.R., Burke K.A., Schoenbaum G. The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes. Neuron 2009, 62:269-280.
89. Takahashi Y.K., Roesch M.R., Wilson R.C., Toreson K., O'Donnell P., Niv Y., Schoenbaum G. Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex. Nat. Neurosci. 2011, 14:1590-1597.
90. Teitelbaum H. A comparison of effects of orbitofrontal and hippocampal lesions upon discrimination learning and reversal in the cat. Exp. Neurol. 1964, 9:452-462.
91. Thorpe S.J., Rolls E.T., Maddison S. The orbitofrontal cortex: neuronal activity in the behaving monkey. Exp. Brain Res. 1983, 49:93-115.
92. Tsuchida A., Doll B.B., Fellows L.K. Beyond reversal: a critical role for human orbitofrontal cortex in flexible learning from probabilistic feedback. J. Neurosci. 2010, 30:16868-16875.
93. Tsujimoto S., Genovesio A., Wise S.P. Monkey orbitofrontal cortex encodes response choices near feedback time. J. Neurosci. 2009, 29:2569-2574.
94. Tsujimoto S., Genovesio A., Wise S.P. Comparison of strategy signals in the dorsolateral and orbital prefrontal cortex. J. Neurosci. 2011, 31:4583-4592.
95. Wallis J.D. Orbitofrontal cortex and its contribution to decision-making. Annu. Rev. Neurosci. 2007, 30:31-56.
96. Wallis J.D. Cross-species studies of orbitofrontal cortex and value-based decision-making. Nat. Neurosci. 2012, 15:13-19.
97. Walton M.E., Behrens T.E.J., Buckley M.J., Rudebeck P.H., Rushworth M.F.S. Separable learning systems in the macaque brain and the role of orbitofrontal cortex in contingent learning. Neuron 2010, 65:927-939.
98. Wilson R.C., Niv Y. Inferring relevance in a changing world. Front Hum Neurosci 2011, 5:189.
99. Wise S.P. Forward frontal fields: phylogeny and fundamental function. Trends Neurosci. 2008, 31:599-608.