Separable Learning Systems in the Macaque Brain and the Role of Orbitofrontal Cortex in Contingent Learning

Neuron

Volumn 65, Issue 6, 2010, Pages 927-939

  Walton, Mark E ^a   Behrens, Timothy E J ^a   Buckley, Mark J ^a   Rudebeck, Peter H ^a   Rushworth, Matthew F S ^a  

^a UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

Sysneuro

Indexed keywords

ADULT ANIMAL; ANIMAL BEHAVIOR; ANIMAL EXPERIMENT; ARTICLE; BRAIN FUNCTION; CONTROLLED STUDY; DECISION MAKING; LEARNING; MACACA; MALE; NONHUMAN; ORBITAL CORTEX; PRIORITY JOURNAL; REINFORCEMENT; TASK PERFORMANCE;

ANIMALS; CEREBRAL CORTEX; CHOICE BEHAVIOR; FRONTAL LOBE; LEARNING; MACACA; MACACA MULATTA; MALE; PSYCHOMOTOR PERFORMANCE;

EID: 77950255924     PISSN: 08966273     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neuron.2010.02.027     Document Type: Article

References (67)

1. Balleine B.W., Dickinson A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology 1998, 37:407-419.
2. Barraclough D.J., Conroy M.L., Lee D. Prefrontal cortex and decision making in a mixed-strategy game. Nat. Neurosci. 2004, 7:404-410.
3. Behrens T.E., Woolrich M.W., Walton M.E., Rushworth M.F. Learning the value of information in an uncertain world. Nat. Neurosci. 2007, 10:1214-1221.
4. Bogacz R., McClure S.M., Li J., Cohen J.D., Montague P.R. Short-term memory traces for action bias in human reinforcement learning. Brain Res. 2007, 1153:111-121.
5. Butter C.M. Perseveration in extinction and in discrimination reversal tasks following selective prefrontal ablations in Macaca mulatta. Physiol. Behav. 1969, 4:163-171.
6. Carmichael S.T., Price J.L. Limbic connections of the orbital and medial prefrontal cortex in macaque monkeys. J. Comp. Neurol. 1995, 363:615-641.
7. Carmichael S.T., Price J.L. Sensory and premotor connections of the orbital and medial prefrontal cortex of macaque monkeys. J. Comp. Neurol. 1995, 363:642-664.
8. Chudasama Y., Robbins T.W. Dissociable contributions of the orbitofrontal and infralimbic cortex to pavlovian autoshaping and discrimination reversal learning: further evidence for the functional heterogeneity of the rodent frontal cortex. J. Neurosci. 2003, 23:8771-8780.
9. Clarke H.F., Robbins T.W., Roberts A.C. Lesions of the medial striatum in monkeys produce perseverative impairments during reversal learning similar to those produced by lesions of the orbitofrontal cortex. J. Neurosci. 2008, 28:10972-10982.
10. Croxson P.L., Johansen-Berg H., Behrens T.E., Robson M.D., Pinsk M.A., Gross C.G., Richter W., Richter M.C., Kastner S., Rushworth M.F. Quantitative investigation of connections of the prefrontal cortex in the human and macaque using probabilistic diffusion tractography. J. Neurosci. 2005, 25:8854-8866.
11. Dias R., Robbins T.W., Roberts A.C. Dissociable forms of inhibitory control within prefrontal cortex with an analog of the Wisconsin Card Sort Test: restriction to novel situations and independence from " on-line" processing. J. Neurosci. 1997, 17:9285-9297.
12. Elliott R., Dolan R.J., Frith C.D. Dissociable functions in the medial and lateral orbitofrontal cortex: evidence from human neuroimaging studies. Cereb. Cortex 2000, 10:308-317.
13. Fellows L.K. The role of orbitofrontal cortex in decision making: a component process account. Ann. N Y Acad. Sci. 2007, 1121:421-430.
14. Fellows L.K., Farah M.J. Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. Brain 2003, 126:1830-1837.
15. Fu W.T., Anderson J.R. Solving the credit assignment problem: explicit and implicit learning of action sequences with probabilistic outcomes. Psychol. Res. 2008, 72:321-330.
16. Gottfried J.A., O'Doherty J., Dolan R.J. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 2003, 301:1104-1107.
17. Hare T.A., O'Doherty J., Camerer C.F., Schultz W., Rangel A. Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. J. Neurosci. 2008, 28:5623-5630.
18. Hikosaka K., Watanabe M. Delay activity of orbital and lateral prefrontal neurons of the monkey varying with different rewards. Cereb. Cortex 2000, 10:263-271.
19. Iversen S.D., Mishkin M. Perseverative interference in monkeys following selective lesions of the inferior prefrontal convexity. Exp. Brain Res. 1970, 11:376-386.
20. Izquierdo A., Suda R.K., Murray E.A. Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J. Neurosci. 2004, 24:7540-7548.
21. Jones B., Mishkin M. Limbic lesions and the problem of stimulus-reinforcement associations. Exp. Neurol. 1972, 36:362-377.
22. Kringelbach M.L., Rolls E.T. The functional neuroanatomy of the human orbitofrontal cortex: evidence from neuroimaging and neuropsychology. Prog. Neurobiol. 2004, 72:341-372.
23. Lara A.H., Kennerley S.W., Wallis J.D. Encoding of gustatory working memory by orbitofrontal neurons. J. Neurosci. 2009, 29:765-774.
24. Lau B., Glimcher P.W. Dynamic response-by-response models of matching behavior in rhesus monkeys. J. Exp. Anal. Behav. 2005, 84:555-579.
25. Lau B., Glimcher P.W. Action and outcome encoding in the primate caudate nucleus. J. Neurosci. 2007, 27:14502-14514.
26. Lewis D.A. The catecholaminergic innervation of primate prefrontal cortex. J. Neural Transm. Suppl. 1992, 36:179-200.
27. Liu Z., Murray E.A., Richmond B.J. Learning motivational significance of visual cues for reward schedules requires rhinal cortex. Nat. Neurosci. 2000, 3:1307-1315.
28. Luk C.H., Wallis J.D. Dynamic encoding of responses and outcomes by neurons in medial prefrontal cortex. J. Neurosci. 2009, 29:7526-7539.
29. Meunier M., Bachevalier J., Mishkin M. Effects of orbital frontal and anterior cingulate lesions on object and spatial memory in rhesus monkeys. Neuropsychologia 1997, 35:999-1015.
30. Morecraft R.J., Geula C., Mesulam M.M. Cytoarchitecture and neural afferents of orbitofrontal cortex in the brain of the monkey. J. Comp. Neurol. 1992, 323:341-358.
31. Murray E.A., O'Doherty J.P., Schoenbaum G. What we know and do not know about the functions of the orbitofrontal cortex after 20 years of cross-species studies. J. Neurosci. 2007, 27:8166-8169.
32. O'Doherty J., Critchley H., Deichmann R., Dolan R.J. Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J. Neurosci. 2003, 23:7931-7939.
33. Ostlund S.B., Balleine B.W. Orbitofrontal cortex mediates outcome encoding in Pavlovian but not instrumental conditioning. J. Neurosci. 2007, 27:4819-4825.
34. Price J.L. Definition of the orbital cortex in relation to specific connections with limbic and visceral structures and other cortical regions. Ann. N Y Acad. Sci. 2007, 1121:54-71.
35. Rangel A., Camerer C., Montague P.R. A framework for studying the neurobiology of value-based decision making. Nat. Rev. Neurosci. 2008, 9:545-556.
36. Rolls E.T., Hornak J., Wade D., McGrath J. Emotion-related learning in patients with social and emotional changes associated with frontal lobe damage. J. Neurol. Neurosurg. Psychiatry 1994, 57:1518-1524.
37. Rudebeck P.H., Walton M.E., Smyth A.N., Bannerman D.M., Rushworth M.F. Separate neural pathways process different decision costs. Nat. Neurosci. 2006, 9:1161-1168.
38. Rudebeck P.H., Behrens T.E., Kennerley S.W., Baxter M.G., Buckley M.J., Walton M.E., Rushworth M.F. Frontal cortex subregions play distinct roles in choices between actions and stimuli. J. Neurosci. 2008, 28:13775-13785.
39. Rushworth M.F., Behrens T.E., Rudebeck P.H., Walton M.E. Contrasting roles for cingulate and orbitofrontal cortex in decisions and social behaviour. Trends Cogn. Sci. 2007, 11:168-176.
40. Saddoris M.P., Gallagher M., Schoenbaum G. Rapid associative encoding in basolateral amygdala depends on connections with orbitofrontal cortex. Neuron 2005, 46:321-331.
41. Schoenbaum G., Nugent S.L., Saddoris M.P., Setlow B. Orbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations. Neuroreport 2002, 13:885-890.
42. Schoenbaum G., Setlow B., Saddoris M.P., Gallagher M. Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron 2003, 39:855-867.
43. Schoenbaum G., Roesch M.R., Stalnaker T.A. Orbitofrontal cortex, decision-making and drug addiction. Trends Neurosci. 2006, 29:116-124.
44. Schoenbaum G., Saddoris M.P., Stalnaker T.A. Reconciling the roles of orbitofrontal cortex in reversal learning and the encoding of outcome expectancies. Ann. N Y Acad. Sci. 2007, 1121:320-335.
45. Schultz W. Multiple dopamine functions at different time courses. Annu. Rev. Neurosci. 2007, 30:259-288.
46. Seo H., Lee D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J. Neurosci. 2007, 27:8366-8377.
47. Seo H., Lee D. Cortical mechanisms for reinforcement learning in competitive games. Philos. Trans. R. Soc. Lond. 2008, 363:3845-3857.
48. Seo H., Lee D. Behavioral and neural changes after gains and losses of conditioned reinforcers. J. Neurosci. 2009, 29:3627-3641.
49. Seo H., Barraclough D.J., Lee D. Dynamic signals related to choices and outcomes in the dorsolateral prefrontal cortex. Cereb. Cortex 2007, 17(Suppl 1):i110-i117.
50. Seo H., Barraclough D.J., Lee D. Lateral intraparietal cortex and reinforcement learning during a mixed-strategy game. J. Neurosci. 2009, 29:7278-7289.
51. Simmons J.M., Richmond B.J. Dynamic changes in representations of preceding and upcoming reward in monkey orbitofrontal cortex. Cereb. Cortex 2008, 18:93-103.
52. Stalnaker T.A., Roesch M.R., Franz T.M., Burke K.A., Schoenbaum G. Abnormal associative encoding in orbitofrontal neurons in cocaine-experienced rats during decision-making. Eur. J. Neurosci. 2006, 24:2643-2653.
53. Stalnaker T.A., Franz T.M., Singh T., Schoenbaum G. Basolateral amygdala lesions abolish orbitofrontal-dependent reversal impairments. Neuron 2007, 54:51-58.
54. Sutton R.S., Barto A.C. Reinforcement Learning: An introduction 1998, MIT Press, London.
55. Takahashi Y.K., Roesch M.R., Stalnaker T.A., Haney R.Z., Calu D.J., Taylor A.R., Burke K.A., Schoenbaum G. The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes. Neuron 2009, 62:269-280.
56. Thorndike E.L. Animal Intelligence: Experimental Studies 1911, Macmillan, New York.
57. Thorndike E.L. A proof of the law of effect. Science 1933, 77:173-175.
58. Tremblay L., Schultz W. Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. J. Neurophysiol. 2000, 83:1877-1885.
59. Tsujimoto S., Genovesio A., Wise S.P. Monkey orbitofrontal cortex encodes response choices near feedback time. J. Neurosci. 2009, 29:2569-2574.
60. Uchida Y., Lu X., Ohmae S., Takahashi T., Kitazawa S. Neuronal activity related to reward size and rewarded target position in primate supplementary eye field. J. Neurosci. 2007, 27:13750-13755.
61. van Duuren E., Lankelma J., Pennartz C.M. Population coding of reward magnitude in the orbitofrontal cortex of the rat. J. Neurosci. 2008, 28:8590-8603.
62. Wallis J.D., Miller E.K. Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. Eur. J. Neurosci. 2003, 18:2069-2081.
63. Walton M.E., Devlin J.T., Rushworth M.F. Interactions between decision making and performance monitoring within prefrontal cortex. Nat. Neurosci. 2004, 7:1259-1265.
64. Weiskrantz L. Problems of learning and memory: one or multiple memory systems?. Philos. Trans. R. Soc. Lond. 1990, 329:99-108.
65. White N.M. Reward or reinforcement: what's the difference?. Neurosci. Biobehav. Rev. 1989, 13:181-186.
66. Wise S.P. Forward frontal fields: phylogeny and fundamental function. Trends Neurosci. 2008, 31:599-608.
67. Yanike M., Wirth S., Smith A.C., Brown E.N., Suzuki W.A. Comparison of associative learning-related signals in the macaque perirhinal cortex and hippocampus. Cereb. Cortex 2009, 19:1064-1078.