Neural Estimates of Imagined Outcomes in the Orbitofrontal Cortex Drive Behavior and Learning

Neuron

Volumn 80, Issue 2, 2013, Pages 507-518

  Takahashi, Yuji K ^a   Chang, Chun Yun ^a   Lucantonio, Federica ^b   Haney, Richard Z ^b   Berg, Benjamin A ^a   Yau, Hau Jie ^a   Bonci, Antonello ^a,c,d   Schoenbaum, Geoffrey ^a,b,c  

^a NATIONAL INSTITUTE ON DRUG ABUSE   (United States)

^b UNIVERSITY OF MARYLAND SCHOOL OF MEDICINE   (United States)

^c JOHNS HOPKINS UNIVERSITY SCHOOL OF MEDICINE   (United States)

^d JOHNS HOPKINS UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ANIMAL EXPERIMENT; ANIMAL TISSUE; ARTICLE; ASSOCIATION; BEHAVIOR; BRAIN FUNCTION; BRAIN NERVE CELL; CONDITIONING; CONTROLLED STUDY; HISTOLOGY; IMAGINATION; LEARNING; MALE; NEUROTRANSMISSION; NONHUMAN; ORBITAL CORTEX; PRIORITY JOURNAL; RAT;

ACTION POTENTIALS; ANIMALS; CONDITIONING, CLASSICAL; CUES; EXTINCTION, PSYCHOLOGICAL; IMAGINATION; MALE; NEURAL INHIBITION; NEURONS; PREFRONTAL CORTEX; RATS;

EID: 84885704049     PISSN: 08966273     EISSN: 10974199     Source Type: Journal    
DOI: 10.1016/j.neuron.2013.08.008     Document Type: Article

References (53)

1. Abe H., Lee D. Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex. Neuron 2011, 70:731-741.
2. Bunsey M., Eichenbaum H. Conservation of hippocampal memory function in rats and humans. Nature 1996, 379:255-257.
3. Burke K.A., Franz T.M., Miller D.N., Schoenbaum G. The role of the orbitofrontal cortex in the pursuit of happiness and more specific rewards. Nature 2008, 454:340-344.
4. Camille N., Griffiths C.A., Vo K., Fellows L.K., Kable J.W. Ventromedial frontal lobe damage disrupts value maximization in humans. J.Neurosci. 2011, 31:7527-7532.
5. Chudasama Y., Wright K.S., Murray E.A. Hippocampal lesions in rhesus monkeys disrupt emotional responses but not reinforcer devaluation effects. Biol. Psychiatry 2008, 63:1084-1091.
6. Critchley H.D., Rolls E.T. Hunger and satiety modify the responses of olfactory and visual neurons in the primate orbitofrontal cortex. J.Neurophysiol. 1996, 75:1673-1686.
7. Gallagher M., McMahan R.W., Schoenbaum G. Orbitofrontal cortex and representation of incentive value in associative learning. J.Neurosci. 1999, 19:6610-6614.
8. Gottfried J.A., O'Doherty J., Dolan R.J. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 2003, 301:1104-1107.
9. Groenewegen H.J., Berendse H.W., Wolters J.G., Lohman A.H.M. The anatomical relationship of the prefrontal cortex with the striatopallidal system, the thalamus and the amygdala: evidence for a parallel organization. Prog. Brain Res. 1990, 85:95-116. discussion 116-118.
10. Hall G., Pearce J.M. Changes in stimulus associability during conditioning: implications for theories of acquisition. Quantitative Analyses of Behavior 1982, 221-239. Ballinger, Cambridge, MA. M.L. Commons, R.J. Herrnstein, A.R. Wagner (Eds.).
11. Haney R.Z., Calu D.J., Takahashi Y.K., Hughes B.W., Schoenbaum G. Inactivation of the central but not the basolateral nucleus of the amygdala disrupts learning in response to overexpectation of reward. J.Neurosci. 2010, 30:2911-2917.
12. Hatfield T., Han J.S., Conley M., Gallagher M., Holland P. Neurotoxic lesions of basolateral, but not central, amygdala interfere with Pavlovian second-order conditioning and reinforcer devaluation effects. J.Neurosci. 1996, 16:5256-5265.
13. Hollland P.C., Rescorla R.A. The effect of two ways of devaluing the unconditioned stimulus after first- and second-order appetitive conditioning. J.Exp. Psychol. Anim. Behav. Process. 1975, 1:355-363.
14. Hornak J., O'Doherty J., Bramham J., Rolls E.T., Morris R.G., Bullock P.R., Polkey C.E. Reward-related reversal learning after surgical excisions in orbito-frontal or dorsolateral prefrontal cortex in humans. J.Cogn. Neurosci. 2004, 16:463-478.
15. Izquierdo A.D., Murray E.A. Bilateral orbital prefrontal cortex lesions disrupt reinforcer devaluation effects in rhesus monkeys. Society for Neuroscience Abstracts 2000, 26:978.
16. Izquierdo A.D., Murray E.A. Opposing effects of amygdala and orbital prefrontal cortex lesions on the extinction of instrumental responding in macaque monkeys. Eur. J. Neurosci. 2005, 22:2341-2346.
17. Izquierdo A.D., Murray E.A. Selective bilateral amygdala lesions in rhesus monkeys fail to disrupt object reversal learning. J.Neurosci. 2007, 27:1054-1062.
18. Izquierdo A.D., Suda R.K., Murray E.A. Bilateral orbital prefrontal cortex lesions in rhesus monkeys disrupt choices guided by both reward value and reward contingency. J.Neurosci. 2004, 24:7540-7548.
19. Johnson A.W., Gallagher M., Holland P.C. The basolateral amygdala is critical to the expression of pavlovian and instrumental outcome-specific reinforcer devaluation effects. J.Neurosci. 2009, 29:696-704.
20. Jones J.L., Esber G.R., McDannald M.A., Gruber A.J., Hernandez A., Mirenzi A., Schoenbaum G. Orbitofrontal cortex supports behavior and learning using inferred but not cached values. Science 2012, 338:953-956.
21. Kahnt T., Chang L.J., Park S.Q., Heinzle J., Haynes J.-D. Connectivity-based parcellation of the human orbitofrontal cortex. J.Neurosci. 2012, 32:6240-6250.
22. Le Pelley M.E. The role of associative history in models of associative learning: a selective review and a hybrid model. Q.J. Exp. Psychol. B 2004, 57:193-243.
23. Levy D.J., Glimcher P.W. Comparing apples and oranges: using reward-specific and reward-general subjective value representation in the brain. J.Neurosci. 2011, 31:14693-14707.
24. Levy D.J., Glimcher P.W. The root of all value: a neural common currency for choice. Curr. Opin. Neurobiol. 2012, 22:1027-1038.
25. Luk C.-H., Wallis J.D. Choice coding in frontal cortex during stimulus-guided or action-guided decision-making. J.Neurosci. 2013, 33:1864-1871.
26. Machado C.J., Bachevalier J. The effects of selective amygdala, orbital frontal cortex or hippocampal formation lesions on reward assessment in nonhuman primates. Eur. J. Neurosci. 2007, 25:2885-2904.
27. Málková L., Gaffan D., Murray E.A. Excitotoxic lesions of the amygdala fail to produce impairment in visual learning for auditory secondary reinforcement but interfere with reinforcer devaluation effects in rhesus monkeys. J.Neurosci. 1997, 17:6011-6020.
28. McDannald M.A., Lucantonio F., Burke K.A., Niv Y., Schoenbaum G. Ventral striatum and orbitofrontal cortex are both required for model-based, but not model-free, reinforcement learning. J.Neurosci. 2011, 31:2700-2705.
29. Montague P.R., Berns G.S. Neural economics and the biological substrates of valuation. Neuron 2002, 36:265-284.
30. Ongür D., Price J.L. The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. Cereb. Cortex 2000, 10:206-219.
31. Padoa-Schioppa C. Range-adapting representation of economic value in the orbitofrontal cortex. J.Neurosci. 2009, 29:14004-14014.
32. Padoa-Schioppa C. Neurobiology of economic choice: a good-based model. Annu. Rev. Neurosci. 2011, 34:333-359.
33. Padoa-Schioppa C., Assad J.A. Neurons in the orbitofrontal cortex encode economic value. Nature 2006, 441:223-226.
34. Padoa-Schioppa C., Assad J.A. The representation of economic value in the orbitofrontal cortex is invariant for changes of menu. Nat. Neurosci. 2008, 11:95-102.
35. Pickens C.L., Saddoris M.P., Gallagher M., Holland P.C. Orbitofrontal lesions impair use of cue-outcome associations in a devaluation task. Behav. Neurosci. 2005, 119:317-322.
36. Pickens C.L., Saddoris M.P., Setlow B., Gallagher M., Holland P.C., Schoenbaum G. Different roles for orbitofrontal cortex and basolateral amygdala in a reinforcer devaluation task. J.Neurosci. 2003, 23:11078-11084.
37. Plassmann H., O'Doherty J., Rangel A. Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. J.Neurosci. 2007, 27:9984-9988.
38. Price J.L. Definition of the orbital cortex in relation to specific connections with limbic and visceral structures and other cortical regions. Ann. N Y Acad. Sci. 2007, 1121:54-71.
39. Rescorla R.A. Reduction in the effectiveness of reinforcement after prior excitatory conditioning. Learn. Motiv. 1970, 1:372-381.
40. Rescorla R.A., Wagner A.R. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. Classical Conditioning II: Current Research and Theory 1972, 64-99. Appleton-Century-Crofts, New York. A.H. Black, W.F. Prokasy (Eds.).
41. Rudebeck P.H., Murray E.A. Amygdala and orbitofrontal cortex lesions differentially influence choices during object reversal learning. J.Neurosci. 2008, 28:8338-8343.
42. Schoenbaum G., Esber G.R. How do you (estimate you will) like them apples? Integration as a defining trait of orbitofrontal function. Curr. Opin. Neurobiol. 2010, 20:205-211.
43. Schoenbaum G., Nugent S.L., Saddoris M.P., Setlow B. Orbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations. Neuroreport 2002, 13:885-890.
44. Schoenbaum G., Setlow B., Nugent S.L., Saddoris M.P., Gallagher M. Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals. Learn. Mem. 2003, 10:129-140.
45. Sul J.H., Kim H., Huh N., Lee D., Jung M.W. Distinct roles of rodent orbitofrontal and medial prefrontal cortex in decision making. Neuron 2010, 66:449-460.
46. Sutton R.S. Learning to predict by the method of temporal difference. Mach. Learn. 1988, 3:9-44.
47. Takahashi Y.K., Roesch M.R., Stalnaker T.A., Haney R.Z., Calu D.J., Taylor A.R., Burke K.A., Schoenbaum G. The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes. Neuron 2009, 62:269-280.
48. Thornton J.A., Malkova L., Murray E.A. Rhinal cortex ablations fail to disrupt reinforcer devaluation effects in rhesus monkeys (Macaca mulatta). Behav. Neurosci. 1998, 112:1020-1025.
49. Tobler P.N., O'doherty J.P., Dolan R.J., Schultz W. Human neural learning depends on reward prediction errors in the blocking paradigm. J.Neurophysiol. 2006, 95:301-310.
50. Tremblay L., Schultz W. Relative reward preference in primate orbitofrontal cortex. Nature 1999, 398:704-708.
51. Wellman L.L., Gale K., Malkova L. GABAA-mediated inhibition of basolateral amygdala blocks reward devaluation in macaques. J.Neurosci. 2005, 25:4577-4586.
52. West E.A., DesJardin J.T., Gale K., Malkova L. Transient inactivation of orbitofrontal cortex blocks reinforcer devaluation in macaques. J.Neurosci. 2011, 31:15128-15135.
53. Wimmer G.E., Shohamy D. Preference by association: how memory mechanisms in the hippocampus bias decisions. Science 2012, 338:270-273.