Orbitofrontal cortical neurons encode expectation-driven initiation of reward-seeking

Journal of Neuroscience

Volumn 34, Issue 31, 2014, Pages 10234-10246

  Moorman, David E ^a   Aston Jones, Gary ^a  

^a MEDICAL UNIVERSITY OF SOUTH CAROLINA   (United States)

Author keywords

Motivation; Orbital cortex; Prefrontal cortex; Response inhibition; Salience; Single neuron

Indexed keywords

MOTIVATION; ORBITAL CORTEX; PREFRONTAL CORTEX; RESPONSE-INHIBITION; SALIENCE; SINGLE-NEURON;

ACTION POTENTIALS; ANIMALS; CONDITIONING, OPERANT; CUES; DISCRIMINATION LEARNING; EXTINCTION, PSYCHOLOGICAL; INHIBITION (PSYCHOLOGY); MALE; MOTIVATION; NEURONS; PREFRONTAL CORTEX; RATS; RATS, LONG-EVANS; REWARD; SUCROSE; TIME FACTORS;

EID: 84904994553     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.3216-13.2014     Document Type: Article

References (83)

1. Ambroggi F, Ishikawa A, Fields HL, Nicola SM (2008) Basolateral amygdala neurons facilitate reward-seeking behavior by exciting nucleus accumbens neurons. Neuron 59: 648-661. CrossRef Medline.
2. Ambroggi F, Ghazizadeh A, Nicola SM, Fields HL (2011) Roles of nucleus accumbens core and shell in incentive-cue responding and behavioral inhibition. J Neurosci 31: 6820-6830. CrossRef Medline.
3. Aron AR, Fletcher PC, Bullmore ET, Sahakian BJ, RobbinsTW (2003) Stopsignal inhibition disrupted by damage to right inferior frontal gyrus in humans. Nat Neurosci 6: 115-116. CrossRef Medline.
4. Burke KA, Franz TM, Miller DN, Schoenbaum G (2008) The role of the orbitofrontal cortex in the pursuit of happiness and more specific rewards. Nature 454: 340-344. CrossRef Medline.
5. Burke KA, Takahashi YK, Correll J, Brown PL, Schoenbaum G (2009) Orbitofrontal inactivation impairs reversal of Pavlovian learning by interfering with "disinhibition" of responding for previously unrewarded cues. Eur J Neurosci 30: 1941-1946. CrossRef Medline.
6. Butter CM, Mishkin M, Rosvold HE (1963) Conditioning and extinction of a food-rewarded response after selective ablations of frontal cortex in rhesus monkeys. Exp Neurol 7: 65-75. CrossRef Medline.
7. Carelli RM, Ijames SG (2000) Nucleus accumbens cell firing during maintenance, extinction, and reinstatement of cocaine self-administration behavior in rats. Brain Res 866: 44-54. CrossRef Medline.
8. Chudasama Y, RobbinsTW (2003) Dissociable contributions of the orbitofrontal and infralimbic cortex to Pavlovian autoshaping and discrimination reversal learning: further evidence for the functional heterogeneity of the rodent frontal cortex. J Neurosci 23: 8771-8780. Medline.
9. Chudasama Y, Passetti F, Rhodes SE, Lopian D, Desai A, RobbinsTW (2003) Dissociable aspects of performance on the 5-choice serial reaction time task following lesions of the dorsal anterior cingulate, infralimbic and orbitofrontal cortex in the rat: differential effects on selectivity, impulsivity and compulsivity. Behav Brain Res 146: 105-119. CrossRef Medline.
10. Eagle DM, Baunez C (2010) Is there an inhibitory-response-control system in the rat? Evidence from anatomical and pharmacological studies of behavioral inhibition. Neurosci Biobehav Rev 34: 50-72. CrossRef Medline.
11. Eagle DM, Baunez C, Hutcheson DM, Lehmann O, Shah AP, Robbins TW (2008) Stop-signal reaction-time task performance: role of prefrontal cortex and subthalamic nucleus. Cereb Cortex 18: 178-188. CrossRef Medline.
12. Esber GR, HaselgroveM (2011) Reconciling the influence of predictiveness and uncertainty on stimulus salience: a model of attention in associative learning. Proc Biol Sci 278: 2553-2561. CrossRef Medline.
13. Feierstein CE, Quirk MC, Uchida N, Sosulski DL, Mainen ZF (2006) Representation of spatial goals in rat orbitofrontal cortex. Neuron 51: 495-507. CrossRef Medline.
14. Finger EC, Mitchell DG, Jones M, Blair RJ (2008) Dissociable roles of medial orbitofrontal cortex in human operant extinction learning. Neuroimage 43: 748-755. CrossRef Medline.
15. Gallagher M, McMahan RW, Schoenbaum G (1999) Orbitofrontal cortex and representation of incentive value in associative learning. J Neurosci 19: 6610-6614. Medline.
16. Ghazizadeh A, Ambroggi F, Odean N, Fields HL (2012) Prefrontal cortex mediates extinction of responding by two distinct neural mechanisms in accumbens shell. J Neurosci 32: 726-737. CrossRef Medline.
17. Ghitza UE, Fabbricatore AT, Prokopenko V, Pawlak AP, West MO (2003) Persistent cue-evoked activity of accumbens neurons after prolonged abstinence from self-administered cocaine. J Neurosci 23: 7239-7245. Medline.
18. Gottfried JA, Dolan RJ (2004) Human orbitofrontal cortex mediates extinction learning while accessing conditioned representations of value. Nat Neurosci 7: 1144-1152. CrossRef Medline.
19. Grabenhorst F, Rolls ET (2011) Value, pleasure and choice in the ventral prefrontal cortex. Trends Cogn Sci 15: 56-67. CrossRef Medline.
20. Hollander JA, Carelli RM (2007) Cocaine-associated stimuli increase cocaine seeking and activate accumbens core neurons after abstinence. J Neurosci 27: 3535-3539. CrossRef Medline.
21. Hollander JA, Ijames SG, Roop RG, Carelli RM (2002) An examination of nucleus accumbens cell firing during extinction and reinstatement of water reinforcement behavior in rats. Brain Res 929: 226-235. CrossRef Medline.
22. Ishikawa A, Ambroggi F, Nicola SM, Fields HL (2008) Dorsomedial prefrontal cortex contribution to behavioral and nucleus accumbens neuronal responses to incentive cues. J Neurosci 28: 5088-5098. CrossRef Medline.
23. Izquierdo A, Murray EA (2005) Opposing effects of amygdala and orbital prefrontal cortex lesions on the extinction of instrumental responding in macaque monkeys. Eur J Neurosci 22: 2341-2346. CrossRef Medline.
24. Jones JL, Esber GR, McDannald MA, Gruber AJ, Hernandez A, Mirenzi A, Schoenbaum G (2012) Orbitofrontal cortex supports behavior and learning using inferred but not cached values. Science 338: 953-956. CrossRef Medline.
25. Kennerley SW, Wallis JD (2009) Encoding of reward and space during a working memory task in the orbitofrontal cortex and anterior cingulate sulcus. J Neurophysiol 102: 3352-3364. CrossRef Medline.
26. Kepecs A, Uchida N, Zariwala HA, Mainen ZF (2008) Neural correlates, computation and behavioural impact of decision confidence. Nature 455: 227-231. CrossRef Medline.
27. Kim J, RagozzinoME (2005) The involvement of the orbitofrontal cortex in learning under changing task contingencies. Neurobiol Learn Mem 83: 125-133. CrossRef Medline.
28. Kravitz AV, Peoples LL (2008) Background firing rates of orbitofrontal neurons reflect specific characteristics of operant sessions and modulate phasic responses to reward-associated cues and behavior. J Neurosci 28: 1009-1018. CrossRef Medline.
29. Krettek JE, Price JL (1977) Projections from the amygdaloid complex to the cerebral cortex and thalamus in the rat and cat. J Comp Neurol 172: 687-722. CrossRef Medline.
30. Luk CH, Wallis JD (2013) Choice coding in frontal cortex during stimulusguided or action-guided decision-making. J Neurosci 33: 1864-1871. CrossRef Medline.
31. Mackintosh NJ (1975) Theory of attention: variations in associability of stimuli with reinforcement. Psychol Rev 82: 276-298. CrossRef.
32. Mainen ZF, Kepecs A (2009) Neural representation of behavioral outcomes in the orbitofrontal cortex. Curr Opin Neurobiol 19: 84-91. CrossRef Medline.
33. McAlonan K, Brown VJ (2003) Orbital prefrontal cortex mediates reversal learning and not attentional set shifting in the rat. Behav Brain Res 146: 97-103. CrossRef Medline.
34. McDannald MA, Lucantonio F, Burke KA, Niv Y, Schoenbaum G (2011) Ventral striatum and orbitofrontal cortex are both required for modelbased, but not model-free, reinforcement learning. J Neurosci 31: 2700-2705. CrossRef Medline.
35. McDannald MA, Takahashi YK, Lopatina N, Pietras BW, Jones JL, Schoenbaum G (2012) Model-based learning and the contribution of the orbitofrontal cortex to the model-free world. Eur J Neurosci 35: 991-996. CrossRef Medline.
36. McDonald AJ (1991a) Organization of amygdaloid projections to the prefrontal cortex and associated striatum in the rat. Neuroscience 44: 1-14. CrossRef Medline.
37. McDonald AJ (1991b) Topographical organization of amygdaloid projections to the caudatoputamen, nucleus accumbens, and related striatal-like areas of the rat brain. Neuroscience 44: 15-33. CrossRef Medline.
38. Milad MR, Quirk GJ (2002) Neurons in medial prefrontal cortex signal memory for fear extinction. Nature 420: 70-74. CrossRef Medline.
39. Nicola SM, Yun IA, Wakabayashi KT, Fields HL (2004a) Cue-evoked firing of nucleus accumbens neurons encodes motivational significance during a discriminative stimulus task. J Neurophysiol 91: 1840-1865. CrossRef Medline.
40. Nicola SM, Yun IA, Wakabayashi KT, Fields HL (2004b) Firing of nucleus accumbens neurons during the consummatory phase of a discriminative stimulus task depends on previous reward predictive cues. J Neurophysiol 91: 1866-1882. CrossRef Medline.
41. Noonan MP, Kolling N, Walton ME, Rushworth MF (2012) Re-evaluating the role of the orbitofrontal cortex in reward and reinforcement. Eur J Neurosci 35: 997-1010. CrossRef Medline.
42. Ogawa M, van der Meer MA, Esber GR, Cerri DH, Stalnaker TA, Schoenbaum G (2013) Risk-responsive orbitofrontal neurons track acquired salience. Neuron 77: 251-258. CrossRef Medline.
43. O'Neill M, Schultz W (2010) Coding of reward risk by orbitofrontal neurons is mostly distinct from coding of reward value. Neuron 68: 789-800. CrossRef Medline.
44. Ostlund SB, Balleine BW (2007) Orbitofrontal cortex mediates outcome encoding in Pavlovian but not instrumental conditioning. J Neurosci 27: 4819-4825. CrossRef Medline.
45. Padoa-SchioppaC (2007) Orbitofrontal cortex and the computation of economic value. Ann N Y Acad Sci 1121: 232-253. CrossRef Medline.
46. Padoa-Schioppa C (2011) Neurobiology of economic choice: a good-based model. Ann Rev Neurosci 34: 333-359. CrossRef Medline.
47. Padoa-Schioppa C, Cai X (2011) The orbitofrontal cortex and the computation of subjective value: consolidated concepts and new perspectives. Ann N Y Acad Sci 1239: 130-137. CrossRef Medline.
48. Panayi MC, Killcross S (2014) Orbitofrontal cortex inactivation impairs between-but not within-session Pavlovian extinction: an associative analysis. Neurobiol Learn Mem 108: 78-87. CrossRef Medline.
49. Pearce JM, Hall G (1980) A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev 87: 532-552. CrossRef Medline.
50. Peters J, Kalivas PW, Quirk GJ (2009) Extinction circuits for fear and addiction overlap in prefrontal cortex. Learn Mem 16: 279-288. CrossRef Medline.
51. Peters YM, O'Donnell P, Carelli RM (2005) Prefrontal cortical cell firing during maintenance, extinction, and reinstatement of goal-directed behavior for natural reward. Synapse 56: 74-83. CrossRef Medline.
52. Reynolds SM, Zahm DS (2005) Specificity in the projections of prefrontal and insular cortex to ventral striatopallidum and the extended amygdala. J Neurosci 25: 11757-11767. CrossRef Medline.
53. Riceberg JS, Shapiro ML (2012) Reward stability determines the contribution of orbitofrontal cortex to adaptive behavior. J Neurosci 32: 16402-16409. CrossRef Medline.
54. Roberts AC (2006) Primate orbitofrontal cortex and adaptive behaviour. Trends Cogn Sci 10: 83-90. CrossRef Medline.
55. Roesch MR, Olson CR (2004) Neuronal activity related to reward value and motivation in primate frontal cortex. Science 304: 307-310. CrossRef Medline.
56. Roesch MR, Olson CR (2005) Neuronal activity in primate orbitofrontal cortex reflects the value of time. J Neurophysiol 94: 2457-2471. CrossRef Medline.
57. Roesch MR, Taylor AR, Schoenbaum G (2006) Encoding of timediscounted rewards in orbitofrontal cortex is independent of value representation. Neuron 51: 509-520. CrossRef Medline.
58. Roesch MR, Stalnaker TA, Schoenbaum G (2007) Associative encoding in anterior piriform cortex versus orbitofrontal cortex during odor discrimination and reversal learning. Cereb Cortex 17: 643-652. CrossRef Medline.
59. Roesch MR, Calu DJ, Esber GR, Schoenbaum G (2010) All that glitters. dissociating attention and outcome expectancy from prediction errors signals. J Neurophysiol 104: 587-595. CrossRef Medline.
60. Roitman JD, RoitmanMF (2010) Risk-preference differentiates orbitofrontal cortex responses to freely chosen reward outcomes. Eur J Neurosci 31: 1492-1500. CrossRef Medline.
61. Rolls ET, Critchley HD, Mason R, Wakeman EA (1996) Orbitofrontal cortex neurons: role in olfactory and visual association learning. J Neurophysiol 75: 1970-1981. Medline.
62. Saddoris MP, Gallagher M, Schoenbaum G (2005) Rapid associative encoding in basolateral amygdala depends on connections with orbitofrontal cortex. Neuron 46: 321-331. CrossRef Medline.
63. Schilman EA, Uylings HB, Galis-de Graaf Y, Joel D, Groenewegen HJ (2008) The orbital cortex in rats topographically projects to central parts of the caudate-putamen complex. Neurosci Lett 432: 40-45. CrossRef Medline.
64. Schoenbaum G, Eichenbaum H (1995) Information coding in the rodent prefrontal cortex: I. Single-neuron activity in orbitofrontal cortex compared with that in pyriform cortex. J Neurophysiol 74: 733-750. Medline.
65. Schoenbaum G, Chiba AA, Gallagher M (1998) Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat Neurosci 1: 155-159. CrossRef Medline.
66. Schoenbaum G, Chiba AA, GallagherM (1999) Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. J Neurosci 19: 1876-1884. Medline.
67. Schoenbaum G, Chiba AA, GallagherM (2000) Changes in functional connectivity in orbitofrontal cortex and basolateral amygdala during learning and reversal training. J Neurosci 20: 5179-5189. Medline.
68. Schoenbaum G, Nugent SL, Saddoris MP, Setlow B (2002) Orbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations. Neuroreport 13: 885-890. CrossRef Medline.
69. Schoenbaum G, Setlow B, Ramus SJ (2003) A systems approach to orbitofrontal cortex function: recordings in rat orbitofrontal cortex reveal interactions with different learning systems. Behav Brain Res 146: 19-29. CrossRef Medline.
70. Schoenbaum G, Roesch MR, Stalnaker TA, Takahashi YK (2009) A new perspective on the role of the orbitofrontal cortex in adaptive behaviour. Nat Rev Neurosci 10: 885-892. CrossRef Medline.
71. Schoenbaum G, Roesch MR, Stalnaker TA, Takahashi YK (2011) Orbitofrontal cortex and outcome expectancies: optimizing behavior and sensory perception. In: Neurobiology of sensation and reward (Gottfried JA, ed). Boca Raton, FL: CRC.
72. Schultz W, O'Neill M, Tobler PN, Kobayashi S (2011) Neuronal signals for reward risk in frontal cortex. Ann N Y Acad Sci 1239: 109-117. CrossRef Medline.
73. Shi CJ, Cassell MD (1998) Cortical, thalamic, and amygdaloid connections of the anterior and posterior insular cortices. J Comp Neurol 399: 440-468. CrossRef Medline.
74. Takahashi YK, Chang CY, Lucantonio F, Haney RZ, Berg BA, Yau HJ, Bonci A, Schoenbaum G (2013) Neural estimates of imagined outcomes in the orbitofrontal cortex drive behavior and learning. Neuron 80: 507-518. CrossRef Medline.
75. Tremblay L, Schultz W (1999) Relative reward preference in primate orbitofrontal cortex. Nature 398: 704-708. CrossRef Medline.
76. Tye KM, Cone JJ, Schairer WW, Janak PH (2010) Amygdala neural encoding of the absence of reward during extinction. J Neurosci 30: 116-125. CrossRef Medline.
77. van Duuren E, Escámez FA, Joosten RN, Visser R, Mulder AB, Pennartz CM (2007) Neural coding of reward magnitude in the orbitofrontal cortex of the rat during a five-odor olfactory discrimination task. Learn Mem 14: 446-456. CrossRef Medline.
78. van Duuren E, Lankelma J, Pennartz CM (2008) Population coding of reward magnitude in the orbitofrontal cortex of the rat. J Neurosci 28: 8590-8603. CrossRef Medline.
79. van Duuren E, van der Plasse G, Lankelma J, Joosten RN, Feenstra MG, Pennartz CM (2009) Single-cell and population coding of expected reward probability in the orbitofrontal cortex of the rat. J Neurosci 29: 8965-8976. CrossRef Medline.
80. Wallis JD (2012) Cross-species studies of orbitofrontal cortex and valuebased decision-making. Nat Neurosci 15: 13-19. CrossRef Medline.
81. Wright CI, Beijer AV, Groenewegen HJ (1996) Basal amygdaloid complex afferents to the rat nucleus accumbens are compartmentally organized. J Neurosci 16: 1877-1893. Medline.
82. Yun IA, Wakabayashi KT, Fields HL, NicolaSM (2004) The ventral tegmental area is required for the behavioral and nucleus accumbens neuronal firing responses to incentive cues. J Neurosci 24: 2923-2933. CrossRef Medline.
83. Zeeb FD, Winstanley CA (2013) Functional disconnection of the orbitofrontal cortex and basolateral amygdala impairs acquisition of a rat gambling task and disrupts animals' ability to alter decision-making behavior after reinforcer devaluation. J Neurosci 33: 6434-6443. CrossRef Medline.