Amygdala-frontal interactions and reward expectancy

Current Opinion in Neurobiology

Volumn 14, Issue 2, 2004, Pages 148-155

  Holland, Peter C ^a   Gallagher, Michela ^a  

^a JOHNS HOPKINS UNIVERSITY   (United States)

Author keywords

Basolateral amygdala; BLA; Conditioned stimulus; CS; fMRI; Functional magnetic resonance imaging; OFC; Orbitofrontal cortex

Indexed keywords

AMYGDALOID NUCLEUS; BEHAVIORAL SCIENCE; BRAIN FUNCTION; BRAIN INJURY; BRAIN NERVE CELL; COGNITION; DISCRIMINATION LEARNING; EMOTION; EXPECTANCY; FRONTAL CORTEX; HUMAN; NERVE PROJECTION; NONHUMAN; ORBITAL CORTEX; PREFRONTAL CORTEX; PRIORITY JOURNAL; REINFORCEMENT; REVIEW; REWARD; SENSORY STIMULATION; STIMULUS RESPONSE; THALAMUS;

AMYGDALA; ANIMALS; COGNITION; DISCRIMINATION LEARNING; EMOTIONS; HUMANS; NEURAL INHIBITION; NEURAL PATHWAYS; PREFRONTAL CORTEX; REWARD;

EID: 1842735212     PISSN: 09594388     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.conb.2004.03.007     Document Type: Review

References (79)

1. Baxter M.G., Murray E.A. The amygdala and reward. Nat Rev Neurosci. 7:2002;563-573 The authors present a brief review of recent evidence, primarily from studies with rats and monkeys, for an important role of the amygdala in processing of positive rewards. The authors contrast amygdala-independent and amygdala-dependent reward processing, emphasizing the role of amygdala in the formation of stimulus value associations in learning about the positive consequences of events.
2. Cardinal R.N., Parkinson J.A., Hall J., Everitt B.J. Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev. 26:2002;321-352 The authors present an extensive review of the contributions of these brain regions to performance in a wide range of learning tasks thought to reveal learning about the affective value of environmental stimuli.
3. Miller E.K. The prefrontal cortex and cognitive control. Nat Rev Neurosci. 1:2002;59-65.
4. Krawczyk D.C. Contributions of the prefrontal cortex to the neural basis of human decision making. Neurosci Biobehav Rev. 26:2002;631-664 This is a comprehensive review, which integrates findings from behavioral, brain imaging, neuropsychological and animal studies. The author proposes that different aspects of decision processes are subserved by three major prefrontal regions: the OFC and ventromedial areas, the dorsolateral areas, and the anterior and ventral cingulate areas.
5. Rescorla R.A. A Pavlovian analysis of goal-directed behavior. Am Psychol. 42:1987;119-129.
6. Holland P.C. Event representation in Pavlovian conditioning: image and action. Cognition. 37:1990;105-131.
7. Rescorla RA, Wagner AR: A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In Classical Conditioning II. Edited by Black AH, Prokasy WF. New York: Appleton-Century- Crofts, 1972:64-99.
8. Balleine B.W., Killcross A.S., Dickinson A. The effect of lesions of the basolateral amygdala on instrumental conditioning. J Neurosci. 23:2003;666-675 The authors report three studies that implicate BLA in the coding of specific reinforcing outcomes. Pre-training lesions of the BLA impaired performance on an instrumental devaluation task (sensory-specific satiety), reduced sensitivity to post-training degradation of response-reinforcer contingencies, and were unable to use properties of the reinforcers to guide correct responding in a discrimination task. The authors suggest that BLA is crucial for "relating the sensory features of nutritive commodities to the consequences induced by their consumption".
9. Blundell P., Hall G., Killcross S.L. Preserved sensitivity to outcome value after lesions of the basolateral amygdala. J Neurosci. 23:2003;7702-7709 Rats with BLA lesions failed to show a devaluation in a Pavlovian approach conditioning task, but showed normal devaluation effects in a within-compound taste learning (sensory preconditioning) task. The authors suggest that the BLA is crucial for learning associations between sensory and incentive stimulus attribute cues but not for learning sensory-sensory associations.
10. Hatfield T., Han J.-S., Conley M., Gallagher M., Holland P. Neurotoxic lesions of the basolateral, but not central, amygdala interfere with Pavlovian second-order conditioning and reinforcer-devaluation effects. J Neurosci. 16:1996;5256-5265.
11. Gallagher M., McMahan R.W., Schoenbaum G. Orbitofrontal cortex and representation of incentive value in associative learning. J Neurosci. 19:1999;6610-6614.
12. Malkova L., Gaffan D., Murray E.A. Excitotoxic lesions of the amygdala fail to produce impairment in visual learning for auditory secondary reinforcement but interfere with reinforcer devaluation effects in rhesus monkeys. J Neurosci. 17:1997;6011-6020.
13. Baxter M.G., Parker A., Lindner C.C.C., Izquierdo A.D., Murray E.A. Control of response selection by reinforcer value requires interaction of amygdala and orbital prefrontal cortex. J Neurosci. 20:2000;4311-4319.
14. LaBar K.S., Gitelman D.R., Parrish T.B., Kim Y.H., Nobre A.C., Mesulam M.M. Hunger selectively modulates corticolimbic activation to food stimuli in humans. Behav Neurosci. 115:2001;493-500.
15. Morris J.S., Dolan R.J. Involvement of human amygdala and orbitofrontal cortex in hunger-enhanced memory for food stimuli. J Neurosci. 21:2001;5304-5310.
16. Kringelbach M.L., O'Doherty J., Rolls E.T., Andrews C. Activation of the human orbitofrontal cortex to a liquid food stimulus is correlated with subjective pleasantness. Cereb Cortex. 13:2003;1064-1071.
17. Kringelbach M.L., O'Doherty J., Rolls E.T., Andrews C. Sensory-specific satiety for the flavor of food is represented in the orbitofrontal cortex. Neuroimage. 11:2000;S767.
18. Gottfried J.A., O'Doherty J., Dolan R.J. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science. 301:2003;1104-1107 In this event-related fMRI study, participants received visual cues, one paired with the odor of ice cream, one with the odor of peanut butter, and one not paired with any odor. They then were allowed to eat either peanut butter or ice cream to satiation and finally were retested. Both amygdala and OFC responded more to the visual CS-odor pair whose corresponding food item was not devalued by satiation than to the other events. In a follow-up study (Gottfried JA, pers comm), testing in the presence of the visual cues only revealed activation of OFC but not amygdala, consistent with the claim that amygdala is crucial for learning about incentive values of cues but not for guiding behavior based on those cue-outcome expectancies.
19. Schoenbaum G., Chiba A.A., Gallagher M. Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat Neurosci. 1:1998;155-159.
20. Schoenbaum G., Chiba A.A., Gallagher M. Neural encoding in orbitofrontal cortex and basolateral amygdala during olfactory discrimination learning. J Neurosci. 19:1999;1876-1884.
21. Schoenbaum G., Chiba A.A., Gallagher M. Changes in functional connectivity in orbitofrontal cortex and basolateral amygdala during learning and reversal training. J Neurosci. 20:2000;5179-5189.
22. Watanabe M. Reward expectancy in primate prefrontal neurons. Nature. 382:1996;629-632.
23. Hikosaka K., Watanabe M. Delay activity of orbital and lateral prefrontal neurons of the monkey varying with different rewards. Cereb Cortex. 10:2000;263-271.
24. Toyomitsu Y., Nishijo H., Uwano T., Kuratsu J., Ono T. Neuronal responses of the rat amygdala during extinction and reassociation learning in elementary and configural associative tasks. Eur J Neurosci. 15:2002;753-768.
25. Tremblay L., Schultz W. Reward-related neuronal activity during go-nogo task performance in primate orbitofrontal cortex. J Neurophysiol. 83:2000;1864-1876.
26. Schoenbaum G., Setlow B., Saddoris M.P., Gallagher M. Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends on output from basolateral amygdala. Neuron. 39:2003;855-867 Rats were trained in a series of go/no-go discrimination tasks in which responding after one odor was reinforced with sucrose and responding after another odor was reinforced with (aversive) quinine. Although rats with bilateral lesions of the basolateral amygdala and sham lesioned control rats showed comparable behavioral accuracy in the behavioral discrimination task and some similarities in OFC neuronal activity, they showed substantially reduced coding of cue-outcome expectancies. Fewer cue-selective OFC neurons were observed in BLA lesioned rats, and unlike in shams, these neurons did not also code the upcoming reinforcer after a response, but rather were more often bound to cue identity. This last characteristic was especially evident in reversal learning, in which OFC neurons of lesioned rats were more likely to maintain the same cue-selectivity after reversal as in the original training.
27. ••]. The authors recorded from BLA neurons in rats with unilateral excitotoxic or sham lesions of the OFC. Lesioned rats showed changes in neural coding in BLA ipsilateral to the OFC lesion. BLA neurons in lesioned rats were less likely to show outcome-specific responding during the delay interval after responding but before reinforcer delivery than neurons in sham rats. Furthermore, odor cue-selective neurons in lesioned rats were much less likely to reverse their cue selectivity after reversal training than comparable neurons in shams.
28. Bohn I., Giertler C., Hauber W. NMDA receptors in the rat orbital frontal prefrontal cortex are involved in guidance of instrumental behavior under reversal conditions. Cereb Cortex. 13:2003;968-976.
29. Mitchell D.G.V., Colledge E., Leonard A., Blair R.J.R. Risky decisions and response reversal: is there evidence of orbitofrontal cortex dysfunction in psychopathic individuals? Neuropsychology. 40:2002;2013-2022.
30. MacAlonan K., Brown V.J. Orbital prefrontal cortex mediates reversal learning and not attentional set-shifting. Behav Brain Res. 146:2003;97-103.
31. Schoenbaum G., Setlow B., Nugent S.L., Saddoris M.P., Gallagher M. Lesions of orbitofrontal cortex and basolateral amygdala complex disrupt acquisition of odor-guided discriminations and reversals. Learn Mem. 10:2003;129-140.
32. Pickens C.P., Saddoris M.P., Setlow B., Gallagher M., Holland P.C., Schoenbaum G. Different roles for orbitofrontal cortex and basolateral amygdala in a reinforcer devaluation task. J Neurosci. 23:2003;11078-11084 Although Pavlovian conditioned responding of rats with lesions of BLA made before training showed no sensitivity to reinforcer devaluation (see text), rats with BLA lesions made after light-food training but before food devaluation were unimpaired. By contrast, responding of rats with OFC lesions made after light-food pairings was not sensitive to subsequent food devaluation. The authors suggested that BLA is crucial for forming representations linking cues to the incentive properties of outcomes but not for the use of the those representations to guide behavior, which is instead dependent on OFC function.
33. Pickens CP, Saddoris MP, Gallagher M, Holland PC: Orbitofrontal lesions impair use of cue-outcome associations in a devaluation task [abstract]. Soc Neurosci 2003: 89.13.
34. Gottfried J.A., O'Doherty J., Dolan R.J. Appetitive and aversive olfactory learning in humans studied using event-related functional magnetic resonance imaging. J Neurosci. 15:2002;10829-10837 In this event-related fMRI study, pictures of faces were paired or unpaired with pleasant, neutral, or unpleasant odors in classical conditioning procedures. Activity of medial and lateral OFC was associated with presentations of faces that signaled pleasant and unpleasant odors, respectively. Amygdala activity was initially related to presentations of signals for pleasant odors, but that response habituated over testing. Activity of the nucleus accumbens increased with presentations of signals for pleasant odors but decreased with signals for unpleasant odors.
35. Arana F.S., Parkinson J.A., Hinton E., Holland A.J., Owen A.M., Roberts A.C. Dissociable contributions of the human amygdala and orbitofrontal cortex to incentive motivation and goal selection. J Neurosci. 29:2003;9632-9638 In this study, food-sated participants were shown restaurant menus that varied in incentive value, based on a prior survey of their food preferences. On half of the trials, the participants were asked to choose from the menus. PET analysis of regional cerebral blood flow showed activation of the amygdala by high-incentive menus, regardless of whether a choice had to be made; that activation was proportional to participants' subjective ratings of menu incentive value. Medial OFC responded to high-incentive menus and on any trials that required a choice response; activation on choice trials was proportional to participants' subjective ratings of choice difficulty. Finally, lateral OFC responded when participants had to choose among similarly desirable alternatives. The authors suggest that amygdala and OFC have specialized roles within a system that underlies the selection of goals based on expectancies of their incentive values. Whereas the amygdala provides magnitude estimates, the OFC "integrate(s) diverse aspects of goal information including value, sensory, experiential and semantic components to guide prospective (expectancy-based) behavior".
36. O'Doherty J., Critchley H., Deichmann R., Dolan R.J. Dissociating valence of outcome from behavioral control in human orbital and ventral prefrontal cortices. J Neurosci. 23:2003;7931-7939 The authors present an fMRI study of normal participants in a two-choice visual discrimination reversal task. In a 'choice' condition, responses to one visual cue yielded a monetary gain 70% of the time and a monetary loss 30% of the time; responses to the other stimulus had the opposite consequences. After a discrimination criterion was met, the discrimination contingencies were reversed. In a response independent condition, participants could not choose their responses, rather reinforcement was determined by the performance of another 'yoked' subject. Outcome-dependent activation of central/medial OFC was found in both choice and response-independent conditions, but was greater in the choice condition. Amygdala and striatal activation to reward versus punishment occurred without regard to choice condition. Finally, enhanced responding of medial and left lateral OFC was observed after both reinforced (correct) and punished (incorrect) trials that were not followed by a change in response on the next trial, relative to the neural response on punished trials that were followed by a subsequent correct response. Thus, this signal reflected a continuing behavioral strategy of response maintenance or persistence rather than the trial outcome alone. The authors conclude that some prefrontal regions "represent valence irrespective of behavioral choice, other regions are sensitive to response maintenance, and other regions are involved in detecting a change in response contingencies".
37. Setlow B., Gallagher M., Holland P.C. The basolateral complex of the amygdala is necessary for acquisition but not expression of CS motivational value in appetitive Pavlovian conditioning. Eur J Neurosci. 15:2002;1841-1853.
38. Schoenbaum G, Setlow B, Gallagher M: Orbitofrontal cortex: modeling prefrontal function in rats. In The Neurospsychology of Memory. Edited by Squire L, Schachter D. New York: Guilford Press, 2002:463-477.
39. Bechara A., Damasio A.R., Damasio H., Anderson S.W. Insensitivity to future consequences following damage to human prefrontal cortex. Cognition. 50:1994;7-15.
40. Dias R., Robbins T.W., Roberts A.C. Dissociable forms of inhibitory control within prefrontal cortex with an analog of the Wisconsin card sort test: restriction to novel situations and independence from 'on-line' processing. J Neurosci. 17:2000;9285-9297.
41. Rescorla R.A. Response-outcome associations remain functional through interference treatments. Anim Learn Behav. 24:1996;450-458.
42. Anderson A.K., Christoff K., Stappen I., Panitz D., Ghahremani D.G., Glover G., Gabrieli J.D., Sobel N. Dissociated neural representations of intensity and valence in human olfaction. Nat Neurosci. 6:2003;196-202.
43. Small D.M., Gregory M.D., Mak Y.E., Gitelman D., Mesulam M.M., Parish T. Dissociation of neural representation of intensity and affective valuation in human gustation. Neuron. 39:2003;701-711 Using fMRI imaging techniques, the authors found that amygdala activation was associated with the intensity but not valence (subjective pleasantness) of taste stimuli, whereas OFC activation was related to the valence of those stimuli, independent of their intensity.
44. Rolls E.T., Kringelbach M.L., De Araujo I.E. Different representations of pleasant and unpleasant odours in the human brain. Eur J Neurosci. 18:2003;695-703.
45. O'Doherty J., Rolls E.T., Francis S., Bowtell R., McGlone F. Representation of pleasant and aversive taste in the human brain. J Neurophysiol. 85:2001;1315-1321.
46. Gottfried J.A., Deichmann R., Winston J.S., Dolan R.J. Functional heterogeneity in human olfactory cortex: an event-related functional magnetic imaging study. J Neurosci. 22:2002;10819-10828.
47. Ueda K., Okamoto Y., Okada G., Yamashita H., Hori T., Yamawaki S. Brain activity during expectancy of emotional stimuli: an fMRI study. Neuroreport. 14:2003;51-55.
48. Breiter H.C., Aharon I., Kahneman D., Dale A., Shizgal P. Functional imaging of neural responses to expectancy and experience of monetary gains and loses. Neuron. 30:2001;619-639.
49. O'Doherty J., Kringelbach M.L., Rolls E.T., Hornak J., Andrews C. Abstract reward and punishment representations in the human orbitofrontal cortex. Nat Neurosci. 4:2001;95-102 In this event-related fMRI study, participants were exposed to a visual reversal task in which correct choices led to higher probability of monetary gain and incorrect choices led to cue to monetary losses. Monetary gains were associated with activation of the lateral OFC and suppression of activity of the medial OFC, and losses were associated with the opposite activation patterns in these two regions. Furthermore, the magnitude of activation of these regions was related to the magnitudes of the gains and losses.
50. O'Doherty J.P., Deichmann R., Critchley H.D., Dolan R.J. Neural responses during anticipation of a primary taste reward. Neuron. 33:2002;815-826.
51. Blundell P., Hall G., Killcross S. Lesions of the basolateral amygdala disrupt selective aspects of reinforcer representation in rats. J Neurosci. 21:2001;9018-9026.
52. Schoenbaum G., Eichenbaum H. Information coding in the rodent prefrontal cortex. I. Single-neuron activity in orbitofrontal cortex compared with that in pyriform cortex. J Neurophysiol. 74:1995;733-750.
53. Elliott R., Dolan R.J., Frith C.D. Dissociable functions in the medial and lateral orbitofrontal cortex: evidence from human neuroimaging studies. Cereb Cortex. 10:2000;308-317.
54. Pearce J.M., Hall G. A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev. 106:1980;532-552.
55. Holland P.C., Gallagher M. Amygdala circuitry in attentional and representational processes. Trends Cogn Sci. 3:1999;65-73.
56. Schultz W., Dickinson A. Neuronal coding of prediction errors. Annu Rev Neurosci. 23:2000;473-500.
57. Elliott R., Newman J.L., Longe O.A., Deakin J.F. Differential response patterns in the striatum and orbitofrontal cortex to financial reward in humans: a parametric functional magnetic resonance imaging study. J Neurosci. 23:2003;303-307.
58. Berns G.S., McClure S.M., Pagnoni G., Montague P.R. Predictability modulates human brain response to reward. J Neurosci. 21:2001;2793-2798.
59. Volkow N.D., Wang G.-J., Ma Y., Fowler J.S., Zhu W., Maynard L., Telang F., Vaska P., Ding Y.-S., Wong C., Swanson J.M. Expectation enhances the regional brain metabolic and the reinforcing effects of stimulants in cocaine abusers. J Neurosci. 23:2003;11461-11468.
60. O'Doherty J.P., Dayan P., Friston K., Critchley H., Dolan R.J. Temporal difference models and reward-related learning in the human brain. Neuron. 38:2003;150-152.
61. Nobre A.C., Coull J.T., Frith C.D., Mesulam M.M. Orbitofrontal cortex is activated during breaches of expectation in tasks of visual attention. Nat Neurosci. 2:1999;11-12.
62. Swanson L.W., Petrovich G.D. What is the amygdala? Trends Neurosci. 21:1998;323-331.
63. Alheid G.F., Heimer L. New perspectives in basal forebrain organization of special relevance for neuropsychiatric disorders: the striatopallidal, amygdaloid, and corticopetal components of substantia innominata. Neuroscience. 1:1988;1-39.
64. Yamasaki H., LaBar K.S., McCarthy G. Dissociable prefrontal brain systems for attention and emotion. Proc Natl Acad Sci U S A. 99:2002;11447-11451.
65. Ghashghaie H.T., Barbas H. Pathways for emotion: interactions of prefrontal and anterior temporal pathways in the amygdala of the rhesus monkey. Neuroscience. 115:2002;1261-1279 The authors report the results of a series of anterograde and retrograde tracer studies of connections among amygdala, prefrontal, and anterior temporal areas, which shed light on how "this intricate network may be recruited in cognitive tasks that are inextricably linked with emotional associations".
66. Corbit L.H., Muir J.L., Balleine B.W. Lesions of mediodorsal thalamus and anterior thalamic nuclei produce dissociable effects on instrumental conditioning in rats. Eur J Neurosci. 18:2003;1286-1294.
67. Setlow B., Schoenbaum G., Gallagher M. Neural encoding in ventral striatum during olfactory discrimination learning. Neuron. 38:2003;625-636.
68. Wallis J.D., Miller E.K. Neuronal activity in primate dorsolateral and orbital prefrontal cortex during performance of a reward preference task. Eur J Neurosci. 18:2003;2069-2081.
69. Rolls E.T. The orbitofrontal cortex and reward. Cereb Cortex. 10:2000;284-294.
70. Montague P.R., Berns G.S. Neural economics and the biological substrates of valuation. Neuron. 36:2002;265-284 The authors propose a 'predictor-valuation model' by which OFC, in concert with its reciprocal connections with striatal circuitry, converts "disparate types of outcomes into a common currency to compare the valuation of future behavioral acts or stimuli".
71. Watanabe M. Cognitive and motivational operations in primate prefrontal neurons. Rev Neurosci. 9:1998;225-241.
72. Hodgson T.L., Mort D., Chamberlain M.M., Hutton S.B., O'Neill K.S., Kennard C. Orbitofrontal cortex mediates inhibition of return. Neuropsychology. 40:2002;1891-1901.
73. Gallagher M: The amygdala and associative learning. In The Amygdala: A Functional Analysis. Edited by Aggleton JP. Oxford: Oxford University Press, 2001:311-330.
74. Everitt BJ, Cardinal RN, Hall J, Parkinson JA, Robbins TW: Differential involvement of amygdala subsystems in appetitive conditioning and drug addiction. In The Amygdala: A Functional Analysis. Edited by Aggleton JP. Oxford: Oxford University Press, 2001:353-390.
75. Bechara A., Damasio H., Tranel D., Damasio A.R. Deciding advantageously before knowing the advantageous strategy. Science. 275:1997;1293-1294.
76. Anderson A.K., Phelps E.A. Lesions of the human amygdala impair enhanced perception of emotionally salient events. Nature. 411:2001;305-309.
77. Bechara A., Damasio H., Damasio A.R., Lee G.P. Different contributions of the human amygdala and ventromedial prefrontal cortex to decision-making. J Neurosci. 19:1999;5473-5481.
78. Dolan R.J. On the neurology of morals. Nat Neurosci. 2:1999;927-929.
79. Stoleru S., Redoute J., Costes N., Lavenne F., Bars D.L., Dechaud H., Forest M.G., Pugeat M., Cinotti L., Pujol J.F. Brain processing of visual sexual stimuli in men with hypoactive sexual desire disorder. Psychiatry Res. 124:2003;67-86.