Neural and psychological mechanisms underlying appetitive learning: Links to drug addiction

Current Opinion in Neurobiology

Volumn 14, Issue 2, 2004, Pages 156-162

  Cardinal, Rudolf N ^a   Everitt, Barry J ^a  

^a UK   (United Kingdom)

Author keywords

Acb; AcbC; CeA; Central nucleus of the amygdala; Conditioned stimulus; CS; DA; Dopamine; N methyl D aspartate; NMDA; Nucleus accumbens; Nucleus accumbens core subregion; PKA; Protein kinase A

Indexed keywords

DOPAMINE 1 RECEPTOR; N METHYL DEXTRO ASPARTIC ACID RECEPTOR;

APPETITE; ASSOCIATION; BEHAVIORAL SCIENCE; BRAIN CORTEX; CONDITIONED REFLEX; DISCRIMINATIVE STIMULUS; DOPAMINERGIC SYSTEM; DOPAMINERGIC TRANSMISSION; DRUG DEPENDENCE; FOOD PREFERENCE; HUMAN; INSTRUMENTAL CONDITIONING; LEARNING; MOTIVATION; NONHUMAN; NUCLEUS ACCUMBENS; PALATABILITY; PRIORITY JOURNAL; RECEPTOR BLOCKING; REINFORCEMENT; REVIEW; SENSITIZATION; STIMULUS RESPONSE;

ANIMALS; CUES; HUMANS; LEARNING; MODELS, NEUROLOGICAL; MOTIVATION; NEURAL PATHWAYS; NUCLEUS ACCUMBENS; REINFORCEMENT (PSYCHOLOGY); REWARD; SUBSTANCE-RELATED DISORDERS;

EID: 1842734671     PISSN: 09594388     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.conb.2004.03.004     Document Type: Review

References (87)

1. Thorndike EL: Animal Intelligence: Experimental Studies. New York: Macmillan; 1911.
2. Dickinson A: Instrumental conditioning. In Animal Learning and Cognition. Edited by Mackintosh NJ. London: Academic Press; 1994:45-79.
3. Dickinson A., Balleine B. Motivational control of goal-directed action. Anim Learn Behav. 22:1994;1-18.
4. Balleine B., Dickinson A. Instrumental performance following reinforcer devaluation depends upon incentive learning. Q J Exp Psychol B. 43:1991;279-296.
5. Garcia J: Food for Tolman: cognition and cathexis in concert. In Aversion, Avoidance and Anxiety. Edited by Archer T, Nilsson L-G. Hillsdale NJ: Erlbaum; 1989:45-85.
6. Adams C.D. Variations in the sensitivity of instrumental responding to reinforcer devaluation. Q J Exp Psychol B. 34:1982;77-98.
7. Cardinal R.N., Parkinson J.A., Hall J., Everitt B.J. Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex. Neurosci Biobehav Rev. 26:2002;321-352 The authors present an extended review of the psychological processes that contribute to motivated behaviour in the rat, and the contribution of the amygdala, ventral striatum, and prefrontal cortex to instrumental responding.
8. Berridge K.C. Modulation of taste affect by hunger, caloric satiety, and sensory-specific satiety in the rat. Appetite. 16:1991;103-120.
9. Hutcheson D.M., Everitt B.J., Robbins T.W., Dickinson A. The role of withdrawal in heroin addiction: enhances reward or promotes avoidance? Nat Neurosci. 4:2001;943-947.
10. Cabanac M. Pleasure: the common currency. J Theor Biol. 155:1992;173-200.
11. Altman J., Everitt B.J., Glautier S., Markou A., Nutt D., Oretti R., Phillips G.D., Robbins T.W. The biological, social and clinical bases of drug addiction: commentary and debate. Psychopharmacology (Berl). 125:1996;285-345.
12. Foltin R.W., Haney M. Conditioned effects of environmental stimuli paired with smoked cocaine in humans. Psychopharmacology (Berl). 149:2000;24-33.
13. Kelley A.E., SmithRoe S.L., Holahan M.R. Response-reinforcement learning is dependent on N-methyl-D-aspartate receptor activation in the nucleus accumbens core. Proc Natl Acad Sci U S A. 94:1997;12174-12179.
14. Smith-Roe S.L., Kelley A.E. Coincident activation of NMDA and dopamine D1 receptors within the nucleus accumbens core is required for appetitive instrumental learning. J Neurosci. 20:2000;7737-7742.
15. Baldwin A.E., Sadeghian K., Holahan M.R., Kelley A.E. Appetitive instrumental learning is impaired by inhibition of cAMP-dependent protein kinase within the nucleus accumbens. Neurobiol Learn Mem. 77:2002;44-62.
16. Hernandez P.J., Sadeghian K., Kelley A.E. Early consolidation of instrumental learning requires protein synthesis in the nucleus accumbens. Nat Neurosci. 5:2002;1327-1331 The authors provide a demonstration that post-trial infusions of the protein synthesis inhibitor anisomycin into the AcbC disrupt consolidation of a recently learned instrumental response (lever-pressing for food), indicating a role for this structure in instrumental acquisition.
17. de Borchgrave R., Rawlins J.N., Dickinson A., Balleine B.W. Effects of cytotoxic nucleus accumbens lesions on instrumental conditioning in rats. Exp Brain Res. 144:2002;50-68.
18. Balleine B., Killcross S. Effects of ibotenic acid lesions of the nucleus accumbens on instrumental action. Behav Brain Res. 65:1994;181-193.
19. Cardinal R.N., Pennicott D.R., Sugathapala C.L., Robbins T.W., Everitt B.J. Impulsive choice induced in rats by lesions of the nucleus accumbens core. Science. 292:2001;2499-2501.
20. Hall J., Parkinson J.A., Connor T.M., Dickinson A., Everitt B.J. Involvement of the central nucleus of the amygdala and nucleus accumbens core in mediating Pavlovian influences on instrumental behaviour. Eur J Neurosci. 13:2001;1984-1992.
21. Wyvell C.L., Berridge K.C. Intra-accumbens amphetamine increases the conditioned incentive salience of sucrose reward: enhancement of reward 'wanting' without enhanced 'liking' or response reinforcement. J Neurosci. 20:2000;8122-8130.
22. Wyvell C.L., Berridge K.C. Incentive sensitization by previous amphetamine exposure: increased cue-triggered 'wanting' for sucrose reward. J Neurosci. 21:2001;7831-7840.
23. Dickinson A., Smith J., Mirenowicz J. Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. Behav Neurosci. 114:2000;468-483.
24. Parkinson J.A., Olmstead M.C., Burns L.H., Robbins T.W., Everitt B.J. Dissociation in effects of lesions of the nucleus accumbens core and shell on appetitive Pavlovian approach behavior and the potentiation of conditioned reinforcement and locomotor activity by d-amphetamine. J Neurosci. 19:1999;2401-2411.
25. Taylor J.R., Robbins T.W. Enhanced behavioural control by conditioned reinforcers following microinjections of d-amphetamine into the nucleus accumbens. Psychopharmacology (Berl). 84:1984;405-412.
26. Taylor J.R., Robbins T.W. 6-Hydroxydopamine lesions of the nucleus accumbens, but not of the caudate nucleus, attenuate enhanced responding with reward-related stimuli produced by intra-accumbens d-amphetamine. Psychopharmacology (Berl). 90:1986;390-397.
27. Cador M., Taylor J.R., Robbins T.W. Potentiation of the effects of reward-related stimuli by dopaminergic-dependent mechanisms in the nucleus accumbens. Psychopharmacology (Berl). 104:1991;377-385.
28. Salamone J.D., Correa M. Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine. Behav Brain Res. 137:2002;3-25 The authors present an excellent review of the nature of 'reinforcement' and the contribution of Acb DA to motivated behaviour.
29. Salamone J.D., Correa M., Mingote S.M., Weber S.M. Nucleus accumbens dopamine and the regulation of effort in food-seeking behavior: implications for studies of natural motivation, psychiatry, and drug abuse. J Pharmacol Exp Ther. 305:2003;1-8.
30. Brown P.L., Jenkins H.M. Auto-shaping of the pigeon's keypeck. J Exp Anal Behav. 11:1968;1-8.
31. Parkinson J.A., Robbins T.W., Everitt B.J. Selective excitotoxic lesions of the nucleus accumbens core and shell differentially affect aversive Pavlovian conditioning to discrete and contextual cues. Psychobiology. 27:1999;256-266.
32. Parkinson J.A., Willoughby P.J., Robbins T.W., Everitt B.J. Disconnection of the anterior cingulate cortex and nucleus accumbens core impairs Pavlovian approach behavior: further evidence for limbic cortical-ventral striatopallidal systems. Behav Neurosci. 114:2000;42-63.
33. •].
34. Cardinal R.N., Parkinson J.A., Djafari Marbini H., Toner A.J., Bussey T.J., Robbins T.W., Everitt B.J. Role of the anterior cingulate cortex in the control over behaviour by Pavlovian conditioned stimuli in rats. Behav Neurosci. 117:2003;566-587.
35. Parkinson J.A., Robbins T.W., Everitt B.J. Dissociable roles of the central and basolateral amygdala in appetitive emotional learning. Eur J Neurosci. 12:2000;405-413.
36. •]) it is demonstrated that the CeA is involved in the learning, but not the performance, of a learned approach response to a CS that predicts an appetitive outcome, and that Acb DA contributes to both processes.
37. Phillips A.G., Ahn S., Howland J.G. Amygdalar control of the mesocorticolimbic dopamine system: parallel pathways to motivated behavior. Neurosci Biobehav Rev. 27:2003;543-554.
38. Reynolds J.N., Hyland B.I., Wickens J.R. A cellular mechanism of reward-related learning. Nature. 413:2001;67-70.
39. Reynolds J.N., Wickens J.R. Dopamine-dependent plasticity of corticostriatal synapses. Neural Netw. 15:2002;507-521 The authors present a review of empirical data and recent theoretical models of the manner in which cortical (presynaptic) and striatal (postsynaptic) activity contribute to altering the strength of corticostriatal synapses, and how this relationship is altered by DA.
40. Baldwin A.E., Sadeghian K., Kelley A.E. Appetitive instrumental learning requires coincident activation of NMDA and dopamine D1 receptors within the medial prefrontal cortex. J Neurosci. 22:2002;1063-1071 The authors found that infusion of the DA D1 receptor antagonist SCH-23390 and the NMDA-type glutamate receptor antagonist AP-5 synergistically disrupted acquisition of a simple lever-press instrumental response when infused into the rat medial prefrontal cortex before training. Infusion of a PKA inhibitor into the PFC similarly disrupted learning. Impairment of learning occurred at drug doses that did not impair performance of a previously learned response. No drug altered primary feeding or locomotor behaviour, although they did suppress rats' tendency to respond at the food alcove to collect reward; this may be a cause or a consequence of their impaired instrumental learning.
41. Balleine B.W., Dickinson A. Goal-directed instrumental action: contingency and incentive learning and their cortical substrates. Neuropharmacology. 37:1998;407-419.
42. Pecina S., Cagniard B., Berridge K.C., Aldridge J.W., Zhuang X. Hyperdopaminergic mutant mice have higher 'wanting' but not 'liking' for sweet rewards. J Neurosci. 23:2003;9395-9402.
43. Pecina S., Berridge K.C., Parker L.A. Pimozide does not shift palatability: separation of anhedonia from sensorimotor suppression by taste reactivity. Pharmacol Biochem Behav. 58:1997;801-811.
44. Berridge K.C., Robinson T.E. What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? Brain Res Brain Res Rev. 28:1998;309-369.
45. Berridge K.C. Measuring hedonic impact in animals and infants: microstructure of affective taste reactivity patterns. Neurosci Biobehav Rev. 24:2000;173-198.
46. Kelley A.E., Berridge K.C. The neuroscience of natural rewards: relevance to addictive drugs. J Neurosci. 22:2002;3306-3311.
47. O'Brien C.P., Childress A.R., Ehrman R., Robbins S.J. Conditioning factors in drug abuse: can they explain compulsion? J Psychopharmacol. 12:1998;15-22.
48. Di Ciano P., Everitt B.J. Differential control over drug-seeking behavior by drug-associated conditioned reinforcers and discriminative stimuli predictive of drug availability. Behav Neurosci. 117:2003;952-960.
49. Everitt B.J., Robbins T.W. Second-order schedules of drug reinforcement in rats and monkeys: measurement of reinforcing efficacy and drug-seeking behaviour. Psychopharmacology (Berl). 153:2000;17-30.
50. Hutcheson D.M., Parkinson J.A., Robbins T.W., Everitt B.J. The effects of nucleus accumbens core and shell lesions on intravenous heroin self-administration and the acquisition of drug-seeking behaviour under a second-order schedule of heroin reinforcement. Psychopharmacology (Berl). 153:2001;464-472.
51. Di Ciano P., Everitt B.J. Dissociable effects of antagonism of NMDA and AMPA/KA receptors in the nucleus accumbens core and shell on cocaine-seeking behavior. Neuropsychopharmacology. 25:2001;341-360.
52. Sokoloff P., Giros B., Martres M.P., Bouthenet M.L., Schwartz J.C. Molecular cloning and characterization of a novel dopamine receptor (D3) as a target for neuroleptics. Nature. 347:1990;146-151.
53. Di Ciano P., Underwood R.J., Hagan J.J., Everitt B.J. Attenuation of cue-controlled cocaine-seeking by a selective D3 dopamine receptor antagonist SB-277011-A. Neuropsychopharmacology. 28:2003;329-338 The authors provide a recent demonstration that the DA D3 receptor antagonist SB-277011-A suppresses cocaine seeking under a second-order schedule in rats, without affecting cocaine taking under a fixed-ratio-1 schedule, or food seeking on the same second-order schedule as used for cocaine. Such compounds may represent a pharmacological intervention that could be of use in drug addiction.
54. Vorel S.R., Ashby C.R. Jr., Paul M., Liu X., Hayes R., Hagan J.J., Middlemiss D.N., Stemp G., Gardner E.L. Dopamine D3 receptor antagonism inhibits cocaine-seeking and cocaine-enhanced brain reward in rats. J Neurosci. 22:2002;9595-9603.
55. Pilla M., Perachon S., Sautel F., Garrido F., Mann A., Wermuth C.G., Schwartz J.C., Everitt B.J., Sokoloff P. Selective inhibition of cocaine-seeking behaviour by a partial dopamine D3 receptor agonist. Nature. 400:1999;371-375.
56. Cervo L., Carnovali F., Stark J.A., Mennini T. Cocaine-seeking behavior in response to drug-associated stimuli in rats: involvement of D3 and D2 dopamine receptors. Neuropsychopharmacology. 28:2003;1150-1159.
57. Rachlin H: Economic concepts in the behavioral study of addiction. In Choice, Behavioral Economics and Addiction. Edited by Heather N, Vuchinich R. Oxford: Elsevier; 2003:129-149.
58. Petry N.M. A behavioral economic analysis of polydrug abuse in alcoholics: asymmetrical substitution of alcohol and cocaine. Drug Alcohol Depend. 62:2001;31-39.
59. Bickel W.K., DeGrandpre R.J., Higgins S.T. The behavioral economics of concurrent drug reinforcers: a review and reanalysis of drug self-administration research. Psychopharmacology (Berl). 118:1995;250-259.
60. Heyman G.M. An economic approach to animal models of alcoholism. Alcohol Res Health. 24:2000;132-139.
61. Di Chiara G. Nucleus accumbens shell and core dopamine: differential role in behavior and addiction. Behav Brain Res. 137:2002;75-114.
62. Will M.J., Franzblau E.B., Kelley A.E. Nucleus accumbens mu-opioids regulate intake of a high-fat diet via activation of a distributed brain network. J Neurosci. 23:2003;2882-2888 The authors demonstrate that administration of a μ-opioid antagonist into the Acb markedly increases the intake of highly palatable high-fat foods in rats. Infusion of the γ-aminobutyric acid (GABA) agonist muscimol into the ventral tegmental area or lateral hypothalamus blocked this effect; indeed, infusion of muscimol into the lateral hypothalamus suppressed high-fat food intake in its own right, whereas muscimol increased fat intake when infused into the dorsomedial hypothalamus.
63. Zhang M., Kelley A.E. Intake of saccharin, salt, and ethanol solutions is increased by infusion of a mu opioid agonist into the nucleus accumbens. Psychopharmacology (Berl). 159:2002;415-423.
64. Kelley A.E., Bakshi V.P., Haber S.N., Steininger T.L., Will M.J., Zhang M. Opioid modulation of taste hedonics within the ventral striatum. Physiol Behav. 76:2002;365-377.
65. Zhang M., Kelley A.E. Enhanced intake of high-fat food following striatal mu-opioid stimulation: microinjection mapping and fos expression. Neuroscience. 99:2000;267-277.
66. Zhang M., Gosnell B.A., Kelley A.E. Intake of high-fat food is selectively enhanced by mu opioid receptor stimulation within the nucleus accumbens. J Pharmacol Exp Ther. 285:1998;908-914.
67. Kelley A.E., Will M.J., Steininger T.L., Zhang M., Haber S.N. Restricted daily consumption of a highly palatable food (chocolate Ensure(R)) alters striatal enkephalin gene expression. Eur J Neurosci. 18:2003;2592-2598.
68. Robinson T.E., Berridge K.C. The neural basis of drug craving - an incentive-sensitization theory of addiction. Brain Res Brain Res Rev. 18:1993;247-291.
69. Olausson P., Jentsch J.D., Taylor J.R. Repeated nicotine exposure enhances reward-related learning in the rat. Neuropsychopharmacology. 28:2003;1264-1271.
70. Taylor J.R., Horger B.A. Enhanced responding for conditioned reward produced by intra-accumbens amphetamine is potentiated after cocaine sensitization. Psychopharmacology (Berl). 142:1999;31-40.
71. Vezina P., Lorrain D.S., Arnold G.M., Austin J.D., Suto N. Sensitization of midbrain dopamine neuron reactivity promotes the pursuit of amphetamine. J Neurosci. 22:2002;4654-4662.
72. American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders, 4th edn. Washington DC: American Psychiatric Association; 2000.
73. Sax K.W., Strakowski S.M. Behavioral sensitization in humans. J Addict Dis. 20:2001;55-65.
74. Robbins TW, Everitt BJ: Drug addiction: bad habits add up. [news] Nature 1999, 398:567-570.
75. Everitt B.J., Dickinson A., Robbins T.W. The neuropsychological basis of addictive behaviour. Brain Res Brain Res Rev. 36:2001;129-138.
76. Everitt B.J., Wolf M.E. Psychomotor stimulant addiction: a neural systems perspective. J Neurosci. 22:2002;3312-3320.
77. O'Brien C.P., McLellan A.T. Myths about the treatment of addiction. Lancet. 347:1996;237-240.
78. Olmstead M.C., Lafond M.V., Everitt B.J., Dickinson A. Cocaine seeking by rats is a goal-directed action. Behav Neurosci. 115:2001;394-402.
79. Miles F.J., Everitt B.J., Dickinson A. Oral cocaine seeking by rats: action or habit? Behav Neurosci. 117:2003;927-938.
80. Dickinson A., Wood N., Smith J.W. Alcohol seeking by rats: action or habit? Q J Exp Psychol B. 55:2002;331-348 The authors trained rats to respond on two levers for food pellets and 10% ethanol solution. Subsequently, either the food pellets or the ethanol was devalued by pairing it with illness induced by lithium chloride. When lever-pressing was subsequently tested (in extinction), rats pressed the pellet lever less if the pellets (but not the ethanol) had been devalued, indicating that responding for pellets was a goal-directed action. By contrast, the rate at which they pressed the ethanol lever was unaffected by devaluation of the reinforcer (although their consumption of the ethanol was reduced). This suggests that responding for ethanol was not a goal-directed action but a stimulus-response habit, whose performance was not controlled by the consequence of the action.
81. Packard M.G., McGaugh J.L. Inactivation of hippocampus or caudate nucleus with lidocaine differentially affects expression of place and response learning. Neurobiol Learn Mem. 65:1996;65-72.
82. Ito R., Dalley J.W., Robbins T.W., Everitt B.J. Dopamine release in the dorsal striatum during cocaine-seeking behavior under the control of a drug-associated cue. J Neurosci. 22:2002;6247-6253.
83. Killcross A.S., Coutureau E. Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb Cortex. 13:2003;400-408 When rats learn to press a lever for food, their actions are initially goal directed and if the value of the reinforcer is subsequently altered, their responding is sensitive to this. Under certain conditions, extended training can render the response habitual and insensitive to reinforcer revaluation. In this study, rats were given either limited or extended training before receiving excitotoxic lesions of either prelimbic or infralimbic cortex. Following prelimbic lesions, rats' behaviour was habitual after only limited training, which suggests that they lacked their goal-directed system. This supports previous evidence demonstrating the necessity of this structure to perceive action-outcome contingencies and thus to execute goal-directed actions. Intriguingly, rats with infralimbic lesions maintained goal-directed responses even after responding had become habitual in sham-operated controls.
84. Abi-Dargham A., Kegeles L.S., Martinez D., Innis R.B., Laruelle M. Dopamine mediation of positive reinforcing effects of amphetamine in stimulant naive healthy volunteers: results from a large cohort. Eur Neuropsychopharmacol. 13:2003;459-468.
85. Volkow N.D., Fowler J.S., Wang G.J. Role of dopamine in drug reinforcement and addiction in humans: results from imaging studies. Behav Pharmacol. 13:2002;355-366.
86. Yin H.H., Knowlton B.J., Balleine B.W. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur J Neurosci. 19:2004;181-189 The authors demonstrate that instrumental responding in rats with lesions of the dorsolateral striatum continues to be sensitive to devaluation of the reinforcer (that is, their actions are goal directed) beyond the point at which normal rats' actions have become habitual and insensitive to such devaluation. This provides strong support for the long-held view that the dorsal striatum is critical for habit formation.
87. ••] by demonstrating that rats can be 'released' from habitual responding by inactivation of the infralimbic cortex; this structure may therefore act to inhibit goal directed action once a habit has been acquired.