Learning and motivational processes contributing to pavlovian–instrumental transfer and their neural bases: Dopamine and beyond

Current Topics in Behavioral Neurosciences

Volumn 27, Issue , 2016, Pages 259-289

  Corbit, Laura H ^a   Balleine, Bernard W ^a  

^a UNIVERSITY OF SYDNEY   (Australia)

Author keywords

Amygdala; Incentive; Instrumental learning; Nucleus accumbens; Pavlovian learning; Reward

Indexed keywords

ACETYLCHOLINE; DOPAMINE; FLUPENTIXOL; MITOGEN ACTIVATED PROTEIN KINASE; RACLOPRIDE;

AMYGDALA; ARTICLE; BEHAVIOR CHANGE; COGNITION; CONDITIONED EMOTIONAL RESPONSE TEST; DECISION MAKING; DORSAL STRIATUM; DRUG ABUSE; DRUG EXPOSURE; HUMAN; LEARNING; MENTAL DISEASE; MOTIVATION; NONHUMAN; NUCLEUS ACCUMBENS; OBESITY; PRIORITY JOURNAL; PROTEIN PHOSPHORYLATION; STRESS; TRAINING; VENTRAL PALLIDUM; VENTRAL TEGMENTUM; ADDICTION; ANIMAL; ANIMAL BEHAVIOR; BRAIN; CONDITIONED REFLEX; DOPAMINERGIC NERVE CELL; FOOD; INSTRUMENTAL CONDITIONING; MENTAL STRESS; NEOSTRIATUM; PHYSIOLOGY;

ANIMALS; BEHAVIOR, ADDICTIVE; BEHAVIOR, ANIMAL; BRAIN; CHOICE BEHAVIOR; CONDITIONING, CLASSICAL; CONDITIONING, OPERANT; DECISION MAKING; DOPAMINE; DOPAMINERGIC NEURONS; FOOD; HUMANS; LEARNING; MOTIVATION; NEOSTRIATUM; NUCLEUS ACCUMBENS; OBESITY; STRESS, PSYCHOLOGICAL; SUBSTANCE-RELATED DISORDERS; TRANSFER (PSYCHOLOGY);

EID: 84975824508     PISSN: 18663370     EISSN: 18663389     Source Type: Book Series    
DOI: 10.1007/7854_2015_388     Document Type: Article

References (115)

1. Apicella P (2007) Leading tonically active neurons of the striatum from reward detection to context recognition. Trends Neurosci 30: 299-306.
2. Balleine BW (1994) Asymmetrical interactions between thirst and hunger in Pavlovian-instrumental transfer. Q J Exp Psychol B 47: 211-231.
3. Balleine BW, Morris RW, Leung BK (2015) Thalamocortical integration of instrumental learning and performance and their disintegration in addiction. Brain Res (Epub ahead of print).
4. Baxter DJ, Zamble E (1982) Reinforcer and response specificity in appetitive transfer of control. Anim Learn Behav 10: 201-210.
5. Beck A, Wustenberg T, Genauck A, Wrase J, Schlagenhauf F, Smolka et al (2012) Effect of brain structure, brain function, and brain connectivity on relapse in alcohol-dependent patients. Arch Gen Psychiatry 69: 842-852.
6. Berridge KC (2007) The debate over dopamine's role in reward: the case for incentive salience. Psychopharmacology 191: 391-431.
7. Bertran-Gonzalez J, Laurent V, Chieng BC, Christie MJ, Balleine BW (2013) Learning-related translocation of δ-opioid receptors on ventral striatal cholinergic interneurons mediates choice between goal-directed actions. J Neurosci 33: 16060-16071.
8. Bindra D (1974) A motivational view of learning, performance, and behaviour modification. Psychol Rev 81: 199-213.
9. Bindra D (1978) How adaptive behavior is produced: a perceptual motivational alternative to response-reinforcement. Behav Brain Sci 1: 41-91.
10. Blundell P, Hall G, Killcross S (2001) Lesions of the basolateral amygdala disrupt selective aspects of reinforcer representation in rats. J Neurosci 21(22): 9018-9026.
11. Brown P (2007) Abnormal oscillatory synchronisation in the motor system leads to impaired movement. Curr Opin Neurobiol 17: 656-664.
12. Brown MT, Tan KR, O'Connor EC, Nikonenko I, Muller D, Lüscher C (2012) Ventral tegmental area GABA projections pause accumbal cholinergic interneurons to enhance associative learning. Nature 492: 452-456.
13. Campese V, McCue M, Lázaro-Muñoz G, Ledoux JE, Cain CK (2013) Development of an aversive Pavlovian-to-instrumental transfer task in rat. Front Behav Neurosci 7: 176.
14. Colwill RM, Rescorla RA (1990) Effect of reinforcer devaluation on discriminative control of instrumental behavior. J Exp Psychol Anim Behav Process 16(1): 40.
15. Colwill RM, Motzkin DK (1994) Encoding of the unconditioned stimulus in Pavlovian conditioning. Anim Learn Behav 22: 384-394.
16. Corbit LH, BalleineBW (2005) Double dissociation of basolateral and central amygdala lesions onthe general and outcome-specific forms of Pavlovian-instrumental transfer. J Neurosci 25: 962-970.
17. Corbit LH, Balleine BW (2011) The general and outcome-specific forms of Pavlovianinstrumental transfer are differentially mediated by the nucleus accumbens core and shell. J Neurosci 31: 11786-11794.
18. Corbit LH, Chieng BC, Balleine BW (2014) Effects of repeated cocaine exposure on habit learning and reversal by N-acetylcysteine. Neuropsychopharmacology 39: 1893-1901.
19. Corbit LH, Janak PH (2007a) Inactivation of the lateral but not medial dorsal striatum eliminates the excitatory impact of Pavlovian stimuli on instrumental responding. J Neurosci 27: 13977-13981.
20. Corbit LH, Janak PH (2007b) Ethanol-associated cues produce general Pavlovian-instrumental transfer. Alcohol Clin Exp Res 31: 766-774.
21. Corbit LH, Janak PH (2010) Posterior dorsomedial striatum is critical for both selective instrumental and Pavlovian reward learning. Eur J Neurosci 31: 1312-1321.
22. Corbit LH, Janak PH, Balleine BW (2007) General and outcome-specific forms of Pavlovian-instrumental transfer: the effect of shifts in motivational state and inactivation of the ventral tegmental area. Eur J Neurosci 26: 3141-3149.
23. Corbit LH, Muir JL, Balleine BW (2001) The role of the nucleus accumbens in instrumental conditioning: evidence for a functional dissociation between core and shell. J Neurosci 21: 3251-3260.
24. Corbit LH, Muir JL, Balleine BW (2003) Lesions of mediodorsal thalamus and anterior thalamic nuclei produce dissociable effects on instrumental conditioning in rats. Eur J Neurosci 18: 1286-1294.
25. Corbit LH, Nie H, Janak PH (2012) Habitual alcohol seeking: time course and the contribution of subregions of the dorsal striatum. Biol Psychiatry 72: 389-395.
26. Cui G, Jun SB, Jin X, Pham MD, Vogel SS, Lovinger DM, Costa RM (2013) Concurrent activation of striatal direct and indirect pathways during action initiation. Nature 494: 238-242.
27. de Borchgrave R, Rawlins JN, Dickinson A, Balleine BW (2002) Effects of cytotoxic nucleus accumbens lesions on instrumental conditioning in rats. Exp Brain Res 144: 50-68.
28. Dias-Ferreira E, Sousa JC, Melo I, Morgado P, Mesquita AR, Cerqueira JJ, Costa RM, Sousa N (2009) Chronic stress causes frontostriatal reorganization and affects decision-making. Science 325: 621-625.
29. Dickinson A (1980) Contemporary animal learning theory. Cambridge University Press, Cambridge.
30. Dickinson A (1985) Actions and habits-the development of behavioral autonomy. Philos Trans R Soc Lond Ser B Biol Sci 308: 67-78.
31. Dickinson A, Balleine BW (1994) Motivational control of goal-directed action. Anim learn Behav 22: 1-18.
32. Dickinson A, Balleine BW (2002) Steven's handbook of experimental psychology: learning, motivation and emotion. In: Gallistel C (ed) The role of learning in the operation of motivational systems, vol 3. Wiley, New York, pp 497-534.
33. Dickinson A, Dawson GR (1987) Pavlovian processes in the motivational control of instrumental performance. Q J Exp Psychol Sect B Comp Physiol Psychol 39: 201-213.
34. Dickinson A, Dearing MF (1979) Appetitive-aversive interactions and inhibitory processes. In: Dickinson A, Boakes RA (eds) Mechanism of learning and motivation. Lawrence Erlbaum Associates, Hillsdale, NJ, pp 203-231.
35. Dickinson A, Smith J, Mirenowicz J (2000) Dissociation of Pavlovian and instrumental incentive learning under dopamine antagonists. Behav Neurosci 114: 468-83.
36. Ding JB, Guzman JN, Peterson JD, Goldberg JA, Surmeier DJ (2010) Thalamic gating of corticostriatal signaling by cholinergic interneurons. Neuron 67: 294-307.
37. Estes WK (1943) Discriminative conditioning I: a discriminative property of conditioned anticipation. J Exp Psychol 32: 150-155.
38. Fuchs RA, Branham RK, See RE (2006) Different neural substrates mediate cocaine seeking after abstinence versus extinction training: a critical role for the dorsolateral caudate-putamen. J Neurosci 26: 3584-3588.
39. Furlong TM, Jayaweera HK, Balleine BW, Corbit LH (2014) Binge-like consumption of a palatable food accelerates habitual control of behavior and is dependent on activation of the dorsolateral striatum. J Neurosci 34: 5012-5022.
40. Ganesan R, Pearce JM (1988) Effect of changing the unconditioned stimulus on appetitive blocking. J Exp Psychol Anim Behav Proc 14(3): 280-291.
41. Garbusow M, Schad DJ, Sebold M, Friedel E, Bernhardt N, Koch SP et al (2015) Pavlovian-to-instrumental transfer effects in the nucleus accumbens relate to relapse in alcohol dependence. Addict Biol (Epub ahead of print).
42. Garbusow M, Schad DJ, Sommer C, Jünger E, Sebold M, Friedel E, Wendt J, Kathmann N, Schlagenhauf F, Zimmermann US, Heinz A, Huys QJ, Rapp MA (2014) Pavlovian-to-instrumental transfer in alcohol dependence: a pilot study. Neuropsychobiology 70: 111-121.
43. Geisler S, Marinelli M, Degarmo B, Becker ML, Freiman AJ, Beales M, Meredith GE, Zahm DS (2008) Prominent activation of brainstem and pallidal afferents of the ventral tegmental area by cocaine. Neuropsychopharmacology 33: 2688-2700.
44. Gerfen CR, Surmeier DJ (2011) Modulation of striatal projection systems by dopamine. Annu Rev Neurosci 34: 441-466.
45. Goldberg JA, Ding JB, Surmeier DJ (2012) Muscarinic modulation of striatal function and circuitry. Handb Exp Pharmacol 208: 223-241.
46. Hall J, Parkinson JA, Connor TM, Dickinson A, Everitt BJ (2001) Involvement of the central nucleus of the amygdala and nucleus accumbens core in mediating Pavlovian influences on instrumental behavior. Eur J Neurosci 13: 1984-1992.
47. Heimer L, Zahm DS, Churchill L, Kalivas PW, Wohltmann C (1991) Specificity in the projection patterns of acumbal core and shell in the rat. Neuroscience 41: 89-125.
48. Heinz A, Siessmeier T, Wrase J, Hermann D, Klein S, Grusser S, Flor H, Braus D, Buchholz HG, Grunder G, Schreckenberger M, Smolka M, Rosch F, Mann K, Bartenstein P (2004) Correlation between dopamine D2 receptors in the ventral striatum and central processing of alcohol cues and craving. Am J Psychiatry 161: 1783-1789.
49. Holland PC (2004) Relations between Pavlovian-instrumental transfer and reinforcer devaluation. J Exp Psychol Anim Behav Process 30: 104-117.
50. Holland PC, Gallagher M (2003) Double dissociation of the effects of lesion of basolateral and central amygdala on conditioned stimulus-potentiated feeding and Pavlovian-instrumental transfer. Eur J Neurosci 17: 1680-1694.
51. Holland PC, Rescorla RA (1975) The effect of two ways of devaluing the unconditioned stimulus after first-and second-order appetitive conditioning. J Exp Psychol Anim Behav Process 1: 355-363.
52. Holmes NM, Marchand AR, Coutureau E (2010) Pavlovian to instrumental transfer: a neurobehavioural perspective. Neurosci Biobehav Rev 34: 1277-1295.
53. Kalivas PW, Churchill L, Romanides A (1999) Involvement of the pallidal-thalamocortical circuit in adaptive behavior. Ann N Y Acad Sci 877: 64-70.
54. Konorski J (1967) Integrative activity of the brain. University of Chicago Press, Chicago.
55. Kravitz AV, Tye LD, Kreitzer AC (2012) Distinct roles for direct and indirect pathway striatal neurons in reinforcement. Nat Neurosci 15: 816-818.
56. Kruse JM, Overmier JB, Konz WA, Rokke E (1983) Pavlovian conditioned stimulus effects upon instrumental choice behavior are reinforcer specific. Learn Motiv 14(2): 165-181.
57. Laurent V, Leung B, Maidment N, Balleine BW (2012) Mu-and delta-opioid-related processes in the accumbens core and shell differentially mediate the influence of reward-guided and stimulus-guided decisions on choice. J Neurosci 32: 1875-1883.
58. Laurent V, Bertran-Gonzalez J, Chieng BC, Balleine BW (2014) Delta-opioid and dopaminergic processes in accumbens shell modulate the cholinergic control of predictive learning and choice. J Neurosci 34: 1358-1369.
59. Laurent V, Wong FL, Balleine BW (2015) δ-opioid receptors in the accumbens shell mediate the influence of both excitatory and inhibitory predictions on choice. Br J Pharmacol 172: 562-570.
60. Le Blanc KH, Maidment NT, Ostlund SB (2013) Repeated cocaine exposure facilitates the expression of incentive motivation and induces habitual control in rats. PLoS One 8(4): e61355.
61. Leung BK, Balleine BW (2013) The ventral striato-pallidal pathway mediates the effect of predictive learning on choice between goal-directed actions. J Neurosci 33: 13848-13860.
62. Leung BK, Balleine BW (2015) Ventral pallidal projections to mediodorsal thalamus and ventral tegmental area play distinct roles in outcome-specific Pavlovian-instrumental transfer. J Neurosci 35: 4953-4964.
63. Lex A, Hauber W (2008) Dopamine D1 and D2 receptors in the nucleus accumbens core and shell mediate Pavlovian-instrumental transfer. Learn Mem. 15: 483-491.
64. Lovibond PF (1983) Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus. J Exp Psychol Anim Behav Process 9: 225-247.
65. Lu XY, Ghasemzadeh MB, Kalivas PW (1998) Expression of D1 receptor, D2 receptor, substance P and enkephalin messenger RNAs in the neurons projecting from the nucleus accumbens. Neuroscience 82: 767-780.
66. Mahler SV, Aston-Jones GS (2012) Fos activation of selective afferents to ventral tegmental area during cue-induced reinstatement of cocaine seeking in rats. J Neurosci 32: 13026-13309.
67. Mahler SV, Vazey EM, Beckley JT, Keistler CR, McGlinchey EM, Kaufling J, Wilson SP, Deisseroth K, Woodward JJ, Aston-Jones G (2014) Designer receptors show role for ventral pallidum input to ventral tegmental area in cocaine seeking. Nat Neurosci 17: 577-585.
68. Martinovic J, Jones A, Christiansen P, Rose AK, Hogarth L, Field M (2014) Electrophysiological responses to alcohol cues are not associated with Pavlovian-to-instrumental transfer in social drinkers. PLoS One 9(4): e94605.
69. McAlonan GM, Robbins TW, Everitt BJ (1993) Effects of medial dorsal thalamic and ventral pallidal lesions on the acquisition of a conditioned place preference: further evidence for the involvement of the ventral striatopallidal system in reward-related processes. Neuroscience 52: 605-620.
70. Mink JW (1996) The basal ganglia: focused selection and inhibition of competing motor programs. Prog Neurobiol 50: 381-425.
71. Mogenson GJ, Jones DL, Yim CY (1980) From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol 14: 69-97.
72. Morgado P, Silva M, Sousa N, Cerqueira JJ (2012) Stress transiently affects Pavlovian-to-instrumental transfer. Front Neurosci 6: 93.
73. Mowrer OH (1960) Learning theory and behavior, Wiley, New York.
74. Nelson A, Killcross S (2006) Amphetamine exposure enhances habit formation. J Neurosci 26: 3805-3812.
75. Murschall A, Hauber W (2006) Inactivation of the ventral tegmental area abolished the general excitatory influence of Pavlovian cues on instrumental performance. Learn Mem 13: 123-126.
76. Ostlund SB, Balleine BW (2007) Instrumental reinstatement depends on sensory-and motivationally-specific features of the instrumental outcome. Learn Behav 35(1): 43-52.
77. Ostlund SB, Balleine BW (2008) Differential involvement of the basolateral amygdala and mediodorsal thalamus in instrumental action selection. J Neurosci 28: 4398-4405.
78. Ostlund SB, Maidment NT (2012) Dopamine receptor blockade attenuates the general incentive motivational effects of noncontingently delivered rewards and reward-paired cues without affecting their ability to bias action selection. Neuropsychopharmacology 37: 508-519.
79. Overmier JB, Lawry JA (1979) Pavlovian conditioning and the mediation of behavior. Psychology of Learning and Motivation 13: 1-55.
80. Pavlov IP (1927) Conditioned reflexes. Oxford University Press, Oxford, UK.
81. Parnaudeau S, Taylor K, Bolkan SS, Ward RD, Balsam PD, Kellendonk C (2015) Mediodorsal thalamus hypofunction impairs flexible goal-directed behavior. Biol Psychiatry 77: 445-453.
82. Peciña S, Schulkin J, Berridge KC (2006) Nucleus accumbens corticotropin-releasing factor increases cue-triggered motivation for sucrose reward: paradoxical positive incentive effects in stress? BMC Biol 4: 8.
83. Peciña S, Berridge KC (2013) Dopamine or opioid stimulation of nucleus accumbens similarly amplify cue-triggered 'wanting' for reward: entire core and medial shell mapped as substrates for PIT enhancement. Eur J Neurosci 37(9): 1529-1540.
84. Pielock SM, Lex B, Hauber W (2011) The role of dopamine in the dorsomedial striatum in general and outcome-selective Pavlovian-instrumental transfer. Eur J Neurosci 33: 717-725.
85. Pool E, Brosch T, Delplanque S, Sander D (2015) Stress increases cue-triggered "wanting" for sweet reward in humans. J Exp Psychol Anim Learn Cogn 41: 128-136.
86. Ray OS, Stein L (1959) Generalization of conditioned suppression. J Exp Anal Behav 2: 357-361.
87. Rescorla RA (1994) Transfer of instrumental control mediated by a devalued outcome. Anim Learn Behav 22: 27-33.
88. Rescorla RA, LoLordo VM (1965) Inhibition of avoidance behavior. J Comp Physiol Psychol 59: 406-412.
89. Rescorla RA, Solomon RL (1967) Two-process learning theory: relationship between Pavlovian conditioning and instrumental learning. Psychol Rev 74: 151-182.
90. Saddoris MP, Stamatakis A, Carelli RM (2011) Neural correlates of Pavlovian-to-instrumental transfer in the nucleus accumbens shell are selectively potentiated following cocaine self-administration. Eur J Neurosci 33: 2274-2287.
91. Salamone JD, Correa M, Farrar A, Mingote SM (2007) Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology 191: 461-482.
92. Scherrer G, Tryoen-Tóth P, Filliol D, Matifas A, Laustriat D, Cao YQ, Basbaum AI, Dierich A, Vonesh JL, Gavériaux-Ruff C, Kieffer BL (2006) Knockin mice expressing fluorescent delta-opioid receptors uncover G protein-coupled receptor dynamics in vivo. Proc Natl Acad Sci 103: 9691-9696.
93. Schwabe L, Wolf OT (2010) Socially evaluated cold pressor stress after instrumental learning favors habits over goal-directed action. Psychoneuroendocrinology. 35: 977-986.
94. Schultz W (2006) Behavioral theories and the neurophysiology of reward. Annu Rev Psychol 2006(57): 87-115.
95. Shiflett MW (2012) The effects of amphetamine exposure on outcome-selective Pavlovian-instrumental transfer in rats. Psychopharmacology 223(3): 361-370.
96. Shiflett MW, Balleine BW (2010) At the limbic-motor interface: disconnection of basolateral amygdala from nucleus accumbens core and shell reveals dissociable components of incentive motivation. Eur J Neurosci 32: 1735-1743.
97. Smith RJ, Lobo MK, Spencer S, Kalivas PW (2013) Cocaine-induced adaptations in D1 and D2 accumbens projection neurons (a dichotomy not necessarily synonymous with direct and indirect pathways). Curr Opin Neurobiol 23: 546-552.
98. Soares-Cunha C, Coimbra B, Borges S, Carvalho MM, Rodrigues AJ, Sousa N (2014) The motivational drive to natural rewards is modulated by prenatal glucocorticoid exposure. Transl Psychiatry. 2014(4): e397.
99. Solomon RL, Turner LH (1962) Discriminative classical conditioning in dogs paralyzed by curare can later control discriminative discriminative avoidance responses in the normal state. Psychol Rev 69: 202-219.
100. Stocco A (2012) Acetylcholine-based entropy in response selection: a model of how striatal interneurons modulate exploration, exploitation, and response variability in decision-making. Front Neurosci 6: 18.
101. Toates FM (1986) Motivational systems. Cambridge University Press, Cambridge.
102. Trapold MA, Overmier JB (1972) The second learning process in instrumental learning. In: Black AA, Prokasy WF (eds) Classical conditioning II: current research and theory. Appleton-Century-Crofts, New York, pp 427-452.
103. Tripathi A, Prensa L, Mengual E (2013) Axonal branching patterns of ventral pallidal neurons in the rat. Brain Struct Funct 218: 1133-1157.
104. Vanderschuren LJ, Di Ciano P, Everitt BJ (2005) Involvement of the dorsal striatum in cue-controlled cocaine seeking. J Neurosci 25: 8665-8670.
105. Wassum KM, Ostlund SB, Loewinger GC, Maidment NT (2013) Phasic mesolimbic dopamine release tracks reward seeking during expression of Pavlovian-to-instrumental transfer. Biol Psychiatry 73(8): 747-755.
106. Watson P, Wiers RW, Hommel B, de Wit S (2014) Working for food you don't desire cues interfere with goal-directed food-seeking. Appetite 79: 139-148.
107. Wiers CE, Stelzel C, Park SQ, Gawron CK, Ludwig VU, Gutwinski S, Heinz A, Lindenmeyer J, Wiers RW, Walter H, Bermpohl F (2014) Neural correlates of alcohol-approach bias in alcohol addiction: the spirit is willing but the flesh is weak for spirits. Neuropsychopharmacology 39: 688-697.
108. Winterbauer NE, Balleine BW (2005) Motivational control of second-order conditioning. J Exp Psychol Anim Behav Process 31(3): 334-340.
109. Wyvell CL, Berridge KC (2000) Intra-accumbens amphetamine increases the conditioned incentive salience of sucrose reward: enhancement of reward "wanting" without enhanced "liking" or response reinforcement. J Neurosci 20: 8122-8130.
110. Wyvell CL, Berridge KC (2001) Incentive sensitization by previous amphetamine exposure: increased cue-triggered "wanting" for sucrose reward. J Neurosci 21: 7831-7840.
111. Yin HH, Ostlund SB, Knowlton BJ, Balleine BW (2005) The role of the dorsomedial striatum in instrumental conditioning. Eur J Neurosci 22: 513-523.
112. Zahm DS (2000) An integrative neuroanatomical perspective on some subcortical substrates of adaptive responding with emphasis on the nucleus accumbens. Neurosci Biobehav Rev 24: 85-105.
113. Zahm DS, Heimer L (1990) Two transpallidal pathways originating in the rat nucleus accumbens. J Comp Neurol 302: 437-446.
114. Zener K (1937) The significance of behavior accompanying conditioned salivary secretion for theories of the conditioned response. Am J Psychol 50: 384-403.
115. Zhou L, Furuta T, Kaneko T (2003) Chemical organization of projection neurons in the rat accumbens nucleus and olfactory tubercle. Neuroscience 120: 783-798.