The origins and organization of vertebrate pavlovian conditioning

Cold Spring Harbor Perspectives in Biology

Volumn 8, Issue 1, 2016, Pages

  Fanselow, Michael S ^a   Wassum, Kate M ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTATION; ANIMAL; BIOLOGICAL MODEL; CONDITIONED REFLEX; FEAR; LEARNING; NERVE TRACT; PHYSIOLOGY; REWARD; VERTEBRATE;

ADAPTATION, PHYSIOLOGICAL; ANIMALS; CONDITIONING, CLASSICAL; FEAR; LEARNING; MODELS, BIOLOGICAL; NEURAL PATHWAYS; REWARD; VERTEBRATES;

EID: 84953320080     PISSN: None     EISSN: 19430264     Source Type: Journal    
DOI: 10.1101/cshperspect.a021717     Document Type: Article

References (232)

1. Anagnostaras SG, Wood SC, Shuman T, Cai DJ, Leduc AD, Zurn KR, Zurn JB, Sage JR, Herrera GM. 2010. Automated assessment of Pavlovian conditioned freezing and shock reactivity in mice using the video freeze system. Front Behav Neurosci 4: 158.
2. AnnauZ, Kamin LJ. 1961. The conditioned emotional response as a function of intensity of the US. J Comp Physiol Psychol 54: 428-432.
3. Balleine B. 1994. Asymmetrical interactions between thirst and hunger in Pavlovian-instrumental transfer. Q J Exp Psychol B 47: 211-231.
4. Balleine BW, Killcross S. 2006. Parallel incentive processing: An integrated viewof amygdala function. TrendsNeurosci 29: 272-279.
5. Berridge KC. 1991. Modulation of taste affect by hunger, caloric satiety, and sensory-specific satiety in the rat. Appetite 16: 103-120.
6. Berridge KC. 2007. The debate over dopamine's role in reward: The case for incentive salience. Psychopharmacology 191: 391-431.
7. Berridge KC, Robinson TE. 2003. Parsing reward. Trends Neurosci 26: 507-513.
8. Betts S, Brandon S, Wagner A. 1996. Dissociation of the blocking of conditioned eyeblink and conditioned fear following a shift in US locus. Anim Learn Behav 24: 459-470.
9. Boakes RA. 1977. Performance on learning to associate a stimulus with positive reinforcement, in operant-Pavlovian interactions. In Operant-Pavlovian interactions (ed. Davis H, Hurwitz HMB), pp. 67-97. Erlbaum, Hillsdale, NJ.
10. Bolles R. 1972. Reinforcement, expectancy, and learning. Psychol Rev 79: 394-409.
11. Bolles RC, Fanselow MS. 1980. A perceptual-defensive-recuperative model of fear and pain. Behav Brain Sci 3: 291-301.
12. Bolles RC, Zeigler HP. 1967. Theory of motivation. Harper & Row, New York.
13. Bolles RC, Stokes LW, Younger MS. 1966. Does CS termination reinforce avoidance behavior? J Comp Physiol Psychol 62: 201-207.
14. Bouton ME. 1993. Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. Psychol Bull 114: 80-99.
15. Bouton ME, Bolles RC. 1980. Conditioned fear assessed by freezing and by the suppression of three different baselines. Anim Learn Behav 8: 429-434.
16. Bray S, Rangel A, Shimojo S, Balleine B, O'Doherty JP. 2008. The neural mechanisms underlying the influence of Pavlovian cues on human decision making. J Neurosci 28: 5861-5866.
17. Brogden WJ. 1939. Sensory pre-conditioning. J Exp Psychol 25: 323-332.
18. Brown PL, Jenkins HM. 1968. Auto-shaping of the pigeons key-peck. J Exp Anal Behav 11: 1-8.
19. Brown HD, McCutcheon JE, Cone JJ, Ragozzino ME, Roitman MF. 2011. Primary food reward and reward-predictive stimuli evoke different patterns of phasic dopamine signaling throughout the striatum. Eur J Neurosci 34: 1997-2006.
20. Bush RR, Mosteller F. 1955. Stochastic models for learning. Wiley, Oxford.
21. Canli T, Donegan NH. 1995.Conditioned diminution of the unconditioned response in rabbit eyeblink conditioning: Identifying neural substrates in the cerebellum and brainstem. Behav Neurosci 109: 874-892.
22. Cardinal RN, Parkinson JA, Marbini HD, Toner AJ, Bussey TJ, Robbins TW, Everitt BJ. 2003. Role of the anterior cingulate cortex in the control over behavior by Pavlovian conditioned stimuli in rats. Behav Neurosci 117: 566- 587.
23. Chang SE,Wheeler DS, Holland PC. 2012. Effects of lesions of the amygdala central nucleus on autoshaped lever pressing. Brain Res 1450: 49-56.
24. Christoph GR, Leonzio RJ,Wilcox KS. 1986. Stimulation of the lateral habenula inhibits dopamine-containing neurons in the substantia nigra and ventral tegmental area of the rat. J Neurosci 6: 613-619.
25. Chuhma N, Tanaka KF, Hen R, Rayport S. 2011. Functional connectome of the striatal medium spiny neuron. J Neurosci 31: 1183-1192.
26. Cleland GG, Davey GC. 1982. The effects of satiation and reinforcer develuation on signal-centered behavior in the rat. Learn Motiv 13: 343-360.
27. Cleland GG, Davey GC. 1983. Autoshaping in the rat: The effects of localizable visual and auditory signals for food. J Exp Anal Behav 40: 47-56.
28. Cohen JY, Haesler S, Vong L, Lowell BB, Uchida N. 2012. Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482: 85-88.
29. Cole S, McNally GP. 2007. Opioid receptors mediate direct predictive fear learning: Evidence from one-trial blocking. LearnMem 14: 229-235.
30. Collier G, Hirsch E, Hamlin PH. 1972. The ecological determinants of reinforcement in the rat. Physiol Behav 9: 705-716.
31. Colwill RM, Motzkin DK. 1994. Encoding of the unconditioned stimulus in Pavlovian conditioning. Anim Learn Behav 22: 384-394.
32. Corbit LH, Balleine BW. 2005. Double dissociation of basolateral and central amygdala lesions on the general and outcome-specific forms of Pavlovian-instrumental transfer. J Neurosci 25: 962-970.
33. Corbit LH, Janak PH. 2007. Inactivation of the lateral but not medial dorsal striatum eliminates the excitatory impact of Pavlovian stimuli on instrumental responding. J Neurosci 27: 13977-13981.
34. Corbit LH, Janak PH. 2010. Posterior dorsomedial striatum is critical for both selective instrumental and Pavlovian reward learning. Eur J Neurosci 31: 1312-1321.
35. Corbit LH, Janak PH, Balleine BW. 2007. General and outcome- specific forms of Pavlovian-instrumental transfer: The effect of shifts in motivational state and inactivation of the ventral tegmental area. Eur J Neurosci 26: 3141- 3149.
36. Dardou D, Datiche F, Cattarelli M. 2006. Fos and Egr1 expression in the rat brain in response to olfactory cue after taste-potentiated odor aversion retrieval. Learn Mem 13: 150-160.
37. Dardou D, Datiche F, Cattarelli M. 2007. Does taste or odor activate the same brain networks after retrieval of taste potentiated odor aversion? Neurobiol Learn Mem 88: 186-197.
38. Davis M. 1989. Neural systems involved in fear-potentiated startle. Ann NYAcad Sci 563: 165-183.
39. Davis M,Walker DL, Lee Y. 1997. Amygdala and bed nucleus of the stria terminalis: Differential roles in fear and anxiety measured with the acoustic startle reflex. Philos Trans R Soc Lond B Biol Sci 352: 1675-1687.
40. Day JJ, Roitman MF, Wightman RM, Carelli RM. 2007. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat Neurosci 10: 1020-1028.
41. Dayan P, Berridge KC. 2014. Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation. Cogn Affect Behav Neurosci 14: 473-492.
42. Delamater AR. 2012. On the nature of CS and US representations in Pavlovian learning. Learn Behav 40: 1-23.
43. Delamater AR, Oakeshott S. 2007. Learning about multiple attributes of reward in Pavlovian conditioning. Ann NY Acad Sci 1104: 1-20.
44. Deniau JM, Chevalier G. 1985. Disinhibition as a basic process in the expression of striatal functions. II: The striatonigral influence on thalamocortical cells of the ventromedial thalamic nucleus. Brain Res 334: 227-233.
45. Dickinson A, Balleine BW. 2002. The role of learning in the operation of motivational systems. In Learning, motivation and emotion, volume 3 of Steven's handbook of experimental psychology (ed. Gallistel CR), pp. 497-533.Wiley, New York.
46. Dickinson A, Dawson GR. 1987. Pavlovian processes in the motivational control of instrumental performance. Q J Exp Psychol (Hove) 39: 201-213.
47. Domjan M. 1994. Formulation of a behavior system for sexual conditioning. Psychon Bull Rev 1: 421-428.
48. Domjan M, O'Vary D, Greene P. 1988. Conditioning of appetitive and consummatory sexual behavior in male Japanese quail. J Exp Anal Behav 50: 505-519.
49. Dwyer DM, Killcross S. 2006. Lesions of the basolateral amygdala disrupt conditioning based on the retrieved representations of motivationally significant events. J Neurosci 26: 8305-8309.
50. Eikelboom R, Stewart J. 1982. Conditioning of drug-induced physiological responses. Psychol Rev 89: 507-528.
51. Estes W. 1948. Discriminative conditioning. II: Effects of a Pavlovian conditioned stimulus upon a subsequently established operant response. J Exp Psychol 38: 173-177.
52. Estes WK, Skinner BF. 1941. Some quantitative properties of anxiety. J Exp Psychol 29: 392-400.
53. Falls WA, Miserendino MJ, Davis M. 1992. Extinction of fear-potentiated startle: Blockade by infusion of an NMDA antagonist into the amygdala. J Neurosci 12: 854-863.
54. Fanselow MS. 1979.Naloxone attenuates rats' preference for signaled shock. Physiol Psychol 7: 70-74.
55. Fanselow MS. 1980a. Conditional and unconditional components of post-shock freezing. Pavlov J Biol Sci 15: 177- 182.
56. Fanselow MS. 1980b. Signaled shock-free periods and preference for signaled shock. J Exp Psychol Anim Behavior Processes 6: 65-80.
57. Fanselow MS. 1981.Naloxone and Pavlovian fear conditioning. Learn Motiv 12: 398-419.
58. Fanselow MS. 1982. The postshock activity burst. Anim Learn Behav 10: 448-454.
59. Fanselow MS. 1984. Opiate modulation of the active and inactive components of the postshock reaction: Parallels between naloxone pretreatment and shock intensity. Behav Neurosci 98: 269-277.
60. Fanselow MS. 1986a. Conditioned fear-induced opiate analgesia: A competing motivational state theory of stress analgesia. Ann NY Acad Sci 467: 40-54.
61. Fanselow MS. 1986b. Associative vs topographical accounts of the immediate shock-freezing deficit in rats: Implications for the response selection rules governing speciesspecific defensive reactions. Learn Motiv 17: 16-39.
62. Fanselow MS. 1989. The adaptive function of conditioned defensive behavior: An ecological approach to Pavlovian stimulus substitution theory. In Ethoexperimental approaches to the study of behavior (ed. Blanchard RJ, et al.), pp. 151-166. Kluwer, Boston.
63. Fanselow MS. 1991. The midbrain periaqueductal gray as a coordinator of action in response to fear and anxiety. In The midbrain periaqueductal gray matter: Functional, anatomical and immunohistochemical organization (ed. Depaulis A, Bandler R), pp. 151-173. Plenum, New York.
64. Fanselow MS. 1994. Neural organization of the defensive behavior system responsible for fear. Psychon Bull Rev 1: 429-438.
65. Fanselow MS. 2010. From contextual fear to a dynamic view of memory systems. Trends Cogn Sci 14: 7-15.
66. Fanselow MS, Baackes MP. 1982. Conditioned fear-induced opiate analgesia on the formalin test: Evidence for two aversive motivational systems. Learn Motiv 13: 200-221.
67. Fanselow MS, Bolles RC. 1979. Triggering of the endorphin analgesic reaction by a cue previously associated with shock: Reversal by naloxone. Bull Psychon Soc 14: 88-90.
68. D,L-2-amino-5-phosphonovaleric acid to the basolateral amygdala. Behav Neurosci 108: 210-212.
69. Fanselow MS, LeDoux JE. 1999. Why we think plasticity underlying Pavlovian fear conditioning occurs in the basolateral amygdala. Neuron 23: 229-232.
70. Fanselow MS, Lester LS. 1988. A functional behavioristic approach to aversively motivated behavior: Predatory imminence as a determinant of the topography of defensive behavior. In Evolution and learning (ed. Bolles RC, Beecher MD), pp. 185-211. Erlbaum, Hillsdale, NJ.
71. Fanselow MS, Landeira-Fernandez J, DeCola JP, Kim JJ. 1994. The immediate-shock deficit and postshock analgesia: Implications for the relationship between the analgesic CR and UR. Anim Learn Behav 22: 72-76.
72. Fendt M, Fanselow MS. 1999. The neuroanatomical and neurochemical basis of conditioned fear. Neurosci Biobehav Rev 23: 743-760.
73. Flaherty CF, Becker HC. 1984. Influence of conditioned stimulus context on hyperglycemic conditioned responses. Physiol Behav 33: 587-593.
74. Freeze BS, Kravitz AV, Hammack N, Berke JD, Kreitzer AC. 2013. Control of basal ganglia output by direct and indirect pathway projection neurons. J Neurosci 33: 18531- 18539.
75. Fudge JL, Haber SN. 2000. The central nucleus of the amygdala projection to dopamine subpopulations in primates. Neuroscience 97: 479-494.
76. Gallagher M, GrahamPW, Holland PC. 1990. The amygdala central nucleus and appetitive Pavlovian conditioning: Lesions impair one class of conditioned behavior. J Neurosci 10: 1906-1911.
77. Gerfen CR, Surmeier DJ. 2011. Modulation of striatal projection systems by dopamine. Annu Rev Neurosci 34: 441-466.
78. Gilboa A, Sekeres M, Moscovitch M, Winocur G. 2014. Higher-order conditioning is impaired by hippocampal lesions. Curr Biol 24: 2202-2207.
79. Goldberg JH, Farries MA, Fee MS. 2013. Basal ganglia output to the thalamus: Still a paradox. Trends Neurosci 36: 695-705.
80. Gonzales C, Chesselet MF. 1990. Amygdalonigral pathway: An anterograde study in the rat with Phaseolus vulgaris leucoagglutinin (PHA-L). J Comp Neurol 297: 182-200.
81. Gormezano I, More JW. 1964. Yoked comparisons of contingent and noncontingent US presentations in human eyelid conditioning. Psychon Sci 1: 231-232.
82. Gottfried JA, O'Doherty J, Dolan RJ. 2003. Encoding predictive reward value in human amygdala and orbitofrontal cortex. Science 301: 1104-1107.
83. Graham JM, Desjardins C. 1980. Classical conditioning: Induction of luteinizing hormone and testosterone secretion in anticipation of sexual activity. Science 210: 1039-1041.
84. Grill HJ, Norgren R. 1978. The taste reactivity test. I: Mimetic responses to gustatory stimuli in neurologically normal rats. Brain Res 143: 263-279.
85. Hall J, Parkinson JA, Connor TM, Dickinson A, Everitt BJ. 2001. Involvement of the central nucleus of the amygdala and nucleus accumbens core in mediating Pavlovian influences on instrumental behaviour. Eur J Neurosci 13: 1984-1992.
86. Han JH, Kushner SA, Yiu AP, Cole CJ, Matynia A, Brown RA, Neve RL, Guzowski JF, Silva AJ, Josselyn SA. 2007. Neuronal competition and selection during memory formation. Science 316: 457-460.
87. Haney RZ, Calu DJ, Takahashi YK, Hughes BW, Schoenbaum G. 2010. Inactivation of the central but not the basolateral nucleus of the amygdala disrupts learning in response to overexpectation of reward. J Neurosci 30: 2911-2917.
88. Hart AS, Rutledge RB, Glimcher PW, Phillips PE. 2014. Phasic dopamine release in the rat nucleus accumbens symmetrically encodes a reward prediction error term. J Neurosci 34: 698-704.
89. Hatfield T, Han JS, Conley M, Gallagher M, Holland P. 1996. Neurotoxic lesions of basolateral, but not central, amygdala interfere with Pavlovian second-order conditioning and reinforcer devaluation effects. J Neurosci 16: 5256- 5265.
90. HaubensakW, Kunwar PS, Cai H, Ciocchi S,Wall NR, Ponnusamy R, Biag J, DongHW, Deisseroth K, Callaway EM, et al. 2010. Genetic dissection of an amygdala microcircuit that gates conditioned fear. Nature 468: 270-276.
91. Helmstetter FJ, Fanselow MS. 1987. Effects of naltrexone on learning and performance of conditional fear-induced freezing and opioid analgesia. Physiol Behav 39: 501-505.
92. Helmstetter FJ, Parsons RG, Gafford GM. 2008. Macromolecular synthesis, distributed synaptic plasticity, and fear conditioning. Neurobiol Learn Mem 89: 324-337.
93. Herkenham M, Nauta WJ. 1979. Efferent connections of the habenular nuclei in the rat. J Comp Neurol 187: 19-47.
94. Herry C, Ciocchi S, Senn V, Demmou L,Müller C, Lüthi A. 2008. Switching on and off fear by distinct neuronal circuits. Nature 454: 600-606.
95. Hikosaka O. 2010. The habenula: From stress evasion to value-based decision-making. Nat Rev Neurosci 11: 503- 513.
96. Holland PC. 1977. Conditioned stimulus as a determinant of the form of the Pavlovian conditioned response. J Exp Psychol Anim Behav Process 3: 77-104.
97. Holland PC. 1979a. Differential effects of omission contingencies on various components of Pavlovian appetitive responding in rats. J Exp Psychol 17: 1680-1694.
98. Holland PC. 1979b. The effects of qualitative and quantitative variation in the US on individual components of Pavlovian appetitive conditioned behavior in rats. Anim Learn Behav 7: 424-432.
99. Holland PC. 1981. The effects of satiation after first- and second-order appetitive conditioning in rats. Pavlov J Biol Sci 16: 18-24.
100. Holland PC. 2004. Relations between Pavlovian-instrumental transfer and reinforcer devaluation. J Exp Psychol Anim Behav Process 30: 104-117.
101. Holland PC. 2008. Cognitive versus stimulus-response theories of learning. Learn Behav 36: 227-241.
102. Holland PC, Gallagher M. 1993. Effects of amygdala central nucleus lesions on blocking and unblocking. Behav Neurosci 107: 235-245.
103. Holland PC, Gallagher M. 2003. Double dissociation of the effects of lesions of basolateral and central amygdala on conditioned stimulus-potentiated feeding and Pavlovian- instrumental transfer. Eur J Neurosci 17: 1680-1694.
104. Holland PC, Rescorla RA. 1975. The effect of two ways of devaluing the unconditioned stimulus after first- and second-order appetitive conditioning. J Exp Psychol Anim Behav Process 1: 355-363.
105. Holland PC, Straub JJ. 1979. Differential effects of two ways of devaluing the unconditioned stimulus after Pavlovian appetitive conditioning. J Exp Psychol Anim Behav Process 5: 65-78.
106. Holland PC, Chik Y, Zhang Q. 2001. Inhibitory learning tests of conditioned stimulus associability in rats with lesions of the amygdala central nucleus. Behav Neurosci 115: 1154-1158.
107. Hollerman JR, Schultz W. 1998. Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1: 304-309.
108. Hollis KL, Cadieux EL, Colbert MM. 1989. The biological function of Pavlovian conditioning: A mechanism for mating success in the blue gourami (Trichogaster trichopterus). J Comp Psychol 103: 115-121.
109. Hull CL. 1942. Conditioning: Outline of a systematic theory of learning. In The forty-first yearbook of the National Society for the Study of Education. Part II: The psychology of learning (ed. Henry NB), pp. 61-95. University of Chicago Press, Chicago.
110. Hull C. 1943. Principles of behavior. Appleton, New York.
111. Ito R, Everitt BJ, Robbins TW. 2005. The hippocampus and appetitive Pavlovian conditioning: Effects of excitotoxic hippocampal lesions on conditioned locomotor activity and autoshaping. Hippocampus 15: 713-721
112. Izquierdo A, Murray EA. 2010. Functional interaction of medial mediodorsal thalamic nucleus but not nucleus accumbens with amygdala and orbital prefrontal cortex is essential for adaptive response selection after reinforcer devaluation. J Neurosci 30: 661-669.
113. A receptor-mediated mechanism. J Neurosci 27: 6923- 6930.
114. Johansen JP, Tarpley JW, LeDoux JE, Blair HT. 2010. Neural substrates for expectation-modulated fear learning in the amygdala and periaqueductal gray. Nat Neurosci 13: 979-986.
115. Kamin LJ. 1969. Predictability, surprise, attention, and conditioning. In Punishment and aversive behavior (ed. Campbell BA, Church RM), pp. 279-296. Appleton- Century-Crofts, New York.
116. Kamin LJ, Gaioni SJ. 1974. Compound conditioned emotional response conditioning with differentially salient elements in rats. J Comp Physiol Psychol 87: 591-597.
117. Kiernan MJ, Westbrook RF, Cranney J. 1995. Immediate shock, passive avoidance, and potentiated startle: Implications for the unconditioned response to shock. Anim Learn Behav 23: 22-30.
118. Kim JJ, Krupa DJ, Thompson RF. 1998. Inhibitory cerebelloolivary projections and blocking effect in classical conditioning. Science 279: 570-573.
119. Koffka K. 1935. The principles of Gestalt psychology. Harcourt Brace, New York. Kohler W. 1940. Dynamics in psychology. Liveright, New York.
120. Konorski J. 1948. Conditioned reflexes and neuron organization. Cambridge University Press, New York.
121. Konorski J. 1967. Integrative activity of the brain: An interdisciplinary approach. University of Chicago Press, Chicago.
122. Konorski J, Miller S. 1937. On two types of conditioned reflex. J Gen Psychol 16: 264-272.
123. Krasne FB, Fanselow MS, Zelikowsky M. 2011. Design of a neurally plausible model of fear learning. Front Behav Neurosci 5: 41.
124. KremerEF. 1978. The Rescorla-Wagner model: Losses in associative strength in compound conditioned stimuli. J Exp Psychol Anim Behav Process 4: 22-36.
125. Kruse JM, Overmier JB, KonzWA, Rokke E. 1983. Pavlovian conditioned stimulus effects upon instrumental choice behavior are reinforcer specific. Learn Motiv 14: 165- 181.
126. LeDoux JE. 2000. The amygdala and emotion: A view through fear. In The Amygdala: A functional analysis (ed. Aggleton JP), pp. 289-310. Oxford University Press, Oxford.
127. Lee HJ, Gallagher M, Holland PC. 2010. The central amygdala projection to the substantia nigra reflects prediction error information in appetitive conditioning. Learn Mem 17: 531-538.
128. Lewin K. 1936. Principles of topological psychology.McGraw- Hill, New York.
129. Lockard JS. 1963. Choice of a warning signal or no warning signal in an unavoidable shock situation. J Comp Physiol Psychol 56: 65-80.
130. Lovibond PF. 1983. Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus. J Exp Psychol Anim Behav Process 9: 225-247.
131. Lubow RE,Moore AU. 1959. Latent inhibition: The effect of nonreinforced pre-exposure to the conditional stimulus. J Comp Physiol Psychol 52: 415-419.
132. Mackintosh NJ. 1975. A theory of attention: Variations in the associability of stimuli with reinforcement. Psychol Rev 82.
133. Mackintosh NJ. 1976. Overshadowing and stimulus intensity. Anim Learn Behav 4: 186-192.
134. Malvaez M, Greenfield VY, Wang AS, Yorita AM, Feng L, Linker KE,Monbouquette HG,Wassum KM. 2015. Basolateral amygdala rapid glutamate release encodes an outcome- specific representation vital for reward-predictive cues to selectively invigorate reward-seeking actions. Sci Rep 5: 12511.
135. MarenS, FanselowMS. 1995. Synaptic plasticity in the basolateral amygdala induced by hippocampal formation stimulation in vivo. J Neurosci 15: 7548-7564.
136. Maren S, Yap SA, Goosens KA. 2001. The amygdala is essential for the development of neuronal plasticity in the medial geniculate nucleus during auditory fear conditioning in rats. J Neurosci 21: RC135.
137. MatsumotoM, Hikosaka O. 2007. Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447: 1111-1115.
138. Matthews RN, Domjan M, Ramsey M, Crews D. 2007. Learning effects on sperm competition and reproductive fitness. Psychol Sci 18: 758-762.
139. McDannald MA, Jones JL, Takahashi YK, Schoenbaum G. 2014a. Learning theory: A driving force in understanding orbitofrontal function. Neurobiol Learn Mem 108: 22-27.
140. McDannald MA, Esber GR, Wegener MA, Wied HM, Liu TL, Stalnaker TA, Jones JL, Trageser J, Schoenbaum G. 2014b. Orbitofrontal neurons acquire responses to "valueless" Pavlovian cues during unblocking. eLife 3: e02653.
141. McNally GP, Pigg M, Weidemann G. 2004a. Opioid receptors in the midbrain periaqueductal gray regulate extinction of Pavlovian fear conditioning. J Neurosci 24: 6912- 6919.
142. McNally GP, Pigg M, Weidemann G. 2004b. Blocking, unblocking, and overexpectation of fear: A role for opioid receptors in the regulation of Pavlovian association formation. Behav Neurosci 118: 111-120.
143. Miller NE. 1948. Studies of fear as an acquirable drive fear as motivation and fear-reduction as reinforcement in the learning of new responses. J Exp Psychol 38: 89-101.
144. Miserendino MJ, Sananes CB, Melia KR, Davis M. 1990. Blocking of acquisition but not expression of conditioned fear-potentiated startle by NMDA antagonists in the amygdala. Nature 345: 716-718.
145. Mitchell AS, Browning PG, Baxter MG. 2007. Neurotoxic lesions of the medial mediodorsal nucleus of the thalamus disrupt reinforcer devaluation effects in rhesus monkeys. J Neurosci 27: 11289-11295.
146. Morgan MM, Whitier KL, Hegarty DM, Aicher SA. 2008. Periaqueductal gray neurons project to spinally projecting GABAergic neurons in the rostral ventromedial medulla. Pain 140: 376-386.
147. Murray EA, O'Doherty JP, Schoenbaum G. 2007. What we know and do not know about the functions of the orbitofrontal cortex after 20 years of cross-species studies. J Neurosci 27: 8166-8169.
148. Orsini CA, Kim JH, Knapska E, Maren S. 2011. Hippocampal and prefrontal projections to the basal amygdala mediate contextual regulation of fear after extinction. J Neurosci 31: 17269-17277.
149. Orsini CA, Yan C, Maren S. 2013. Ensemble coding of context- dependent fear memory in the amygdala. Front Behav Neurosci 7: 199.
150. OstlundSB, Balleine BW. 2007. Orbitofrontal cortex mediates outcome encoding in Pavlovian but not instrumental conditioning. J Neurosci 27: 4819-4825.
151. Ostlund SB, Balleine BW. 2008. Differential involvement of the basolateral amygdala and mediodorsal thalamus in instrumental action selection. J Neurosci 28: 4398-4405.
152. Paré D, Quirk GJ, Ledoux JE. 2004. New vistas on amygdala networks in conditioned fear. J Neurophysiol 92: 1-9.
153. Parkes SL, Balleine BW. 2013. Incentive memory: Evidence the basolateral amygdala encodes and the insular cortex retrieves outcome values to guide choice between goaldirected actions. J Neurosci 33: 8753-8763.
154. Parkinson JA, Robbins TW, Everitt BJ. 2000. Dissociable roles of the central and basolateral amygdala in appetitive emotional learning. Eur J Neurosci 12: 405-413.
155. Parkinson JA, Roberts AC, Everitt BJ, Di Ciano P. 2005. Acquisition of instrumental conditioned reinforcement is resistant to the devaluation of the unconditioned stimulus. Q J Exp Psychol B 58: 19-30.
156. Parsons RG, Riedner BA, Gafford GM, Helmstetter FJ. 2006. The formation of auditory fear memory requires the synthesis of protein and mRNA in the auditory thalamus. Neuroscience 141: 1163-1170.
157. Pavlov IP. 1927. Conditioned Reflexes, an investigation of the psychological activity of the cerebral cortex. Oxford University Press, New York.
158. Pavlov I. 1932. The reply of a physiologist to psychologists. Psychol Rev 39: 91-127.
159. Pearce JM, Hall G. 1980. A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol Rev 87.
160. Pecinã S, Berridge KC. 2013. Dopamine or opioid stimulation of nucleus accumbens similarly amplify cue-triggered "wanting" for reward: Entire core and medial shell mapped as substrates for PITenhancement. Eur J Neurosci 37: 1529-1540.
161. Pickens CL. 2008. A limited role for mediodorsal thalamus in devaluation tasks. Behav Neurosci 122: 659-676.
162. Pickens CL, Holland PC. 2004. Conditioning and cognition. Neurosci Biobehav Rev 28: 651-661.
163. Pickens CL, Saddoris MP, Setlow B, Gallagher M, Holland PC, Schoenbaum G. 2003. Different roles for orbitofrontal cortex and basolateral amygdala in a reinforcer devaluation task. J Neurosci 23: 11078-11084.
164. Rescorla RA. 1968. Probability of shock in the presence and absence of CS in fear conditioning. JComp Physiol Psychol 66: 1-5.
165. Rescorla RA. 1971. Summation and retardation tests of latent inhibition. J Comp Physiol Psychol 75: 77-81.
166. Rescorla R. 1973a. Mechanisms of formation and inhibition of conditioned reflex. Academy of Sciences of the U.S.S.R., Moscow.
167. Rescorla RA. 1973b. Effect of US habituation following conditioning. J Comp Physiol Psychol 82: 137-143.
168. Rescorla RA. 1974. Effect of inflation of the unconditioned stimulus value following conditioning. J Comp Physiol Psychol 86: 101-106.
169. Rescorla RA. 1980. Simultaneous and successive associations in sensory preconditioning. J Exp Psychol Anim Behav Process 6: 207-216.
170. Rescorla RA. 1988. Pavlovian conditioning. It's not what you think it is. Am Psychol 43: 151.
171. RescorlaRA. 1994. Transfer of instrumental control mediated by a devalued outcome. Anim Learn Behav 22: 27- 33.
172. Rescorla RA, Solomon RL. 1967. Two-process learning theory: Relationships between Pavlovian conditioning and instrumental learning. Psychol Rev 74: 151-182.
173. Rescorla RA, Wagner AR. 1972. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and non-reinforcement. In Classical conditioning II (ed. Black AH, Prokasy WF), pp. 64-99. Appleton-Century Crofts, New York.
174. Rizley RC, Rescorla RA. 1972. Associations in second-order conditioning and sensory preconditioning. J Comp Physiol Psychol 81: 1-11.
175. Robinson TE, Berridge KC. 1993. The neural basis of drug craving: An incentive-sensitization theory of addiction. BrainRes Brain Res Rev , 18, 247-291
176. Robinson MJ, Berridge KC. 2013. Instant transformation of learned repulsion into motivational "wanting." Curr Biol 23: 282-289.
177. Roitman MF, Wheeler RA, Wightman RM, Carelli RM. 2008. Real-time chemical responses in the nucleus accumbens differentiate rewarding and aversive stimuli. Nat Neurosci 11: 1376-1377.
178. Romanski LM, Clugnet MC, Bordi F, LeDoux JE. 1993. Somatosensory and auditory convergence in the lateral nucleus of the amygdala. Behav Neurosci 107: 444-450.
179. Rouillard C, Freeman AS. 1995. Effects of electrical stimulation of the central nucleus of the amygdala on the in vivo electrophysiological activity of rat nigral dopaminergic neurons. Synapse 21: 348-356.
180. Rozeboom WW. 1958. What is learned? An empirical enigma. Psychol Rev 65: 22-33.
181. Saddoris MP, Gallagher M, Schoenbaum G. 2005. Rapid associative encoding in basolateral amygdala depends on connections with orbitofrontal cortex. Neuron 46: 321-331.
182. Saddoris MP, Holland PC, Gallagher M. 2009. Associatively learned representations of taste outcomes activate tasteencoding neural ensembles in gustatory cortex. JNeurosci 29: 15386-15396.
183. Schoenbaum G, Setlow B, Saddoris MP, Gallagher M. 2003. Encoding predicted outcome and acquired value in orbitofrontal cortex during cue sampling depends upon input from basolateral amygdala. Neuron 39: 855-867.
184. SchultzW. 2002. Getting formalwith dopamine and reward. Neuron 36: 241-263.
185. Siegel S. 1975a. Evidence from rats that morphine tolerance is a learned response. J Comp Physiol Psychol 89: 498-506.
186. Siegel S. 1975b. Conditioning insulin effects. J Comp Physiol Psychol 89: 89-99. 1.
187. Siegel S. 1991. Tolerance: Role of conditioning processes. NIDA Res Monogr 1991: 213-229.
188. Sigmundi RA, Bouton ME, Bolles RC. 1980. Conditioned freezing in the rat as a function of shock intensity and CS modality. Bull Psychon Soc 15: 254-256.
189. Singh T, McDannald MA, Haney RZ, Cerri DH, Schoenbaum G. 2010. Nucleus accumbens core and shell are necessary for reinforcer devaluation effects on Pavlovian conditioned responding. Front Integr Neurosci 4: 7126.
190. SkinnerBF. 1938. The behavior of organisms: An experimental analysis. Appleton-Century, Oxford.
191. Skinner BF. 1950. Are theories of learning necessary? Psychol Rev 57: 193-216.
192. Small DM, Veldhuizen MG, Felsted J, Mak YE, McGlone F. 2008. Separable substrates for anticipatory and consummatory food chemosensation. Neuron 57: 786-797.
193. Smith KS, Tindell AJ, Aldridge JW, Berridge KC. 2009. Ventral pallidum roles in reward and motivation. Behav Brain Res 196: 155-167.
194. Spence KW. 1956. Behavior theory and conditioning. Yale University Press, New Haven.
195. Steinberg EE, Keiflin R, Boivin JR,Witten IB, Deisseroth K, Janak PH. 2013. A causal link between prediction errors, dopamine neurons and learning. Nat Neurosci 16: 966- 973.
196. Surmeier DJ, Ding J, Day M,Wang Z, ShenW. 2007. D1 and D2 dopamine-receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons. Trends Neurosci 30: 228-235.
197. Sutherland NS,Mackintosh NJ. 1971.Mechanisms of animal discrimination learning. Academic, London.
198. Sutton RS. 1988. Learning to predict by methods of temporal differences. Machine Learning 3: 9-44.
199. Sutton R, Barto A. 1990. Time-derivative models of Pavlovian reinforcement. In Learning and computational neuroscience: Foundations of adaptive networks (ed. Gabriel M, Moore J), pp. 497-537. MIT Press, Boston.
200. Swanson LW, Petrovich GD. 1998. What is the amygdala? Trends Neurosci 21: 323-331.
201. Talk A, Kashef A, Gabriel M. 2004. Effects of conditioning during amygdalar inactivation on training-induced neuronal plasticity in the medial geniculate nucleus and cingulate cortex in rabbits (Oryctolagus cuniculus). Behav Neurosci 118: 944-955.
202. Thompson RF. 1988. The neural basis of basic associative learning of discrete behavioral responses. Trends Neurosci 11: 152-155.
203. Thorndike EL. 1898. Animal intelligence: An experimental study of the associative processes in animals. Psychol Rev 2(Suppl): i-109.
204. Timberlake W. 1994. Behavior systems, associationism, and Pavlovian conditioning. Psychon Bull Rev 1: 405-420.
205. TimberlakeW, Fanselow MS. 1994. Symposiumon behavior systems: Learning, neurophysiology, and development. PsychonBull Rev 1: 403-404.
206. Tobler PN, Dickinson A, Schultz W. 2003. Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J Neurosci 23: 10402- 10410.
207. Tolman EC. 1949. There is more than one kind of learning. Psychol Rev 56: 144-155.
208. Tolman EC. 1951. Purposive behavior in animals and men. University of California Press, Berkeley, CA. Ungless MA
209. Magill PJ, Bolam JP. 2004. Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science 303: 2040-2042.
210. Vandercar DH, Schneiderman N. 1967. Interstimulus interval functions in different response systems during classical conditioning. Psychon Sci 9: 9-10.
211. van Vulpen EH, Verwer RW. 1989. Organization of projections from the mediodorsal nucleus of the thalamus to the basolateral complex of the amygdala in the rat. Brain Res 500: 389-394.
212. Veldhuizen MG, Bender G, Constable RT, Small DM. 2007. Trying to detect taste in a tasteless solution: Modulation of early gustatory cortex by attention to taste. Chem Senses 32: 569-581.
213. Waddell J, Morris RW, Bouton ME. 2006. Effects of bed nucleus of the stria terminalis lesions on conditioned anxiety: Aversive conditioning with long-duration conditional stimuli and reinstatement of extinguished fear. Behav Neurosci 120: 324-336.
214. Waelti P, Dickinson A, SchultzW. 2001. Dopamine responses comply with basic assumptions of formal learning theory. Nature 412: 43-48.
215. Wagner AR, Brandon SE. 1989. Evolution of a structured connectionist model of Pavlovian conditioning (AESOP). In Contemporary learning theories: Pavlovian conditioning and the status of traditional learning theory (ed. Klein SB,Mowrer RR), pp. 149-189. Erlbaum, Hillsdale, NJ.
216. Wagner AR, Brandon SE. 2001. A componential theory of Pavlovian conditioning. In Handbook of contemporary learning theories (ed. Mower RR, Klein SB), pp. 23-64. Erlbaum, Mahwah, NJ.
217. Wagner AR, Logan FA, Haberlandt K, Price T. 1968. Stimulus selection in animal discrimination learning. J Exp Psychol 76: 171-180.
218. Wassum KM, Cely IC, Balleine BW, Maidment NT. 2009. Distinct opioid circuits determine the palatability and the desirability of rewarding events. Proc Natl Acad Sci 106: 12512-12517.
219. Wassum KM, Ostlund SB, Balleine BW, Maidment NT. 2011a. Differential dependence of Pavlovian incentive motivation and instrumental incentive learning processes on dopamine signaling. Learn Mem 18: 475-483.
220. Wassum KM, Cely IC, Balleine BW, Maiment NT. 2011b. μ Opioid receptor activation in the basolateral amygdala mediates the learning of increases but not decreases in the incentive value of a food reward. J Neurosci 31: 1583- 1599.
221. Wassum KM, Ostlund SB, Maidment NT. 2012. Phasic mesolimbic dopamine signaling precedes and predicts performance of a self-initiated action sequence task. Biol Psychiatry 71: 846-854.
222. Wassum KM, Ostlund SB, Loewinger GC, Maidment NT. 2013. Phasic mesolimbic dopamine release tracks reward seeking during expression of Pavlovian-to-instrumental transfer. Biol Psychiatry 73: 747-755.
223. Watson JB. 1916. The place of the conditioned-reflex in psychology. Psychol Rev 23: 89.
224. WatsonJB, Rayner R. 1920. Conditioned emotional reactions. Am Psychol 15: 313-317.
225. Williams BA. 1994. Conditioned reinforcement: Experimental and theoretical issues. Behav Anal 17: 261-285.
226. Woods SC, MakousW, Hutton RA. 1969. Temporal parameters of conditioned hypoglycemia. J Comp Physiol Psychol 69: 301-307.
227. Wyvell CL, Berridge KC. 2000. Intra-accumbens amphetamine increases the conditioned incentive salience of sucrose reward: Enhancement of reward "wanting" without enhanced "liking" or response reinforcement. J Neurosci 20: 8122-8130.
228. Young SL, Fanselow MS. 1992. Associative regulation of Pavlovian fear conditioning: Unconditional stimulus intensity, incentive shifts, and latent inhibition. J Exp Psychol Anim Behav Process 18: 400-413.
229. Zamble E, Hadad GM, Mitchell JB, Cutmore TR. 1985. Pavlovian conditioning of sexual arousal: First- and second- order effects. J Exp Psychol Anim Behav Process 11: 598-610.
230. Zelikowsky M, Fanselow MS. 2010. Opioid regulation of Pavlovian overshadowing in fear conditioning. Behav Neurosci 124: 510-519.
231. Zener K, McCurdy HG. 1939. Analysis of motivational factors in conditioned behavior. I: The differential effect of changes in hunger upon conditioned, unconditioned, and spontaneous salivary secretion. J Psychol 8: 321-350.
232. Zentall TR, Hogan DE. 1975. Key pecking in pigeons produced by pairing keylight with inaccessible grain. J Exp Anal Behav 23: 199-206.