Tripartite mechanism of extinction suggested by dopamine neuron activity and temporal difference model

Journal of Neuroscience

Volumn 28, Issue 39, 2008, Pages 9619-9631

  Pan, Wei Xing ^a,c   Schmidt, Robert ^a,d   Wickens, Jeffery R ^a,b   Hyland, Brian I ^a  

^a UNIVERSITY OF OTAGO   (New Zealand)

^b OKINAWA INSTITUTE OF SCIENCE AND TECHNOLOGY GRADUATE UNIVERSITY   (Japan)

^c NATIONAL INSTITUTE ON DRUG ABUSE   (United States)

^d HUMBOLDT UNIVERSITY   (Germany)

Author keywords

Classical conditioning; Dopaminergic neuron; Extinction; Extracellular recording; Reward learning; Temporal difference model

Indexed keywords

4 AMINOBUTYRIC ACID A RECEPTOR; 4 AMINOBUTYRIC ACID RECEPTOR BLOCKING AGENT; BICUCULLINE; DOPAMINE;

ANIMAL EXPERIMENT; ARTICLE; CONDITIONING; CONTROLLED STUDY; DOPAMINERGIC NERVE CELL; ELECTROENCEPHALOGRAM; LEARNING; MALE; MATHEMATICAL MODEL; MESENCEPHALON; NONHUMAN; PREDICTION; PRIORITY JOURNAL; PROTEIN EXPRESSION; RAT; REINFORCEMENT; ACTION POTENTIAL; ALGORITHM; ANIMAL; ANIMAL BEHAVIOR; ASSOCIATION; AUDITORY STIMULATION; BIOLOGICAL MODEL; CYTOLOGY; DRUG EFFECT; METABOLISM; METHODOLOGY; NERVE CELL; PHYSIOLOGY; SUBSTANTIA NIGRA; VENTRAL TEGMENTUM; WISTAR RAT;

ACOUSTIC STIMULATION; ACTION POTENTIALS; ALGORITHMS; ANIMALS; BEHAVIOR, ANIMAL; BICUCULLINE; CONDITIONING (PSYCHOLOGY); CUES; DOPAMINE; EXTINCTION, PSYCHOLOGICAL; GABA ANTAGONISTS; MALE; MODELS, NEUROLOGICAL; NEURONS; RATS; RATS, WISTAR; SUBSTANTIA NIGRA; VENTRAL TEGMENTAL AREA;

EID: 55749102442     PISSN: 02706474     EISSN: 02706474     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.0255-08.2008     Document Type: Article

References (48)

1. Aebischer P, Schultz W (1984) The activity of pars compacta neurons of the monkey substantia nigra is depressed by apomorphine. Neurosci Lett 50:25-29.
2. Bouton ME, Moody EW (2004) Memory processes in classical conditioning. Neurosci Biobehav Rev 28:663-674.
3. Chiodo LA, Bannon MJ, Grace AA, Roth RH, Bunney BS (1984) Evidence for the absence of impulse-regulating somatodendritic and synthesis-modulating nerve terminal autoreceptors on subpopulations of mesocortical dopamine neurons. Neuroscience 12:1-16.
4. Comoli E, Coizet V, Boyes J, Bolam JP, Canteras NS, Quirk RH, Overton PG, Redgrave P (2003) A direct projection from superior colliculus to substantia nigra for detecting salient visual events. Nat Neurosci 6:974-980.
5. Delamater AR (2004) Experimental extinction in pavlovian conditioning: behavioural and neuroscience perspectives. Q J Exp Psychol B 57:97-132.
6. Diamond DM, Weinberger NM (1986) Classical conditioning rapidly induces specific changes in frequency receptive fields of single neurons in secondary and ventral ectosylvian auditory cortical fields. Brain Res 372:357-360.
7. Froemke RC, Merzenich MM, Schreiner CE (2007) A synaptic memory trace for cortical receptive field plasticity. Nature 450:425-429.
8. Harris JA, Westbrook RF (1998) Evidence that GABA transmission mediates context-specific extinction of learned fear. Psychopharmacology (Berl) 140:105-115.
9. Heil P (1997) Auditory cortical onset responses revisited. I. First-spike timing. J Neurophysiol 77:2616-2641.
10. Hendersen RW (1978) Forgetting of conditioned fear inhibition. Learn Motiv 9:16-30.
11. Hollerman JR, Schultz W (1998) Dopamine neurons report an error in the temporal prediction of reward during learning. Nat Neurosci 1:304-309.
12. Hull CL (1943) Principles of behavior. New York: Appleton-Century-Crofts.
13. Hyland BI, Reynolds JN, Hay J, Perk CG, Miller R (2002) Firing modes of midbrain dopamine cells in the freely moving rat. Neuroscience 114:475-492.
14. Ji H, Shepard PD (2007) Lateral habenula stimulation inhibits rat midbrain dopamine neurons through a GABA(A) receptor-mediated mechanism. J Neurosci 27:6923-6930.
15. Klopf AH (1988) A neuronal model of classical conditioning. Psychobiology 16:85-125.
16. Kobayashi Y, Okada K (2007) Reward prediction error computation in the pedunculopontine tegmental nucleus neurons. Ann N Y Acad Sci 1104:310-323.
17. Komura Y, Tamura R, Uwano T, Nishijo H, Kaga K, Ono T (2001) Retrospective and prospective coding for predicted reward in the sensory thalamus. Nature 412:546-549.
18. Konorski J (1948) Conditioned reflexes and neuron organisation. New York: Cambridge UP.
19. Lammel S, Hetzel A, Häckel O, Jones I, Liss B, Roeper J (2008) Unique properties of mesoprefrontal neurons within a dual mesocorticolimbic dopamine system. Neuron 57:760-773.
20. Matsumoto M, Hikosaka O (2007) Lateral habenula as a source of negative reward signals in dopamine neurons. Nature 447:1111-1115.
21. Mauk MD, Ohyama T (2004) Extinction as new learning versus unlearning: considerations from a computer simulation of the cerebellum. Learn Mem 11:566-571.
22. McClelland JL, Rumelhart DE (1985) Distributed memory and the representation of general and specific information. J Exp Psychol Gen 114:159-197.
23. Medina JF, Garcia KS, Mauk MD (2001) A mechanism for savings in the cerebellum. J Neurosci 21:4081-4089.
24. Medina JF, Nores WL, Mauk MD (2002) Inhibition of climbing fibres is a signal for the extinction of conditioned eyelid responses. Nature 416:330-333.
25. Mirenowicz J, Schultz W (1996) Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379:449-451.
26. Miyashita T, Kubik S, Lewandowski G, Guzowski JF (2007) Networks of neurons, networks of genes: an integrated view of memory consolidation. Neurobiol Learn Mem 89:269-284.
27. Montague PR, Dayan P, Sejnowski TJ (1996) A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J Neurosci 16:1936-1947.
28. Myers KM, Davis M (2002) Behavioral and neural analysis of extinction. Neuron 36:567-584.
29. Myers KM, Ressler KJ, Davis M (2006) Different mechanisms of fear extinction dependent on length of time since fear acquisition. Learn Mem 13:216-223.
30. Pan WX, Hyland BI (2005) Pedunculopontine tegmental nucleus controls conditioned responses of midbrain dopamine neurons in behaving rats. J Neurosci 25:4725-4732.
31. Pan WX, Schmidt R, Wickens JR, Hyland BI (2005) Dopamine cells respond to predicted events during classical conditioning: evidence for eligibility traces in the reward-learning network. J Neurosci 25:6235-6242.
32. Parker D, Grillner S (1999) Activity-dependent metaplasticity of inhibitory and excitatory synaptic transmission in the lamprey spinal cord locomotor network. J Neurosci 19:1647-1656.
33. Pavlov IP (1927) Conditioned reflexes. Oxford: Oxford UP.
34. Paxinos G, Watson C (1997) The rat brain in stereotaxic coordinates, Ed 3. London: Academic.
35. Rescorla RA (2001) Retraining of extinguished pavlovian stimuli. J Exp Psychol Anim Behav Process 27:115-124.
36. Rescorla RA (2004) Spontaneous recovery. Learn Mem 11:501-509.
37. Rescorla RA, Wagner AR (1972) A theory of pavlovian conditioning: variations in the effectiveness of reinforcement and non-reinforcement. In: Classical conditioning II: current research and theory (Black AH, Prokasy WF, eds), pp 64-99. New York: Appleton-Century-Crofts.
38. Roesch MR, Calu DJ, Schoenbaum G (2007) Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat Neurosci 10:1615-1624.
39. Satoh T, Nakai S, Sato T, Kimura M (2003) Correlated coding of motivation and outcome of decision by dopamine neurons. J Neurosci 23:9913-9923.
40. Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80:1-27.
41. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593-1599.
42. Sutton RS (1988) Learning to predict by the methods of temporal differences. Mach Learn 3:9-44.
43. Tepper JM, Lee CR (2007) GABAergic control of substantia nigra dopaminergic neurons. In: GABA and the basal ganglia: from molecules to systems (Tepper JM, Abercrombie ED, Bolam JP, eds). Amsterdam: Elsevier.
44. Tobler PN, Dickinson A, Schultz W (2003) Coding of predicted reward omission by dopamine neurons in a conditioned inhibition paradigm. J Neurosci 23:10402-10410.
45. Toyomitsu Y, Nishijo H, Uwano T, Kuratsu J, Ono T (2002) Neuronal responses of the rat amygdala during extinction and reassociation learning in elementary and configural associative tasks. Eur J Neurosci 15:753-768.
46. Waelti P, Dickinson A, Schultz W (2001) Dopamine responses comply with basic assumptions of formal learning theory. Nature 412:43-48.
47. Wheeler DS, Stout SC, Miller RR (2004) Interaction of retention interval with CS-preexposure and extinction treatments: symmetry with respect to primacy. Learn Behav 32:335-347.
48. Wixted JT (2004) The psychology and neuroscience of forgetting. Annu Rev Psychol 55:235-269.