Dynamic shaping of dopamine signals during probabilistic Pavlovian conditioning

Neurobiology of Learning and Memory

Volumn 117, Issue , 2015, Pages 84-92

  Hart, Andrew S ^a,b   Clark, Jeremy J ^a,b   Phillips, Paul E M ^a,b  

^a UNIVERSITY OF WASHINGTON SCHOOL OF MEDICINE   (United States)

^b UNIVERSITY OF WASHINGTON   (United States)

Author keywords

Dopamine; Pavlovian; Reinforcement learning; Reward prediction error; Uncertainty

Indexed keywords

CARBON FIBER; DOPAMINE;

ANIMAL BEHAVIOR; ANIMAL EXPERIMENT; ANIMAL TISSUE; ARTICLE; ASSOCIATION; CONDITIONING; CYCLIC POTENTIOMETRY; DOPAMINE RELEASE; DOPAMINERGIC ACTIVITY; DOPAMINERGIC NERVE CELL; DOPAMINERGIC TRANSMISSION; ELECTRODE; EVOKED RESPONSE; LEARNING; MALE; NONHUMAN; NUCLEUS ACCUMBENS; PROBABILISTIC PAVLOVIAN CONDITIONING; QUANTITATIVE ANALYSIS; RAT; REINFORCEMENT; REWARD; ANIMAL; CONDITIONED REFLEX; METABOLISM; PHYSIOLOGY; PROBABILITY; SPRAGUE DAWLEY RAT;

ANIMALS; CONDITIONING, CLASSICAL; CUES; DOPAMINE; MALE; NUCLEUS ACCUMBENS; PROBABILITY; RATS; RATS, SPRAGUE-DAWLEY; REINFORCEMENT (PSYCHOLOGY); REWARD;

EID: 84920984160     PISSN: 10747427     EISSN: 10959564     Source Type: Journal    
DOI: 10.1016/j.nlm.2014.07.010     Document Type: Article

References (24)

1. Bayer H.M., Glimcher P.W. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron 2005, 47:129-141.
2. Bayer H.M., Lau B., Glimcher P.W. Statistics of midbrain dopamine neuron spike trains in the awake primate. Journal of Neurophysiology 2007, 98:1428-1439.
3. Clark J.J., Collins A.L., Sanford C.A., Phillips P.E.M. Dopamine encoding of Pavlovian incentive stimuli diminishes with extended training. Journal of Neuroscience 2013, 33(8):3526-3532.
4. Clark J.J., Sandberg S.G., Wanat M.J., Gan J.O., Horne E.A., Hart A.S., et al. Chronic microsensors for longitudinal, subsecond dopamine detection in behaving animals. Nature Methods 2010, 7:126-129.
5. Fiorillo C.D. Transient activation of midbrain dopamine neurons by reward risk. Neuroscience 2011, 197:162-171.
6. Fiorillo C.D., Tobler P.N., Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science 2003, 299:1898-1902.
7. Fiorillo C.D., Tobler P.N., Schultz W. Evidence that the delay-period activity of dopamine neurons corresponds to reward uncertainty rather than backpropagating TD errors. Behavioral and Brain Functions 2005, 1:7.
8. Flagel S.B., Clark J.J., Robinson T.E., Mayo L., Czuj A., Willuhn I., et al. A selective role for dopamine in stimulus-reward learning. Nature 2011, 469:53-57.
9. Gan J.O., Walton M.E., Phillips P.E.M. Dissociable cost and benefit encoding of future rewards by mesolimbic dopamine. Nature Neuroscience 2010, 13:25-27.
10. Garris P.A., Ciolkowski E.L., Pastore P., Wightman R.M. Efflux of dopamine from the synaptic cleft in the nucleus accumbens of the rat brain. Journal of Neuroscience 1994, 14:6084-6093.
11. Jackson T.E., Mudholkar G.S. Control procedures for residuals associated with principal component analysis. Technometrics 1979, 21:341-349.
12. Keithley R.B., Carelli R.M., Wightman R.M. Rank estimation and the multivariate analysis of in vivo fast-scan cyclic voltammetric data. Analytical Chemistry 2010, 82(13):5541-5551.
13. Nakahara H., Itoh H., Kawagoe R., Takikawa Y., Hikosaka O. Dopamine neurons can represent context-dependent prediction error. Neuron 2004, 41:269-280.
14. Niv Y., Duff M.O., Dayan P. Dopamine, uncertainty, and TD learning. Behavioral and Brain Functions 2005, 1:6.
15. Pan W.X., Schmidt R., Wickens J.R., Hyland B.I. Dopamine cells respond to predicted events during classical conditioning: Evidence for eligibility traces in the reward-learning network. Journal of Neuroscience 2005, 25:6235-6242.
16. Paxinos G., Watson C. The rat brain in stereotaxic coordinates 2005, Elsevier Academic Press, London. (5th ed.).
17. Preuschoff K., Bossaerts P., Quartz S.R. Neural differentiation of expected reward and risk in human subcortical structures. Neuron 2006, 51:381-390.
18. Rescorla R.A., Wagner A.R. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. Classical conditioning II: Current research and theory 1972, 64-99. Appleton-Century-Crofts, New York, NY. A.H. Black, W.F. Prokasy (Eds.).
19. Rutledge R.B., Dean M., Caplin A., Glimcher P.W. Testing the reward prediction error hypothesis with an axiomatic model. Journal of Neuroscience 2010, 30(40):13525-13536.
20. Schultz W., Dayan P., Montague P.R. A neural substrate of prediction and reward. Science 1997, 275:1593-1599.
21. Stuber G.D., Klanker M., de Ridder B., Bowers M.S., Joosten R.N., Feenstra M.G., et al. Reward-predictive cues enhance excitatory synaptic strength onto midbrain dopamine neurons. Science 2008, 321(5896):1690-1692.
22. Sutton R.S. Learning to predict by the methods of temporal differences. Machine Learning 1988, 3:9-44.
23. Tobler P.N., Fiorillo C.D., Schultz W. Adaptive coding of reward value by dopamine neurons. Science 2005, 307:1642-1645.
24. Waelti P., Dickinson A., Schultz W. Dopamine responses comply with basic assumptions of formal learning theory. Nature 2001, 412:43-48.