Dopamine neurons report an error in the temporal prediction of reward during learning

Nature Neuroscience

Volumn 1, Issue 4, 1998, Pages 304-309

  Hollerman, Jeffrey R ^a,b   Schultz, Wolfram ^a  

^a UNIVERSITY OF FRIBOURG   (Switzerland)

^b ALLEGHENY COLLEGE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DOPAMINE;

ANIMAL; ARTICLE; CYTOLOGY; ELECTROPHYSIOLOGY; FORECASTING; LEARNING; MACACA; NERVE CELL; PHOTOSTIMULATION; PHYSIOLOGY; REWARD; SUBSTANTIA NIGRA; TEGMENTUM; TIME PERCEPTION;

ANIMALS; DOPAMINE; ELECTROPHYSIOLOGY; FORECASTING; LEARNING; MACACA FASCICULARIS; NEURONS; PHOTIC STIMULATION; REWARD; SUBSTANTIA NIGRA; TEGMENTUM MESENCEPHALI; TIME PERCEPTION;

EID: 33644688754     PISSN: 10976256     EISSN: None     Source Type: Journal    
DOI: 10.1038/1124     Document Type: Article

References (22)

1. Rescorla, R. A. & Wagner, A. R. in Classical Conditioning II: Current Research and Theory (eds Black, A. H. & Prokasy, W. F.) 64-99 (Appleton Century Crofts, New York, 1972).
2. Dickinson, A. Contemporary Animal Learning Theory (Cambridge Univ. Press, Cambridge, 1980).
3. Mackintosh, N. J. A theory of attention: Variations in the associability of stimulus with reinforcement. Psychol. Rev. 82, 276-298 (1975).
4. Pearce, J. M. & Hall, G. A model for Pavlovian conditioning: variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychol. Rev. 87, 532-552 (1980).
5. Sutton, R. S. & Barto, A. G. Toward a modern theory of adaptive networks: expectation and prediction. Psychol. Rev. 88, 135-170 (1981).
6. Smith, M. C. CS-US interval and US intensity in classical conditioning of the rabbit's nictitating membrane response. J. Comp. Physiol. Psychol. 66, 679-687 (1968).
7. Dickinson, A., Hall, G. & Mackintosh, N. J. Surprise and the attenuation of blocking. J. Exp. Psychol. Anim. Behav. Proc. 2, 313-322 (1976).
8. Sutton, R. S. Learning to predict by the method of temporal difference. Machine Learning 3, 9-44 (1988).
9. Barto, A. G., Sutton, R. S. & Anderson, C. W. Neuronlike adaptive elements that can solve difficult learning problems. IEEE Trans Syst. Man Cybernet. SMC 13, 834-846 (1983).
10. Tesauro, G. TD-Gammon, a self-teaching backgammon program, achieves master-level play. Neural Comp. 6, 215-219 (1994).
11. Wise, R. A. Neuroleptics and operant behavior: The anhedonia hypothesis. Behav. Brain Sci. 5, 39-87 (1982).
12. Fibiger, H. C. & Phillips, A. G. in Handbook of Physiology - The Nervous System, vol IV (ed. Bloom, F. E.) 647-675 (Williams & Wilkins, Baltimore, 1986).
13. Robbins, T. W. & Everitt, B. J. Neurobehavioural mechanisms of reward and motivation. Curr. Opin. Neurobiol. 6, 228-236 (1996).
14. Mirenowicz, J. & Schultz, W. Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol. 72, 1024-1027 (1994).
15. Schultz, W., Dayan, P. & Montague, R. R. A neural substrate of prediction and reward. Science 275, 1593-1599 (1997).
16. Ljungberg, T., Apicella, P. & Schultz, W. Responses of monkey dopamine neurons during learning of behavioral reactions. J. Neurophysiol. 67, 145-163 (1992).
17. Schultz, W., Apicella, P. & Ljungberg, T. Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci. 13, 900-913 (1993).
18. Steinfels, G. F., Heym, J., Strecker, R. E & Jacobs, B. L. Behavioral correlates of dopaminergic unit activity in freely moving cats. Brain Res. 258, 217-228 (1983).
19. Schultz, W. & Romo, R. Dopamine neurons of the monkey midbrain: Contingencies of responses to stimuli eliciting immediate behavioral reactions. J. Neurophysiol. 63, 607-624 (1990).
20. Mirenowicz, J. & Schultz, W. Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379, 449-451 (1996).
21. Horvitz, J. C., Stewart, T. & Jacobs, B. L. Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat. Brain Res. 759, 251-258 (1997).
22. Houk, J. C., Adams, J. L. & Barto, A. G. in Models of Information Processing in the Basal Ganglia (eds Houk, J. C., Davis, J. L. & Beiser, D. G.) 249-270 (MIT Press, Cambridge, 1995).