Temporal difference models and reward-related learning in the human brain

Neuron

Volumn 38, Issue 2, 2003, Pages 329-337

  O'Doherty, John P ^a   Dayan, Peter ^b   Friston, Karl ^a   Critchley, Hugo ^a   Dolan, Raymond J ^a  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CONDITIONING; CONTROLLED STUDY; CORPUS STRIATUM; HUMAN; HUMAN TISSUE; LEARNING; MODEL; NUCLEAR MAGNETIC RESONANCE IMAGING; ONSET AGE; ORBITAL CORTEX; PREDICTION; PRIORITY JOURNAL; REGRESSION ANALYSIS; STIMULUS RESPONSE; TEMPORAL LOBE;

EID: 0037987978     PISSN: 08966273     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0896-6273(03)00169-7     Document Type: Article

References (34)

1. Apicella P., Scarnati E., Ljungberg T., Schultz W. Neuronal activity in monkey striatum related to the expectation of predictable environmental events. J. Neurophysiol. 68:1992;945-960.
2. Berns G.S., McClure S.M., Pagnoni G., Montague P.R. Predictability modulates human brain response to reward. J. Neurosci. 21:2001;2793-2798.
3. Breiter H.C., Aharon I., Kahneman D., Dale A., Shizgal P. Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron. 30:2001;619-639.
4. Dayan P., Kakade S., Montague P.R. Learning and selective attention. Nat. Neurosci. Suppl. 3:2000;1218-1223.
5. Delgado M.R., Nystrom L.E., Fissell C., Noll D.C., Fiez J.A. Tracking the hemodynamic responses to reward and punishment in the striatum. J. Neurophysiol. 84:2000;3072-3077.
6. Duvernoy H.M. The Human Brain Stem and Cerebellum. 1995;Springer-Verlag, Vienna.
7. Duvernoy H.M. The Human Brain. 1999;Springer-Verlag, Vienna.
8. Elliott R., Friston K.J., Dolan R.J. Dissociable neural responses in human reward systems. J. Neurosci. 20:2000;6159-6165.
9. Fletcher P.C., Anderson J.M., Shanks D.R., Honey R., Carpenter T.A., Donovan T., Papadakis N., Bullmore E.T. Responses of human frontal cortex to surprising events are predicted by formal associative learning theory. Nat. Neurosci. 4:2001;1043-1048.
10. Francis S., Rolls E.T., Bowtell R., McGlone F., O'Doherty J., Browning A., Clare S., Smith E. The representation of the pleasantness of touch in the human brain, and its relation to taste and olfactory areas. Neuroreport. 10:1999;453-459.
11. Friston K.J., Tononi G., Reeke G.N. Jr., Sporns O., Edelman G.M. Value-dependent selection in the brain. simulation in a synthetic neural model Neuroscience. 59:1994;229-243.
12. Friston K.J., Ashburner J., Poline J.B., Frith C.D., Heather J.D., Frackowiak R.S. Spatial registration and normalisation of images. Hum. Brain Mapp. 2:1995;165-189.
13. Hikosaka K., Watanabe M. Delay activity of orbital and lateral prefrontal neurons of the monkey varying with different rewards. Cereb. Cortex. 10:2000;263-271.
14. Holt D.J., Graybiel A.M., Saper C.B. Neurochemical architecture of the human striatum. J. Comp. Neurol. 384:1997;1-25.
15. Karachi C., Francois C., Parain K., Bardinet E., Tande D., Hirsch E., Yelnik J. Three-dimensional cartography of functional territories in the human striatopallidal complex by using calbindin immunoreactivity. J. Comp. Neurol. 450:2002;122-134.
16. Knutson B., Fong G.W., Adams C.M., Varner J.L., Hommer D. Dissociation of reward anticipation and outcome with event-related fMRI. Neuroreport. 12:2001;3683-3687. a.
17. Knutson B., Adams C.M., Fong G.W., Hommer D. Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J. Neurosci. 21:2001;RC159. b.
18. Ljungberg T., Apicella P., Schultz W. Responses of monkey dopamine neurons during learning of behavioral reactions. J. Neurophysiol. 67:1992;145-163.
19. Mirenowicz J., Schultz W. Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol. 72:1994;1024-1027.
20. Mirenowicz J., Schultz W. Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature. 379:1996;449-451.
21. Nobre A.C., Coull J.T., Frith C.D., Mesulam M.M. Orbitofrontal cortex is activated during breaches of expectation in tasks of visual attention. Nat. Neurosci. 2:1999;11-12.
22. O'Doherty J., Rolls E.T., Francis S., Bowtell R., McGlone F. Representation of pleasant and aversive taste in the human brain. J. Neurophysiol. 85:2001;1315-1321.
23. O'Doherty J., Deichmann R., Crtichley H.D., Dolan R.J. Neural responses during anticipation of a primary taste reward. Neuron. 33:2002;815-826.
24. Oades R.D., Halliday G.M. Ventral tegmental (A10) system. neurobiology. 1. Anatomy and connectivity Brain Res. 434:1987;117-165.
25. Pagnoni G., Zink C.F., Montague P.R., Berns G.S. Activity in human ventral striatum locked to errors of reward prediction. Nat. Neurosci. 5:2002;97-98.
26. Pearce J.M., Hall G. A model for Pavlovian learning. Variations in the effectiveness of conditioned but not of unconditioned stimuli Psychol. Rev. 87:1980;532-552.
27. Ploghaus A., Tracey I., Clare S., Gati J.S., Rawlins J.N., Matthews P.M. Learning about pain. the neural substrate of the prediction error for aversive events Proc. Natl. Acad. Sci. USA. 97:2000;9281-9286.
28. Rescorla R.A., Wagner A.R. A theory of Pavlovian conditioning. variations in the effectiveness of reinforcement and nonreinforcement Black A.H., Prokasy W.F. Classical Conditioning II. Current Research and Theory:1972;Appleton Century Crofts, New York. 64-99.pp.
29. Schoenbaum G., Chiba A.A., Gallagher M. Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nat. Neurosci. 1:1998;155-159.
30. Schultz W. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80:1998;1-27.
31. Schultz W., Dayan P., Montague P.R. A neural substrate of prediction and reward. Science. 275:1997;1593-1599.
32. Schultz W., Tremblay L., Hollerman J.R. Reward prediction in primate basal ganglia and frontal cortex. Neuropharmacology. 37:1998;421-429.
33. Sutton R.S. Learning to predict by the methods of temporal differences. Machine Learning. 3:1988;9-44.
34. Sutton R.S., Barto A.G. Time derivative models of Pavlovian reinforcement. Gabriel M., Moore J. Learning and Computational Neuroscience. Foundations of Adaptive Networks:1990;MIT Press, Cambridge, MA. 539-602.pp.