Dopamine, uncertainty and TD learning

Behavioral and Brain Functions

Volumn 1, Issue , 2005, Pages

  Niv, Yael ^a,b   Duff, Michael O ^b   Dayan, Peter D ^b  

^a HEBREW UNIVERSITY OF JERUSALEM   (Israel)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

DOPAMINE;

ANALYTICAL ERROR; ARTICLE; CELL ACTIVITY; CONDITIONING; DOPAMINERGIC NERVE CELL; LEARNING; MATHEMATICAL ANALYSIS; MESENCEPHALON; NERVE CELL STIMULATION; NONHUMAN; PREDICTION; PRIMATE; PRIORITY JOURNAL; PROBABILITY; REINFORCEMENT; REWARD; TIME PERCEPTION; UNCERTAINTY;

EID: 26444446315     PISSN: 17449081     EISSN: None     Source Type: Journal    
DOI: 10.1186/1744-9081-1-6     Document Type: Article

References (26)

1. Ljungberg T, Apicella P, Schultz W: Responses of monkey dopamine neurons during learning of behavioral reactions. Journal Neurophysiol 1992, 67:145-163.
2. Schultz W: Predictive reward signal of dopamine neurons. Journal of Neurophysiology 1998, 80:1-27 [http://jn.physiology.org/cgi/content/full/ 80/1/1].
3. O'Doherty J, Dayan P, Friston K, Critchley H, Dolan R: Temporal difference models and reward-related learning in the human brain. Neuron 2003, 38:329-337.
4. Seymour B, O'Doherty J, Dayan P, Koltzenburg M, Jones A, Dolan R, Friston K, Frackowiak R: Temporal difference models describe higher order learning in humans. Nature 2004, 429:664-667.
5. Montague PR, Hyman SE, Cohan JD: Computational roles for dopamine in behavioural control. Nature 2004, 431:760-767.
6. Montague, PR, Dayan P, Sejnowski TJ: A framework for mesencephalic dopamine systems based on predictive Hebbian learning. The Journal of Neuroscience 1996, 16:1936-1947.
7. Schultz W, Dayan P, Montague PR: A neural substrate of prediction and reward. Science 1997, 275:1593-1599.
8. Sutton RS: Learning to predict by the method of temporal difference. Machine Learning 1988, 3:9-44.
9. Sutton RS, Barto AG: Reinforcement learning: An introduction 1998 [http://www.cs.ualberta.ca/~sutton/book/ebook/the-book.html]. MIT Press
10. Hollerman J, Schultz W: Dopamine neurons report an error in the temporal prediction of reward during learning. Nature Neuroscience 1998, 1:304-309.
11. Schultz W, Apicella P, Ljungberg T: Responses of monkey dopamine neurons to reward and conditioned stimuli during succesive steps of learning a delayed response task. The Journal of Neuroscience 1993, 13:900-913.
12. Tobler P, Dickinson A, Schultz W: Coding of Predicted Reward Omission by Dopamine Neurons in a Conditioned Inhibition Paradigm. Journal of Neuroscience 2003, 23(32):10402-10410.
13. Takikawa Y, Kawagoe R, Hikosaka O: A possible role of midbrain dopamine neurons in short- and long-term adaptation of saccades to position-reward mapping. Journal of Neurophysiology 2004, 92:2520-2529.
14. Bayer H: A role for the substantia nigra in learning and motor control. PhD thesis; New York University 2004.
15. Fiorillo C, Tobler P, Schultz W: Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons. Science 2003. 299(5614):1898-1902.
16. Morris G, Arkadir D, Nevet A, Vaadia E, Bergman H: Coincident but distinct messages of midbrain dopamine and striatal tonically active neurons. Neuron 2004, 43:133-143.
17. Barto A, Sutton R, Watkins C: Learning and sequntial decision making. In Learning and Computational Neuroscience: Foundations of Adaptive Networks Edited by: Gabriel M, Moore J. Cambridge, MA: MIT Press; 1990:539-602.
18. Nakahara H, Itoh H, Kawagoe R, Takikawa Y, Hikosaka O: Dopamine neurons can represent context-dependent prediction error. Neuron 2004, 41:269-280.
19. Gallistel CR, Gibbon J: Time, rate and conditioning. Psychological Review 2000, 107:289-344.
20. Daw ND, Kakade S, Dayan P: Opponent interactions between serotonin and dopamine. Neural Networks 2002, 15(4-6):603-616
21. Suri RE, Schultz W: A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience 1999, 91:871-890.
22. Pearce JM, Hall G: A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review 1980, 87:532-552.
23. Dayan P, Kakade S, Montague PR: Learning and selective attention. Nature Neuroscience 2000, 3:1218-1223.
24. Dayan P, Yu A: Expected and unexpected uncertainty: Ach and NE in the neocortex. Advances in Neural Information Processing Sysytems 2002, 14:189-196 [http://books.nips.ce/papers/files/nips15/NS08.pdf]. Cambridge, MA: MIT Press
25. Daw N, Niv Y, Dayan P: Actions, Policies, Values, and the Basal Ganglia. In Recent Breakthroughs in Basal Ganglia Research Edited by: Bezard E. New York, USA: Nova Science Publishers, Inc in press.
26. Wickens J, Kötter R: Cellular models of reinforcememnt. In Models of Information Processing in the Basal Ganglia Edited by: Houk JC, Davis JL, Beiser DG. MIT Press; 1995:187-214.