Hyperbolically discounted temporal difference learning

Neural Computation

Volumn 22, Issue 6, 2010, Pages 1511-1527

  Alexander, William H ^a   Brown, Joshua W ^a  

^a INDIANA UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACTION POTENTIAL; ALGORITHM; ANIMAL; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; BRAIN; COMPUTER SIMULATION; DECISION MAKING; MATHEMATICAL PHENOMENA; NERVE CELL; NERVE CELL NETWORK; PHYSIOLOGY; TIME; TIME PERCEPTION;

ACTION POTENTIALS; ALGORITHMS; ANIMALS; ARTIFICIAL INTELLIGENCE; BRAIN; COMPUTER SIMULATION; DECISION MAKING; MATHEMATICAL CONCEPTS; NERVE NET; NEURAL NETWORKS (COMPUTER); NEURONS; TIME FACTORS; TIME PERCEPTION;

EID: 77953497419     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/neco.2010.08-09-1080     Document Type: Article

References (18)

1. Amador, N., Schlag-Rey, M., & Schlag, J. (2000). Reward-predicting and rewarddetecting neuronal activity in the primate supplementary eye field. J. Neurophysiol., 84(4), 2166-2170.
2. Brunner, D. (1999). Preference for sequences of rewards: Further tests of a parallel discounting model. Behavioural Processes, 45(1-3), 87-99.
3. Daw, N. D., & Touretzky, D. S. (2000). Behavioral considerations suggest an average reward TD model of the dopamine system. Neurocomputing: An International Journal, 32-33, 679-684.
4. Daw, N. D., & Touretzky, D. S. (2002). Long-term reward prediction in TD models of the dopamine system. Neural Comput., 14(11), 2567-2587.
5. Green, L., & Myerson, J. (1996). Exponential versus hyperbolic discounting of delayed outcomes: Risk and waiting time. Amer. Zool., 36(4), 496-505.
6. Kacelnik, A., & Bateson,M. (1996). Risky theories-The effects of variance on foraging decisions. Amer. Zool., 36(4), 402-434.
7. Kobayashi, S., & Schultz, W. (2008). Influence of reward delays on responses of dopamine neurons. J. Neurosci., 28(31), 7837-7846.
8. Mazur, J. E. (1987). An adjusting procedure for studying delayed reinforcement. In M. L. Commons, J. E.Mazur, J. A. Nevin,&H. Rachlin (Eds.), The effect of delay and intervening events on reinforcement value (Vol. 5, pp. 55-73).Mahwah, NJ: Erlbaum.
9. Myerson, J., & Green, L. (1995). Discounting of delayed rewards: Models of individual choice. J. Exp. Anal. Behav., 64(3), 263-276.
10. Schultz, W., Apicella, P., Ljungberg, T., Romo, R., & Scarnati, E. (1993). Rewardrelated activity in the monkey striatum and substantia nigra. Prog. Brain Res., 99, 227-235.
11. Schultz, W., Apicella, P., Scarnati, E., & Ljungberg, T. (1992). Neuronal activity in monkey ventral striatum related to the expectation of reward. J. Neurosci., 12(12), 4595-4610.
12. Schultz, W., Tremblay, L., & Hollerman, J. R. (2000). Reward processing in primate orbitofrontal cortex and basal ganglia. Cereb. Cortex, 10(3), 272-284.
13. Schweighofer, N., Bertin, M., Shishida, K., Okamoto, Y., Tanaka, S. C., Yamawaki, S., et al. (2008). Low-serotonin levels increase delayed reward discounting in humans. J. Neurosci., 28(17), 4528-4532.
14. Sozou, P. D. (1998). On hyperbolic discounting and uncertain hazard rates. Proceedings of the Royal Society of London. Series B: Biological Sciences, 265(1409), 2015-2020.
15. Suri, R. E., & Schultz,W. (2001). Temporal difference model reproduces anticipatory neural activity. Neural Comput., 13(4), 841-862.
16. Sutton, R. S., & Barto, A. G. (1990). Time-derivative models of Pavlovian reinforcement. In M. Gabriel & J. Moore (Eds.), Learning and computational neuroscience (pp. 497-537). Cambridge, MA: MIT Press.
17. Tsitsiklis, J. N., & Van Roy, B. (1999). Average cost temporal-difference learning. Automatica, 35(11), 1799-1808.
18. Tsitsiklis, J.N.,&Van Roy, B. (2002).On average versus discounted reward temporaldifference learning. Machine Learning, 49(2-3), 179-191.