A theoretical and empirical analysis of expected sarsa

2009 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, ADPRL 2009 - Proceedings

Volumn , Issue , 2009, Pages 177-184

  Harm van, Seijen ^a   Hado van, Hasselt ^b   Whiteson, Shimon ^c   Wiering, Marco ^d  

^a TNO   (Netherlands)

^b UTRECHT UNIVERSITY   (Netherlands)

^c UNIVERSITY OF AMSTERDAM   (Netherlands)

^d UNIVERSITY OF GRONINGEN   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

BEHAVIOR POLICY; DIFFERENCE METHOD; EMPIRICAL ANALYSIS; HIGHER LEARNING; LEARNING RATES; MODEL FREE; MULTIPLE DOMAINS; Q-LEARNING; STOCHASTICITY; ZERO VARIANCE;

DYNAMIC PROGRAMMING; LEARNING ALGORITHMS; REINFORCEMENT; REINFORCEMENT LEARNING; SYSTEMS ENGINEERING;

EDUCATION;

EID: 67650505307     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ADPRL.2009.4927542     Document Type: Conference Paper

References (12)

1. L. P. Kaelbling, M. L. Littman, and A. P. Moore, "Reinforcement learning: A survey," Journal of Artificial Intelligence Research, Vol. 4, pp. 237-285, 1996. (Pubitemid 126646155)
2. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, Massachussets: MIT Press, 1998.
3. R. E. Bellman, "A Markov decision process," Journal of Methematical Mechanics, Vol. 6, pp. 679-684, 1957.
4. R. E. Bellman, Dynamic Programming. Princeton, NJ.: Princeton University Press, 1957.
5. R. S. Sutton, "Learning to predict by the methods of temporal differences," Machine Learning, Vol. 3, pp. 9-44, 1988.
6. C. Watkins and P. Dayan, "Q-learning," Machine Learning, Vol. 8(3-4), pp. 9-44, 1992.
7. J. A. Boyan and A. W. Moore, "Generalization in reinforcement learning: Safely approximating the value function," in Advances in Neural Information Processing Systems 7, G. Tesauro, D. S. Touretzky, and T. K. Leen, Eds. MIT Press Cambridge MA, 1995, pp. 369-376.
8. G. Gordon, "Stable function approximation in dynamic programming," in Machine Learning: Proceedings of the Twelfth International Conference, A. Prieditis and S. Russell, Eds. Morgan Kaufmann Publishers, San Francisco, CA, 1995, pp. 261-268.
9. L. Baird, "Residual algorithms: Reinforcement learning with function approximation," in Machine Learning: Proceedings of the Twelfth International Conference, A. Prieditis and S. Russell, Eds. Morgan Kaufmann Publishers, San Francisco, CA, 1995, pp. 30-37.
10. G. Rummery and M. Niranjan, "On-line Q-learning using connectionist systems," Cambridge University, Tech. Rep. CUED/F-INFENG/TR 166, 1994.
11. R. Sutton, "Generalization in reinforcement learning: Successful examples using sparse coarse coding," in Advances in Neural Information Processing Systems 8, 1996, pp. 1038-1044.
12. S. Singh, T. Jaakkola, M. L. Littman, and C. Szepesvári, "Convergence results for single-step on-policy reinforcement-learning algorithms," Machine Learning, Vol. 38, no. 3, pp. 287-308, 2000. (Pubitemid 30572449)