Convergence and divergence in standard and averaging reinforcement learning

Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science)

Volumn 3201, Issue , 2004, Pages 477-488

  Wiering, Marco A ^a  

^a UTRECHT UNIVERSITY   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION THEORY; DECISION THEORY; DYNAMIC PROGRAMMING; FUNCTIONS; ITERATIVE METHODS; MARKOV PROCESSES; PROBABILITY; ARTIFICIAL INTELLIGENCE; LEARNING SYSTEMS;

FUNCTION APPROXIMATORS (FA); LINEAR FUNCTIONS; REINFORCEMENT LEARNING; VALUE ITERATION (VI); CONVENTIONAL REINFORCEMENT LEARNING; DISCOUNT FACTORS; FUNCTION APPROXIMATORS; OPTIMAL POLICIES; REINFORCEMENT LEARNING METHOD; VALUE FUNCTIONS; VALUE ITERATION;

LEARNING SYSTEMS; REINFORCEMENT LEARNING;

EID: 22944460232     PISSN: 03029743     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1007/978-3-540-30115-8_44     Document Type: Conference Paper

References (18)

1. J. S. Albus. A theory of cerebellar function. Mathematical Biosciences, 10:25-61, 1975.
2. L. Baird. Residual algorithms: Reinforcement learning with function approximation. In A. Prieditis and S. Russell, editors, Machine Learning: Proceedings of the Twelfth International Conference, pages 30-37. Morgan Kaufmann Publishers, San Francisco, CA, 1995.
3. R. Bellman. Dynamic Programming. Princeton University Press, 1957.
4. J. A. Boyan and A. W. Moore. Generalization in reinforcement learning: Safely approximating the value function. In G. Tesauro, D. S. Touretzky, and T. K. Leen, editors, Advances in Neural Information Processing Systems 7, pages 369-376. MIT Press, Cambridge MA, 1995.
5. G.J. Gordon. Stable function approximation in dynamic programming. Technical Report CMU-CS-95-103, Carnegie Mellon University, 1995.
6. T. Jaakkola, M. I. Jordan, and S. P. Singh. On the convergence of stochastic iterative dynamic programming algorithms. Neural Computation, 6:1185-1201, 1994.
7. L. P. Kaelbling, M. L. Littman, and A. W. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237-285, 1996.
8. T.J. Perkins and D. Precup. A convergent form of approximate policy iteration. In Todd K. Leen, Thomas G. Dietterich, and Volker Tresp, editors, Advances in Neural Information Processing Systems 13. MIT Press, 2002.
9. S.I. Reynolds. The stability of general discounted reinforcement learning with linear function approximation. In Proceedings of the UK Workshop on Computational Intelligence (UKCI-02), pages 139-146, 2002.
10. G.A. Rummery and M. Niranjan. On-line Q-learning using connectionist sytems. Technical Report CUED/F-INFENG-TR 166, Cambridge University, UK, 1994.
11. S.P. Singh, T. Jaakkola, M.L. Littman, and C. Szepesvari. Convergence results for single-step on-policy reinforcement-learning algorithms. Machine Learning, 38(3):287-308, 2000.
12. R. S. Sutton. Generalization in reinforcement learning: Successful examples using sparse coarse coding. In D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, editors, Advances in Neural Information Processing Systems 8, pages 1038-1045. MIT Press, Cambridge MA, 1996.
13. R. S. Sutton and A. G. Barto. Reinforcement Learning: An Introduction. The MIT press, Cambridge MA, A Bradford Book, 1998.
14. C. Szepesvari and W.D. Smart. Convergent value function approximation methods. 2004. Accepted in the International Conference om Machine Learning (ICML'04).
15. G.J. Tesauro. Temporal difference learning and TD-Gammon. Communications of the ACM, 38:58-68, 1995.
16. J. N Tsitsiklis. Asynchronous stochastic approximation and Q-learning. Machine Learning, 16:185-202, 1994.
17. C. J. C. H. Watkins. Learning from Delayed Rewards. PhD thesis, King's College, Cambridge, England, 1989.
18. C. J. C. H. Watkins and P. Dayan. Q-learning. Machine Learning, 8:279-292, 1992.