An analysis of reinforcement learning with function approximation

Proceedings of the 25th International Conference on Machine Learning

Volumn , Issue , 2008, Pages 664-671

  Melo, Francisco S ^a   Meyn, Sean P ^b   Ribeiro, M Isabel ^c  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^c INSTITUTE FOR SYSTEMS AND ROBOTICS   (Portugal)

Author keywords

[No Author keywords available]

Indexed keywords

LEARNING ALGORITHMS; MACHINE LEARNING; STOCHASTIC SYSTEMS; FUNCTIONS; LEARNING SYSTEMS; PROBABILITY DENSITY FUNCTION; REINFORCEMENT; REINFORCEMENT LEARNING; ROBOT LEARNING;

APPROXIMATE METHODS; CONVERGENCE PROPERTIES; FUNCTION APPROXIMATION; INFINITE STATE SPACE; MARKOV DECISION PROBLEM; REINFORCEMENT LEARNING WITH FUNCTION APPROXIMATIONS; RELATED WORKS; STOCHASTIC CONTROL;

REINFORCEMENT LEARNING; EDUCATION;

APPROXIMATE METHODS; CONVERGENCE PROPERTIES; FUNCTION APPROXIMATIONS; INFINITE STATE; MARKOV DECISION PROBLEMS; Q FUNCTIONS; Q-LEARNING; REINFORCEMENT LEARNING WITH FUNCTION APPROXIMATIONS; STOCHASTIC CONTROLS; TD-LEARNING;

EID: 56449091120     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1390156.1390240     Document Type: Conference Paper

References (24)

1. Baird, L. (1995). Residual algorithms: Reinforcement learning with function approximation. Proc. 12th Int. Conf. Machine Learning (pp. 30-37).
2. Benveniste, A., Métivier, M., & Priouret, P. (1990). Adaptive algorithms and stochastic approximations, vol. 22. Springer-Verlag.
3. Bertsekas, D., & Tsitsiklis, J. (1996). Neuro-dynamic programming. Athena Scientific.
4. Borkar, V. (1997). Stochastic approximation with two time scales. Systems & Control Letters, 29, 291-294.
5. Borkar, V. (2000). A learning algorithm for discrete-time stochastic control. Probability in the Engineering and Informational Sciences, 14, 243-258.
6. de Farias, D., & Van Roy, B. (2000). On the existence of fixed points for approximate value iteration and temporal-difference learning. Journal of Optimization Theory and Applications, 105, 589-608.
7. Diaconis, P., & Saloff-Coste, L. (1996). Logarithmic Sobolev inequalities for finite Markov chains. Annals of Applied Probability, 6, 695-750.
8. Gordon, G. (1995). Stable function approximation in dynamic programming (Technical Report CMU-CS-95-103). School of Computer Science, Carnegie Mellon University.
9. Gordon, G. (1996). Chattering in SARSA(λ). (Technical Report). CMU Learning Lab Internal Report.
10. Meyn, S., & Tweedie, R. (1993). Markov chains and stochastic stability. Springer-Verlag.
11. Meyn, S., & Tweedie, R. (1994). Computable bounds for geometric convergence rates of Markov chains. Annals of Applied Probability, 4, 981-1011.
12. Ormoneit, D., & Sen, S. (2002). Kernel-based reinforcement learning. Machine Learning, 49, 161-178.
13. Perkins, T., & Pendrith, M. (2002). On the existence of fixed-points for Q-learning and SARSA in partially observable domains. Proc. 19th Int. Conf. Machine Learning (pp. 490-497).
14. Perkins, T., & Precup, D. (2003). A convergent form of approximate policy iteration. Adv. Neural Information Proc. Systems (pp. 1595-1602).
15. Precup, D., Sutton, R., & Dasgupta, S. (2001). Off-policy temporal-difference learning with function approximation. Proc. 18th Int. Conf. Machine Learning (pp. 417-424).
16. Ribeiro, C., & Szepesvári, C. (1996). Q-learning combined with spreading: Convergence and results. Proc. ISRF-IEE Int. Conf. Intelligent and Cognitive Systems (pp. 32-36).
17. Rosenthal, J. (2002). Quantitative convergence rates of Markov chains: A simple account. Electronic Communications in Probability, 7, 123-128.
18. Singh, S., Jaakkola, T., & Jordan, M. (1994). Reinforcement learning with soft state aggregation. Adv. Neural Information Proc. Systems (pp. 361-368).
19. Sutton, R. (1999). Open theoretical questions in reinforcement learning. Lecture Notes in Computer Science, 1572, 11-17.
20. Szepesvári, C., & Smart, W. (2004). Interpolation-based Q-learning. Proc. 21st Int. Conf. Machine learning (pp. 100-107).
21. Tadić, V. (2001). On the convergence of temporal-difference learning with linear function approximation. Machine Learning, 42, 241-267.
22. Tsitsiklis, J., & Van Roy, B. (1996a). An analysis of temporal-difference learning with function approximation. IEEE Trans. Automatic Control, 42, 674-690.
23. Tsitsiklis, J., & Van Roy, B. (1996b). Feature-based methods for large scale dynamic programming. Machine Learning, 22, 59-94.
24. Watkins, C. (1989). Learning from delayed rewards. Doctoral dissertation, King's College, University of Cambridge.