The asymptotic convergence-rate of Q-learning

Advances in Neural Information Processing Systems

Volumn , Issue , 1998, Pages 1064-1070

  Szepesvari C ^a,b  

^a UNIVERSITY OF SZEGED   (Hungary)

^b Associative Computing Ltd   (Hungary)

Author keywords

[No Author keywords available]

Indexed keywords

ASYMPTOTIC RATE; DISCOUNT FACTORS; ONLINE LEARNING; Q-LEARNING; STATIONARY DISTRIBUTION;

PROBABILITY DISTRIBUTIONS;

EID: 84898998140     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (17)

1. Bertsekas, D. and Tsitsiklis, J. (1996). Neuro-Dynamic Programming. Athena Scientific, Belmont, MA.
2. Hordjik, A. and Tijms, H. (1975). A modified form of the iterative method of dynamic programming. Annals of Statistics, 3:203-208.
3. Jaakkola, T., Jordan, M., and Singh, S. (1994). On the convergence of stochastic iterative dynamic programming algorithms. Neural Computation, 6(6):1185- 1201.
4. Littman, M. (1994). Markov games as a framework for multi-agent reinforcement learning. In Proc. of the Eleventh International Conference on Machine Learning, pages 157-163, San Francisco, CA. Morgan Kauffman.
5. Littman, M. and Szepesvri, C. (1996). A Generalized Reinforcement Learning Model: Convergence and applications. In Int. Conf. on Machine Learning. http://iserv.iki.kfki.hu/asl-publs.html.
6. Mahadevan, S. (1994). To discount or not to discount in reinforcement learning: A case study comparing R learning and Q learning. In Proceedings of the Eleventh International Conference on Machine Learning, pages 164-172, San Francisco, CA. Morgan Kaufmann.
7. Mahadevan, S. (1996). Average reward reinforcement learning: Foundations, algorithms, and empirical results. Machine Learning, 22(1, 2, 3) :124-158.
8. Major, P. (1993). A law of the iterated logarithm for the Robbins-Monro method. Studia Scientiarum Mathematicarum Hungarica, 8:95-102.
9. Poljak, B. and Tsypkin, Y. (1983). Pseudogradient adaption and training algorithms. Automation and Remote Control, 12:83-94.
10. Puterman, M. L. (1994). Markov Decision Processes - Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc., New York, NY.
11. Ross, S. (1970). Applied Probability Models with Optimization Applications. Holden Day, San Francisco, California.
12. Singh, S., Jaakkola, T., and Jordan, M. (1995). Reinforcement learning with soft state aggregation. In Proceedings of Neural Information Processing Systems.
13. Singh, S., Jaakkola, T., Littman, M., and Csaba Szepesvari (1997). On the convergence of single-step on-policy reinforcement-learning al gorithms. Machine Learning, in preparation.
14. Szepesvari, C. and Littman, M. (1996). Generalized Markov Decision Processes: Dynamic programming and reinforcement learning algorithms. Machine Learning. in preparation, available as TR CS96-10, Brown Univ.
15. Tsitsiklis, J. (1994). Asynchronous stochastic approximation and q-learning. Machine Learning, 8(3-4):257-277.
16. Wasan, T. (1969). Stochastic Approximation. Cambridge University Press, London.
17. Watkins, C. (1990). Learning from Delayed Rewards. PhD thesis, King's College, Cambridge. QLEARNING.