PAC bounds for discounted MDPs

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 7568 LNAI, Issue , 2012, Pages 320-334

  Lattimore, Tor ^a   Hutter, Marcus ^a  

^a AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

exploration exploitation; Markov decision processes; PAC MDP; Reinforcement learning; sample complexity

Indexed keywords

FINITE-STATE; LOWER BOUNDS; MARKOV DECISION PROCESSES; PAC BOUNDS; PAC-MDP; SAMPLE-COMPLEXITY; TRANSITION MATRICES; TRANSITION PROBABILITIES; UPPER AND LOWER BOUNDS;

MARKOV PROCESSES;

REINFORCEMENT LEARNING;

EID: 84867877076     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-34106-9_26     Document Type: Conference Paper

References (15)

1. Auer, P., Jaksch, T., Ortner, R.: Near-optimal regret bounds for reinforcement learning. J. Mach. Learn. Res. 99, 1563-1600 (2010)
2. Azar, M., Munos, R., Kappen, B.: On the sample complexity of reinforcement learning with a generative model. In: Proceedings of the 29th International Conference on Machine Learning. ACM, New York (2012)
3. Auer, P., Ortner, R.: Logarithmic online regret bounds for undiscounted reinforcement learning. In: Advances in Neural Information Processing Systems 19, pp. 49-56. MIT Press (2007)
4. Auer, P.: Upper confidence reinforcement learning. Unpublished, keynote at European Workshop of Reinforcement Learning (2011)
5. Chung, F., Lu, L.: Concentration inequalities and martingale inequalities a survey. Internet Mathematics 3, 1 (2006)
6. Kakade, S.: On The Sample Complexity of Reinforcement Learning. PhD thesis, University College London (2003)
7. Lattimore, T., Hutter, M.: PAC bounds for discounted MDPs. Technical report (2012), http://arxiv.org/abs/1202.3890
8. Lai, T., Robbins, H.: Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics 6(1), 4-22 (1985)
9. Mannor, S., Tsitsiklis, J.: The sample complexity of exploration in the multi-armed bandit problem. J. Mach. Learn. Res. 5, 623-648 (2004)
10. Strehl, A., Littman, M.: A theoretical analysis of model-based interval estimation. In: Proceedings of the 22nd International Conference on Machine Learning, ICML 2005, pp. 856-863 (2005)
11. Strehl, A., Littman, M.: An analysis of model-based interval estimation for Markov decision processes. Journal of Computer and System Sciences 74(8), 1309-1331 (2008)
12. Strehl, A., Li, L., Littman, M.: Reinforcement learning in finite MDPs: PAC analysis. J. Mach. Learn. Res. 10, 2413-2444 (2009)
13. Strehl, A., Li, L., Wiewiorac, E., Langford, J., Littman, M.: PAC model-free reinforcement learning. In: Proceedings of the 23rd International Conference on Machine Learning, ICML 2006, pp. 881-888. ACM, New York (2006)
14. Sobel, M.: The variance of discounted Markov decision processes. Journal of Applied Probability 19(4), 794-802 (1982)
15. Szita, I., Szepesvári, C.: Model-based reinforcement learning with nearly tight exploration complexity bounds. In: Proceedings of the 27th International Conference on Machine Learning, pp. 1031-1038. ACM, New York (2010)