Adaptive aggregation for reinforcement learning with efficient exploration: Deterministic domains

21st Annual Conference on Learning Theory, COLT 2008

Volumn , Issue , 2008, Pages 323-334

  Bernstein, Andrey ^a   Shimkin, Nahum ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE AGGREGATION; COARSER RESOLUTION; CONTINUOUS STATE SPACE; DETERMINISTIC DOMAINS; EXPLORATION TECHNIQUES; ONLINE LEARNING; STATE AGGREGATION; UNCERTAINTY INTERVALS;

ALGORITHMS;

REINFORCEMENT LEARNING;

EID: 84898060153     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. A. Antos, R. Munos, and C. Szepesvari. Fitted Qiteration in continuous action-space MDPs. In Proceedings of Neural Information Processing Systems Conference (NIPS), 2007.
2. P. Auer and R. Ortner. Logarithmic online regret bounds for undiscounted reinforcement learning. In Proceedings of Neural Information Processing Systems Conference (NIPS), 2006.
3. A. Bernstein. Adaptive state aggregation for reinforcement learning. Master's thesis, Technion - Israel Institute of Technology, 2007. URL: http://tx.technion.ac.il/~andreyb/MSc-Thesis-final.pdf.
4. D. P. Bertsekas. Dynamic Programming and Optimal Control, vol. 2. Athena Scientific, Belmont, MA, third edition, 2007.
5. A. Bonarini, A. Lazaric, and M. Restelli. LEAP: an adaptive multi-resolution reinforcement learning algorithm. To appear.
6. C. Boutilier, T. Dean, and S. Hanks. Decision-theoretic planning: Structural assumptions and computational leverage. Journal of Artificial Intelligence Research, 11: 1-94, 1999.
7. R. I. Brafman and M. Tennenholtz. R-MAX - a general polynomial time algorithm for near-optimal reinforcement learning. Journal of Machine Learning Research, 3: 213-231, 2002.
8. H. Chapman. Global confidence bound algorithms for the exploration-exploitation tradeoff in reinforcement learning. Master's thesis, Technion - Israel Institute of Technology, 2007.
9. C.-S Chow and J.N. Tsitsiklis. An optimal oneway multigrid algorithm for discrete-time stochastic control. IEEE Transactions on Automatic Control, 36(8): 898-914, 1991.
10. C. Diuk, A. L. Strehl, and M. L. Littman. A hierarchical approach to efficient reinforcement learning in deterministic domains. In Proceedings of the 5th International Joint Conference on Autonomous Agents and Multiagent Systems, pages 313-319, 2006.
11. G. J. Gordon. Reinforcement learning with function approximation converges to a region. In Advances in Neural Information Processing Systems (NIPS) 12, pages 1040-1046, 2000.
12. S. M. Kakade. On the Sample Complexity of Reinforcement Learning. PhD thesis, Gatsby Computational Neuroscience Unit, University College London, UK, 2003.
13. M. Kearns and S. P. Singh. Near-optimal reinforcement learning in polynomial time. Machine Learning, 49: 209-232, 2002.
14. I. Menache, S. Mannor, and N. Shimkin. Q-Cut - dynamic discovery of sub-goals in reinforcement learning. In Proceedings of the 13th European Conference on Machine Learning (ECML 2002), pages 187-195, 2002.
15. A. W. Moore and C. G. Atkeson. The parti-game algorithm for variable resolution reinforcement learning in multidimensional state-spaces. Machine Learning, 21: 199-233, 1995.
16. R. Munos and A. W. Moore. Variable resolution discretization in optimal control. Machine Learning, 49: 291-323, 2002.
17. M. L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc., New York, NY, USA, 1994.
18. A. L. Strehl and M. L. Littman. A theoretical analysis of model-based interval estimation. In Proceedings of the 22nd International Conference on Machine Learning, pages 857-864, 2005.
19. A. L. Strehl, E. Wiewiora, J. Langford, and M. L. Littman. PAC model-free reinforcement learning. In Proceedings of the 23nd International Conference on Machine Learning, pages 881-888, 2006.
20. A. Tewari and P. L. Bartlett. Optimistic linear programming gives logarithmic regret for irreducible MDPs. In Proceedings of Neural Information Processing Systems Conference (NIPS), 2007.
21. W. Whitt. Approximations of dynamic programs, I. Mathematics of Operations Research, 3(3): 231-243, 1978.