Learning is planning: Near Bayes-optimal reinforcement learning via Monte-Carlo tree search

Proceedings of the 27th Conference on Uncertainty in Artificial Intelligence, UAI 2011

Volumn , Issue , 2011, Pages 19-26

  Asmuth, John ^a   Littman, Michael ^a  

^a RUTGERS UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

MARKOV PROCESSES; MONTE CARLO METHODS;

BAYES-OPTIMAL; BELIEF SPACE; INFINITE STATE SPACE; MARKOV DECISION PROCESSES; MONTE CARLO TREE SEARCH (MCTS); MONTE-CARLO TREE SEARCHES; POLYNOMIAL NUMBER; SPARSE SAMPLING;

REINFORCEMENT LEARNING;

EID: 80053158617     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. Asmuth, J., Li, L., Littman, M., Nouri, A., & Wingate, D. (2009). A Bayesian sampling approach to exploration in reinforcement learning. Proceedings of the 25th Conference on Uncertainty in Artifical Intelligence (UAI-09).
2. Asmuth, J., & Littman, M. (2011). Appendix (Technical Report DCS-tr-687). Rutgers University department of Computer Science.
3. Auer, P., Cesa-Bianchi, N., & Fischer, P. (2002). Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47, 235-256. (Pubitemid 34126111)
4. Brafman, R. I., & Tennenholtz, M. (2002). R-MAX-A general polynomial time algorithm for near-optimal reinforcement learning. Journal of Machine Learning Research, 3, 213-231.
5. Duff, M. (2003). Design for an optimal probe. Proceedings of the 20th International Conference on Machine Learning.
6. Duff, M. O., & Barto, A. G. (1997). Local bandit approximation for optimal learning problems. Advances in Neural Information Processing Systems (pp. 1019-1025). The MIT Press.
7. Gelly, S., & Silver, D. (2008). Achieving master level play in 9×9 computer go. Proceedings of the 23rd national conference on Artificial intelligence - Volume 3 (pp. 1537-1540). Chicago, Illinois: AAAI Press.
8. Kakade, S. M. (2003). On the sample complexity of reinforcement learning. Doctoral dissertation, Gatsby Computational Neuroscience Unit, University College London.
9. Kearns, M., Mansour, Y., & Ng, A. Y. (1999). A sparse sampling algorithm for near-optimal planning in large Markov decision processes. Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence (IJCAI-99) (pp. 1324-1331).
10. Kocsis, L., & Szepesvari, C. (2006). Bandit based Monte-Carlo planning. Proceedings of the 17th European Conference on Machine Learning (ECML- 06) (pp. 282-293). Springer Berlin / Heidelberg. (Pubitemid 44618839)
11. Kolter, J. Z., & Ng, A. Y. (2009). Near-Bayesian exploration in polynomial time. Proceedings of the 26th Annual International Conference on Machine Learning (pp. 513-520). New York, NY, USA: ACM.
12. Leffler, B. R., Littman, M. L., & Edmunds, T. (2007). Efficient reinforcement learning with relocatable action models. Proceedings of the Twenty-Second Conference on Artificial Intelligence (AAAI-07).
13. Li, L. (2009). A unifying framework for computational reinforcement learning theory (pp 78-79). Doctoral dissertation, Rutgers University.
14. MacEachern, S. N., & Muller, P. (1998). Estimating mixture of Dirichlet process models. Journal of Computational and Graphical Statistics, 7, 223-238.
15. Neal, R. M. (2000). Markov chain sampling methods for dirichlet process mixture models. Journal of Computational and Graphical Statistics, Vol. 9, pp. 249-265.
16. Poupart, P., Vlassis, N., Hoey, J., & Regan, K. (2006). An analytic solution to discrete Bayesian reinforcement learning. Proceedings of the 23rd International Conference on Machine Learning (pp. 697-704).
17. Puterman, M. L. (1994). Markov decision processes- discrete stochastic dynamic programming. New York, NY: John Wiley & Sons, Inc.
18. Russell, S. J., & Norvig, P. (1994). Artificial intelligence: A modern approach. Englewood Cliffs, NJ: Prentice-Hall.
19. Sorg, J., Singh, S., & Lewis, R. L. (2010). Variance- based rewards for approximate Bayesian reinforcement learning. Proceedings of the 26th Conference on Uncertainty in Artifical Intelligence (UAI-10).
20. Strehl, A. L., Diuk, C., & Littman, M. L. (2007). Efficient structure learning in factored-state MDPs. Proceedings of the Twenty-Second National Conference on Artificial Intelligence (AAAI-07).
21. Strehl, A. L., & Littman, M. L. (2008). An analysis of modelbased interval estimation for Markov decision processes. Journal of Computer and System Sciences, 74, 1309-1331. Special Issue on Learning Theory.
22. Strens, M. J. A. (2000). A Bayesian framework for reinforcement learning. Proceedings of the Seventeenth International Conference on Machine Learning (ICML 2000) (pp. 943-950).
23. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. The MIT Press.
24. Walsh, T., Goschin, S., & Littman, M. (2010). Integrating samplebased planning and model-based reinforcement learning. Proceedings of the Association for the Advancement of Artificial Intelligence.
25. Walsh, T. J., Szita, I., Diuk, C., & Littman, M. L. (2009). Exploring compact reinforcement-learning representations with linear regression. Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (pp. 591-598). Arlington, Virginia, United States: AUAI Press.
26. Wang, T., Lizotte, D., Bowling, M., & Schuurmans, D. (2005). Bayesian sparse sampling for on-line reward optimization. ICML '05: Proceedings of the 22nd International Conference on Machine Learning (pp. 956-963). New York, NY, USA: ACM.
27. Wilson, A., Fern, A., Ray, S., & Tadepalli, P. (2007). Multi-task reinforcement learning: A hierarchical Bayesian approach. Machine Learning, Proceedings of the Twenty-Fourth International Conference (ICML 2007) (pp. 1015-1022).