(More) efficient reinforcement learning via posterior sampling

Advances in Neural Information Processing Systems

Volumn , Issue , 2013, Pages

  Osband, Ian ^a   Van Roy, Benjamin ^a   Russo, Daniel ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

LEARNING ALGORITHMS; MARKOV PROCESSES;

ACTION SPACES; CARDINALITIES; COMPUTATIONALLY EFFICIENT; MARKOV DECISION PROCESSES; PRIOR DISTRIBUTION; PRIOR KNOWLEDGE; REGRET BOUNDS; STATE OF THE ART;

REINFORCEMENT LEARNING;

EID: 84899019264     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (22)

1. A. N. Burnetas and M. N. Katehakis. Optimal adaptive policies for markov decision processes. Mathematics of Operations Research, 22(1):222-255, 1997.
2. P. R. Kumar and P. Varaiya. Stochastic systems: estimation, identification and adaptive control. Prentice-Hall, Inc., 1986.
3. T.L. Lai and H. Robbins. Asymptotically efficient adaptive allocation rules. Advances in applied mathematics, 6(1):4-22, 1985.
4. T. Jaksch, R. Ortner, and P. Auer. Near-optimal regret bounds for reinforcement learning. The Journal of Machine Learning Research, 99:1563-1600, 2010.
5. P. L. Bartlett and A. Tewari. Regal: A regularization based algorithm for reinforcement learning in weakly communicating mdps. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, pages 35-42. AUAI Press, 2009.
6. R. I. Brafman and M. Tennenholtz. R-max-A general polynomial time algorithm for nearoptimal reinforcement learning. The Journal of Machine Learning Research, 3:213-231, 2003.
7. S. M. Kakade. On the sample complexity of reinforcement learning. PhD thesis, University of London, 2003.
8. M. Kearns and S. Singh. Near-optimal reinforcement learning in polynomial time. Machine Learning, 49(2-3):209-232, 2002.
9. W. R. Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3/4):285-294, 1933.
10. O. Chapelle and L. Li. An empirical evaluation of Thompson sampling. In Neural Information Processing Systems (NIPS), 2011.
11. S.L. Scott. A modern Bayesian look at the multi-Armed bandit. Applied Stochastic Models in Business and Industry, 26(6):639-658, 2010.
12. S. Agrawal and N. Goyal. Further optimal regret bounds for Thompson sampling. arXiv preprint arXiv:1209.3353, 2012.
13. S. Agrawal and N. Goyal. Thompson sampling for contextual bandits with linear payoffs. arXiv preprint arXiv:1209.3352, 2012.
14. E. Kauffmann, N. Korda, and R. Munos. Thompson sampling: an asymptotically optimal finite time analysis. In International Conference on Algorithmic Learning Theory, 2012.
15. D. Russo and B. Van Roy. Learning to optimize via posterior sampling. CoRR, abs/1301.2609, 2013.
16. M. Strens. A Bayesian framework for reinforcement learning. In Proceedings of the 17th International Conference on Machine Learning, pages 943-950, 2000.
17. J. Z. Kolter and A. Y. Ng. Near-Bayesian exploration in polynomial time. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 513-520. ACM, 2009.
18. T. Wang, D. Lizotte, M. Bowling, and D. Schuurmans. Bayesian sparse sampling for on-line reward optimization. In Proceedings of the 22nd international conference on Machine learning, pages 956-963. ACM, 2005.
19. A. Guez, D. Silver, and P. Dayan. Efficient bayes-Adaptive reinforcement learning using samplebased search. arXiv preprint arXiv:1205.3109, 2012.
20. J. Asmuth and M. L. Littman. Approaching bayes-optimalilty using monte-carlo tree search. In Proc. 21st Int. Conf. Automat. Plan. Sched., Freiburg, Germany, 2011.
21. A. L. Strehl and M. L. Littman. An analysis of model-based interval estimation for markov decision processes. Journal of Computer and System Sciences, 74(8):1309-1331, 2008.
22. S. Filippi, O. Cappé, and A. Garivier. Optimism in reinforcement learning based on kullbackleibler divergence. CoRR, abs/1004.5229, 2010.