Further optimal regret bounds for thompson sampling

Journal of Machine Learning Research

Volumn 31, Issue , 2013, Pages 99-107

  Agrawal, Shipra ^a   Goyal, Navin ^a  

^a MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; PROBABILITY; STATISTICS;

ANALYSIS TECHNIQUES; BETA DISTRIBUTIONS; EMPIRICAL PERFORMANCE; MULTI-ARMED BANDIT PROBLEM; OPTIMAL PROBLEMS; RANDOMIZED ALGORITHMS; STATE-OF-THE-ART METHODS; THOMPSON SAMPLINGS;

OPTIMIZATION;

EID: 84898938874     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (24)

1. M. Abramowitz and I. A. Stegun. Handbook of Math- ematical Functions with Formulas, Graphs, and Mathematical Tables. Dover, New York, 1964.
2. S. Agrawal and N. Goyal. Analysis of Thompson Sampling for the Multi-armed Bandit Problem. In COLT, 2012a.
3. S. Agrawal and N. Goyal. Thompson sampling for contextual bandits with linear payoffs. Manuscript, 2012b.
4. J.-Y. Audibert and S. Bubeck. Minimax Policies for Adversarial and Stochastic Bandits. In COLT, 2009.
5. P. Auer, N. Cesa-Bianchi, and P. Fischer. Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47(2-3): 235-256, 2002.
6. S. Bubeck and N. Cesa-Bianchi. Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems. CoRR, 2012.
7. O. Chapelle and L. Li. An Empirical Evaluation of Thompson Sampling. In NIPS, pages 2249-2257, 2011.
8. O. Chapelle and L. Li. Open Problem: Regret Bounds for Thompson Sampling. In COLT, 2012.
9. A. Garivier and O. Cappé. The KL-UCB Algorithm for Bounded Stochastic Bandits and Beyond. In Con- ference on Learning Theory (COLT), 2011.
10. J. C. Gittins. Multi-armed Bandit Allocation Indices. Wiley Interscience Series in Systems and Optimization. John Wiley and Son, 1989.
11. T. Graepel, J. Q. Candela, T. Borchert, and R. Herbrich. Web-Scale Bayesian Click-Through rate Prediction for Sponsored Search Advertising in Microsoft's Bing Search Engine. In ICML, pages 13-20, 2010.
12. O.-C. Granmo. Solving Two-Armed Bernoulli Bandit Problems Using a Bayesian Learning Automaton. International Journal of Intelligent Computing and Cybernetics (IJICC), 3(2): 207-234, 2010.
13. E. Jerábek. Dual weak pigeonhole principle, Boolean complexity, and derandomization. Annals of Pure and Applied Logic, 129(1-3): 1-37, October 2004.
14. E. Kaufmann, O. Cappé, and A. Garivier. On Bayesian Upper Confidence Bounds for Bandit Problems. In Fifteenth International Conference on Artificial Intelligence and Statistics (AISTAT), 2012a.
15. E. Kaufmann, N. Korda, and R. Munos. Thompson Sampling: An Optimal Finite Time Analysis. In International Conference on Algorithmic Learning Theory (ALT), 2012b.
16. T. L. Lai and H. Robbins. Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics, 6: 4-22, 1985.
17. O.-A. Maillard, R. Munos, and G. Stoltz. Finitetime analysis of multi-armed bandits problems with Kullback-Leibler divergences. In Conference on Learning Theory (COLT), 2011.
18. B. C. May and D. S. Leslie. Simulation studies in optimistic Bayesian sampling in contextual-bandit problems. Technical Report 11: 02, Statistics Group, Department of Mathematics, University of Bristol, 2011.
19. B. C. May, N. Korda, A. Lee, and D. S. Leslie. Optimistic Bayesian sampling in contextual-bandit problems. Technical Report 11: 01, Statistics Group, Department of Mathematics, University of Bristol, 2011.
20. P. A. Ortega and D. A. Braun. Linearly parametrized bandits. Journal of Artificial Intelligence Research, 38: 475-511, 2010.
21. S. Scott. A modern Bayesian look at the multi-armed bandit. Applied Stochastic Models in Business and Industry, 26: 639-658, 2010.
22. M. J. A. Strens. A Bayesian Framework for Reinforcement Learning. In ICML, pages 943-950, 2000.
23. W. R. Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3-4): 285-294, 1933.
24. J. Wyatt. Exploration and Inference in Learning from Reinforcement. PhD thesis, Department of Artificial Intelligence, University of Edinburgh, 1997.