Analysis of thompson sampling for the multi-armed bandit problem

Journal of Machine Learning Research

Volumn 23, Issue , 2012, Pages

  Agrawal, Shipra ^a   Goyal, Navin ^a  

^a MICROSOFT RESEARCH   (United States)

Author keywords

Bayesian algorithm; Multi armed bandit; Online learning; Thompson sampling

Indexed keywords

DECISION THEORY; LEARNING ALGORITHMS; OPTIMIZATION; PROBABILITY; STOCHASTIC SYSTEMS;

BAYESIAN ALGORITHMS; CONSTANT FACTORS; EXPLORATION/EXPLOITATION; MULTI ARMED BANDIT; MULTI-ARMED BANDIT PROBLEM; ONLINE LEARNING; SEQUENTIAL DECISIONS; THOMPSON SAMPLINGS;

STATISTICS;

EID: 84874084136     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (14)

1. P. Auer, N. Cesa-Bianchi, and P. Fischer. Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47(2-3):235-256, 2002.
2. O. Chapelle and L. Li. An empirical evaluation of thompson sampling. In NIPS, 2011.
3. A. Garivier and O. Capṕe. The KL-UCB algorithm for bounded stochastic bandits and beyond. In Conference on Learning Theory (COLT), 2011.
4. J. C. Gittins. Multi-armed Bandit Allocation Indices. Wiley Interscience Series in Systems and Optimization. John Wiley and Son, 1989.
5. T. Graepel, J. Q. Candela, T. Borchert, and R. Herbrich. Web-scale bayesian click-through rate prediction for sponsored search advertising in microsoft's bing search engine. In ICML, pages 13-20, 2010.
6. O.-C. Granmo. Solving two-armed bernoulli bandit problems using a bayesian learning automaton. International Journal of Intelligent Computing and Cybernetics (IJICC), 3(2):207-234, 2010.
7. K. Jogdeo and S. M. Samuels. Monotone Convergence of Binomial Probabilities and A Generalization of Ramanujan's equation. The Annals of Mathematical Statistics, (4):1191-1195, 1968.
8. E. Kaufmann, O. Capṕe, and A. Garivier. On bayesian upper confidence bounds for bandit problems. In Fifteenth International Conference on Artificial Intelligence and Statistics (AISTAT), 2012.
9. T. L. Lai and H. Robbins. Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics, 6:4-22, 1985.
10. O.-A. Maillard, R.Munos, and G. Stoltz. Finite-time analysis of multi-armed bandits problems with kullback-leibler divergences. In Conference on Learning Theory (COLT), 2011.
11. B. C. May and D. S. Leslie. Simulation studies in optimistic bayesian sampling in contextual-bandit problems. Technical Report 11:02, Statistics Group, Department of Mathematics, University of Bristol, 2011.
12. B. C. May, N. Korda, A. Lee, and D. S. Leslie. Optimistic bayesian sampling in contextual-bandit problems. Technical Report 11:01, Statistics Group, Department of Mathematics, University of Bristol, 2011.
13. S. Scott. A modern bayesian look at the multi-armed bandit. Applied Stochastic Models in Business and Industry, 26:639-658, 2010.
14. W. R. Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3-4):285-294, 1933.