Exploration-exploitation tradeoff using variance estimates in multi-armed bandits

Theoretical Computer Science

Volumn 410, Issue 19, 2009, Pages 1876-1902

  Audibert, Jean Yves ^a,b   Munos, Rémi ^c   Szepesvári, Csaba ^d  

^a UNIVERSITÉ PARIS EST   (France)

^b ECOLE NORMALE SUPÉRIEURE   (France)

^c INRIA   (France)

^d UNIVERSITY OF ALBERTA   (Canada)

Author keywords

Bernstein's inequality; Exploration exploitation tradeoff; High probability bound; Multi armed bandits; Risk analysis

Indexed keywords

ALGORITHMS; AMBER; COMMUNICATION CHANNELS (INFORMATION THEORY); RISK ASSESSMENT; SAFETY FACTOR;

BANDIT PROBLEMS; BERNSTEIN'S INEQUALITY; COMPETING ALGORITHMS; DECISION MAKERS; EXPLORATION AND EXPLOITATIONS; EXPLORATION-EXPLOITATION TRADEOFF; HIGH-PROBABILITY BOUND; MULTI-ARMED BANDIT PROBLEMS; MULTI-ARMED BANDITS; UPPER CONFIDENCE BOUNDS; VARIANCE ESTIMATES;

RISK ANALYSIS;

EID: 62949181077     PISSN: 03043975     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tcs.2009.01.016     Document Type: Article

References (13)

1. Agrawal R. Sample mean based index policies with O (log n) regret for the multi-armed bandit problem. Advances in Applied Probability 27 (1995) 1054-1078
2. J.-Y. Audibert, PAC-Bayesian statistical learning theory, Ph.D. Thesis, Laboratoire de Probabilités et Modèles Aléatoires, Universités Paris 6 and Paris 7, 2004. http://certis.enpc.fr/~audibert/ThesePack.zip
3. Auer P., Cesa-Bianchi N., and Fischer P. Finite time analysis of the multiarmed bandit problem. Machine Learning 47 2-3 (2002) 235-256
4. P. Auer, N. Cesa-Bianchi, J. Shawe-Taylor, Exploration versus exploitation challenge, in: 2nd PASCAL Challenges Workshop, Pascal Network, 2006
5. Freedman D.A. On tail probabilities for martingales. The Annals of Probability 3 1 (1975) 100-118
6. S. Gelly, Y. Wang, R. Munos, O. Teytaud, Modification of UCT with patterns in Monte-Carlo go, Technical Report, INRIA RR-6062, 2006
7. Gittins J.C. Multi-Armed Bandit Allocation Indices. Wiley-Interscience Series in Systems and Optimization (1989), Wiley, Chichester, NY
8. Hoeffding W. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association 58 (1963) 13-30
9. L. Kocsis, Cs. Szepesvári, Bandit based Monte-Carlo planning, in: Proceedings of the 17th European Conference on Machine Learning, ECML-2006, 2006, pp.282-293
10. Lai T.L., and Robbins H. Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics 6 (1985) 4-22
11. Lai T.L., and Yakowitz S. Machine learning and nonparametric bandit theory. IEEE Transactions on Automatic Control 40 (1995) 1199-1209
12. Robbins H. Some aspects of the sequential design of experiments. Bulletin of the American Mathematical Society 58 (1952) 527-535
13. Thompson W.R. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25 (1933) 285-294