The KL-UCB algorithm for bounded stochastic bandits and beyond

Journal of Machine Learning Research

Volumn 19, Issue , 2011, Pages 359-376

  Garivier, Aurélien ^a   Cappé, Olivier ^a  

^a INSTITUT TELECOM   (France)

Author keywords

[No Author keywords available]

Indexed keywords

STOCHASTIC SYSTEMS;

BANDIT PROBLEMS; EXPONENTIAL FAMILY; FINITE-TIME ANALYSIS; INDEX POLICIES; LOWER BOUNDS; REGRET BOUNDS; SPECIFIC CLASS; TIME HORIZONS;

ALGORITHMS;

EID: 84898437076     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (14)

1. R. Agrawal. Sample mean based index policies with O(log n) regret for the multi-armed bandit problem. Advances in Applied Probability, 27(4):1054-1078, 1995.
2. J-Y. Audibert and S. Bubeck. Regret bounds and minimax policies under partial monitoring. Journal of Machine Learning Resaerch, 11:2785-2836, 2010.
3. J-Y. Audibert, R. Munos, and Cs. Szepesvári. Exploration- exploitation trade-off using variance estimates in multi-armed bandits. Theoretical Computer Science, 410(19), 2009.
4. P. Auer, N. Cesa-Bianchi, and P. Fischer. Finite-time analysis of the multiarmed bandit problem. Machine Learning, 47(2):235-256, 2002. (Pubitemid 34126111)
5. A.N. Burnetas and M.N. Katehakis. Optimal adaptive policies for Markov decision processes. Mathematics of Operations Research, pages 222-255, 1997. (Pubitemid 127621321)
6. C.J. Clopper and E.S. Pearson. The use of confidence of fiducial limits illustration in the case of the binomial. Biometrika, 26:404-413, 1934.
7. E. Even-Dar, S. Mannor, and Y. Mansour. PAC bounds for multi-armed bandit and Markov decision processes. In Conf. Comput. Learning Theory (Sydney, Australia, 2002), volume 2375 of Lecture Notes in Comput. Sci., pages 255-270. Springer, Berlin, 2002.
8. S. Filippi. Optimistic strategies in Reinforcement Learning (in French). PhD thesis, Telecom ParisTech, 2010. URL http://tel.archives-ouvertes.fr/tel- 00551401/.
9. S. Filippi, O. Cappé, and A. Garivier. Optimism in reinforcement learning and KullbackLeibler divergence. In Allerton Conf. on Communication, Control, and Computing, Monticello, US, 2010.
10. J.C. Gittins. Bandit processes and dynamic allocation indices. Journal of the Royal Statistical Society, Series B, 41(2):148-177, 1979.
11. J. Honda and A. Takemura. An asymptotically optimal bandit algorithm for bounded support models. In T. Kalai and M. Mohri, editors, Conf. Comput. Learning Theory, Haifa, Israel, 2010.
12. T. L. Lai and H. Robbins. Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics, 6(1):4, 1985.
13. O-A. Maillard, R. Memos, and G. Stoltz. A finite-time analysis of multi-armed bandits problems with kullback-leibler divergences. In Conf. Comput. Learning Theory, Budapest, Hungary, 2011.
14. P. Massart. Concentration inequalities and model selection, volume 1896 of Lecture Notes in Mathematics. Springer, Berlin, 2007. Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour, July 6-23, 2003.