A finite-time analysis of multi-armed bandits problems with Kullback-Leibler divergences

Journal of Machine Learning Research

Volumn 19, Issue , 2011, Pages 497-514

  Maillard, Odalric Ambrym ^a   Munos, Rémi ^a   Stoltz, Gilles ^b,c  

^a INRIA   (France)

^b ECOLE NORMALE SUPÉRIEURE   (France)

^c HEC PARIS   (France)

Author keywords

Finite time analysis; Kullback Leibler divergence; Multi armed bandit problem; Sanov's lemma

Indexed keywords

STATISTICS;

FINITE SUPPORTS; FINITE-TIME ANALYSIS; KULLBACK LEIBLER DIVERGENCE; LOWER BOUNDS; MULTI ARMED BANDIT; MULTI-ARMED BANDIT PROBLEM; SANOV'S LEMMA;

ALGORITHMS;

EID: 84874038864     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (15)

1. J-Y. Audibert, R. Munos, and C. Szepesvari. Exploration-exploitation trade-off using variance estimates in multi-armed bandits. Theoretical Computer Science, 410:1876-1902, 2009.
2. J.Y. Audibert and S. Bubeck. Regret bounds and minimax policies under partial monitoring. Journal of Machine Learning Research, 11:2635-2686, 2010.
3. P. Auer and R. Ortner. UCB revisited: Improved regret bounds for the stochastic multiarmed bandit problem. Periodica Mathematica Hungarica, 61(1-2) :555, 2010.
4. P. Auer, N. Cesa-Bianchi, and P. Fischer. Finite time analysis of the multiarmed bandit problem. Machine Learning, 47(2-3):235-256, 2002. (Pubitemid 34126111)
5. A.N. Burnetas and M.N. Katehakis. Optimal adaptive policies for sequential allocation problems. Advances in Applied Mathematics, 17(2) :122-142, 1996. (Pubitemid 126160351)
6. Y. Chow and H. Teicher. Probability Theory. Springer, 1988.
7. I. H. Dinwoodie. Mesures dominantes et théorème de Sanov. Annales de l'Institut Henri Poincaré - Probabilités et Statistiques, 28(3):365-373, 1992.
8. S. Filippi. Stratégies optimistes en apprentissage par renforcement. PhD thesis, Télécom ParisTech, 2010.
9. A. Garivier and O. Cappè. The KL-UCB algorithm for bounded stochastic bandits and beyond. In Proceedings of COLT, 2011.
10. A. Garivier and F. Leonardi. Context tree selection: A unifying view. Stochastic Processes and their Applications, 2011. In press.
11. J. Honda and A. Takemura. An asymptotically optimal bandit algorithm for bounded support models. In Proceedings of COLT, pages 67-79, 2010a.
12. J. Honda and A. Takemura. An asymptotically optimal policy for finite support models in the multiarmed bandit problem. arXiv:0905.2776, 2010b.
13. T. L. Lai and H. Robbins. Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics, 6:4-22, 1985.
14. O.-A. Maillard, R. Munos, and G. Stoltz. A finite-time analysis of multi-armed bandits problems with Kullback-Leibler divergences. 2011. URL http://hai.archives-ouvertes. fr/inria-00574987/.
15. H. Robbins. Some aspects of the sequential design of experiments. Bulletin of the American Mathematics Society, 58:527-535, 1952.