The multi-armed bandit problem with covariates

Annals of Statistics

Volumn 41, Issue 2, 2013, Pages 693-721

  Perchet, Vianney ^a   Rigollet, Philippe ^b  

^a UNIVERSITÉ PARIS DESCARTES   (France)

^b Princeton University   (United States)

Author keywords

Adaptive partition; Contextual bandit; Multi armed bandit; Nonparametric bandit; Regret bounds; Sequential allocation; Successive elimination

Indexed keywords

EID: 84879129093     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/13-AOS1101     Document Type: Article

References (25)

1. AUDIBERT, J.-Y. and BUBECK, S. (2010). Regret bounds and minimax policies under partial monitoring. J. Mach. Learn. Res. 11 2785-2836. MR2738783.
2. AUDIBERT, J.-Y. and TSYBAKOV, A. B. (2007). Fast learning rates for plug-in classifiers. Ann. Statist. 35 608-633. MR2336861.
3. AUDIBERT, J. Y. and TSYBAKOV, A. B. B. (2005). Fast learning rates for plug-in classifiers under the margin condition. Preprint, Laboratoire de Probabilités et Modèles Aléatoires, Univ. Paris VI and VII. Available at arXiv:math/0507180.
4. AUER, P., CESA-BIANCHI, N. and FISCHER, P. (2002). Finite-time analysis of the multiarmed bandit problem. Mach. Learn. 47 235-256.
5. AUER, P. and ORTNER, R. (2010). UCB revisited: Improved regret bounds for the stochastic multi-armed bandit problem. Period. Math. Hungar. 61 55-65. MR2728432.
6. BATHER, J. A. (1981). Randomized allocation of treatments in sequential experiments. J. R. Stat. Soc. Ser. B Stat. Methodol. 43 265-292. MR0637940.
7. CESA-BIANCHI, N. and LUGOSI, G. (2006). Prediction, Learning, and Games. Cambridge Univ. Press, Cambridge. MR2409394.
8. EVEN-DAR, E., MANNOR, S. and MANSOUR, Y. (2006). Action elimination and stopping conditions for the multi-armed bandit and reinforcement learning problems. J. Mach. Learn. Res. 7 1079-1105. MR2274398.
9. GOLDENSHLUGER, A. and ZEEVI, A. (2009). Woodroofe's one-armed bandit problem revisited. Ann. Appl. Probab. 19 1603-1633. MR2538082.
10. GOLDENSHLUGER, A. and ZEEVI, A. (2011). A note on performance limitations in bandit problems with side information. IEEE Trans. Inform. Theory 57 1707-1713. MR2815844.
11. HAZAN, E. and MEGIDDO, N. (2007). Online learning with prior knowledge. In Learning Theory. Lecture Notes in Computer Science 4539 499-513. Springer, Berlin. MR2397608.
12. JUDITSKY, A., NAZIN, A. V., TSYBAKOV, A. B. and VAYATIS, N. (2008). Gap-free bounds for stochastic multi-armed bandit. In Proceedings of the 17th IFAC World Congress.
13. KAKADE, S., SHALEV-SHWARTZ, S. and TEWARI, A. (2008). Efficient bandit algorithms for online multiclass prediction. In Proceedings of the 25th Annual International Conference on Machine Learning (ICML 2008) (A. McCallum and S. Roweis, eds.) 440-447. Omnipress, Helsinki, Finland.
14. LAI, T. L. and ROBBINS, H. (1985). Asymptotically efficient adaptive allocation rules. Adv. in Appl. Math. 6 4-22. MR0776826.
15. LANGFORD, J. and ZHANG, T. (2008). The epoch-greedy algorithm for multi-armed bandits with side information. In Advances in Neural Information Processing Systems 20 (J. C. Platt, D. Koller, Y. Singer and S. Roweis, eds.) 817-824. MIT Press, Cambridge, MA.
16. LU, T., PÁL, D. and PÁL, M. (2010). Showing relevant ads via Lipschitz context multi-armed bandits. JMLR: Workshop and Conference Proceedings 9 485-492.
17. MAMMEN, E. and TSYBAKOV, A. B. (1999). Smooth discrimination analysis. Ann. Statist. 27 1808-1829. MR1765618.
18. RIGOLLET, P. and ZEEVI, A. (2010). Nonparametric bandits with covariates. In COLT (A. Tauman Kalai and M. Mohri, eds.) 54-66. Omnipress, Haifa, Israel.
19. ROBBINS, H. (1952). Some aspects of the sequential design of experiments. Bull. Amer. Math. Soc. (N.S.) 58 527-535. MR0050246.
20. SLIVKINS, A. (2011). Contextual bandits with similarity information. JMLR: Workshop and Conference Proceedings 19 679-701.
21. TSYBAKOV, A. B. (2004). Optimal aggregation of classifiers in statistical learning. Ann. Statist. 32 135-166. MR2051002.
22. VOGEL, W. (1960). An asymptotic minimax theorem for the two armed bandit problem. Ann. Math. Statist. 31 444-451. MR0116443.
23. WANG, C.-C., KULKARNI, S. R. and POOR, H. V. (2005). Bandit problems with side observations. IEEE Trans. Automat. Control 50 338-355. MR2123095.
24. WOODROOFE, M. (1979). A one-armed bandit problem with a concomitant variable. J. Amer. Statist. Assoc. 74 799-806. MR0556471.
25. YANG, Y. and ZHU, D. (2002). Randomized allocation with nonparametric estimation for a multi-armed bandit problem with covariates. Ann. Statist. 30 100-121. MR1892657.