Contextual multi-armed bandits

Journal of Machine Learning Research

Volumn 9, Issue , 2010, Pages 485-492

  Lu, Tyler ^a   Pál, Dávid ^b   Pál, Martin ^c  

^a UNIVERSITY OF TORONTO   (Canada)

^b UNIVERSITY OF ALBERTA   (Canada)

^c GOOGLE INC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BOUNDED SUBSET; CLICK-THROUGH RATE; CONTEXT-FREE; EUCLIDEAN SPACES; INTERNET SEARCH ENGINE; LIPSCHITZ; LIPSCHITZ CONDITIONS; LOWER BOUNDS; METRIC SPACES; MULTI ARMED BANDIT; MULTI-ARMED BANDIT PROBLEM; ON-LINE ALGORITHMS; PACKING DIMENSION; PAYOFF FUNCTION; SEARCH QUERIES; SIDE INFORMATION; UPPER AND LOWER BOUNDS; WEB SEARCHES;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; INFORMATION RETRIEVAL; SEARCH ENGINES; SET THEORY; TOPOLOGY; WORLD WIDE WEB;

STATISTICS;

EID: 84862301554     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (20)

1. R. Agrawal. The continuum-armed bandit problem. SIAM J. Control and Optimization, 33:1926-1951, 1995.
2. Peter Auer and Ronald Ortner. Logarithmic online regret bounds for undiscounted reinforcement learning. In Ad vances in Neural Information Processing Systems 19, (NIPS 2007), pages 49-56. MIT Press, 2007.
3. Peter Auer, Nicolò Cesa-Bianchi, and Paul Fischer. Finitetime analysis of the multiarmed bandit problem. Machine Learning, 47(2-3):235-256, 2002.
4. Peter Auer, Nicolò Cesa-Bianchi, Yoav Freund., and Robert E. Schapire. The nonstochastic multiarmed bandit problem. SIAM Journal on Computing, 32(1):48-77,2003.
5. Peter Auer, Ronald Ortner, and Csaba Szepesvári. Improved rates for the stochastic continuum-armed bandit problem. In Proceedings of the 20th Annual Conference on Learning Theory, (COLT 2007), pages 454-468.Springer, 2007.
6. Sébastien Bubeck, Rémi Munos, Gilles Stoltz, and Csaba Szepesvári. Online optimization in x-armed bandits. In NIPS, pages 201-208, 2008.
7. Nicolò Cesa-Bianchi and Gábor Lugosi. Prediction, Learning, and Games. Cambridge University Press, 2006.
8. Luc Devroye and Gábor Lugosi. Combinatorial Methods in Density Estimation. Springer, 2001.
9. Eyal Even-Dar, Shie Mannor, and Yishay Mansour. Action elimination and stopping conditions for the multi-armed bandit and reinforcement learning problems. Journal of Machine Learning Research, 7:1079-1105, 2006.
10. Abraham D. Flaxman, Adam T. Kalai, and H. Brendan McMahan. Online convex optimization in the bandit setting: gradient descent without a gradient. In Proceedings of the sixteenth annual ACM-SIAM symposium on Discrete algorithms (SODA 2005), pages 385-394. Society for Industrial and Applied Mathematics Philadelphia, PA, USA, 2005.
11. Alexander Goldenshluger and Assaf Zeevi. Performance limitations in bandit problems with side observations. manuscript, 2007.
12. Wassily Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13-30, 1963.
13. Robert D. Kleinberg. Nearly tight bounds for the continuum-armed bandit problem. In Lawrence K. Saul, Yair Weiss, and Léon Bottou, editors, Advances in Neural Information Processing Systems 17, (NIPS 2005), pages 697-704. MIT Press, 2005a.
14. Robert D. Kleinberg. Online Decision Problems with Large Strategy Sets. PhD thesis, Massachusetts Institute of Technology, June 2005b.
15. Robert D. Kleinberg, Aleksandrs Slivkins, and Eli Upfal. Multi-armed bandits in metric spaces. In Proceedings of the 40th Annual ACM Symposium, STOC 2008, pages 681-690. Association for Computing Machinery, 2008.
16. T. L. Lai and Herbert Robbins. Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics, 6(1):4-22, 1985.
17. John Langford. How do we get weak action dependence for learning with partial observations? Blog post: http://hunch.net/?p=421, September 2008.
18. John Langford and Tong Zhang. The epoch-greedy algorithm for multi-armed bandits with side information. In NIPS, 2007.
19. Richard S. Sutton and Andrew G. Barto. Reinforcement Learning. MIT Press, 1998.
20. Chih-Chun Wang, Sanjeev R. Kulkarni, and H. Vincent Poor. Bandit problems with side observations. IEEE Transactions on Automatic Control, 50(3):338-355, May 2005.