Improved algorithms for linear stochastic bandits

Advances in Neural Information Processing Systems 24: 25th Annual Conference on Neural Information Processing Systems 2011, NIPS 2011

Volumn , Issue , 2011, Pages

  Abbasi Yadkori, Yasin ^a   Pál, Dávid ^a   Szepesvári, Csaba ^a  

^a UNIVERSITY OF ALBERTA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; PROBABILITY;

BANDIT PROBLEMS; CONFIDENCE SETS; EMPIRICAL PERFORMANCE; HIGH PROBABILITY; IMPROVED * ALGORITHM; MULTIARMED BANDIT PROBLEMS (MABP); PERFORMANCE OF ALGORITHM; REGRET BOUNDS; SIMPLE MODIFICATIONS; STOCHASTICS;

STOCHASTIC SYSTEMS;

EID: 85162561761     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. Y. Abbasi-Yadkori, A. Antos, and Cs. Szepesvári. Forced-exploration based algorithms for playing in stochastic linear bandits. In COLTWorkshop on On-line Learning with Limited Feedback, 2009.
2. N. Abe, A. W. Biermann, and P. M. Long. Reinforcement learning with immediate rewards and linear hypotheses. Algorithmica, 37:263293, 2003.
3. A. Antos, V. Grover, and Cs. Szepesvári. Active learning in heteroscedastic noise. Theoretical Computer Science, 411(29-30):2712-2728, 2010.
4. J.-Y. Audibert, R. Munos, and Csaba Szepesvári. Exploration-exploitation tradeoff using variance estimates in multi-armed bandits. Theoretical Computer Science, 410(19):1876-1902, 2009.
5. P. Auer. Using upper confidence bounds for online learning. In FOCS, pages 270-279, 2000.
6. P. Auer. Using confidence bounds for exploitation-exploration trade-offs. JMLR, 2002.
7. P. Auer, N. Cesa-Bianchi, and P. Fischer. Finite time analysis of the multiarmed bandit problem. Machine Learning, 47(2-3):235-256, 2002.
8. S. Bubeck, R. Munos, G. Stoltz, and Cs. Szepesvári. Online optimization in X-armed bandits. In NIPS, pages 201-208, 2008.
9. N. Cesa-Bianchi and G. Lugosi. Prediction, Learning, and Games. 2006.
10. W. Chu, L. Li, L. Reyzin, and R. E. Schapire. Contextual bandits with linear payoff functions. In AISTATS, 2011.
11. P.-A. Coquelin and R. Munos. Bandit algorithms for tree search. In UAI, 2007.
12. V. Dani, T. P. Hayes, and S. M. Kakade. Stochastic linear optimization under bandit feedback. In Rocco Servedio and Tong Zhang, editors, COLT, pages 355-366, 2008.
13. V. H. de la Peña, M. J. Klass, and T. L. Lai. Self-normalized processes: exponential inequalities, moment bounds and iterated logarithm laws. Annals of Probability, 32(3):1902-1933, 2004.
14. V. H. de la Peña, T. L. Lai, and Q.-M. Shao. Self-normalized processes: Limit theory and Statistical Applications. Springer, 2009.
15. O. Dekel, C. Gentile, and K. Sridharan. Robust selective sampling from single and multiple teachers. In COLT, 2010.
16. D. A. Freedman. On tail probabilities for martingales. The Annals of Probability, 3(1):100-118, 1975.
17. A. Garivier and E. Moulines. On upper-confidence bound policies for non-stationary bandit problems. Technical report, LTCI, 2008.
18. R. Kleinberg, A. Niculescu-Mizil, and Y. Sharma. Regret bounds for sleeping experts and bandits. Machine learning, pages 1-28, 2008.
19. T. L. Lai and H. Robbins. Asymptotically efficient adaptive allocation rules. Advances in Applied Mathematics, 6:4-22, 1985.
20. T. L. Lai and C. Z. Wei. Least squares estimates in stochastic regression models with applications to identification and control of dynamic systems. The Annals of Statistics, 10(1):154-166, 1982.
21. T. L. Lai, H. Robbins, and C. Z. Wei. Strong consistency of least squares estimates in multiple regression. Proceedings of the National Academy of Sciences, 75(7):3034-3036, 1979.
22. L. Li, W. Chu, J. Langford, and R. E. Schapire. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th International Conference on World Wide Web (WWW 2010), pages 661-670. ACM, 2010.
23. V. Mnih, Cs. Szepesvári, and J.-Y. Audibert. Empirical Bernstein stopping. pages 672-679, 2008.
24. H. Robbins. Some aspects of the sequential design of experiments. Bulletin of the American Mathematical Society, 58:527-535, 1952.
25. P. Rusmevichientong and J. N. Tsitsiklis. Linearly parameterized bandits. Mathematics of Operations Research, 35(2):395-411, 2010.
26. G. W. Stewart and J.-G. Sun. Matrix Perturbation Theory. Academic Press, 1990.
27. T. J. Walsh, I. Szita, C. Diuk, and M. L. Littman. Exploring compact reinforcement-learning representations with linear regression. In UAI, pages 591-598. AUAI Press, 2009