On upper-confidence bound policies for switching bandit problems

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 6925 LNAI, Issue , 2011, Pages 174-188

  Garivier, Aurélien ^a   Moulines, Eric ^a  

^a INSTITUT TELECOM   (France)

Author keywords

[No Author keywords available]

Indexed keywords

ABRUPT CHANGE; BANDIT PROBLEMS; COGNITIVE RADIO; NONSTATIONARY; PARAMETER CONTROL; RANDOM NUMBERS; SCANNING TUNNELLING MICROSCOPES; SLIDING-WINDOW; SUMMANDS;

NUMERICAL METHODS;

ALGORITHMS;

EID: 80054097465     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-24412-4_16     Document Type: Conference Paper

References (25)

1. Agrawal, R.: Sample mean based index policies with O(log n) regret for the multiarmed bandit problem. Adv. in Appl. Probab. 27(4), 1054-1078 (1995)
2. Audibert, J.Y., Munos, R., Szepesvari, A.: Tuning bandit algorithms in stochastic environments. In: Hutter, M., Servedio, R.A., Takimoto, E. (eds.) ALT 2007. NCS (LNAI), vol. 4754, pp. 150-165. Springer, Heidelberg (2007)
3. Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM J. Comput. 32(1), 48-77 (2002)
4. Auer, P.: Using confidence bounds for exploitation-exploration trade-offs. J. Mach. Learn. Res. 3(Spec. Issue Comput. Learn. Theory), 397-422 (2002)
5. Auer, P., Cesa-Bianchi, N., Fischer, P.: Finite-time analysis of the multiarmed bandit problem. Machine Learning 47(2/3), 235-256 (2002)
6. Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press, New York (2006)
7. Cesa-Bianchi, N., Lugosi, G.: On prediction of individual sequences. Ann. Statist. 27(6), 1865-1895 (1999)
8. Cesa-Bianchi, N., Lugosi, G., Stoltz, G.: Regret minimization under partial monitoring. Math. Oper. Res. 31(3), 562-580 (2006)
9. Cesa-Bianchi, N., Lugosi, G., Stoltz, G.: Competing with typical compound actions (2008)
10. Devroye, L., Györfi, L., Lugosi, G.: A probabilistic theory of pattern recognition. Applications of Mathematics, vol. 31. Springer, New York (1996)
11. Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. System Sci. 55(1, part 2), 119-139 (1997);
12. In: Vitányi, P.M.B. (ed.) EuroCOLT 1995. LNCS, vol. 904. Springer, Heidelberg (1995)
13. Fuh, C.D.: Asymptotic operating characteristics of an optimal change point detection in hidden Markov models. Ann. Statist. 32(5), 2305-2339 (2004)
14. Garivier, A., Cappé, O.: The kl-ucb algorithm for bounded stochastic bandits and beyond. In: Proceedings of the 24rd Annual International Conference on Learning Theory (2011)
15. Hartland, C., Gelly, S., Baskiotis, N., Teytaud, O., Sebag, M.: Multi-armed bandit, dynamic environments and meta-bandits. In: nIPS-2006 Workshop, Online Trading Between Exploration and Exploitation, Whistler, Canada (2006)
16. Herbster, M., Warmuth, M.: Tracking the best expert. Machine Learning 32(2), 151-178 (1998)
17. Honda, J., Takemura, A.: An asymptotically optimal bandit algorithm for bounded support models. In: Proceedings of the 23rd Annual International Conference on Learning Theory (2010)
18. Kocsis, L., Szepesvári, C.: Discounted UCB. In: 2nd PASCAL Challenges Workshop, Venice, Italy (April 2006)
19. Koulouriotis, D.E., Xanthopoulos, A.: Reinforcement learning and evolutionary algorithms for non-stationary multi-armed bandit problems. Applied Mathematics and Computation 196(2), 913-922 (2008)
20. Lai, L., El Gamal, H., Jiang, H., Poor, H.V.: Cognitive medium access: Exploration, exploitation and competition (2007)
21. Lai, T.L., Robbins, H.: Asymptotically efficient adaptive allocation rules. Adv. in Appl. Math. 6(1), 4-22 (1985)
22. Mei, Y.: Sequential change-point detection when unknown parameters are present in the pre-change distribution. Ann. Statist. 34(1), 92-122 (2006)
23. Slivkins, A., Upfal, E.: Adapting to a changing environment: the brownian restless bandits. In: Proceedings of the Conference on 21st Conference on Learning Theory, pp. 343-354 (2008)
24. Whittle, P.: Restless bandits: activity allocation in a changing world. J. Appl. Probab. Special 25A, 287-298 (1988) a celebration of applied probability
25. Yu, J.Y., Mannor, S.: Piecewise-stationary bandit problems with side observations. In: ICML 2009: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 1177-1184. ACM, New York (2009)