Online algorithms for the multi-armed bandit problem with Markovian rewards

2010 48th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2010

Volumn , Issue , 2010, Pages 1675-1682

  Tekin, Cem ^a   Liu, Mingyan ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

INDEX POLICIES; LEARNING PROBLEM; MARKOVIAN; MULTI-ARMED BANDIT PROBLEM; NUMBER OF STATE; ON-LINE ALGORITHMS; OPTIMALITY; SAMPLE MEANS; STATE TRANSITION PROBABILITIES; STATE-DEPENDENT;

EID: 79952397795     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ALLERTON.2010.5707118     Document Type: Conference Paper

References (12)

1. H. Robbins, Some aspects of the sequential design of experiments, Bull. Amer. Math. Soc., 55, pp. 527-535, 1952
2. P. Auer, N. Cesa-Bianchi, Y. Freund, and R.E. Schapire, The non-stochastic multiarmed bandit problem, SIAM Journal on Computing, 32:48-77, 2002.
3. T. Lai, H. Robbins, Asymptotically efficient adaptive allocation rules, Advances in Applied Mathematics, 1985, 6 ,4-22
4. V. Anantharam, P. Varaiya, J . Walrand, Asymptotically efficient allocation rules for the multiarmed bandit problem with multiple plays-Part I: IID rewards, IEEE Trans. Automat. Contr., pp. 968-975, 1987
5. V. Anantharam, P. Varaiya, J . Walrand, Asymptotically efficient allocation rules for the multiarmed bandit problem with multiple plays-Part II: Markovian rewards, IEEE Trans. Automat. Contr., pp. 977-982, November 1987
6. R. Agrawal, Sample Mean Based Index Policies with O(log n) Regret for the Multi-Armed Bandit Problem, Advances in Applied Probability, Vol. 27, No. 4, pp. 1054-1078, December 1995.
7. P. Auer, N. Cesa-Bianchi, and P. Fischer. Finite-time analysis of the multiarmed bandit problem, Machine Learning, 47(2/3):235256, 2002.
8. K. Liu and Q. Zhao, Distributed Learning in Multi-Armed Bandit with Multiple Players, available at http://arxiv.org/abs/0910.2065.
9. A. Anandkumar, N. Michael, AK Tang, Opportunistic Spectrum Access with Multiple Players: Learning under Competition, Proc. of IEEE INFOCOM, March 2010.
10. J.C. Gittins, Bandit Processes and Dynamic Allocation Indices, Journal of the Royal Statistical Society, Series B, Vol. 41, No.2, pp 148-177, 1979.
11. D. Gillman, A chernoff bound for random walks on expander graphs, Proc. 34th IEEE Symp. on Foundations of Computer Science (FOCS93), vol. SIAM J. Comp., Vol 27, No 4, 1998.
12. P. Lezaud, Chernoff-type Bound for Finite Markov Chains, Ann. Appl. Prob. 8, pp 849-867, 1998.