Adaptive learning of uncontrolled restless bandits with logarithmic regret

2011 49th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2011

Volumn , Issue , 2011, Pages 983-990

  Tekin, Cem ^a   Liu, Mingyan ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE LEARNING; AVERAGE REWARD; FINITE HORIZONS; INFINITE HORIZONS; OPTIMAL POLICIES; OPTIMAL STATIONARY POLICY; RESTLESS BANDIT; STATE TRANSITIONS; TRANSITION PROBABILITIES;

COMMUNICATION; LEARNING ALGORITHMS;

STRUCTURAL OPTIMIZATION;

EID: 84856091352     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/Allerton.2011.6120273     Document Type: Conference Paper

References (21)

1. L. K. Platzman, "Optimal infinite-horizon undiscounted control of finite probabilistic systems," SIAM J. Control Optim., vol. 18, pp. 362-380, 1980.
2. S. P. Hsu, D. M. Chuang, and A. Arapostathis, "On the existence of stationary optimal policies for partially observed mdps under the long-run average cost criterion," Systems and Control Letters, vol. 55, pp. 165-173, 2006. (Pubitemid 41745435)
3. T. Lai and H. Robbins, "Asymptotically efficient adaptive allocation rules," Advances in Applied Mathematics, vol. 6, pp. 4-22, 1985.
4. R. Agrawal, "Sample mean based index policies with o(log n) regret for the multi-armed bandit problem," Advances in Applied Probability, vol. 27, no. 4, pp. 1054-1078, December 1995.
5. V. Anantharam, P. Varaiya, and J. . Walrand, "Asymptotically efficient allocation rules for the multiarmed bandit problem with multiple plays-part i: Iid rewards," IEEE Trans. Automat. Contr., pp. 968-975, November 1987.
6. -, "Asymptotically efficient allocation rules for the multiarmed bandit problem with multiple plays-part ii: Markovian rewards," IEEE Trans. Automat. Contr., pp. 977-982, November 1987.
7. P. Auer, N. Cesa-Bianchi, and P. Fischer, "Finite-time analysis of the multiarmed bandit problem," Machine Learning, vol. 47, p. 235256, 2002.
8. C. Tekin and M. Liu, "Online algorithms for the multi-armed bandit problem with markovian rewards," in Proceedings of the 48th Annual Allerton Conference on Communication, Control, and Computation, September.
9. -, "Online learning in opportunistic spectrum access: A restless bandit approach," in 30th IEEE International Conference on Computer Communications (INFOCOM), April 2011.
10. -, "Online learning of rested and restless bandits,"http:// arxiv.org/abs/1102.3508v1.
11. -, "Performance and convergence of multi-user online learning,"in 2nd International ICST Conference on Game Theory for Networks (GAMENETS), April 2011.
12. K. Liu and Q. Zhao, "Distributed learning in multi-armed bandit with multiple players, http://arxiv.org/abs/0910.2065."
13. A. Anandkumar, N. Michael, and A. Tang, "Opportunistic spectrum access with multiple players: Learning under competition," in Proc. of IEEE INFOCOM, March 2010.
14. A. N. Burnetas and M. N. Katehakis, "Optimal adaptive policies for markov decision processes," Mathematics of Operations Research, vol. 22, no. 1, pp. 222-255, 1997.
15. A. Tewari and P. Bartlett, "Optimistic linear programming gives logarithmic regret for irreducible mdps," Advances in Neural Information Processing Systems, vol. 20, pp. 1505-1512, 2008.
16. C. Tekin and M. Liu, "Adaptive learning of uncontrolled restless bandits with logarithmic regret," http://arxiv.org/abs/1107.4042.
17. A. Y. Mitrophanov, "Senstivity and convergence of uniformly ergodic markov chains," J. Appl. Prob., vol. 42, pp. 1003-1014, 2005.
18. C. H. Papadimitriou and J. N. Tsitsiklis, "The complexity of optimal queuing network control," Math. Oper. Res., vol. 24, no. 2, pp. 293-305, 1999.
19. P. Whitlle, "Restless bandits," J. Appl. Prob., pp. 301-313, 1988.
20. S. Guha, K. Mungala, and P. Shi, "Approximation algorithms for restless bandit problems," 20th ACM-SIAM Symp. on Discrete Algorithms (SODA), pp. 28-37, 2009.
21. S. H. A. Ahmad, M. Liu, T. Javidi, Q. Zhao, and B. Krishnamachari, "Optimality of myopic sensing in multi-channel opportunistic access,"IEEE Transactions on Information Theory, vol. 55, no. 9, pp. 4040-4050, September 2009.