Approximately optimal adaptive learning in opportunistic spectrum access

Proceedings - IEEE INFOCOM

Volumn , Issue , 2012, Pages 1548-1556

  Tekin, Cem ^a   Liu, Mingyan ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Approximate optimality; online learning; opportunistic spectrum access; restless bandits

Indexed keywords

ADAPTIVE LEARNING; ADAPTIVE LEARNING ALGORITHM; AVERAGE REWARD CRITERIA; CHANNEL CONDITIONS; COMPUTATIONALLY EFFICIENT; DISCRETE TIME MARKOV CHAINS; GILBERT-ELLIOT CHANNEL MODEL; INFINITE HORIZONS; MARKOV DECISION PROBLEM; ONLINE LEARNING; OPPORTUNISTIC SPECTRUM ACCESS; OPPORTUNISTIC SPECTRUM ACCESSES (OSA); OPTIMAL POLICIES; OPTIMALITY; POLYNOMIAL COMPLEXITY; RESTLESS BANDITS; TRANSITION PROBABILITIES; TRANSITION STRUCTURES; TWO-STATE; USER ACTIVITY;

LEARNING ALGORITHMS; SPECTROSCOPY;

OPTIMIZATION;

EID: 84861588214     PISSN: 0743166X     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/INFCOM.2012.6195522     Document Type: Conference Paper

References (21)

1. C. H. Papadimitriou and J. N. Tsitsiklis, "The complexity of optimal queuing network control," Math. Oper. Res., vol. 24, no. 2, pp. 293-305, 1999.
2. P. Whitlle, "Restless bandits," J. Appl. Prob., pp. 301-313, 1988.
3. S. Guha, K. Mungala, and P. Shi, "Approximation algorithms for restless bandit problems," Journal of the ACM, vol. 58, December 2010.
4. S. H. A. Ahmad, M. Liu, T. Javidi, Q. Zhao, and B. Krishnamachari, "Optimality of myopic sensing in multi-channel opportunistic access,"IEEE Transactions on Information Theory, vol. 55, no. 9, pp. 4040-4050, September 2009.
5. H. Robbins, "Some aspects of the sequential design of experiments,"Bull. Amer. Math. Soc., vol. 55, pp. 527-535, 1952.
6. T. Lai and H. Robbins, "Asymptotically efficient adaptive allocation rules," Advances in Applied Mathematics, vol. 6, pp. 4-22, 1985.
7. R. Agrawal, "Sample mean based index policies with o(log n) regret for the multi-armed bandit problem," Advances in Applied Probability, vol. 27, no. 4, pp. 1054-1078, December 1995.
8. V. Anantharam, P. Varaiya, and J. . Walrand, "Asymptotically efficient allocation rules for the multiarmed bandit problem with multiple plays-part i: Iid rewards," IEEE Trans. Automat. Contr., pp. 968-975, November 1987.
9. -, "Asymptotically efficient allocation rules for the multiarmed bandit problem with multiple plays-part ii: Markovian rewards," IEEE Trans. Automat. Contr., pp. 977-982, November 1987.
10. P. Auer, N. Cesa-Bianchi, and P. Fischer, "Finite-time analysis of the multiarmed bandit problem," Machine Learning, vol. 47, p. 235256, 2002.
11. C. Tekin and M. Liu, "Online algorithms for the multi-armed bandit problem with markovian rewards," in Proceedings of the 48th Annual Allerton Conference on Communication, Control, and Computation, September.
12. -, "Online learning in opportunistic spectrum access: A restless bandit approach," in 30th IEEE International Conference on Computer Communications (INFOCOM), April 2011.
13. -, "Online learning of rested and restless bandits," submitted to IEEE Transactions on Information Theory, under revision.
14. -, "Performance and convergence of multi-user online learning,"in 2nd International ICST Conference on Game Theory for Networks (GAMENETS), April 2011.
15. K. Liu and Q. Zhao, "Distributed learning in multi-armed bandit with multiple players," IEEE Transactions on Signal Processing, vol. 58, pp. 5667 - 5681, November 2010.
16. A. Anandkumar, N. Michael, and A. Tang, "Opportunistic spectrum access with multiple players: Learning under competition," in Proc. of IEEE INFOCOM, March 2010.
17. A. N. Burnetas and M. N. Katehakis, "Optimal adaptive policies for markov decision processes," Mathematics of Operations Research, vol. 22, no. 1, pp. 222-255, 1997.
18. A. Tewari and P. Bartlett, "Optimistic linear programming gives logarithmic regret for irreducible mdps," Advances in Neural Information Processing Systems, vol. 20, pp. 1505-1512, 2008.
19. C. Tekin and M. Liu, "Adaptive learning of uncontrolled restless bandits with logarithmic regret," in Proc. Forty-Ninth Annual Allerton Conference on Communication, Control, and Computing, September 2011.
20. A. Y. Mitrophanov, "Senstivity and convergence of uniformly ergodic markov chains," J. Appl. Prob., vol. 42, pp. 1003-1014, 2005.
21. P. Auer, N. Cesa-Bianchi, Y. Freund, and R. Schapire, "The nonstochastic multiarmed bandit problem," SIAM Journal on Computing, vol. 32, pp. 48-77, 2002.