Logarithmic weak regret of non-Bayesian restless multi-armed bandit

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2011, Pages 1968-1971

  Liu, Haoyang ^a   Liu, Keqin ^a   Zhao, Qing ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

logarithmic order; non Bayesian formulation; regret; Restless multi armed bandit

Indexed keywords

AVERAGE REWARD; COGNITIVE RADIO NETWORK; FADING ENVIRONMENT; FINANCIAL INVESTMENTS; K-OUT-OF-N; LOGARITHMIC ORDER; MARKOVIAN; MARKOVIAN DYNAMICS; NON-BAYESIAN FORMULATION; OPPORTUNISTIC COMMUNICATIONS; POTENTIAL APPLICATIONS; REGRET; RESTLESS MULTI-ARMED BANDIT; UPPER CONFIDENCE BOUND;

FADING (RADIO); SIGNAL PROCESSING;

SPEECH COMMUNICATION;

EID: 80051636024     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2011.5946895     Document Type: Conference Paper

References (17)

1. T. Lai and H. Robbins, "Asymptotically Efficient Adaptive Allocation Rules," Advances in Applied Mathematics, Vol. 6, No. 1, pp. 4C22, 1985.
2. V. Anantharam, P. Varaiya, J. Walrand, "Asymptotically Efficient Allocation Rules for the Multiarmed Bandit Problem with Multiple Plays-Part I: I.I.D. Rewards," IEEE Transaction on Automatic Control, Vol. AC-32 ,No.11 , pp. 968-976, Nov., 1987.
3. V. Anantharam, P. Varaiya, J. Walrand, "Asymptotically Efficient Allocation Rules for the Multiarmed Bandit Problem with Multiple Plays- Part II: Markovian Rewards," IEEE Transaction on Automatic Control, Vol. AC-32 ,No.11 ,pp. 977-982, Nov., 1987.
4. R. Agrawal, "Sample Mean Based Index Policies With O(log n) Regret for the Multi-armed Bandit Problem," Advances in Applied Probability, Vol. 27, pp. 1054C1078, 1995.
5. P. Auer, N. Cesa-Bianchi, P. Fischer, "Finite-time Analysis of the Multiarmed Bandit Problem," Machine Learning, 47, 235-256, 2002. (Pubitemid 34126111)
6. C. Tekin, M. Liu, "Online Algorithms for the Multi-Armed Bandit Problem With Markovian Rewards," Proc. of Allerton Conference on Communications, Control, and Computing, Sep., 2010.
7. C. Papadimitriou, J. Tsitsiklis, "The Complexity of Optimal Queuing Network Control," Mathematics of Operations Research, Vol. 24, No. 2, pp. 293-305, May 1999.
8. H. Liu, K. Liu, and Q. Zhao "Learning and Sharing in A Changing World: Non-Bayesian Restless Bandit with Multiple Players," Proc. of Information Theory and Applications Workshop (ITA), February 2011.
9. K. Liu, Q. Zhao, "Distributed Learning in Multi-Armed Bandit with Multiple Players," Transations on Signal Processing, Vol. 58, No. 11, pp. 5667-5681, Nov. 2010.
10. C. Tekin, M. Liu, "Online Learning in Opportunistic Spectrum Access: A Restless Bandit Approach," Proc. of International Conference on Computer Communications (INFOCOM), April, 2011.
11. W. Dai, Y. Gai, B. Krishnamachari, Q. Zhao "The Non-Bayesian Restless Multi-armed Bandit: A Case Of Near-Logarithmic Regret," Proc. of Internanional Conference on Acoustics, Speech and Signal Processing (ICASSP), May, 2011.
12. P. Whittle, "Restless Bandits: Activity Allocation in a Changing World," Journal of Applied Probability, Vol. 25, pp. 287-298, 1988.
13. J. Gittins, "Bandit processes and dynamic allocation indicies (with discussion)," J. R. Statist. Soc., B, 41, pp. 148-177, 1979
14. R. Weber and G. Weiss, "On an Index Policy for Restless Bandits," Journal of Applied Probability, Vol. 27, No. 3, pp. 637-648, Sep., 1990.
15. K. Liu and Q. Zhao "Indexability of Restless Bandit Problems and Optimality of Whittle Index for Dynamic Multichannel Access," IEEE Transactions on Information Theory, Vol. 55, No. 11, pp. 5547-5567, Nov. 2010.
16. H. Liu, K. Liu, Q. Zhao, "Learning in A Changing World: Non-Bayesian Restless Multi-Armed Bandit," Technical Report, Oct. 2010. Available at http://arxiv.org/abs/1011.4969
17. D. Gillman, "A Chernoff Bound for Random Walks on Expander Graphs," Proc. 34th IEEE Symp. on Foundatioins of Computer Science (FOCS93),vol. SIAM J. Comp.,Vol. 27, No. 4, 1998.