Distributed algorithms for learning and cognitive medium access with logarithmic regret

IEEE Journal on Selected Areas in Communications

Volumn 29, Issue 4, 2011, Pages 731-745

  Anandkumar, Animashree ^a   Michael, Nithin ^b   Tang, Ao Kevin ^b   Swami, Ananthram ^c  

^a University of California at Irvine   (United States)

^b School of Operations Research and Industrial Engineering   (United States)

^c U S ARMY RESEARCH LABORATORY   (United States)

Author keywords

Cognitive medium access control; distributed algorithms; logarithmic regret; multi armed bandits

Indexed keywords

ACCESS DECISION; ACCESS POLICIES; CHANNEL ACCESS; COGNITIVE MEDIUM ACCESS CONTROL; COGNITIVE NETWORK; DISTRIBUTED ALGORITHM; DISTRIBUTED ALGORITHMS; DISTRIBUTED LEARNING; EXPLICIT INFORMATION; LOGARITHMIC REGRET; LOWER BOUNDS; MEDIUM ACCESS; MULTI ARMED BANDIT; OPTIMAL REGRET; SELF-PLAY; SYSTEM THROUGHPUT;

ALGORITHMS; COGNITIVE SYSTEMS; NETWORK PROTOCOLS; OPTIMIZATION; RADIO COMMUNICATION; SECURITY SYSTEMS;

MEDIUM ACCESS CONTROL;

EID: 79953194834     PISSN: 07338716     EISSN: None     Source Type: Journal    
DOI: 10.1109/JSAC.2011.110406     Document Type: Article

References (25)

1. A. Anandkumar, N. Michael, and A. Tang, "Opportunistic Spectrum Access with Multiple Users: Learning under Competition," in Proc. IEEE INFOCOM, San Deigo, USA, March 2010.
2. Q. Zhao and B. Sadler, "A Survey of Dynamic Spectrum Access," IEEE Signal Proc. Mag., vol. 24, no. 3, pp. 79-89, 2007. (Pubitemid 46840190)
3. L. Kaelbling, M. Littman, and A. Moore, "Reinforcement learning: A Survey," J. Artificial Intelligence research, vol. 4, no. 237-285, pp. 102- 138, 1996.
4. K. Liu and Q. Zhao, "Decentralized Multi-Armed Bandit with Multiple Distributed Players," submitted to IEEE Trans. Signal Process., Oct. 2009.
5. A. Konrad, B. Zhao, A. Joseph, and R. Ludwig, "A Markov-based channel model algorithm for wireless networks," Wireless Networks, vol. 9, no. 3, pp. 189-199, 2003.
6. S. Geirhofer, L. Tong, and B. Sadler, "Cognitive Medium Access: Constraining Interference Based on Experimental Models," IEEE J. Sel. Areas Commun., vol. 26, no. 1, p. 95, 2008.
7. V. Anantharam, P. Varaiya, and J. Walrand, "Asymptotically Efficient Allocation Rules for the Multiarmed Bandit Problem with Multiple Plays-Part II: Markovian Rewards," IEEE Tran. Autom. Control, vol. 32, no. 11, pp. 977-982, 1987. (Pubitemid 18521626)
8. T. Lai and H. Robbins, "Asymptotically efficient adaptive allocation rules," Advances in Applied Mathematics, vol. 6, no. 1, pp. 4-22, 1985.
9. V. Anantharam, P. Varaiya, and J. Walrand, "Asymptotically Efficient Allocation Rules for the Multiarmed Bandit Problem with Multiple Plays-Part I: IID rewards," IEEE Tran. Autom. Control, vol. 32, no. 11, pp. 968-976, 1987. (Pubitemid 18521625)
10. R. Agrawal, "Sample Mean Based Index Policies with O(log n) Regret for the Multi-Armed Bandit Problem," Advances in Applied Probability, vol. 27, no. 4, pp. 1054-1078, 1995.
11. P. Auer, N. Cesa-Bianchi, and P. Fischer, "Finite-time Analysis of the Multiarmed Bandit Problem," Machine Learning, vol. 47, no. 2, pp. 235-256, 2002. (Pubitemid 34126111)
12. Q. Zhao, Y. Chen, and A. Swami, "Cognitive MAC Protocols for Dynamic Spectrum Access," in Cognitive Wireless Communication Networks. Springer, 2007, pp. 271-301.
13. K. Liu and Q. Zhao, "A restless bandit formulation of opportunistic access: Indexablity and index policy," Submitted IEEE Trans. Inf. Theory, available on Arxiv, 2008.
14. H. Liu, B. KrishnaMAChari, and Q. Zhao, "Cooperation and Learning in multiuser opportunistic spectrum access," in IEEE Intl. Conf. on Comm. (ICC), Beijing, China, May 2008.
15. F. Fu and M. van der Schaar, "Learning to compete for resources in wireless stochastic games," IEEE Trans. Veh. Technol., vol. 58, no. 4, pp. 1904-1919, May 2009.
16. H. Gang, Z. Qian, and X. Ming, "Contention-Aware Spectrum Sensing and Access Algorithm of Cognitive Network," in Intl. Conf. on Cognitive Radio Oriented Wireless Networks and Comm., Singapore, May 2008.
17. H. Liu, L. Huang, B. KrishnaMAChari, and Q. Zhao, "A Negotiation Game for Multichannel Access in Cognitive Radio Networks," in Proc. Intl. Conf. on Wireless Internet, Las Vegas, NV, Nov. 2008.
18. H. Li, "Multi-agent Q-Learning of Channel Selection in Multi-user Cognitive Radio Systems: A Two by Two Case," in IEEE Conf. on System, Man and Cybernetics, Istanbul, Turkey, 2009.
19. M. Maskery, V. Krishnamurthy, and Q. Zhao, "Game Theoretic Learning and Pricing for Dynamic Spectrum Access in Cognitive Radio," in Cognitive Wireless Comm. Networks. Springer, 2007.
20. R. Kleinberg, G. Piliouras, and E. Tardos, "Multiplicative Updates Outperform Generic No-regret Learning in Congestion Games," in Proc. ACM Symp. on theory of computing (STOC), Bethesda, MD, May-June 2009.
21. Y. Gai, B. KrishnaMAChari, and R. Jain, "Learning Multiuser Channel Allocations in Cognitive Radio Networks: A Combinatorial Multi- Armed Bandit Formulation," in IEEE Symp. on Dynamic Spectrum Access Networks (DySPAN), Singapore, April 2010.
22. K. Liu, Q. Zhao, and B. KrishnaMAChari, "Distributed learning under imperfect sensing in cognitive radio networks," in Submitted to Proc. IEEE Asilomar Conf. on Signals, Sys., and Comp., Monterey, CA, Oct. 2010.
23. T. Cover and J. Thomas, Elements of Information Theory. John Wiley & Sons, Inc., 1991.
24. M. Bona, A Walk Through Combinatorics: An Introduction to Enumeration and Graph Theory. World Scientific Pub. Co. Inc., 2006.
25. M. Shaked and J. Shanthikumar, Stochastic Orders. Springer, 2007.