Learning to compete for resources in wireless stochastic games

IEEE Transactions on Vehicular Technology

Volumn 58, Issue 4, 2009, Pages 1904-1919

  Fu, Fangwen ^a   van der Schaar, Mihaela ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Delay Sensitive Transmission; Delay sensitive transmission; Interactive learning; Interactive Learning; Multi user Resource Management; Multiuser resource management; Reinforcement Learning; Reinforcement learning; Stochastic Games; Stochastic games; Wireless networks; Wireless Networks

Indexed keywords

APPLICATION QUALITY; BIDDING POLICIES; BIDDING STRATEGY; COGNITIVE RADIO NETWORK; DELAY-SENSITIVE TRANSMISSION; DEVELOPING SOLUTIONS; FUTURE STRATEGY; INTERACTIVE LEARNING; MULTI-USER RESOURCE MANAGEMENT; SIMULATION RESULT; STOCHASTIC GAME; STOCHASTIC GAMES; TIME VARYING; WIRELESS USERS;

AUTONOMOUS AGENTS; EDUCATION; GAME THEORY; KNOWLEDGE MANAGEMENT; LEARNING ALGORITHMS; NATURAL RESOURCES MANAGEMENT; PACKET NETWORKS; PLANNING; RANDOM PROCESSES; REINFORCEMENT; REINFORCEMENT LEARNING; RESOURCE ALLOCATION; TIME VARYING NETWORKS;

WIRELESS NETWORKS;

EID: 66149152720     PISSN: 00189545     EISSN: None     Source Type: Journal    
DOI: 10.1109/TVT.2008.2002917     Document Type: Article

References (32)

1. Fed. Commun. Comm., Spectrum Policy Task Force, Nov. 2002. Rep. ET Docket No. 02-135.
2. S. Haykin, "Cognitive radio: Brain-empowered wireless communications," IEEE J. Sel. Areas Commun., vol. 23, no. 2, pp. 201-220, Feb. 2005.
3. I. F. Akyildiz, W. Y. Lee, M. C. Vuran, and S. Mohanty, "NeXt generation/dynamic spectrum access/cognitive radio wireless networks: A survey," Comput. Netw., vol. 50, no. 13, pp. 2127-2159, Sep. 2006.
4. C. Kloeck, H. Jaekel, and F. Jondral, "Auction sequence as a new resource allocation mechanism," in Proc. VTC, Dallas, TX, Sep. 2005, pp. 240-244.
5. F. Fu, A. R. Fattahi, and M. van der Schaar, "Game-theoretic paradigm for resource management in spectrum agile wireless networks," in Proc. IEEE ICME, 2006, pp. 873-876.
6. J. Huang, R. Berry, and M. L. Honig, "Auction-based spectrum sharing," ACM Mobile Netw. Appl. J. (MONET), vol. 11, no. 3, pp. 405-418, Jun. 2006.
7. Y. Xing, R. Chandramouli, and C. M. Cordeiro, "Price dynamics in competitive agile spectrum access markets," IEEE J. Sel. Areas Commun., vol. 25, no. 3, pp. 613-621, Apr. 2007.
8. L. Berlemann, S. Mangold, G. R. Hiertz, and B. H.Walke, "Policy defined spectrum sharing and medium access for cognitive radios," J. Commun. vol. 1, no. 1, pp. 1-12, Apr. 2006.
9. C. T. Chou, S. Shankar N, H. Kim, and K. Shin, "What and how much to gain by spectrum agility?" IEEE J. Sel. Areas Commun., vol. 25, no. 3, pp. 576-588, Apr. 2007.
10. S. Shankar, C. T. Chou, K. Challapali, and S. Mangold, "Spectrum agile radio: Capacity and QoS implications of dynamic spectrum assignment," in Proc. Global Telecommun. Conf., Nov. 2005, pp. 2510-2516.
11. D. Bertsekas and R. Gallager, Data Networks. Upper Saddle River, NJ: Prentice-Hall, 1987.
12. M. van der Schaar and S. Shankar, "Cross-layer wireless multimedia transmission: Challenges, principles, and new paradigms," IEEE Wireless Commun., vol. 12, no. 4, pp. 50-58, Aug. 2005.
13. Wireless Medium Access Control (MAC) and Physical Layer (PHY) Specifications: Medium Access Control (MAC) Enhancements for Quality of Service (QoS), Draft Supplement, IEEE Std. 802.11e/D5.0, Jun. 2003.
14. R. W. Lucky, "Tragedy of the commons," IEEE Spectr., vol. 43, no. 1, p. 88, Jan. 2006.
15. R. G. Gallager, Discrete Stochastic Processes. Norwell, MA: Kluwer, 1996.
16. L. S. Shapley, "Stochastic games," Proc. Nat. Acad. Sci. U.S.A., vol. 39, no. 10, pp. 1095-1100, Oct. 1953.
17. C. Watkins and P. Dayan, "Q-learning, technical note," Mach. Learn. vol. 8, no. 3/4, pp. 279-292, May 1992.
18. M. Bowling and M. Veloso, "Rational and convergent learning in stochastic games," in Proc. 17th IJCAI, Aug. 2001, pp. 1021-1026.
19. P. Klemperer, "Auction theory: A guide to the literature," J. Econ. Surv., vol. 13, no. 3, pp. 227-286, Jul. 1999.
20. S. P. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
21. F. Fu and M. van der Schaar, "Noncollaborative resource management for wireless multimedia applications using mechanism design," IEEE Trans. Multimedia, vol. 9, no. 4, pp. 851-868, Jun. 2007.
22. D. Fudenberg and D. K. Levine, The Theory of Learning in Games. Cambridge, MA: MIT Press, 1999.
23. M. Jackson, "Mechanism theory," in Encyclopedia of Life Support Systems. Oxford, U.K.: EOLSS, 2003.
24. Q. Zhang and S. A. Kassam, "Finite-state Markov model for Rayleigh fading channels," IEEE Trans. Commun., vol. 47, no. 11, pp. 1688-1692, Nov. 1999.
25. A. Ortega, "Variable bit-rate video coding," in Compressed Video Over Networks, M.-T. Sun and A. R. Reibman, Eds. New York: Marcel Dekker, 2000, pp. 343-382.
26. S. Lal and E. S. Sousa, "Distributed resource allocation for DS-CDMA-based multimedia ad hoc wireless LANs," IEEE J. Sel. Areas Commun., vol. 17, no. 5, pp. 947-967, May 1999.
27. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction Cambridge, MA: MIT Press, 1998.
28. M. Chiang, S. H. Low, A. R. Calderbank, and J. C. Doyle, "Layering as optimization decomposition: A mathematical theory of network architectures," Proc. IEEE, vol. 95, no. 1, pp. 255-312, Jan. 2007.
29. F. Kelly, A. Maulloo, and D. Tan, "Rate control for communication networks: Shadow prices, proportional fairness and stability," J. Oper. Res. Soc., vol. 49, no. 3, pp. 237-252, Mar. 1998.
30. X. Zhu, P. Agrawal, J. P. Singh, T. Alpcan, and B. Girod, "Rate allocation for multi-user video streaming over heterogeneous access networks," in Proc. ACM MM, Sep. 2007, pp. 37-46.
31. D. Krishnaswamy and J. Vicente, "Scalable adaptive wireless networks for multimedia in the proactive enterprise," Intel Technol. J., vol. 8, no. 4, pp. 291-301, Nov. 2004. [Online]. Available: http://developer.intel.com/ technology/itj/2004/volume08issue04/ art04_scalingwireless/p01_abstract.htm
32. D. S. Bernstein, E. A. Hansen, S. Zilberstein, and C. Amato, "Dynamic programming for partially observable stochastic games," in Proc. AAAI Spring Symp. Bridging Multi-Agent Multi-Robot. Res. Gap, Stanford, CA, Mar. 2004.