Reinforcement learning for repeated power control game in cognitive radio networks

IEEE Journal on Selected Areas in Communications

Volumn 30, Issue 1, 2012, Pages 54-69

  Zhou, Pan ^a   Chang, Yusun ^b   Copeland, John A ^a  

^a GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

^b SOUTHERN POLYTECHNIC STATE UNIVERSITY   (United States)

Author keywords

Bush Mosteller; Cognitive Radio; Lagrangian Multiplier; Nash Equilibrium; Power Control; Reinforcement Learning; Repeated Games; Stochastic Optimization

Indexed keywords

BUSH MOSTELLER; COGNITIVE RADIO; LAGRANGIAN MULTIPLIERS; NASH EQUILIBRIUM; REPEATED GAMES; STOCHASTIC OPTIMIZATIONS;

COMPUTER SIMULATION; CONSTRAINED OPTIMIZATION; GAME THEORY; LEARNING ALGORITHMS; POWER CONTROL; RADIO; REINFORCEMENT; SIGNAL INTERFERENCE; STOCHASTIC SYSTEMS; WIRELESS TELECOMMUNICATION SYSTEMS;

REINFORCEMENT LEARNING;

EID: 84855426943     PISSN: 07338716     EISSN: None     Source Type: Journal    
DOI: 10.1109/JSAC.2012.120106     Document Type: Article

References (40)

1. Akyildiz, I. F., Lee, W.Y., Vuran, M.C. and Mohanty, S., "NeXt Generation/ Dynamic Spectrum Access/Cognitive Radio Wireless Networks: A Survey," Computer Networks Journal, (Elsevier), vol. 50, pp.2127-2159, September 2006. (Pubitemid 43975031)
2. B. Wang, Y. Wu and K.J. R. Liu, "Game theory for cognitive radio networks: An overview," Computer Networks Journal, (Elsevier), vol. 54, iss.14, pp. 2537-2561, Oct. 2010.
3. M. Maskery, V. Krishnamurthy, and Q. Zhao, "Decentralized Dynamic Spectrum Access for Cognitive Radios: Cooperative Design of a Noncooperative Game, "in IEEE Trans. Commun., vol. 57, No. 2, pp. 459-469, February, 2009.
4. J., Shu and P., Varaiya, "Pricing Network Services, "In Proceedings of IEEE Infocom 2003, San Francisco, CA, pp. 1221-1230, April, 2003.
5. L. Qiu, Y. Yang, Y. Zhang, and S. Shenker, "On Selfish Routing in Internet-Like Environments," In Proceedings of ACM SIGCOMM, Karlsruhe, Germany, pp. 12-25, August 2003.
6. A. Al Daoud, T. Alpcan, S. Agarwal, and M. Alanyali, "A Stackelberg game for pricing uplink power in wide-band cognitive radio networks," in Proc. 47th IEEE Conf. Decision Control (CDC 2008), pp. 1422-1427, Dec. 2008.
7. A. Al-Daoud, M. Alanyali, and D. Starobinski, "Secondary pricing of spectrum in cellular CDMA networks," in Proc. IEEE DySPAN, pp. 535-542, Nov. 2007. (Pubitemid 47431694)
8. Y. Xing, R. Chandramouli, and C. M. Cordeiro, "Price dynamics in a competitive agile secondary spectrum access market," IEEE J. Sel. Areas Commun., vol. 25, no. 3, pp. 613-621, Apr. 2007. (Pubitemid 46620955)
9. D. Niyato and E. Hossain, "Competitive pricing for spectrum sharing in cognitive radio networks: Dynamic game, inefficiency of nash equilibrium, and collusion," IEEE J. Sel. Areas Commun., vol. 26, no. 1, pp. 192-202, Jan. 2008.
10. H. Yu, L.Gao, Z. Li, X. Wang and E. Hossain, "Pricing for uplink power control in cognitive radio networks," IEEE Trans. Veh. Technol., Vol. 59, Iss. 4, pp. 1769 - 1778, 2010.
11. P. Zhou, W. Yuan, W. Liu and W. Cheng, "Joint power and rate control in cognitive radio networks: a game-theoretical approach," IEEE International Conference on Communications (ICC 2008), pp. 3296-3301, 19-23 May 2008.
12. Y. Wu and D. H. K. Tsang, "Distributed multi-channel power allocation algorithm for spectrum sharing cognitive radio networks with QoS guarantee," The 28th IEEE Conference on Computer Communications (INFOCOM), 2009.
13. R. Zhang, "Optimal power control over fading cognitive radio channel by exploiting primary user CSI," IEEE Global Telecommunications Conference (GLOBECOM 2008), pp. 1-5, Nov. 30-Dec. 4, 2008.
14. X. Kang, R. Zhang, Y. C. Liang and H.K. Garg, "On Outage Capacity of Secondary Users in Fading Cognitive Radio Networks with Primary User's Outage Constraint," IEEE Global Telecommunications Conference (GLOBECOM 2009), pp. 1-5, Nov. 30-Dec. 4, 2009.
15. S. Huang, X. Liu, and Z. Ding,"Decentralized Cognitive Radio Control based on Inference from Primary Link Control Information," to appear in IEEE J. Sel. Areas Commun., Special Issue on Advances in Cognitive Radio Networking and Communications.
16. F. E. Lapiccirella, S. Huang, X. Liu, and Z. Ding, "Feedback-based access and power control for distributed multiuser cognitive networks," Information Theory and Application (ITA) workshop, UCSD, 2009.
17. P. Zhou, Y. Chang and J. Copeland, "Asynchronous Power Control Game with Channel Outage Constraints for Cognitive Radio Networks," to appeaer in IEEE International Conference on Communications (IEEE ICC 2011), Tokyo, Japan, 2011.
18. J. Huang, R. Berry, and M. L. Honig, "Auction-based spectrum sharing," ACM Mobile Netw. Appl. J., vol. 11, no. 3, pp. 405-418, Jun. 2006. (Pubitemid 43724282)
19. D. Fudenberg, D. K. Levine, The Theory of Learning in Games. Cambridge, MA: MIT Press, 1998.
20. A.J., Goldsmith and P.P., Varaiya, "Capacity, mutual information, and coding for finite-state Markov channels," IEEE Trans. Inf. Theory, vol. 42, iss.3, pp. 868-886, 1996. (Pubitemid 126770540)
21. A.J., Goldsmith, Wireless Communication, Cambridge University Press, 2005.
22. H. S. Wang and N. Moayeri, "Finite-state Markov channel-a useful model for radio communication channels," IEEE Trans. Veh. Technol., Vol. 44, iss. 1, pp. 163-171.
23. A.N., Shiryaev, Probability, second edition, Springer Press, 2005.
24. P., Hall and C.C. Heyde, Martingale Limmit Theory and Its Applications, Academic Press, 1980.
25. C. Long, Q. Zhang, B. Li, H. Yang, and X. Guan, "Non-Cooperative Power Control for Wireless Ad Hoc Networks with Repeated Games", in IEEE J. Sel. Areas Commun., special issue on Non-Cooperative Behavior in Networking, Aug. 2007.
26. Y. Xing and R. Chandramouli, "Stochastic Learning Solution for Distributed Discrete Power Control Game in Wireless Data Networks", IEEE/ACM Trans. Netw., vol: 16, iss.4, 2008, pp: 932-944.
27. M. Chiang, "Balancing transport and physical layers in wireless multihop networks: Jointly optimal congestion control and power control," IEEE J. Sel. Areas Commun., Vol.23, pp. 104-116, Jan., 2005. (Pubitemid 40088661)
28. P. Zhou, Y. Chang and J. Copeland, "Learning through reinforcement for repeated power control game in cognitive radio networks," IEEE Global Telecommunications Conference (IEEE GLOBECOM 2010), pp. 1157-1162, Dec. 5-Dec. 10, Florida, Dec. 2010.
29. J. Nash, "Equilibrium points in n-person games," Proc. Nat. Acad. USA, vol. 36, pp. 48C49, 1950.
30. A. S. Poznyak, K. Najim, and E. Gomez, Self-Learning Control for Finite Markov Chains. New York: Marcel Dekker, 2000
31. K. J. Arrow, L. Hurwicz, and H. Uzawa, "Constraint qualifications in maximization problems," Nav. Res. Logist. Q., vol. 8, pp. 175-191, 1961.
32. Bush, R. and Mosteller, F., "Stochastic Models of Learning, " John Wiliey & Son, New York, 1995.
33. J. B. Rosen, "Existence and uniqueness of equilibrium points for concave N-persons games," Econometrica, vol. 33, pp. 520-534, 1965.
34. J. P. Aubin, Mathematical Methods of Game and Economic Theory, Amsterdam, The Netherlands: North Holland, 1979.
35. N. N. Vorob́ev, Foundations of Game Theory: Noncooperative Games, Basel: Birkháuser, 1994.
36. Whittle P., Optimization under Constriants, Wiley-Interscience, New York, 1971.
37. A. S. Poznyak and K. Najim, Learning Automata and Stochastic Optimization, New York: Springer-Verlag, 1997.
38. Baba N., New Topics in Learning Automata Theory and Applications, Springer-Verlag, Berlin, 1984.
39. K. Najim and A. S. Poznyak, Learning Automata: Theory and Applications. New York: Pergamon, Springer-Verlag, 1994.
40. H. Robbins and D. Siegmund, "A convergence theorem for non negative almost supermar- tingales and some applications," In Optimizing Methods in Statitsics. (JS Rustagi, ed.), pp. 233-257, 1971.