Traffic offloading in two-tier multi-mode small cell networks over unlicensed bands: A hierarchical learning framework

KSII Transactions on Internet and Information Systems

Volumn 9, Issue 11, 2015, Pages 4291-4310

  Sun, Youming ^a,b   Shao, Hongxiang ^b,c   Liu, Xin ^d   Zhang, Jian ^a   Qiu, Junfei ^b   Xu, Yuhua ^b  

^a NATIONAL DIGITAL SWITCHING SYSTEM ENGINEERING AND TECHNOLOGICAL RESEARCH CENTER   (China)

^b PLA UNIVERSITY OF SCIENCE AND TECHNOLOGY   (China)

^c LUOYANG INSTITUTE OF SCIENCE AND TECHNOLOGY   (China)

^d Outpatient Department   (China)

Author keywords

Q learning; Small cells; Stackelberg game; Traffic offloading

Indexed keywords

BASE STATIONS; DEEP LEARNING; LEARNING SYSTEMS; REINFORCEMENT LEARNING;

CONVERGENCE PERFORMANCE; HIERARCHICAL LEARNING; NUMERICAL EXPERIMENTS; Q-LEARNING; Q-LEARNING ALGORITHMS; SMALL CELLS; STACKELBERG EQUILIBRIUM; STACKELBERG GAMES;

LEARNING ALGORITHMS;

EID: 84948666925     PISSN: 19767277     EISSN: 22881468     Source Type: Journal    
DOI: 10.3837/tiis.2015.11.002     Document Type: Article

References (40)

1. J. G. Andrews, H. Claussen, M. Dohler, et al. “Femtocells: past, present, and future,” IEEE J. Sel. Areas Commun., vol. 30, no. 3, pp. 497-508, Apr. 2012
2. J. G. Andrews, S. Buzzi, W. Choi, et al. “What will 5G be?,” IEEE J. Sel. Areas Commun., vol. 32, no. 6, pp. 1065-1082, Jun. 2014
3. Y. Xu, J. Wang, Q. Wu, et al. “A game theoretic perspective on self-organizing optimization for cognitive small cells,” IEEE Commun. Mag., vol. 53, no. 7, pp. 100-108, Jul. 2015.
4. M. Jo, T. Maksymyuk, B. Strykhalyuk, et al. “Device-to-Device Based Heterogeneous Radio Access Network Architecture for Mobile Cloud Computing,” IEEE Wireless Commun., vol. 22, no.3, pp. 50-58, Jun. 2015
5. Z. Du, Q. Wu, P. Yang, et al. “Exploiting user demand diversity in heterogeneous wireless networks,” IEEE Trans. Wireless Commun., vol. 14, no. 8, pp. 4142-4155, Aug. 2015.
6. Z. Du, Q. Wu, P. Yang, et al. “Dynamic user demand driven online network selection,” IEEE Trans. Veh. Tech. vol. 18, no. 3, pp. 1089-7798. Mar. 2014
7. F. Liu, E. Bala, E. Erkip, et al. “Small cell traffic balancing over licensed and unlicensed Bands,” IEEE Trans. Veh. Tech., to appear
8. M. Bennis, M. Simsek, A. Czylwik, et al. “When cellular meets WiFi in wireless small cell networks,” IEEE Commun. Mag., vol. 51, no. 6, pp. 44-50. Jun. 2013
9. Qualcomm. “LTE in unlicensed spectrum: harmonious coexistence with Wi-Fi,” white paper, Jun. 2014
10. Huawei, “U-LTE: unlicensed spectrum utilization of LTE,” white paper, Feb. 2014.
11. M. Mustonen, T. Chen, H. Saarnisaari, et al, “Cellular architecture enhancement for supporting the european licensed shared access concept,” IEEE Wireless Mag., vol. 21, no. 3, pp. 37-43. Jun. 2014
12. Federal Communications Commission, Unlicensed operation in the TV broadcast bands, 47 CFR Part 15, Federal Register, vol. 74, no. 30, Feb. 2009
13. M. Iwamura, K. Etemad, M.-H. Fong, R. Nory, and R. Love, “Carrier aggregation framework in 3GPP LTE-advanced [WiMAX/LTE update],” IEEE Commun. Mag., vol. 48, no. 8, pp. 60-67, Aug. 2010
14. A. al-Dulaimi, S. al-rubaye, Q. Ni, et al. “Pursuit of more capacity triggers LTE in unlicensed band,” IEEE vehicular Technology Mag., vol. 10, no. 1, pp. 43-51. 2015
15. 3GPP-TSG-RAN-Meetin65, “Study on licensed assisted access using LTE,” Tech. Rep. RP141664, Sep. 2014
16. L. Zhou, M. Chen, Y. Qian, et al. “Fairness Resource Allocation in Blind Wireless Multimedia Communications,” IEEE Transactions on Multimedia, vol. 15, no. 4, pp. 946-956, Jun. 2013
17. L. Zhou, Z. Yang, Y. Wen, et al. “Resource Allocation with Incomplete Information for QoE-driven Multimedia Communications,” IEEE Transactions on Wireless Communications, vol. 12, no. 8, pp. 3733-3745, Aug. 2013
18. Y. Xu, A. Anpalagan, Q. Wu, et al. “Decision-theoretic distributed channel selection for opportunistic spectrum access: Strategies, challenges and solutions,” IEEE Commun. Surveys and Tutorials, vol. 15, no. 4, pp. 1689-1713, Fourth Quarter, 2013
19. Y. Xu, J. Wang, Q. Wu, et al. “Opportunistic spectrum access in cognitive radio networks: Global optimization using local interaction games,” IEEE Journal of Selected Topics in Signal Processing, vol. 6, no. 2, pp. 180-194, Apr. 2012
20. Y. Xu, J. Wang, Q. Wu, et al. “Opportunistic spectrum access in unknown dynamic environment: A game-theoretic stochastic learning solution,” IEEE Transactions on Wireless Communications, vol. 11, no. 4, pp. 1380-1391, Apr. 2012
21. X. Kang, R. Zhang, and M. Motani, “Price-based resource allocation for spectrum-sharing femtocell networks: a Stackelberg game approach,” IEEE J. Sel. Areas Commun., vol. 30, no. 3, pp. 538-549, Apr. 2012
22. S. Guruacharya, D. Niyato, D. I. Kim, and E. Hossain, “Hierarchical competition for downlink power allocation in OFDMA femtocell networks,” IEEE Trans. Wireless Commun., vol. 12, no. 4, pp. 1543-1553, Apr. 2013
23. Q. Han, B. Yang, X. Wang, et al. “Hierarchical-game-based uplink power control in femtocell networks,” IEEE Trans. Veh. Technol., vol. 63, no. 6, pp. 2819-2835, Jul. 2014.
24. S. Bu, F. R. Yu, Y. Cai and H. Yanikomeroglu. “Interference-aware energy-effcient resource allocation for OFDMA-based heterogeneous networks with incomplete channel state information.” IEEE Trans. Veh. Technol., vol. 64, no. 3, pp. 1036-1050, Jun. 2014.
25. Y. Sun, H. Shao, J. Qiu, et al. “Capacity offloading in two-tier small cell networks over unlicensed band: a hierarchical learning framework,” in Proc. of the IEEE International Conference on Wireless Communications and Signal Processing(WCSP 15), Nanjing, Oct. 2015, to appear.
26. Y. Chiang and W. Liao. “Genie: An optimal green policy for energy saving and traffic offloading in heterogeneous cellular networks,” in Proc. of the IEEE International Conference on Communications (ICC), Budapest , Jun. 2013, pp. 6230-6234
27. X. Chen, J. Wu, Y. Cai et al. “Energy-efficiency oriented traffic offloading in wireless networks: a brief survey and a learning approach for heterogeneous cellular networks,” IEEE J. Sel. Areas Commun., vol. 33, no. 4, pp. 627-640, Apr. 2015
28. X. Kang, Y. Chia, S. Sun et al. “Mobile data offloading through a third-party WiFi access point: An operator’s perspective,” IEEE Trans. Wireless Commun., vol. 13, no. 10, pp. 5340-5351, Oct. 2014
29. L. Gao, G. Iosifidis, J. Huang, and L. Tassiulas, “Economics of mobile data offloading,” in Proc. of IEEE INFOCOM SDP Workshop, Turin, Italy, Apr. 2013, pp. 3303-3308.
30. M. Bennis, S. M. Perlaza, P. Blasco, et al. “Self-organization in small cell networks: a reinforcement learning approach,” IEEE Trans. Wireless Commun., vol. 12, no. 7, pp. 3202-3212, Jul. 2013
31. F. Zhang, W. Zhang and Q. Ling. “Non-cooperative game for capacity offload.” IEEE Trans. Wireless Commun., vol. 11, no. 4, pp. 1565-1575, Apr. 2012
32. R. Mahindra, H. Viswanathan, K. Sundaresan, et al. “A practical traffic management system for integrated LTE-WiFi networks,” in Proc. of the 20th annual international conference on Mobile computing and networking(Mobicom 14), Hawaii,USA , Sep. 2014, pp. 189-200.
33. J.B. Rosen. “Existence and uniqueness of equilibrium points for concave N-person games,” Ecnometrica, vol. 33, no. 3, pp. 520-534, Jul. 1965
34. D. Fudenburg and J. Tirole. “Game theory,” The MIT press, 1991
35. C. J. C. H. Watkins and P. Dayan. “Q-learning,” Mach. Learn., vol. 8, pp. 279-292, 1992.
36. Z. Du, Q. Wu and P. Yang. “Dynamic user demand driven online network selection,” IEEE Commun. Letters, vol. 18, no. 3, pp. 419-422, Mar. 2014
37. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA, USA: MIT Press, 1998
38. X. Chen, H. Zhang, T. Chen, et al. “Improving energy efficiency in femtocell networks: a hierarchical reinforcement learning framework,” in Proc. of the IEEE International Conference on Communications (ICC), Budapest, Jun. 2013, pp. 2241-2245
39. A. Kianercy and A. Galstyan. “Dynamics of Boltzmann Q-learning in two-player two-action games,” Physical Review E, vol. 85, no. 4, pp.1-10, Apr. 2012
40. P. S. Sastry, V. V. Phansalkar, and M. Thathachar, “Decentralized learning of nash equilibria in multi-person stochastic games with incomplete information,” IEEE Trans. on Systems, Man and Cybernetics., vol. 24, no. 5, pp. 769-777, May. 1994