Green Resource Allocation Based on Deep Reinforcement Learning in Content-Centric IoT

IEEE Transactions on Emerging Topics in Computing

Volumn 8, Issue 3, 2020, Pages 781-796

  He, Xiaoming ^a   Wang, Kun ^a,d   Huang, Huawei ^b   Miyazaki, Toshiaki ^c   Wang, Yixuan ^a   Guo, Song ^b  

^a NANJING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS   (China)

^b HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

^c UNIVERSITY OF AIZU   (Japan)

^d SHANGHAI JIAO TONG UNIVERSITY   (China)

Author keywords

content centric computing; deep reinforcement learning; Green resource allocation; IoT; QoE

Indexed keywords

DEEP LEARNING; ELECTRONIC MAIL; GREEN COMPUTING; HEURISTIC ALGORITHMS; LEARNING ALGORITHMS; MACHINE LEARNING; QUALITY OF SERVICE; REINFORCEMENT LEARNING; RESOURCE ALLOCATION; WIRELESS NETWORKS;

COMPUTATIONAL MODEL; CONTENT CENTRIC; DYNAMIC APPROACHES; MEAN OPINION SCORES; QUALITY OF EXPERIENCE (QOE); RESOURCE ALLOCATION ALGORITHMS; RESOURCE ALLOCATION PROBLEM; RESOURCE MANAGEMENT;

INTERNET OF THINGS;

EID: 85042136773     PISSN: None     EISSN: 21686750     Source Type: Journal    
DOI: 10.1109/TETC.2018.2805718     Document Type: Article

References (59)

1. K. Wang, and Y. Yu, "A query-matching mechanism over out-of-order event stream in IoT, " Int. J. Ad Hoc Ubiquitous Comput. , vol. 13, no. 3/4, pp. 197-208, Jul. 2013.
2. Z. Su, and Q. Xu, "Content distribution over content centricmobile social networks in 5G, " IEEE Commun. Mag. , vol. 53, no. 6, pp. 66-72, Jun. 2015.
3. K. Wang, L. Gu, S. Guo, H. Chen, V. C. M. Leung, and Y. Sun, "Crowd-Sourcing-based content-centric network: A social perspective, " IEEE Netw. , vol. 31, no. 5, pp. 28-34, Sep. 2017.
4. L. Zhou, M. Chen, Y. Qian, and H. Chen, "Fairness resource allocation in blind wireless multimedia communications, " IEEE Trans. Multimedia, vol. 15, no. 4, pp. 946-956, Jun. 2013.
5. L. Qian, Z. Cheng, Z. Fang, L. Ding, F. Yang, and W. Huang, "AQoE-driven encoder adaptation scheme for multi-user video streaming in wireless networks, " IEEE Trans. Broadcast. , vol. 63, no. 1, pp. 20-31, Mar. 2017.
6. Z. Deng, Y. Liu, J. Liu, X. Zhou, and S. Ci, "QoE-oriented rate allocation for multipath high-definition video streaming over heterogeneous wireless access networks, " IEEE Syst. J. , vol. 11, no. 4, pp. 2524-2535, Dec. 2017.
7. K. Wang, X. Qi, L. Shu, D. Deng, and J. Rodrigues, "Toward trustworthy crowdsourcing in social internet of things, " IEEE Wireless Commun. Mag. , vol. 30, no. 5, pp. 30-36, Oct. 2016.
8. K. Eshraghian, K. Cho, O. Kavehei, S. Kang, D. Abbott, and S. Kang, "Memristor MOS content addressable memory (MCAM): Hybrid architecture for future high performance search engines, " IEEE Trans. Very Large Scale Integr. Syst. , vol. 19, no. 8, pp. 1407-1417, Aug. 2011.
9. X. Zhou, K. Wang, W. Jia, and M. Guo, "Reinforcement learning-based adaptive resource management of differentiated services in geo-distributed data centers, " in Proc. IEEE/ACM Int. Symp. Quality Ser. , 2017, pp. 1-6.
10. R. Mok, R. Chang, and W. Li, "Detecting low-quality workers in QoE crowdtesting: A worker behavior-based approach, " IEEE Trans. Multimedia, vol. 19, no. 3, pp. 530-543, Oct. 2017.
11. X. Li, X. Ge, X. Wang, and V. Leung, "Energy efficiency optimization: Joint antenna subcarrier power allocation in OFDM-DASs, " IEEE Trans. Wireless Commun. , vol. 15, no. 11, pp. 7470-7483, Nov. 2016.
12. Z. Ye, S. Mistry, A. Bouguettaya, and H. Dong, "Long-term QoS-aware cloud service composition using multivariate time series analysis, " IEEE Trans. Serv. Comput. , vol. 9, no. 3, pp. 382-393, May 2016.
13. K. Wang, Y. Wang, Y. Sun, S. Guo, and J. Wu, "Green industrial Internet of Things architecture: An energy-efficient perspective, " IEEE Commun. Mag. , vol. 54, no. 12, pp. 48-54, Dec. 2016.
14. Y. Wang, et al. , "A data-driven architecture for personalized QoE management in 5G wireless networks, " IEEE Wireless Commun. Mag. , vol. 24, no. 1, pp. 102-110, Feb. 2017.
15. Y. Lu, M. Motani, and W. Wong, "A QoE-aware resource distribution framework incentivizing context sharing and moderate competition, " IEEE/ACM Trans. Netw. , vol. 24, no. 3, pp. 1364-1377, Jun. 2016.
16. Y. Wang, Z. Li, G. Tyson, S. Uhlig, and G. Xi, "Design and evaluation of the optimal cache allocation for content-centric networking, " IEEE Trans. Comput. , vol. 65, no. 1, pp. 95-107, Jan. 2016.
17. K. Gai, M. Qiu, H. Zhao, L. Tao, and Z. Zong, "Dynamic energy-aware cloudlet-based mobile cloud computing model for green computing, " J. Netw. Comput. Appl. , vol. 59, pp. 46-54, Jan. 2016.
18. H. Zhu, Y. Cao, W. Wang, B. Liu, and T. Jiang, "QoE-aware resource allocation for adaptive device-to-device video streaming, " IEEE Netw. , vol. 29, no. 6, pp. 6-12, Dec. 2015.
19. G. Li, M. Dong, K. Ota, J. Wu, J. Li, and T. Ye, "Towards QoE named content-centric wireless multimedia sensor networks with mobile sinks, " in Proc. Int. Conf. Commun. , 2017, pp. 1-6.
20. X. Liu, et al. , "RFID Estimation with blocker tags, " IEEE/ACM Trans. Netw. , vol. 25, no. 1, pp. 224-237, Feb. 2017.
21. W. Rahman, D. Yun, and K. Chung, "A client side buffer management algorithm to improve QoE, " IEEE Trans. Consum. Electron. , vol. 62, no. 4, pp. 371-379, Feb. 2016.
22. H. Tran, S. Hoceini, A. Mellouk, J. Perez, and S. Zeadally, "QoE-based server selection for content distribution networks, " IEEE Trans. Comput. , vol. 63, no. 11, pp. 2803-2815, Feb. 2014.
23. L. Qian, Z. Chen, Z. Fang, L. Ding, F. Yang, and W. Huang, "AQoE-driven encoder adaptation scheme for multi-user video streaming in wireless networks, " IEEE Trans. Broadcast. , vol. 63, no. 1, pp. 20-31, Mar. 2017.
24. K. Gai, M. Qiu, and H. Zhao, "Energy-aware task assignment for mobile cyber-enabled applications in heterogeneous cloud computing, " J. Parallel Distrib. Comput. , vol. 11, pp. 126-135, Jan. 2018.
25. M. Afshang, H. Dhillon, and P. Chong, "Fundamentals of cluster-centric content placement in cache-enabled device-to-device networks, " IEEE Trans. Commun. , vol. 64, no. 6, pp. 2511-2526, Jun. 2016.
26. Y. Li, H. Xie, Y. Wen, C. Chow, and Z. Zhang, "How much to coordinate ? optimizing in-network caching in content-centric networks, " IEEE Trans. Netw. Serv. Manag. , vol. 12, no. 3, pp. 420-434, Sep. 2015.
27. K. Wang, J. Yu, Y. Yu, Y. Qian, D. Zeng, S. Guo, Y. Xiang, and J. Wu, "A survey on Energy Internet: Architecture, approach and emerging technologies, " IEEE Syst. J. , vol. PP, no. 99, pp. 1-14, Jan. 2017.
28. X. Liu, X. Xie, K. Wang, S. Guo, J. Cao, and K. Li, "Pinpointing anomaly RFID tags: Situation and opportunities, " IEEE Netw. , vol. 31, no. 6, pp. 40-47, Nov. 2017.
29. V. Pacifici and G. Dan, "Coordinated selfish distributed caching for peering content-centric networks, " IEEE/ACM Trans. Netw. , vol. 24, no. 6, pp. 3690-3701, Dec. 2016.
30. B. Zhou, Y. Cui, and M. Tao, "Stochastic content-centric multicast scheduling for cache-enabled heterogeneous cellular networks, " IEEE Trans. Wireless Commun. , vol. 15, no. 9, pp. 6284-6297, Sep. 2017.
31. Z. Zhou, K. Ota, M. Dong, and C. Xu, "Energy-efficient matching for resource allocation in D2D enabled cellular networks, " IEEE Trans. Veh. Technol. , vol. 66, no. 6, pp. 5256-5268, Jun. 2017.
32. Z. Zhou, M. Dong, K. Ota, G. Wang, and L. Yang, "Energy-efficient resource allocation for D2D communications underlaying cloud-RAN-based LTE-A networks, " IEEE Internet Things J. , vol. 3, no. 3, pp. 428-438, Jun. 2016.
33. M. Gao, K. Wang, and L. He, "Probabilistic model checking and scheduling implementation of energy router system in energy internet for green cities, " IEEE Trans. Ind. Informat. , vol. PP, no. 99, pp. 1-10, Jan. 2018.
34. P. Truong and B. Mathieu, "Economic incentives for deploying localaware ICN-based content delivery, " in Proc. 35th Annu. IEEE Int. Conf. Comput. Commun. , 2016, pp. 352-357.
35. X. Liu, et al. , "Top-k queries for categorized RFID systems, " IEEE/ACM Trans. Netw. , vol. 25, no. 5, pp. 2587-2600, Oct. 2017.
36. S. Nikolaou, R. Renesse, and N. Schiper, "Proactive cache placement on cooperative client caches for online social networks, " IEEE Trans. Parallel Distrib. Syst. , vol. 27, no. 4, pp. 1174-1186, Apr. 2016.
37. C. Xu, K. Wang, and M. Guo, "Intelligent resource management in blockchain based cloud data centers, " IEEE Cloud Comput. , vol. 4, no. 6, pp. 50-59, Dec. 2017.
38. D. Applegate, A. Archer, V. Gopalakrishnan, S. Lee, and K. Ramakrishnan, "Optimal content placement for a large-scale VoD system, " IEEE Trans. Netw. , vol. 24, no. 4, pp. 2114-2127, Aug. 2016.
39. J. Chakareski, "Joint source-channel rate allocation and client clustering for scalable multistream IPTV, " IEEE Trans. Image Process. , vol. 24, no. 8, pp. 2429-2439, Aug. 2015.
40. N. Kumar, S. Zeadally, and J. Rodrigues, "QoS-aware hierarchical web caching scheme for online video streaming applications in internet-based vehicular ad hoc networks, " IEEE Trans. Ind. Electron. , vol. 62, no. 12, pp. 7892-7900, Dec. 2015.
41. X. Liu, S. Zhang, B. Xiao, and K. Bu, "Flexible and time-efficient tag scanning with handheld readers, " IEEE Trans. Mobile Comput. , vol. 15, no. 4, pp. 840-854, Apr. 2016.
42. H. Shao, H. Zhao, Y. Sun, J. Zhang, and Y. Xu, "QoE-aware downlink user-cell association in small cell networks: A transfer-matching game theoretic solution with peer effects, " IEEE Access, vol. 4, pp. 10029-10041, Nov. 2016.
43. M. Venkataraman and M. Chatterjee, "Effects of internet path selection on video-QoE: Analysis and improvements, " IEEE Trans. Netw. , vol. 22, no. 3, pp. 689-702, Jun. 2014.
44. J. Song, F. Yang, Y. Zhou, S. Wan, and H. Wu, "QoE evaluation of multimedia services based on audiovisual quality and user interest, " IEEE Trans. Multimedia, vol. 18, no. 3, pp. 444-457, Mar. 2016.
45. X. Liu, S. Zhang, and K. Bu, "A locality-based range-free localization algorithm for anisotropic wireless sensor networks, " Telecommun. Syst. , vol. 62, no. 1, pp. 3-13, 2016.
46. P. Sermpezis, and T. Spyropoulos, "Effects of content popularity on the performance of content-centric opportunistic networking: An analytical approach and applications, " IEEE/ACM Trans. Netw. , vol. 24, no. 6, pp. 3354-3368, Dec. 2016.
47. K. Mitra, A. Zaslavsky, and C. Ahlund, "Context-aware QoE modelling, measurement, and prediction in mobile computing systems, " IEEE Trans. Mobile Comput. , vol. 14, no. 5, pp. 920-936, May 2015.
48. R. Vaze, "Online knapsack problem and budgeted truthful bipartite matching, " in Proc. IEEE Conf. Comput. Commun. , 2017, pp. 1-9.
49. O. Granmo, B. Oommen, S. Myrer, and M. Olsen, "Learning automatabased solutions to the nonlinear fractional knapsack problem with applications to optimal resource allocation, " IEEE Trans. Syst. , Man, and Cybern. , vol. 37, no. 1, pp. 166-175, Jan. 2007.
50. O. Cihan and M. Akar, "Fastest mixing reversible markov chains on graphs with degree proportional stationary distributions, " IEEE Trans. Autom. Control, vol. 60, no. 1, pp. 227-232, Jan. 2015.
51. L. Chen, H. Shen, and K. Sapra, "Distributed autonomous virtual resource management in datacenters using finite-markov decision process, " in Proc. ACM Symp. Cloud Comput. , Nov. 2014, pp. 1-13.
52. Z. Han, H. Tan, G. Chen, R. Wang, Y. Chen, and F. Lau, "Dynamic virtual machine management via approximate markov decision process, " in Proc. 35th Annu. IEEE Int. Conf. Comput. Commun. , 2016, pp. 1-9.
53. L. Xiao, Y. Li, G. Han, H. Dai, and H. Vincent, "A secure mobile crowdsensing game with deep reinforcement learning, " IEEE Trans. Inform. Forensics and Security, vol. 13, no. 1, pp. 35-47, Jan. 2018.
54. N. Liu, et al. , "A hierarchical framework of cloud resource allocation and power management using deep reinforcement learning, " in Proc. IEEE Int. Conf. Distrib. Comput. Syst. , 2017, pp. 372-382.
55. H. Hasselt, A. Guez, and D. Silver, "Deep reinforcement learning with double Q-learning, " in Proc. AAAI Conf. Artif. Intell. , 2016, pp. 2094-2100.
56. L. Ji, "Reinforcement learning for robots using neural networks, " PhD thesis, Carnegie Mellon University, Pittsburgh. , Jan. 1993.
57. H. Modares, I. Ranatunga, F. Lewis, and D. Popa, "Optimized assistive humanCrobot interaction using reinforcement learning, " IEEE Trans. Cybern. , vol. 46, no. 3, pp. 655-667, Mar. 2016.
58. Q. Meng, K. Wang, B. Liu, T. Miyazaki, and X. He, "QoE-based big data analysis with deep learning in pervasive edge environment, " in Proc. Int. Conf. Commun. , 2018.
59. Q. Wu, Z. Li, and G. Xie, "Codingcache: Multipath-aware CCN cache with network coding, " in Proc. 5th Annu. ACM Special Interest Group Data Commun. , 2013, pp. 41-42.