Integrated networking, caching, and computing for connected vehicles: A deep reinforcement learning approach

IEEE Transactions on Vehicular Technology

Volumn 67, Issue 1, 2018, Pages 44-55

  He, Ying ^a   Zhao, Nan ^a   Yin, Hongxi ^a  

^a DALIAN UNIVERSITY OF TECHNOLOGY   (China)

Author keywords

Caching; Deep reinforcement learning; Edge computing; Vehicular networks

Indexed keywords

CLOUD COMPUTING; DEEP LEARNING; NETWORK FUNCTION VIRTUALIZATION; OPTIMIZATION; QUALITY OF SERVICE; REINFORCEMENT LEARNING; RESOURCE ALLOCATION;

CACHING; EDGE COMPUTING; LEARNING (ARTIFICIAL INTELLIGENCE); MOBILE COMMUNICATIONS; RESOURCE MANAGEMENT; VEHICULAR NETWORKS; VIDEOS;

DISTRIBUTED COMPUTER SYSTEMS;

EID: 85031782882     PISSN: 00189545     EISSN: None     Source Type: Journal    
DOI: 10.1109/TVT.2017.2760281     Document Type: Article

References (41)

1. C. Liang and F. R. Yu, "Wireless network virtualization: A survey, some research issues and challenges, " IEEE Commun. Surveys Tut., vol. 17, no. 1, pp. 358-380, Jan.-Mar. 2015.
2. L. Cui, F. R. Yu, and Q. Yan, "When big data meets software-defined networking (SDN): SDN for big data and big data for SDN, " IEEE Netw., vol. 30, no. 1, pp. 58-65, Jan./Feb. 2016.
3. Y.Cai, F. R.Yu, C. Liang, B. Sun, and Q.Yan, "Software defined device-Todevice (D2D) communications in virtual wireless networks with imperfect network state information (NSI), " IEEE Trans. Veh. Technol., vol. 65, no. 9, pp. 7349-7360, Sep. 2016.
4. K. Liu, J. K. Y. Ng, V. C. S. Lee, S. H. Son, and I. Stojmenovic, "Cooperative data scheduling in hybrid vehicular ad hoc networks: VANET as a software defined network, " IEEE/ACM Trans. Netw., vol. 24, no. 3, pp. 1759-1773, Jun. 2016.
5. Q. Zheng, K. Zheng, H. Zhang, and V. C. M. Leung, "Delay-optimal virtualized radio resource scheduling in software-defined vehicular networks via stochastic learning, " IEEE Trans. Veh. Technol., vol. 65, no. 4, pp. 7857-7867, Oct. 2016.
6. R. dos Reis Fontes, C. Campolo, C. E. Rothenberg, and A. Molinaro, "From theory to experimental evaluation: Resource management in software-defined vehicular networks, " IEEE Access, vol. 5, pp. 3069- 3076, Feb. 2017.
7. C. Fang, F. R. Yu, T. Huang, J. Liu, and Y. Liu, "A survey of green information-centric networking: Research issues and challenges, " IEEE Commun. Surveys Tut., vol. 17, no. 3, pp. 1455-1472, Jul.-Sep. 2015.
8. K. Liu, J. K. Y. Ng, J. Wang, V. C. S. Lee, W. Wu, and S. H. Son, "Network-coding-Assisted data dissemination via cooperative vehicle-Tovehicle/- infrastructure communications, " IEEE Trans. Intell. Transp. Syst., vol. 17, no. 6, pp. 1509-1520, Jun. 2016.
9. C. Liang, F. R. Yu, and X. Zhang, "Information-centric network function virtualization over 5G mobile wireless networks, " IEEE Netw., vol. 29, no. 3, pp. 68-74, May/Jun. 2015.
10. M. Amadeo, C. Campolo, and A. Molinaro, "Information-centric networking for connected vehicles: A survey and future perspectives, " IEEE Commun. Mag., vol. 54, no. 2, pp. 98-104, Feb. 2016.
11. M. Armbrust, A. Fox, R. Griffith, and A. D. Joseph, "A view of cloud computing, " Commun. ACM, vol. 53, no. 4, pp. 50-58, Apr. 2010.
12. H. Zhang, Q. Zhang, and X.Du, "Toward vehicle-Assisted cloud computing for smartphones, " IEEE Trans. Veh. Technol., vol. 64, no. 12, pp. 5610- 5618, Dec. 2015.
13. ETSI, "Mobile-edge computing-Introductory technical white paper, " ETSI White Paper, Sep. 2014.
14. F. R. Yu, C. Liang, Y. He, and N. Zhao, "Energy-efficient resource allocation in software-defined mobile networks with mobile edge computing and caching, " in Proc. IEEE Infocom Workshops, May 2017.
15. X. Hou, Y. Li, M. Chen, D. Wu, D. Jin, and S. Chen, "Vehicular fog computing: A viewpoint of vehicles as the infrastructures, " IEEE Trans. Veh. Technol., vol. 65, no. 6, pp. 3860-3873, Jun. 2016.
16. N. Kumar, S. Zeadally, and J. J. P. C. Rodrigues, "Vehicular delay-Tolerant networks for smart grid data management using mobile edge computing, " IEEE Commun. Mag., vol. 54, no. 10, pp. 60-66, Oct. 2016.
17. V. Mnih et al., "Human-level control through deep reinforcement learning, " Nature, vol. 518, no. 7540, pp. 529-533, 2015.
18. Y. He, F. R. Yu, N. Zhao, V. C. M. Leung, and H. Yin, "Software-defined networks with mobile edge computing and caching for smart cities: A big data deep reinforcement learning approach, " IEEE Commun. Mag., vol. 55, Dec. 2017.
19. Y. He et al., "Deep reinforcement learning-based optimization for cacheenabled opportunistic interference alignment wireless networks, " IEEE Trans. Veh. Technol., to be published, doi: 10.1109/TVT.2017.2751641.
20. C. Campoloa, A.Molinaroa, and R. Scopigno, "From today's vanets to tomorrow's planning and the bets for the day after, " Elsevier Veh. Commun., vol. 2, no. 3, pp. 158-171, 2015.
21. K. Abboud, H. A. Omar, and W. Zhuang, "Interworking of DSRC and cellular network technologies for V2X communications: A survey, " IEEE Trans. Veh. Technol., vol. 65, no. 12, pp. 9457-9470, Dec. 2016.
22. L. Ma, F. Yu, V. C. M. Leung, and T. Randhawa, "Anew method to support UMTS/WLAN vertical handover using SCTP, " IEEE Wireless Commun., vol. 11, no. 4, pp. 44-51, Aug. 2004.
23. F. Yu and V. Krishnamurthy, "Optimal joint session admission control in integrated WLAN and CDMA cellular networks with vertical handoff, " IEEE Trans. Mobile Comput., vol. 6, no. 1, pp. 126-139, Jan. 2007.
24. L. Zhang et al. "Named data networking, " ACM SIGCOMM Comput. Commun. Rev., vol. 44, no. 3, pp. 66-73, Jul. 2014.
25. G. Xylomenos et al., "A survey of information-centric networking research, " IEEE Commun. Surveys Tuts., vol. 16, no. 2, pp. 1024-1049, Apr.-Jun. 2014.
26. Y. He, F. R. Yu, N. Zhao, H. Yin, H. Yao, and R. C. Qiu, "Big data analytics in mobile cellular networks, " IEEE Access, vol. 4, pp. 1985- 1996, Mar. 2016.
27. Y. Wei, F. R. Yu, and M. Song, "Distributed optimal relay selection in wireless cooperative networks with finite-state Markov channels, " IEEE Trans. Veh. Technol., vol. 59, no. 5, pp. 2149 -2158, Jun. 2010.
28. H. Wang, F. R. Yu, L. Zhu, T. Tang, and B. Ning, "Finite-state Markov modeling for wireless channels in tunnel communication-based train control (CBTC) systems, " IEEE Trans. Intell. Transp. Syst., vol. 15, no. 3, pp. 1083-1090, Jun. 2014.
29. H. Gomaa, G. G. Messier, C. Williamson, and R. Davies, "Estimating instantaneous cache hit ratio using Markov chain analysis, " IEEE/ACM Trans. Netw., vol. 21, no. 5, pp. 1472-1483, Oct. 2013.
30. L. Breslau, P. Cao, L. Fan, G. Phillips, and S. Shenker, "Web caching and Zipf-like distributions: Evidence and implications, " in Proc. IEEE Annu. Joint Conf. IEEE Comput. Commun. Soc., pp. 126-134, vol. 1, 1999.
31. R. S. Sutton, "Introduction: The challenge of reinforcement learning, " in Reinforcement Learning. New York, NY, USA: Springer, 1992, pp. 1-3.
32. H. Y. Ong, K. Chavez, and A. Hong, "Distributed deep Q-learning, " arXiv:1508.04186, 2015.
33. Z. Wang, N. de Freitas, and M. Lanctot, "Dueling network architectures for deep reinforcement learning, " arXiv:1511.06581, 2015.
34. M. Hausknecht and P. Stone, "Deep reinforcement learning in parameterized action space, " arXiv:1511.04143, 2015.
35. T. P. Lillicrap et al., "Continuous control with deep reinforcement learning, " arXiv:1509.02971, 2015.
36. S. Gu, T. Lillicrap, I. Sutskever, and S. Levine, "Continuous deep Q-learning with model-based acceleration, " in Proc. 33th Int. Conf.Mach. Learn., vol. 48, pp. 2829-2838, 2016.
37. S. Levine, C. Finn, T. Darrell, and P. Abbeel, "End-To-end training of deep visuomotor policies, " J. Mach. Learn. Res., vol. 17, no. 39, pp. 1-40, 2016.
38. M. Deisenroth and C. E. Rasmussen, "Pilco: A model-based and dataefficient approach to policy search, " in Proc. 28th Int. Conf.Mach. Learn., 2011, pp. 465-472.
39. V. Mnih et al., "Playing Atari with deep reinforcement learning, " arXiv:1312.5602, 2013.
40. H. Van Hasselt, A. Guez, and D. Silver, "Deep reinforcement learning with double q-learning, " in Proc. 30th AAAI Conf. Artificial Intell., 2016, pp. 2094-2100.
41. M. Abadi et al., "Tensorflow: Large-scale machine learning on heterogeneous systems, " arXiv:1603.04467, Nov. 2016.