Real-time task assignment approach leveraging reinforcement learning with evolution strategies for long-term latency minimization in fog computing

Sensors (Switzerland)

Volumn 18, Issue 9, 2018, Pages

  Mai, Long ^a   Dao, Nhu Ngoc ^b   Park, Minho ^a  

^a Soongsil University   (South Korea)

^b Chung Ang University   (South Korea)

Author keywords

Evolution strategies; Fog computing; Long term latency minimization; Real time task assignment; Reinforcement learning

Indexed keywords

FOG; INTERNET OF THINGS; LEARNING ALGORITHMS; REINFORCEMENT LEARNING;

COMMUNICATION RESOURCES; COMPUTING INFRASTRUCTURES; EVOLUTION STRATEGIES; INTERNET OF THING (IOT); LOW COMPUTATIONAL COMPLEXITY; REAL-TIME TASKS; REINFORCEMENT LEARNING APPROACH; SERVICES AND APPLICATIONS;

FOG COMPUTING;

EID: 85052605784     PISSN: 14248220     EISSN: None     Source Type: Journal    
DOI: 10.3390/s18092830     Document Type: Article

References (40)

1. Mouradian, C.; Naboulsi, D.; Yangui, S.; Glitho, R.H.; Morrow, M.J.; Polakos, P.A. A Comprehensive Survey on Fog Computing: State-of-the-art and Research Challenges. IEEE Commun. Surv. Tutor. 2017, 20, 416–464. [CrossRef]
2. Abbas, N.; Zhang, Y.; Taherkordi, A.; Skeie, T. Mobile edge computing: A survey. IEEE Internet Things J. 2018, 5, 450–465. [CrossRef]
3. Dao, N.N.; Vu, D.N.; Lee, Y.; Park, M.; Cho, S. MAEC-X: DDoS prevention leveraging multi-access edge computing. In Proceedings of the IEEE International Conference on Information Networking (ICOIN), Chiang Mai, Thailand, 10–12 January 2018; pp. 245–248.
4. Shih, Y.Y.; Chung, W.H.; Pang, A.C.; Chiu, T.C.; Wei, H.Y. Enabling low-latency applications in fog-radio access networks. IEEE Netw. 2017, 31, 52–58. [CrossRef]
5. Rahman, G.S.; Peng, M.; Zhang, K.; Chen, S. Radio Resource Allocation for Achieving Ultra-Low Latency in Fog Radio Access Networks. IEEE Access 2018, 6, 17442–17454. [CrossRef]
6. Dao, N.N.; Lee, Y.; Cho, S.; Kim, E.; Chung, K.S.; Keum, C. Multi-tier multi-access edge computing: The role for the fourth industrial revolution. In Proceedings of the IEEE International Conference on Information and Communication Technology Convergence (ICTC), Jeju, Korea, 18–20 October 2017; pp. 1280–1282.
7. Peng, M.; Yan, S.; Zhang, K.; Wang, C. Fog-computing-based radio access networks: Issues and challenges. IEEE Netw. 2016, 30, 46–53. [CrossRef]
8. Vu, D.N.; Dao, N.N.; Jang, Y.; Na, W.; Kwon, Y.B.; Kang, H.; Jung, J.J.; Cho, S. Joint energy and latency optimization for upstream IoT offloading services in fog radio access networks. Trans. Emerg. Telecommun. Technol. 2018, 29, e3497. [CrossRef]
9. Sutton, R.S.; Barto, A.G. Reinforcement Learning: An Introduction; MIT Press: Cambridge, UK, 1998; Volume 1.
10. He, Y.; Yu, F.R.; Zhao, N.; Leung, V.C.; Yin, H. Software-defined networks with mobile edge computing and caching for smart cities: A big data deep reinforcement learning approach. IEEE Commun. Mag. 2017, 55, 31–37. [CrossRef]
11. He, Y.; Zhao, N.; Yin, H. Integrated Networking, Caching, and Computing for Connected Vehicles: A Deep Reinforcement Learning Approach. IEEE Trans. Veh. Technol. 2018, 67, 44–55. [CrossRef]
12. Michalewicz, Z. Evolution strategies and other methods. In Genetic Algorithms + Data Structures = Evolution Programs; Springer: Berlin, Heidelberg: 1996; pp. 159–177.
13. Salimans, T.; Ho, J.; Chen, X.; Sutskever, I. Evolution strategies as a scalable alternative to reinforcement learning. arXiv 2017, arXiv:1703.03864.
14. Bishop, C.; Bishop, C.M. Neural Networks for Pattern Recognition; Oxford University Press: Oxford, UK, 1995.
15. Nasrabadi, N.M. Pattern recognition and machine learning. J. Electron. Imaging 2007, 16, 049901.
16. Mahmud, R.; Kotagiri, R.; Buyya, R. Fog computing: A taxonomy, survey and future directions. In Internet of Everything; Springer: Berlin, Heidelberg: 2018; pp. 103–130.
17. Mukherjee, M.; Shu, L.; Wang, D. Survey of Fog Computing: Fundamental, Network Applications, and Research Challenges. IEEE Commun. Surv. Tutor. 2018, 20, 1826–1857. [CrossRef]
18. Mach, P.; Becvar, Z. Mobile edge computing: A survey on architecture and computation offloading. IEEE Commun. Surv. Tutor. 2017, 19, 1628–1656. [CrossRef]
19. Markakis, E.K.; Karras, K.; Sideris, A.; Alexiou, G.; Pallis, E. Computing, Caching, and Communication at the Edge: The Cornerstone for Building a Versatile 5G Ecosystem. IEEE Commun. Mag. 2017, 55, 152–157. [CrossRef]
20. Kim, S. 5G Network Communication, Caching, and Computing Algorithms Based on the Two-Tier Game Model. ETRI J. 2018, 40, 61–71. [CrossRef]
21. Paščinski, U.; Trnkoczy, J.; Stankovski, V.; Cigale, M.; Gec, S. QoS-aware orchestration of network intensive software utilities within software defined data centres. J. Grid Comput. 2018, 16, 85–112. [CrossRef]
22. Hu, Y.; Wang, J.; Zhou, H.; Martin, P.; Taal, A.; de Laat, C.; Zhao, Z. Deadline-aware deployment for time critical applications in clouds. In Proceedings of the European Conference on Parallel Processing, Compostela, Spain, 28 August–1 September 2017; pp. 345–357.
23. Chamola, V.; Tham, C.K.; Chalapathi, G.S. Latency aware mobile task assignment and load balancing for edge cloudlets. In Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), Kona, HI, USA, 13–17 March 2017; pp. 587–592.
24. Dao, N.N.; Vu, D.N.; Lee, Y.; Cho, S.; Cho, C.; Kim, H. Pattern-Identified Online Task Scheduling in Multitier Edge Computing for Industrial IoT Services. Mob. Inf. Syst. 2018, 2018, 2101206. [CrossRef]
25. Ali, M.; Riaz, N.; Ashraf, M.I.; Qaisar, S.; Naeem, M. Joint Cloudlet Selection and Latency Minimization in Fog Networks. IEEE Trans. Ind. Inform. 2018, 99. [CrossRef]
26. Dao, N.N.; Lee, J.; Vu, D.N.; Paek, J.; Kim, J.; Cho, S.; Chung, K.S.; Keum, C. Adaptive resource balancing for serviceability maximization in fog radio access networks. IEEE Access 2017, 5, 14548–14559. [CrossRef]
27. Witten, I.H.; Frank, E.; Hall, M.A.; Pal, C.J. Data Mining: Practical Machine Learning Tools and Techniques; Morgan Kaufmann: Burlington, MA, USA, 2016.
28. Mnih, V.; Kavukcuoglu, K.; Silver, D.; Rusu, A.A.; Veness, J.; Bellemare, M.G.; Graves, A.; Riedmiller, M.; Fidjeland, A.K.; Ostrovski, G.; et al. Human-level control through deep reinforcement learning. Nature 2015, 518, 529. [CrossRef] [PubMed]
29. Van Hasselt, H.; Guez, A.; Silver, D. Deep Reinforcement Learning with Double Q-Learning. In Proceedings of the AAAI, Phoenix, AZ, USA, 12–17 February 2016; Volume 16, pp. 2094–2100.
30. Wang, Z.; Schaul, T.; Hessel, M.; Hasselt, H.; Lanctot, M.; Freitas, N. Dueling Network Architectures for Deep Reinforcement Learning. In Proceedings of the International Conference on Machine Learning, New York, NY, USA, 19–24 June 2016; pp. 1995–2003.
31. Hornik, K.; Stinchcombe, M.; White, H. Multilayer feedforward networks are universal approximators. Neural Netw. 1989, 2, 359–366. [CrossRef]
32. Silver, D.; Huang, A.; Maddison, C.J.; Guez, A.; Sifre, L.; Van Den Driessche, G.; Schrittwieser, J.; Antonoglou, I.; Panneershelvam, V.; Lanctot, M.; et al. Mastering the game of Go with deep neural networks and tree search. Nature 2016, 529, 484–489. [CrossRef] [PubMed]
33. LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature 2015, 521, 436. [CrossRef] [PubMed]
34. Goodfellow, I.; Bengio, Y.; Courville, A.; Bengio, Y. Deep Learning; MIT Press: Cambridge, MA, USA, 2016; Volume 1.
35. Such, F.P.; Madhavan, V.; Conti, E.; Lehman, J.; Stanley, K.O.; Clune, J. Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning. arXiv 2017, arXiv:1712.06567.
36. Mnih, V.; Kavukcuoglu, K.; Silver, D.; Graves, A.; Antonoglou, I.; Wierstra, D.; Riedmiller, M. Playing atari with deep reinforcement learning. arXiv 2013, arXiv:1312.5602.
37. Schwarzkopf, M.; Konwinski, A.; Abd-El-Malek, M.; Wilkes, J. Omega: Flexible, scalable schedulers for large compute clusters. In Proceedings of the 8th ACM European Conference on Computer Systems, Prague, Czech Republic, 14–17 April 2013; pp. 351–364.
38. Hindman, B.; Konwinski, A.; Zaharia, M.; Ghodsi, A.; Joseph, A.D.; Katz, R.H.; Shenker, S.; Stoica, I. Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center. In Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation, Boston, MA, USA, 30 March–1 April 2011; Volume 11, p. 22.
39. Bernstein, D. Containers and cloud: From lxc to docker to kubernetes. IEEE Cloud Comput. 2014, 1, 81–84. [CrossRef]
40. Vecchiola, C.; Chu, X.; Buyya, R. Aneka: A software platform for.NET-based cloud computing. High Speed Large Scale Sci. Comput. 2009, 18, 267–295.