An adaptive RL based approach for dynamic resource provisioning in Cloud virtualized data centers

Computing

Volumn 97, Issue 12, 2015, Pages 1209-1234

  Bahrpeyma, Fouad ^a   Haghighi, Hassan ^a   Zakerolhosseini, Ali ^a  

^a SHAHID BEHESHTI UNIVERSITY   (Iran)

Author keywords

Adaptive control; Cloud computing; Dynamic resource provisioning; Inverse sequential neural fitted Q; Neural networks; Q learning

Indexed keywords

CLOUD COMPUTING; CONTROLLERS; DISTRIBUTED COMPUTER SYSTEMS; ENERGY UTILIZATION; NEURAL NETWORKS; REINFORCEMENT LEARNING;

ADAPTIVE CONTROL; CONTINUOUS REINFORCEMENT; DYNAMIC RESOURCE PROVISIONING; DYNAMIC RESOURCES; ENERGY CONSUMPTION OPTIMIZATION; INVERSE SEQUENTIAL NEURAL FITTED Q; Q-LEARNING; VIRTUALIZED DATA CENTERS;

ADAPTIVE CONTROL SYSTEMS;

EID: 84945437298     PISSN: 0010485X     EISSN: None     Source Type: Journal    
DOI: 10.1007/s00607-015-0455-8     Document Type: Article

References (33)

1. McKnight W (2014) Cloud computing: on-demand elasticity. Morgan Kaufmann, Burlington
2. Gupta P, Seetharaman A, Raj JR (2013) The usage and adoption of cloud computing by small and medium businesses. Int J Inf Manag 33:861–874
3. Maurer M, Brandic I, Sakellariou R (2013) Adaptive resource configuration for Cloud infrastructure management. Future Gener Comput Syst 29:472–487
4. Wang X, Dub Z, Chen Y (2012) An adaptive model-free resource and power management approach for multi-tier cloud environments. J Syst Softw 85:1135–1146
5. Zhang W, Liu J, Song Y, Zhu M, Xiao L, Sun Y et al (2011) Dynamic resource allocation based on user experience in virtualized servers. Adv Control Eng Inf Sci 15:3780–3784
6. Buyya R, Yeo CS, Venugopal S, Broberg J, Brandic I (2009) Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener Comput Syst 25:599–616
7. Huang Y, Lan Y, Thomson SJ, Fang A, Hoffmann WC, Lacey RE (2010) Development of soft computing and applications in agricultural and biological engineering. Comput Electron Agric 71:107–127
8. Bagherjeiran A, Eick CF, Chen CS, Vilalta R (2005) Adaptive clustering: obtaining better clusters using feedback and past experience. In Proceedings of the fifth IEEE international conference on data mining (ICDM-05)
9. Salehie M, Tahvildari L (2009) Self-adaptive software: Landscape and research challenges. In: ACM transactions on autonomous and adaptive systems (TAAS), vol 4
10. Taylor ME, Stone P (2009) Transfer learning for reinforcement learning domains: a survey. J Mach Learn Res 10:1633–1685
11. Zhang Q, Zhu Q, Boutaba R (2011) Dynamic resource allocation for spot markets in cloud computing environments. In: Fourth IEEE international conference on utility and cloud computing (UCC), pp 178–185
12. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, Cambridge
13. Wang X-S, Cheng Y-H, Yi J-Q (2007) A fuzzy actor-critic reinforcement learning network. Inf Sci 177:3764–3781
14. Bahrpeyma F, Zakerolhosseini A, Haghighi H (2015) Using IDS fitted Q to develop a real-time adaptive controller for dynamic resource provisioning in Cloud’s virtualized environment. Appl Soft Comput 26:285–298
15. Islam S, Keung J, Lee K, Liu A (2012) Empirical prediction models for adaptive resource provisioning in the cloud. Future Gener Comput Syst 28:155–162
16. Bahrpeyma F, Golchin B, Cranganu C (2013) Fast fuzzy modeling method to estimate missing logsin hydrocarbon reservoirs. J Petrol Sci Eng 112:310–321
17. Farahnakian F, Liljeberg P, Plosila J (2014) Energy-efficient virtual machines consolidation in cloud data centers using reinforcement learning. In: 22nd euromicro international conference on parallel, distributed and network-based processing (PDP), Torino
18. Yang B, Xu X, Tan F, Park DH (2011) An utility-based job scheduling algorithm for Cloud computing considering reliability factor. In: 2011 international conference on cloud and service computing (CSC), Hong Kong
19. Guo Y, Lama P, Jiang C, Zhou X (2014) Automated and agile server parametertuning by coordinated learning and control. IEEE Trans Parallel Distrib Syst 25:876–886
20. Bu X, Rao J, Xu C-Z (2013) Coordinated self-configuration of virtual machines and appliances using a model-free learning approach. In: IEEE transactions on parallel and distributed systems, vol 24, pp 681–690
21. Hussin M, Lee YC, Zomaya AY (2011) Efficient energy management using adaptive reinforcement learning-based scheduling in large-scale distributed systems. In: International conference on parallel processing (ICPP), pp 385–393
22. Barto AG, Sutton R (1996) Reinforcement learning: an introduction, adaptive computation and machine learning
23. Khan SG, Herrmann G, Lewis FL, Pipe T, Melhuish C (2012) Reinforcement learning and optimal adaptive control: an overview and implementation examples. Annu Rev Control 36:42–59
24. Lin YP, Li XY (2003) Reinforcement learning based on local state feature learning and policy adjustment. Inf Sci
25. Watkins C (1989) Learning from delayed rewards. PhD, University of Cambridge
26. Lin L, Xie H, Zhang D, Shen L (2010) Supervised neural $${\rm Q}\_$$Q_learning based motion control for bionic underwater robots. J Bionic Eng 7:177–184
27. Tesauro GJ (1992) Practical issues in temporal-difference learning. Mach Learn 8:257–277
28. Ilg W, Berns K, Miihlfriedel T, Dillmann R (1997) Hybrid learning concepts based on self-organizing neural networks for adaptive control of walking machines. Robot Autonom Syst 22:317–327
29. Gabel T, Lutz C, Riedmiller M (2011) Improved neural fitted Q iteration applied to a novel computer gaming and learning benchmark. In: 2011 IEEE symposium on adaptive dynamic programming and reinforcement learning (ADPRL), Paris
30. Watkins C, Dayan P (1992) Q-learning. Mach Learn 8:279–292
31. Rodrigues J, Nogueira A, Salvador P (2010) Improving the traffic prediction capability of neural networks using sliding window and multi-task learning mechanisms presented at the second international conference on evolving internet (INTERNET)
32. Levenberg K (1944) A method for the solution of certain non-linear problems in least squares. Q Appl Math 2:164–168
33. Zhang Q, Faten ZM, Zhang S, Zhu Q, Boutaba R, Hellerstein JL (2012) Dynamic energy-aware capacity provisioning for cloud computing environments. In: ICAC’12, California, USA, San Jose