Applying reinforcement learning towards automating resource allocation and application scalability in the cloud

Concurrency and Computation: Practice and Experience

Volumn 25, Issue 12, 2013, Pages 1656-1674

  Barrett, Enda ^a   Howley, Enda ^a   Duggan, Jim ^a  

^a NATIONAL UNIVERSITY OF IRELAND   (Ireland)

Author keywords

cloud computing; reinforcement learning; resource scaling

Indexed keywords

BEHAVIORAL RESEARCH; CLOUD COMPUTING; DECISION MAKING; INFRASTRUCTURE AS A SERVICE (IAAS); INTELLIGENT COMPUTING; MARKOV PROCESSES; NETWORK SECURITY; REINFORCEMENT LEARNING; RESOURCE ALLOCATION; SCALABILITY; VIRTUAL MACHINE;

APPLICATION SCALABILITY; COMPUTATIONAL RESOURCES; CURSE OF DIMENSIONALITY; DECISION MAKING UNDER UNCERTAINTY; MARKOV DECISION PROCESSES; NON-STATIONARY ENVIRONMENT; REINFORCEMENT LEARNING TECHNIQUES; RESOURCE SCALING;

LEARNING ALGORITHMS;

EID: 84880333735     PISSN: 15320626     EISSN: 15320634     Source Type: Journal    
DOI: 10.1002/cpe.2864     Document Type: Article

References (34)

1. Buyya R, Yeo CS, Venugopal S, Broberg J, Brandic I,. Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility. Future Generation Computer Systems 2009; 25 (6): 599-616.
2. Barham P, Dragovic B, Fraser K, Hand S, Harris T, Ho A, Neugebauer R, Pratt I, Warfield A,. Xen and the art of virtualization. In Proceedings of the nineteenth acm symposium on operating systems principles, SOSP '03. ACM: New York, NY, USA, 2003; 164-177.
3. Vmware virtualisation software. (Available from: http://www.vmware.com/) [Accessed date:4 January 2012].
4. Koh Y, Knauerhase R, Brett P, Bowman M, Wen Z, Pu C,. An analysis of performance interference effects in virtual environments. In Proceedings of the IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), San Jose, California, USA, 2007.
5. Armbrust M, Fox A, Griffith R, Joseph AD, Katz RH, Konwinski A, Lee G, Patterson DA, Rabkin A, Stoica I, Zaharia M,. Above the clouds: A berkeley view of cloud computing. Technical Report UCB/EECS-2009-28, EECS Department, University of California, Berkeley, Feb 2009.
6. Padala P, Hou KY, Shin KG, Zhu X, Uysal M, Wang Z, Singhal S, Merchant A,. Automated control of multiple virtualized resources. In Proceedings of the 4th ACM European Conference on Computer Systems, EuroSys '09. ACM: New York, NY, USA, 2009; 13-26.
7. Padala P, Shin KG, Mustafa XZ, Wang UZ, Arif SS,. Adaptive control of virtualized resources in utility computing environments. In Proceedings of the European Conference on Computer Systems, Lisbon, Portugal, 2007; 289-302.
8. Tesauro G, Jong NK, Das R, Bennani MN,. On the use of hybrid reinforcement learning for autonomic resource allocation. Cluster Computing 2007; 10 (3): 287-299.
9. Iosup A, Ostermann S, Yigitbasi N, Prodan R, Fahringer T, Epema D,. Performance analysis of cloud computing services for many-tasks scientific computing. Transactions on Parallel and Distributed Systems June 2011; 22: 931-945.
10. Pu X, Liu L, Mei Y, Sivathanu S, Koh Y, Pu C,. Understanding performance interference of i/o workload in virtualized cloud environments. In Proceedings of the 2010 IEEE 3rd International Conference on Cloud Computing, CLOUD '10. IEEE Computer Society: Washington, DC, USA 2010; 51-58.
11. Amazon elastic compute cloud (amazon ec2). (Available from: http://aws.amazon.com/ec2/) [Accessed date: 4 January 2012].
12. Autoscale. (Available from: http://aws.amazon.com/autoscaling/) [Accessed date: 4 January 2012].
13. Cloudwatch. (Available from: http://aws.amazon.com/cloudwatch/) [Accessed date:4 January 2012].
14. Rightscale. (Available from:http://www.rightscale.com/) [Accessed date: 4 January 2012].
15. Rodero-Merino L, Vaquero LM, Gil V, Galán F, Fontán J, Montero RS, Llorente IM,. From infrastructure delivery to service management in clouds. Future Generation Computer Systems Oct. 2010; 26 (8): 1226-1240.
16. Morn D, Vaquero LM, Galán F,. Elastically ruling the cloud: Specifying application's behavior in federated clouds. In IEEE CLOUD, Liu L, Parashar M, (eds). IEEE: Washington DC, USA, July 2011; 89-96.
17. Dutreilh X, Rivierre N, Moreau A, Malenfant J, Truck I,. From data center resource allocation to control theory and back. Cloud Computing (CLOUD), 2010 IEEE 3rd International Conference on, july 2010; 410-417.
18. Rolia JA, Cherkasova L, McCarthy C,. Configuring workload manager control parameters for resource pools. In IEEE NOMS. Vancouver, Canada, 2006; 127-137.
19. Lim HC, Babu S, Chase JS,. Automated control for elastic storage. In Proceeding of the 7th International Conference on Autonomic Computing, ICAC '10. ACM: New York, NY, USA, 2010; 1-10.
20. Vengerov D,. A reinforcement learning approach to dynamic resource allocation. Engineering Applied Artifical Intelligence 2007; 20 (3): 383-390.
21. Perez J, Germain-Renaud C, Kégl B, Loomis C,. Grid differentiated services: A reinforcement learning approach. In CCGRID '08: Proceedings of the 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid. IEEE Computer Society: Washington, DC, USA, 2008; 287-294.
22. Foster I, Kesselman C, Tuecke S,. The anatomy of the grid: Enabling scalable virtual organizations. International Journal of High Performance Computing Applications 2001; 15 (3): 200-222.
23. Galstyan A, Czajkowski K, Lerman K,. Resource allocation in the grid using reinforcement learning. In AAMAS '04: Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems. IEEE Computer Society: Washington, DC, USA, 2004; 1314-1315.
24. Dutreilh X, Kirgizov S, Melekhova O, Malenfant J, Rivierre N, Truck I,. Using reinforcement learning for autonomic resource allocation in clouds: towards a fully automated workflow. In Seventh International Conference on Autonomic and Autonomous Systems, ICAS 2011, IEEE, May 2011; 67-74. MoVe INT LIP6.
25. Rao J, Bu X, Xu CZ, Wang L, Yin G,. Vconf: a reinforcement learning approach to virtual machines auto-configuration. In Proceedings of the 6th International Conference on Autonomic Computing, ICAC '09. ACM: New York, NY, USA, 2009; 137-146.
26. Chevaleyre Y, Dunne PE, Endriss U, Lang J, Lemaître M, Maudet N, Padget J, Phelps S, Rodríguez-aguilar JA, Sousa P,. Issues in multiagent resource allocation. Informatica 2006; 30: 2006.
27. Jacyno M, Bullock S, Luck M, Payne T,. Understanding decentralised control of resource allocation in a minimal multi-agent system. InThe 6th International Joint Conference on Autonomous Agents and Multi-Agent Systems, Honolulu, Hawai'i, 2007.
28. Scalas E, Gallegati M, Guerci E, Mas D, Tedeschi A,. Growth and allocation of resources in economics: The agent-based approach, arXiv.org, 2006.
29. Wolski R, Brevik J, Plank JS, Bryan T,. Grid resource allocation and control using computational economies. In Grid computing: Making the global infrastructure a reality. John Wiley & Sons: San Francisco, CA, 2003; 747-772.
30. Nau D, Ghallab M, Traverso P,. Automated Planning: Theory & Practice. Morgan Kaufmann Publishers Inc.: San Francisco, CA, USA, 2004.
31. Watkins C,. Learning from delayed rewards. Ph.D. Thesis, 1989.
32. Littman ML,. Algorithms for sequential decision making, 1996.
33. Kretchmar RM,. Parallel reinforcement learning. In the 6th World Conference on Systemics, Cybernetics, and Informatics, Orlando, USA, 2002.
34. Tesauro G,. Online resource allocation using decompositional reinforcement learning. Proc. AAAI-05, 2005; 9-13.