URL: A unified reinforcement learning approach for autonomic cloud management

Journal of Parallel and Distributed Computing

Volumn 72, Issue 2, 2012, Pages 95-105

  Xu, Cheng Zhong ^a   Rao, Jia ^a   Bu, Xiangping ^a  

^a Wayne State University   (United States)

Author keywords

Cloud computing; Reinforcement learning; Virtual machine autoconfiguration

Indexed keywords

AUTO-CONFIGURATION; CHANGING WORKLOAD; CLOUD DYNAMICS; CONFIGURATION PROCESS; PARAMETER SETTING; PERFORMANCE DATA; REINFORCEMENT LEARNING APPROACH; RESOURCE BUDGET; RUNNING-IN; SERVICE AVAILABILITY; SERVICE DEMAND; SERVICE ORIENTED COMPUTING; SERVICE QUALITY ASSURANCES; SLA OPTIMIZATION; VIRTUAL MACHINES;

COMPUTER SIMULATION; OPTIMIZATION; QUALITY ASSURANCE; QUALITY OF SERVICE; REINFORCEMENT; REINFORCEMENT LEARNING;

CLOUD COMPUTING;

EID: 84855359756     PISSN: 07437315     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jpdc.2011.10.003     Document Type: Article

References (39)

1. T.F. Abdelzaher, K.G. Shin, and N. Bhatti Performance guarantees for web server end-systems: a control-theoretical approach IEEE Transactions on Parallel and Distributed Systems 13 1 2002 80 96 (Pubitemid 35037242)
2. C.G. Atkeson, J.C. Santamar'ia, A comparison of direct and model-based reinforcement learning. in: In International Conference on Robotics and Automation, 1997.
3. A. Bar-Hillel, A. Di-Nur, L. Ein-Dor, R. Gilad-Bachrach, Y. Ittach, Workstation capacity tuning using reinforcement learning. in: SC, 2007.
4. T. Bittman, The future of infrastructure and operations: the engine of cloud computing, in: Gartner 27th Annual Data Center Conference, December 2008.
5. X. Bu, J. Rao, C.-Z. Xu, A reinforcement learning approach to online web system autoconfiguration, in: Proc. of Int. Conf. on Distributed Computing Systems (ICDCS), 2009.
6. X. Bu, J. Rao, C.-Z. Xu, A model-free learning approach for coordinated configuration of virtual machines and appliances, in: Proc. of MASCOTS, 2011.
7. I.-H. Chung, J.K. Hollingsworth, Automated cluster-based web service performance tuning, in: HPDC, pp. 3644, 2004.
8. C. Clark, K. Fraser, S. Hand, J.G. Hansen, E. Jul, C. Limpach, I. Pratt, A. Warfield, Live migration of virtual machines, in: NSDI, 2005.
9. Y. Diao, J. Hellerstein, S. Parekh, H. Shaikh, M. Surendra, Controlling quality of service in multi-tier web applications, in: Proc. of ICDCS, 2006.
10. D. Gupta, L. Cherkasova, R. Gardner, A. Vahdat, Enforcing performance isolation across virtual machines in xen, in: Middleware, 2006.
11. M.C. Huebscher, and J.A. Mccann A survey of autonomic computing: degrees, models, and applications ACM Computing Surveys 40 3 2008
12. IDC. Virtualization and multicore innovations disrupt the worldwide server market. document number: 206035, 2007.
13. E. Ipek, O. Mutlu, J.F. Martinez, R. Caruana, Self-optimizing memory controllers: a reinforcement learning approach, in: ISCA, 2008.
14. L.P. Kaelbling, M.L. Littman, and A.W. Moore Reinforcement learning: a survey Journal of Artificial Intelligence Research 4 1996 237 285 (Pubitemid 126646155)
15. A. Kamra, V. Misra, E. Nahum, Yaksha: a self-tuning controller for managing the performance of 3-tiered websites, in: Proc. of IWQoS'04, pp. 4756, 2004.
16. J.O. Kephart, and D.M. Chess The vision of autonomic computing IEEE Computer 2003
17. E. Kwan, S. Lightstone, A. Storm, A. Wu, Automatic configuratoin of ibm db2 universal database, 2002.
18. X. Liu, L. Sha, Y. Diao, S. Froehlich, J.L. Hellerstein, S.S. Parekh, Online response time optimization of apache web server, in: IWQoS, pp. 461478, 2003.
19. C. Lu, Y. Lu, T. Abdelzaher, J. Stankovic, and S. Son Feedback control architecture and design methodology for service delay guarantees in web servers IEEE Transactions on Parallel and Distributed Systems 17 9 2006 1014 1027 (Pubitemid 44263829)
20. D. Ongaro, A.L. Cox, S. Rixner, Scheduling i/o in virtual machine monitors, in: Poc. of ACM/Usenix Int. Conf. on Virtual Execution Environment (VEE), 2008.
21. P. Padala, K.G. Shin, X. Zhu, M. Uysal, Z. Wang, S. Singhal, A. Merchant, K. Salem, Adaptive control of virtualized resources in utility computing environments, in: EuroSys, 2007.
22. J. Rao, X. Bu, C.-Z. Xu, K. Wang, A distributed self-learning approach for elastic provisioning of virtualized cloud resources, in: Proc. of MASCOTS, 2011.
23. J. Rao, X. Bu, C.-Z. Xu, L.Y. Wang, G. Yin, Vconf: a reinforcement learning approach for virtual machine autoconfiguration, in: Proc. of Int. Conf. on Autonomic Computing (ICAC), 2009.
24. J. Rao, C.-Z. Xu, Online measurement of the capacity of multi-tier websites using hardware counters, in: Proc. of ICDCS, 2008. (An extended version is to appear in IEEE Trans. on Parallel and Distributed Systems).
25. L. Roderick, E. Zamost, J. Anderson, Vmmark: a scalable benchmark for virtualized systems. Technical Report VMware-TR-2006-002, VMware, Inc., 2006. www.vmware.com/products/vmmark.
26. A.A. Soror, U.F. Minhas, A. Aboulnaga, K. Salem, P. Kokosielis, S. Kamath, Automatic virtual machine configuration for database workloads, in: SIGMOD Conference, 2008.
27. R.S. Sutton, and A.G. Barto Reinforcement Learning: An Introduction 1998 MIT Press
28. G. Tesauro, Online resource allocation using decompositional reinforcement learning, in: AAAI, 2005.
29. G. Tesauro, R. Das, H. Chan, J. Kephart, D. Levine, F. Rawson, and C. Lefurgy Managing power consumption and performance of computing systems using reinforcement learning Advances in Neural Information Processing Systems 20 2007
30. G. Tesauro, N.K. Jong, R. Das, and M.N. Bennani On the use of hybrid reinforcement learning for autonomic resource allocation Cluster Computing 2007
31. J. Wei, and C.-Z. Xu eQoS: provisioning of client-perceived end-to-end QoS guarantees in web servers IEEE Transactions on Computers 55 12 2006 1543 1556 (Pubitemid 46432391)
32. J. Wei, X. Zhou, and C.-Z. Xu Robust processing rate allocation for proportional slowdown differentiation on Internet servers IEEE Transactions on Computers 54 8 2005 964 977 (Pubitemid 41235934)
33. A. Whitaker, R.S. Cox, S.D. Gribble, Configuration debugging as search: finding the needle in the haystack, in: OSDI, 2004.
34. B. Xi, Z. Liu, M. Raghavachari, C.H. Xia, L. Zhang, A smart hill-climbing algorithm for application server configuration, in: WWW, pp. 287296, 2004.
35. C.-Z. Xu, J. Wei, and F. Liu Model predictive feedback control for end-to-end qos guarantees in web servers IEEE Computer 41 3 2008 66 72
36. G. Yin, C.-Z. Xu, and L. Wang Q-learning algorithms with random truncation bounds and applications to effective parallel computing Journal of Optimization and Applications 137 2 2008 435 452
37. Y. Zhang, W. Qu, A. Liu, Automatic performance tuning for j2ee application server systems, in: WISE, pp. 520527, 2005.
38. W. Zheng, R. Bianchini, T.D. Nguyen, Automatic configuration of internet services, in: EuroSys, pp. 219229, 2007.
39. X. Zhou, J. Wei, and C.-Z. Xu Resource allocation for session-based 2d service differentiation on e-commerce servers IEEE Transactions on Parallel and Distributed Systems 17 8 2006 838 850 (Pubitemid 44070152)