A Hierarchical Framework of Cloud Resource Allocation and Power Management Using Deep Reinforcement Learning

Proceedings - International Conference on Distributed Computing Systems

Volumn , Issue , 2017, Pages 372-382

  Liu, Ning ^a   Li, Zhe ^a   Xu, Jielong ^a   Xu, Zhiyuan ^a   Lin, Sheng ^a   Qiu, Qinru ^a   Tang, Jian ^a   Wang, Yanzhi ^a  

^a Syracuse University   (United States)

Author keywords

Deep reinforcement learning; Distributed algorithm; Hierarchical framework; Resource allocation

Indexed keywords

CLOUD COMPUTING; COOLING SYSTEMS; DECISION MAKING; DEEP LEARNING; ECONOMIC AND SOCIAL EFFECTS; ELECTRIC POWER UTILIZATION; ENERGY MANAGEMENT; GREEN COMPUTING; HIERARCHICAL SYSTEMS; NETWORK FUNCTION VIRTUALIZATION; PARALLEL ALGORITHMS; POWER MANAGEMENT; REINFORCEMENT LEARNING; RESOURCE ALLOCATION;

DISTRIBUTED POWER MANAGEMENTS; DISTRIBUTED SERVERS; EFFECTIVE DYNAMICS; HIERARCHICAL FRAMEWORK; HIGH POWER CONSUMPTION; MANAGEMENT FRAMEWORKS; POWER-MANAGEMENT PROBLEM; RESOURCE ALLOCATION PROBLEM;

DISTRIBUTED COMPUTER SYSTEMS;

EID: 85027244897     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDCS.2017.123     Document Type: Conference Paper

References (37)

1. A. K. Mishra, J. L. Hellerstein, W. Cirne, and C. R. Das, "Towards characterizing cloud backend workloads: insights from google compute clusters, " ACM SIGMETRICS Performance Evaluation Review, vol. 37, no. 4, pp. 34-41, 2010.
2. A. Khan, X. Yan, S. Tao, and N. Anerousis, "Workload characterization and prediction in the cloud: A multiple time series approach, " in 2012 IEEE Network Operations and Management Symposium. IEEE, 2012, pp. 1287-1294.
3. R. S. Sutton and A. G. Barto, Reinforcement learning: An introduction. MIT press Cambridge, 1998, vol. 1.
4. X. Dutreilh, S. Kirgizov, O. Melekhova, J. Malenfant, N. Rivierre, and I. Truck, "Using reinforcement learning for autonomic resource allocation in clouds: towards a fully automated workflow, " in ICAS 2011, The Seventh International Conference on Autonomic and Autonomous Systems, 2011, pp. 67-74.
5. E. Barrett, E. Howley, and J. Duggan, "Applying reinforcement learning towards automating resource allocation and application scalability in the cloud, " Concurrency and Computation: Practice and Experience, vol. 25, no. 12, pp. 1656-1674, 2013.
6. A. Galstyan, K. Czajkowski, and K. Lerman, "Resource allocation in the grid using reinforcement learning, " in Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems-Volume 3. IEEE Computer Society, 2004, pp. 1314-1315.
7. G. Tesauro, R. Das, H. Chan, J. Kephart, D. Levine, F. Rawson, and C. Lefurgy, "Managing power consumption and performance of computing systems using reinforcement learning, " in Advances in Neural Information Processing Systems, 2007, pp. 1497-1504.
8. X. Lin, Y. Wang, and M. Pedram, "A reinforcement learning-based power management framework for green computing data centers, " in 2016 IEEE International Conference on Cloud Engineering (IC2E). IEEE, 2016, pp. 135-138.
9. L. Benini, A. Bogliolo, and G. De Micheli, "A survey of design techniques for system-level dynamic power management, " IEEE transactions on very large scale integration (VLSI) systems, vol. 8, no. 3, pp. 299-316, 2000.
10. G. Dhiman and T. S. Rosing, "Dynamic power management using machine learning, " in Proceedings of the 2006 IEEE/ACM international conference on Computer-aided design. ACM, 2006, pp. 747-754.
11. H. Jung and M. Pedram, "Dynamic power management under uncertain information, " in 2007 Design, Automation & Test in Europe Conference & Exhibition. IEEE, 2007, pp. 1-6.
12. D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. Van Den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot et al., "Mastering the game of go with deep neural networks and tree search, " Nature, vol. 529, no. 7587, pp. 484-489, 2016.
13. V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, "Playing atari with deep reinforcement learning, " arXiv preprint arXiv:1312. 5602, 2013.
14. Y. Bengio, "Learning deep architectures for ai, " Foundations and trends R in Machine Learning, vol. 2, no. 1, pp. 1-127, 2009.
15. S. Hochreiter and J. Schmidhuber, "Long short-term memory, " Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997.
16. C. Reiss, J. Wilkes, and J. L. Hellerstein, "Google cluster-usage traces: format + schema, " Google Inc., Mountain View, CA, USA, Technical Report, Nov. 2011, revised 2012. 03. 20. Posted at urlhttp:// code. google. com/p/googleclusterdata/wiki/TraceVersion2.
17. C. J. Watkins and P. Dayan, "Q-learning, " Machine learning, vol. 8, no. 3-4, pp. 279-292, 1992.
18. S. J. Duff and O. Bradtke Michael, "Reinforcement learning methods for continuous-time markov decision problems, " Adv Neural Inf Process Syst, vol. 7, p. 393, 1995.
19. Y. Wang, Q. Xie, A. Ammari, and M. Pedram, "Deriving a near-optimal power management policy using model-free reinforcement learning and Bayesian classification, " in Proceedings of the 48th Design Automation Conference. ACM, 2011, pp. 41-46.
20. R. S. Sutton, "Generalization in reinforcement learning: Successful examples using sparse coarse coding, " Advances in neural information processing systems, pp. 1038-1044, 1996.
21. X. Fan, W.-D. Weber, and L. A. Barroso, "Power provisioning for a warehouse-sized computer, " vol. 35, no. 2, pp. 13-23, 2007.
22. D. Meisner, B. T. Gold, and T. F. Wenisch, "Powernap: eliminating server idle power, " in ACM Sigplan Notices, vol. 44, no. 3. ACM, 2009, pp. 205-216.
23. J. T. Connor, R. D. Martin, and L. E. Atlas, "Recurrent neural networks and robust time series prediction, " IEEE transactions on neural networks, vol. 5, no. 2, pp. 240-254, 1994.
24. Y. Tan, W. Liu, and Q. Qiu, "Adaptive power management using reinforcement learning, " in Proceedings of the 2009 International Conference on Computer-Aided Design. ACM, 2009, pp. 461-467.
25. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al., "Human-level control through deep reinforcement learning, " Nature, vol. 518, no. 7540, pp. 529-533, 2015.
26. J. Rao, X. Bu, C.-Z. Xu, L. Wang, and G. Yin, "Vconf: a reinforcement learning approach to virtual machines auto-configuration, " in Proceedings of the 6th international conference on Autonomic computing. ACM, 2009, pp. 137-146.
27. D. Kingma and J. Ba, "Adam: A method for stochastic optimization, " arXiv preprint arXiv:1412. 6980, 2014.
28. D. Gross, Fundamentals of queueing theory. John Wiley & Sons, 2008.
29. Y. LeCun and Y. Bengio, "Convolutional networks for images, speech, and time series, " The handbook of brain theory and neural networks, vol. 3361, no. 10, p. 1995, 1995.
30. M. B. Srivastava, A. P. Chandrakasan, and R. W. Brodersen, "Predictive system shutdown and other architectural techniques for energy efficient programmable computation, " IEEE Trans. Very Large Scale Integr. Syst., vol. 4, no. 1, pp. 42-55, Mar. 1996. [Online]. Available: http://dx. doi. org/10. 1109/92. 486080
31. C.-H. Hwang and A. C. H. Wu, "A predictive system shutdown method for energy saving of event-driven computation, " in 1997 Proceedings of IEEE International Conference on Computer Aided Design (ICCAD), Nov 1997, pp. 28-32.
32. N. Kalchbrenner, I. Danihelka, and A. Graves, "Grid long short-term memory, " arXiv preprint arXiv:1507. 01526, 2015.
33. D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization, " CoRR, vol. abs/1412. 6980, 2014. [Online]. Available: http://arxiv. org/abs/1412. 6980
34. T. Simunic, L. Benini, P. Glynn, and G. D. Micheli, "Event-driven power management, " IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 20, no. 7, pp. 840-857, Jul 2001.
35. Q. Qiu, Y. Tan, and Q. Wu, "Stochastic modeling and optimization for robust power management in a partially observable system, " in Proceedings of the conference on Design, automation and test in Europe. EDA Consortium, 2007, pp. 779-784.
36. J. D. Little and S. C. Graves, "Little's law, " in Building intuition. Springer, 2008, pp. 81-100.
37. Z. Wang, N. de Freitas, and M. Lanctot, "Dueling network architectures for deep reinforcement learning, " arXiv preprint arXiv:1511. 06581, 2015.