Cost-effective cloud HPC resource provisioning by building semi-elastic virtual clusters

International Conference for High Performance Computing, Networking, Storage and Analysis, SC

Volumn , Issue , 2013, Pages

  Niu, Shuangcheng ^a   Zhai, Jidong ^a   Ma, Xiaosong ^b,c   Tang, Xiongchao ^a   Chen, Wenguang ^a  

^a TSINGHUA UNIVERSITY   (China)

^b North Carolina State University   (United States)

^c OAK RIDGE NATIONAL LABORATORY   (United States)

Author keywords

Cloud computing; Job scheduling; Resource provisioning

Indexed keywords

CLOUD COMPUTING; COMPUTER SIMULATION; COSTS; SCHEDULING; SCHEDULING ALGORITHMS;

BATCH-SCHEDULING; CLOUD ENVIRONMENTS; JOB SCHEDULING; ON-DEMAND RESOURCE PROVISIONING; PARALLEL SIMULATIONS; RESOURCE PROVISIONING; TRACE DRIVEN SIMULATION; VIRTUAL CLUSTERS;

CLUSTER COMPUTING;

EID: 84899702246     PISSN: 21674329     EISSN: 21674337     Source Type: Conference Proceeding    
DOI: 10.1145/2503210.2503236     Document Type: Conference Paper

References (38)

1. Parallel Workloads Archive. http: //www. cs. huji. ac. il/labs/parallel/workload/, 2012.
2. SLURM: A Highly Scalable Resource Manager. https://computing. llnl. gov/linux/slurm/, 2012.
3. StarCluster. http://web. mit. edu/star/cluster/, 2012.
4. C. Alex and E. Mark. An In-Depth Look Inside the Twitter World. http://blog. rjmetrics. com/ new-data-on-twitters-users-and-engagement/, 2009.
5. A. Andrzejak, D. Kondo, and S. Yi. Decision model for cloud computing under sla constraints. In Modeling, Analysis & Simulation of Computer and Telecommunication Systems (MASCOTS), pages 257-266. IEEE, 2010.
6. D. Ardagna, S. Casolari, and B. Panicucci. Flexible distributed capacity allocation and load redirect algorithms for cloud systems. In Cloud Computing (CLOUD), pages 163-170. IEEE, 2011.
7. B. Billah, M. L. King, R. D. Snyder, and A. B. Koehler. Exponential smoothing model selection for forecasting. International Journal of Forecasting, 22(2):239-247, 2006.
8. F. Doelitzscher, M. Held, C. Reich, and A. Sulistio. Viteraas: Virtual cluster as a service. In Cloud Computing Technology and Science (CloudCom). IEEE, 2011.
9. T. V. T. Duy, Y. Sato, and Y. Inoguchi. Performance evaluation of a green scheduling algorithm for energy savings in cloud computing. In Parallel & Distributed Processing, Workshops and Phd Forum (IPDPSW), pages 1-8. IEEE, 2010.
10. R. Figueiredo, P. Dinda, and J. Fortes. A case for grid computing on virtual machines. In Distributed Computing Systems. IEEE, 2003.
11. E. S. Gardner Jr. Exponential smoothing: The state of the art. Journal of Forecasting, 4(1):1-28, 1985.
12. S. Genaud and J. Gossa. Cost-wait trade-os in client-side resource provisioning with elastic clouds. In Cloud Computing (CLOUD). IEEE, 2011.
13. A. Inc. Amazon EC2 Reserved Instance Marketplace. http://aws. amazon. com/ec2/reserved-instances/ marketplace/, 2012.
14. A. Inc. Amazon EC2 Spot Instances. http://aws. amazon. com/ec2/spot-instances/, 2012.
15. A. C. E. Inc. MOAB workload manager. http://www. supercluster. org/moab/, 2012.
16. A. C. E. Inc. PBS/Torque user manual. http://www. clusterresources. com/torquedocs21/ usersmanual. shtml, 2012.
17. G. Inc. Google Trends. http://www. google. com/trends/, 2013.
18. P. C. Inc. Platform LSF. http://www. platform. com/products/LSFfamily/, 2012.
19. D. Jackson, Q. Snell, and M. Clement. Core algorithms of the Maui scheduler. In Job Scheduling Strategies for Parallel Processing. Springer, 2001.
20. B. Javadi, R. Thulasiramy, and R. Buyya. Statistical modeling of spot instance prices in public cloud environments. In Utility and Cloud Computing (UCC). IEEE, 2011.
21. E. Kalyvianaki, T. Charalambous, and S. Hand. Self-adaptive and self-congured cpu resource provisioning for virtualized servers using kalman lters. In Proceedings of the 6th international conference on Autonomic computing, pages 117-126. ACM, 2009.
22. I. Krsul, A. Ganguly, J. Zhang, J. Fortes, and R. Figueiredo. Vmplants: Providing and managing virtual machine execution environments for grid computing. In Proceedings of the ACM/IEEE SC04 Conference, 2004.
23. J. Lucas-Simarro, R. Moreno-Vozmediano, R. Montero, and I. Llorente. Scheduling strategies for optimal service deployment across multiple clouds. Future Generation Computer Systems, 2012.
24. S. Majumdar, D. Eager, and R. Bunt. Scheduling in multiprogrammed parallel systems, volume 16. ACM, 1988.
25. S. Makridakis, A. Andersen, R. Carbone, R. Fildes, M. Hibon, R. Lewandowski, J. Newton, E. Parzen, and R. Winkler. The accuracy of extrapolation (time series) methods: results of a forecasting competition. Journal of forecasting, 1(2):111-153, 1982.
26. S. Makridakis and M. Hibon. The m3-competition: results, conclusions and implications. International Journal of Forecasting, 16(4):451-476, 2000.
27. A. Matsunaga, M. Tsugawa, S. Adabala, R. Figueiredo, H. Lam, and J. Fortes. Science gateways made easy: The in-vigo approach. Concurrency and Computation: Practice and Experience, 19(6), 2007.
28. C. McCann, R. Vaswani, and J. Zahorjan. A dynamic processor allocation policy for iviukiprogrammed shared-memory multiprocessors. ACM Transactions on Computer Systems, 11(2), 1993.
29. I. Moschakis and H. Karatza. Evaluation of gang scheduling performance and cost in a cloud computing system. The Journal of Supercomputing, 2011.
30. J. Skovira, W. Chan, H. Zhou, and D. Lifka. The EASY-LoadLeveler API project. In Job Scheduling Strategies for Parallel Processing. Springer, 1996.
31. J. Stein. New Data on Twitter's Users and Engagement. http://blog. rjmetrics. com/ new-data-on-twitters-users-and-engagement/, 2010.
32. D. Trigg and A. Leach. Exponential smoothing with an adaptive response rate. OR, pages 53-59, 1967.
33. W. Voorsluys, S. Garg, and R. Buyya. Provisioning spot market cloud resources to create cost-e ective virtual clusters. Algorithms and Architectures for Parallel Processing, 2011.
34. E. Walker. Benchmarking Amazon EC2 for High-Performance Scientic Computing. Login, 33(5), 2008.
35. S. Yi, D. Kondo, and A. Andrzejak. Reducing costs of spot instances via checkpointing in the amazon elastic compute cloud. In Cloud Computing (CLOUD). IEEE, 2010.
36. Y. Zhai, M. Liu, J. Zhai, X. Ma, and W. Chen. Cloud versus in-house cluster: evaluating amazon cluster compute instances for running mpi applications. In State of the Practice Reports, page 11. ACM, 2011.
37. H. Zhao, M. Pan, X. Liu, X. Li, and Y. Fang. Optimal resource rental planning for elastic applications in cloud market. In Parallel & Distributed Processing Symposium (IPDPS), pages 808-819. IEEE, 2012.
38. T. Zheng, J. Yang, M. Woodside, M. Litoiu, and G. Iszlai. Tracking time-varying parameters in software systems with extended kalman lters. In Proceedings of the 2005 conference of the Centre for Advanced Studies on Collaborative research, pages 334-345. IBM Press, 2005.