Online cost-efficient scheduling of deadline-constrained workloads on hybrid clouds

Future Generation Computer Systems

Volumn 29, Issue 4, 2013, Pages 973-985

  Van Den Bossche, Ruben ^a   Vanmechelen, Kurt ^a   Broeckhove, Jan ^a  

^a UNIVERSITY OF ANTWERP   (Belgium)

Author keywords

Cloud computing; Cost; Hybrid clouds; Runtime estimation errors; Scheduling; Simulation

Indexed keywords

APPLICATION REQUIREMENTS; BAG-OF-TASKS APPLICATIONS; BANDWIDTH CONSTRAINT; CLOUD MODELS; CLOUD PROVIDERS; COST FACTORS; COST SAVING; COST-EFFICIENT; DATA CONFIDENTIALITY; DATA REQUIREMENTS; DEADLINE CONSTRAINT; ECONOMIC ASPECTS; ESTIMATION ERRORS; NETWORK BANDWIDTH; NETWORK LATENCIES; PERFORMANCE METRICS; PRICE UNCERTAINTY; PRICING MODELS; RUNTIME ESTIMATES; SIMULATION; SUNK COST; TECHNICAL ASPECTS;

ALGORITHMS; CLOUD COMPUTING; COST ESTIMATING; COSTS; DATA TRANSFER; ERRORS; SCHEDULING;

COST BENEFIT ANALYSIS;

EID: 84872290086     PISSN: 0167739X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.future.2012.12.012     Document Type: Article

References (47)

1. K. Keahey, M. Tsugawa, A. Matsunaga, and J. Fortes Sky computing IEEE Internet Computing 13 2009 43 51
2. R. Buyya, C.S. Yeo, S. Venugopal, J. Broberg, and I. Brandic Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility Future Generation Computer Systems 25 2009 599 616
3. P. Mell, T. Grace, The NIST definition of cloud computing, Technical Report 800-145, National Institute of Standards and Technology (NIST), Information Technology Laboratory, Computer Security Division, 2011.
4. "Amazon Web Services", Amazon EC2 reserved instance marketplace - beta, 2012. http://aws.amazon.com/ec2/reserved-instances/marketplace/ (accessed 19.11.12).
5. B. Sotomayor, R.S. Montero, I.M. Llorente, and I. Foster Virtual infrastructure management in private and hybrid clouds IEEE Internet Computing 13 2009 14 22
6. M.D. Assunção, A. Costanzo, and R. Buyya A cost-benefit analysis of using cloud computing to extend the capacity of clusters Cluster Computing 13 2010 335 347 10.1007/s10586-010-0131-x
7. D. Petcu, G. Macariu, S. Panica, and C. Crciun Portable cloud applications - from theory to practice Future Generation Computer Systems 2012
8. A. Edmonds, T. Metsch, A. Papaspyrou, and A. Richardson Toward an open cloud standard IEEE Internet Computing 16 2012 15 25
9. The Apache Software Foundation, libcloud, a unified interface to the cloud, 2012. http://libcloud.apache.org/.
10. Apache Software Foundation, Deltacloud API, 2012. http://deltacloud. apache.org/.
11. Rightscale Inc., Rightscale multi-cloud engine, 2012. http://www.rightscale.com.
12. J. Tordsson, R.S. Montero, R. Moreno-Vozmediano, and I.M. Llorente Cloud brokering mechanisms for optimized placement of virtual machines across multiple providers Future Generation Computer Systems 28 2012 358 367
13. W. Li, J. Tordsson, E. Elmroth, Modeling for dynamic cloud scheduling via migration of virtual machines, in: IEEE Third International Conference on Cloud Computing Technology and Science, CloudCom 2011, pp. 163-171.
14. J.L. Lucas-Simarro, R. Moreno-Vozmediano, R.S. Montero, and I.M. Llorente Scheduling strategies for optimal service deployment across multiple clouds Future Generation Computer Systems 2012
15. D. Breitgand, A. Maraschini, J. Tordsson, Policy-driven service placement optimization in federated clouds, Technical Report, IBM Research Division, 2011.
16. J. Strebel, A. Stage, An economic decision model for business software application deployment on hybrid cloud environments, M. Schumann, L.M. Kolbe, M.H. Breitner, A. Frerichs, Multikonferenz Wirtschaftsinformatik 2010, Universitätsverlag Göttingen, Göttingen, 2010, pp. 195-206.
17. A. Andrzejak, D. Kondo, S. Yi, Decision model for cloud computing under sla constraints, in: 2010 IEEE International Symposium on Modeling, Analysis Simulation of Computer and Telecommunication Systems, MASCOTS, pp. 257-266.
18. S. Kailasam, N. Gnanasambandam, J. Dharanipragada, and N. Sharma Optimizing service level agreements for autonomic cloud bursting schedulers International Conference on Parallel Processing Workshops 2010 IEEE Computer Society Los Alamitos, CA, USA 285 294
19. U. Lampe, M. Siebenhaar, D. Schuller, R. Steinmetz, A cloud-oriented broker for cost-minimal software service distribution, in: W.Z. Rosa M. Badia (Eds.), Proceedings of the Second Optimising Cloud Services Workshop, OCS 2011.
20. B. Javadi, J. Abawajy, and R. Buyya Failure-aware resource provisioning for hybrid cloud infrastructure Journal of Parallel and Distributed Computing 72 2012 1318 1331
21. R. Van den Bossche, K. Vanmechelen, J. Broeckhove, Cost-optimal scheduling in hybrid IaaS clouds for deadline constrained workloads, in: IEEE 3rd International Conference on Cloud Computing, CLOUD, 2010, pp. 228-235.
22. R. Van den Bossche, K. Vanmechelen, and J. Broeckhove Cost-efficient scheduling heuristics for deadline constrained workloads on hybrid clouds Proceedings of the 3rd IEEE International Conference on Cloud Computing Technology and Science 2011 IEEE Computer Society 320 327
23. P. Brucker Scheduling Algorithms 2004 Springer-Verlag Berlin Heidelberg
24. S. Verboven, P. Hellinckx, F. Arickx, and J. Broeckhove Runtime prediction based grid scheduling of parameter sweep jobs Journal of Internet and Technology 11 2010 47 54
25. M.A. Iverson, F. Özgüner, and G.J. Follen Run-time statistical estimation of task execution times for heterogeneous distributed computing Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing 1996 IEEE Computer Society 263 270
26. M.A. Iverson, F. Özgüner, and L.C. Potter Statistical prediction of task execution times through analytic benchmarking for scheduling in a heterogeneous environment IEEE Transactions on Computers 48 1999 1374 1379 (Pubitemid 30553396)
27. F. Nadeem, M.M. Yousaf, R. Prodan, and T. Fahringer Soft benchmarks-based application performance prediction using a minimum training set Proceedings of the Second IEEE International Conference on e-Science and Grid Computing E-SCIENCE '06 2006 IEEE Computer Society Washington, DC, USA 71
28. T.D. Nguyen, R. Vaswani, and J. Zahorjan Parallel application characterization for multiprocessor scheduling policy design D.G. Feitelson, L. Rudolph, Job Scheduling Strategies for Parallel Processing Lecture Notes in Computer Science vol. 1162 1996 Springer-Verlag 105 118
29. A.B. Downey, A model for speedup of parallel programs, Technical Report UCB/CSD-97-933, EECS Department, University of California, Berkeley, 1997.
30. S.-H. Chiang, R.K. Mansharamani, and M.K. Vernon Use of application characteristics and limited preemption for run-to-completion parallel processor scheduling policies ACM SIGMETRICS Performance Evaluation Review 22 1 1994 33 44
31. L.W. Dowdy, On the partitioning of multi-processor systems, Technical Report 88-06, Vanderbilt University, 1988.
32. A.W. Mualem, and D.G. Feitelson Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling IEEE Transactions on Parallel and Distributed Systems 12 2001 529 543 (Pubitemid 32611060)
33. D. Tsafrir, Y. Etsion, and D.G. Feitelson Backfilling using system-generated predictions rather than user runtime estimates IEEE Transactions on Parallel and Distributed Systems 18 2007 789 803 (Pubitemid 46767742)
34. R. Prasad, C. Dovrolis, M. Murray, and K. Claffy Bandwidth estimation: metrics, measurement techniques, and tools IEEE Network 17 2003 27 35
35. A. Shriram, M. Murray, Y. Hyun, N. Brownlee, A. Broido, M. Fomenkov, and k. claffy Comparison of public end-to-end bandwidth estimation tools on high-speed links C. Dovrolis, Passive and Active Network Measurement Lecture Notes in Computer Science vol. 3431 2005 Springer Berlin, Heidelberg 306 320 (Pubitemid 41252755)
36. S.-J. Lee, P. Sharma, S. Banerjee, S. Basu, and R. Fonseca Measuring bandwidth between planetlab nodes C. Dovrolis, Passive and Active Network Measurement Lecture Notes in Computer Science vol. 3431 2005 Springer Berlin, Heidelberg 292 305 (Pubitemid 41252754)
37. N. Hu, and P. Steenkiste Evaluation and characterization of available bandwidth probing techniques IEEE Journal on Selected Areas in Communications 21 6 2003 879 894
38. D. England, and J.B. Weissman Costs and benefits of load sharing in the computational grid Proceedings of the 10th International Conference on Job Scheduling Strategies for Parallel Processing JSSPP'04 2005 Springer-Verlag Berlin, Heidelberg 160 175 (Pubitemid 41190426)
39. K. Ranganathan, and I. Foster Decoupling computation and data scheduling in distributed data-intensive applications Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing HPDC '02 2002 IEEE Computer Society Washington, DC, USA 352 358
40. M. Tang, B.-S. Lee, X. Tang, and C.-K. Yeo The impact of data replication on job scheduling performance in the data grid Future Generation Computer Systems 22 2006 254 268 (Pubitemid 41689807)
41. A. Chervenak, I. Foster, C. Kesselman, C. Salisbury, and S. Tuecke The data grid: towards an architecture for the distributed management and analysis of large scientific datasets Journal of Network and Computer Applications 23 2000 187 200
42. D. Feitelson, L. Rudolph, U. Schwiegelshohn, K. Sevcik, and P. Wong Theory and practice in parallel job scheduling D. Feitelson, L. Rudolph, Job Scheduling Strategies for Parallel Processing Lecture Notes in Computer Science vol. 1291 1997 Springer Berlin, Heidelberg 1 34 (Pubitemid 127119055)
43. D. Feitelson, L. Rudolph, and U. Schwiegelshohn Parallel job scheduling a status report D. Feitelson, L. Rudolph, U. Schwiegelshohn, Job Scheduling Strategies for Parallel Processing Lecture Notes in Computer Science vol. 3277 2005 Springer Berlin Heidelberg 1 16 (Pubitemid 41190418)
44. A. Iosup, O. Sonmez, S. Anoep, and D. Epema The performance of bags-of-tasks in large-scale distributed systems Proceedings of the 17th International Symposium on High Performance Distributed Computing HPDC '08 2008 ACM New York, NY, USA 97 108
45. A. AuYoung, A. Vahdat, and A. Snoeren Evaluating the impact of inaccurate information in utility-based scheduling Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis SC '09 2009 ACM New York, NY, USA 38:1 38:12
46. C. Bailey Lee, Y. Schwartzman, J. Hardy, and A. Snavely Are user runtime estimates inherently inaccurate? D. Feitelson, L. Rudolph, U. Schwiegelshohn, Job Scheduling Strategies for Parallel Processing Lecture Notes in Computer Science vol. 3277 2005 Springer Berlin, Heidelberg 253 263 (Pubitemid 41190431)
47. S.-H. Chiang, A. Arpaci-Dusseau, and M. Vernon The impact of more accurate requested runtimes on production job scheduling performance D. Feitelson, L. Rudolph, U. Schwiegelshohn, Job Scheduling Strategies for Parallel Processing Lecture Notes in Computer Science vol. 2537 2002 Springer Berlin, Heidelberg 103 127