Cost-aware DAG scheduling algorithms for minimizing execution cost on cloud resources

Journal of Supercomputing

Volumn 72, Issue 3, 2016, Pages 985-1012

  Convolbo, Moïse W ^a   Chou, Jerry ^a  

^a NATIONAL TSING HUA UNIVERSITY   (Taiwan)

Author keywords

Cloud computing; Cost optimization; DAG scheduling; Virtual machine

Indexed keywords

CLOUD COMPUTING; CLUSTERING ALGORITHMS; COSTS; DIRECTED GRAPHS; HEURISTIC ALGORITHMS; OPTIMIZATION; SCHEDULING; SCHEDULING ALGORITHMS; VIRTUAL MACHINE;

COMPLEX APPLICATIONS; COST OPTIMIZATION; DAG SCHEDULING; DIRECTED ACYCLIC GRAPH SCHEDULING; HEURISTIC SCHEDULING ALGORITHMS; MULTIPROCESSOR COMPUTERS; RESOURCE PROVISIONING; SCIENTIFIC APPLICATIONS;

JOB SHOP SCHEDULING;

EID: 84955299137     PISSN: 09208542     EISSN: 15730484     Source Type: Journal    
DOI: 10.1007/s11227-016-1637-7     Document Type: Article

References (37)

1. Adam TL, Chandy KM, Dickson JR (1974) A comparison of list schedules for parallel processing systems. Commun ACM 17(12):685–690
2. Ahmad I, Yu-Kwong K (1998) On exploiting task duplication in parallel program scheduling. IEEE Trans Parallel Distrib Syst 9(9):872–892
3. Almeida VAF, Vasconcelos IMM, Arabe J, Menasce D (1992) Using random task graphs to investigate the potential benefits of heterogeneity in parallel systems. In: ACM/IEEE conference on supercomputing, pp 683–691
4. Arabnejad H, Barbosa J (2014) List scheduling algorithm for heterogeneous systems by an optimistic cost table. IEEE Trans Parallel Distrib Syst 25(3):682–694
5. Armbrust M, Fox A, Griffith R, Joseph AD, Katz R, Konwinski A, Lee G, Patterson D, Rabkin A, Stoica I, Zaharia M (2010) A view of cloud computing. Commun ACM 53(4):50–58
6. Bajaj R, Agrawal D (2004) Improving scheduling of tasks in a heterogeneous environment. IEEE Trans Parallel Distrib Syst 15(2):107–118
7. Bittencourt L, Madeira E (2011) HCOC: a cost optimization algorithm for workflow scheduling in hybrid clouds. J Internet Serv Appl 2(3):207–227
8. Canon L, Jeannot E (2010) Evaluation and optimization of the robustness of dag schedules in heterogeneous environments. IEEE Trans Parallel Distrib Syst 21(4):532–546
9. Casavant TL, Kuhl JG (1988) A taxonomy of scheduling in general-purpose distributed computing systems. IEEE Trans Softw Eng 14(2):141–154
10. Chen G (2010) Simplified particle swarm optimization algorithm based on particles classification. Int Conf Nat Comput 5:2701–2705
11. Chen H, Shirazi B, Kavi K, Hurson A (1993) Static scheduling using linear clustering and task duplication. In: Parallel and distributed computing and systems, pp 285–290
12. Colin J, Chretienne P (1991) C.P.M. scheduling with small computation delays and task duplication. Oper Res 39:680–684
13. Cordeiro D, Mounié G, Perarnau S, Trystram D, Vincent J-M, Wagner F (2010) Random graph generation for scheduling simulations. In: Proceedings of the 3rd international ICST conference on simulation tools and techniques, pp 60:1–60:10
14. Darbha S, Agrawal DP (1998) Optimal scheduling algorithm for distributed-memory machines. IEEE Trans Parallel Distrib Syst 9(1):87–95
15. Dean J, Ghemawat S (2008) MapReduce: simplified data processing on large clusters. Commun ACM 51(1):107–113
16. Fujimoto N, Hagihara K (2003) Near-optimal dynamic task scheduling of precedence constrained coarse-grained tasks onto a computational grid. In: Proceedings of international conference on parallel and distributed computing, pp 80–87
17. Graham R, Lawler L, Lenstra J, Kan A (1979) Optimization and approximation in deterministic sequencing and scheduling: a survey. Ann Discrete Math 5:287–326
18. Hu T (1961) Parallel sequencing and assembly line problems. Oper Res 9(6):841–848
19. Juve G, Chervenak A, Deelman E, Bharathi S, Mehta G, Vahi K (2013) Characterizing and profiling scientific workflows. Future Gener Comput Syst 29(3):682–692
20. Kvas A, Ojstersek M, Zumer V (1994) Evaluation of static program allocation schemes for macro data-flow computer. In: Proceedings of the 20th EUROMICRO conference, pp 573–580
21. Kwok Y-K, Ahmad I (1996) Dynamic critical-path scheduling: an effective technique for allocating task graphs to multiprocessors. IEEE Trans Parallel Distrib Syst 7(5):506–521
22. Mao M, Humphrey M (2011) Auto-scaling to minimize cost and meet application deadlines in cloud workflows. ACM/IEEE Conf Supercomput 49(1–49):12
23. Olston C, Reed B, Srivastava U, Kumar R, Tomkins A (2008) Pig latin: a not-so-foreign language for data processing. In: ACM SIGMOD, pp 1099–1110
24. Radulescu A, van Gemund A (2000) Fast and effective task scheduling in heterogeneous systems. In: Proceedings of 9th heterogeneous computing workshop, 2000 (HCW 2000), pp 229–238
25. Raju R, Amudhavel J, Pavithra M, Anuja S, Abinaya B (2014) A heuristic fault tolerant “MapReduce framework for minimizing makespan in hybrid cloud environment. In: International conference on green computing communication and electrical engineering (ICGCCEE), pp 1–4
26. Rodriguez MA, Buyya R (2014) Deadline based resource provisioningand scheduling algorithm for scientific workflows on clouds. IEEE Trans Cloud Comput 2(2):222–235
27. Shirazi B, Wang M, Pathak G (1990) Analysis and evaluation of heuristic methods for static task scheduling. J Parallel Distrib Comput 10(3):222–2232
28. Thusoo A, Sarma JS, Jain N, Shao Z, Chakka P, Anthony S, Liu H, Wyckoff P, Murthy R (2009) Hive: a warehousing solution over a Map-Reduce framework. Proc VLDB Endow 2(2):1626–1629
29. Topcuouglu H, Hariri S, Wu M-Y (2002) Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans Parallel Distrib Syst 13(3):260–274
30. Wu M-Y, Gajski D (1990) Hypertool: a programming aid for message-passing systems. IEEE Trans Parallel Distrib Syst 1(3):330–343
31. Yang C-H, Lee P, Chung Y-C (2007) Improving static task scheduling in heterogeneous and homogeneous computing systems. In: ICPP, pp 45–45
32. Yang T, Gerasoulis A (1994) DSC: scheduling parallel tasks on an unbounded number of processors. IEEE Trans Parallel Distrib Syst 5(9):951–967
33. Yu WH (1984) Lu decomposition on a multiprocessing system with communications delay, Ph.D. thesis. University of California, Berkeley. AAI8427141
34. Yu Y, Isard M, Fetterly D, Budiu M, Erlingsson U, Gunda PK, Currey J (2008) DryadLINQ: a system for general-purpose distributed data-parallel computing using a high-level language. In: Proceedings of the USENIX conference on operating systems design and implementation, pp 1–14
35. Zaharia M, Chowdhury M, Franklin MJ, Shenker S, Stoica I (2010) Spark: cluster computing with working sets. In: Proceedings of the 2Nd USENIX conference on hot topics in cloud computing, HotCloud’10. USENIX Association, Berkeley, p 10
36. Zhang C, De Sterck H (2010) Cloudbatch: a batch job queuing system on clouds with hadoop and hbase. In: IEEE international conference on cloud computing technology and science (CloudCom), pp 368–375
37. Zhou AC, He B (2014) Transformation-based monetary cost optimizations for workflows in the cloud. IEEE Trans Cloud Comput 2(1):85–98