Cost-efficient task scheduling for executing large programs in the cloud

Parallel Computing

Volumn 39, Issue 4-5, 2013, Pages 177-188

  Su, Sen ^a   Li, Jian ^a   Huang, Qingjia ^a   Huang, Xiao ^a   Shuang, Kai ^a   Wang, Jie ^b  

^a BEIJING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS   (China)

^b University of Massachusetts Lowell   (United States)

Author keywords

Cloud computing; Cost efficient scheduling; Directed acyclic graph; Parallel task scheduling

Indexed keywords

CLOUD COMPUTING; COST REDUCTION; COSTS; DIRECTED GRAPHS; ECONOMICS; GRAPH THEORY; SCHEDULING ALGORITHMS;

COST-EFFICIENT; DIRECTED ACYCLIC GRAPH (DAG); MINIMIZING MAKESPAN; MULTIPLE SEQUENCES; NUMERICAL EXPERIMENTS; PARALLEL TASK; PRECEDENCE CONSTRAINTS; TASK-SCHEDULING ALGORITHMS;

MULTITASKING;

EID: 84887451527     PISSN: 01678191     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.parco.2013.03.002     Document Type: Article

References (36)

1. G. Malewicz, M. Austern, A. Bik, J. Dehnert, I. Horn, N. Leiser, G. Czajkowski, Pregel: a system for large-scale graph processing, in: Proceedings of the 2010 International Conference on Management of Data, ACM, 2010, pp. 135-146.
2. D. Warneke, O. Kao, Nephele: efficient parallel data processing in the cloud, in: Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers, ACM, 2009, p. 8.
3. M. Armbrust, A. Fox, R. Griffith, A. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, et al., Above the clouds: a berkeley view of cloud computing, Tech. rep., Technical Report UCB/EECS-2009-28, EECS Department, University of California, Berkeley, 2009.
4. R. Buyya, C. Yeo, S. Venugopal, J. Broberg, I. Brandic, Cloud computing and emerging it platforms: Vision hype and reality for delivering computing as the 5th utility, Future Generation computer systems 25 (6) (2009) 599-616.
5. M. Isard, M. Budiu, Y. Yu, A. Birrell, D. Fetterly, Dryad: distributed data-parallel programs from sequential building blocks, ACM SIGOPS Operating Systems Review 41 (3) (2007) 59-72. (Pubitemid 47281575)
6. J. Ullman, Np-complete scheduling problems, Journal of Computer and System Sciences 10 (3) (1975) 384-393.
7. E. Hou, N. Ansari, H. Ren, A genetic algorithm for multiprocessor scheduling, IEEE Transactions on Parallel and Distributed Systems 5 (2) (1994) 113-120.
8. Y. Kwok, I. Ahmad, Dynamic critical-path scheduling: an effective technique for allocating task graphs to multiprocessors, IEEE Transactions on Parallel and Distributed Systems 7 (5) (1996) 506-521. (Pubitemid 126782383)
9. H. Topcuoglu, S. Hariri, M. Wu, Performance-effective and low-complexity task scheduling for heterogeneous computing, IEEE Transactions on Parallel and Distributed Systems 13 (3) (2002) 260-274. (Pubitemid 34448780)
10. R. Bajaj, D. Agrawal, Improving scheduling of tasks in a heterogeneous environment, IEEE Transactions on Parallel and Distributed Systems 15 (2) (2004) 107-118.
11. D. Bozdag, F. Ozguner, U. Catalyurek, Compaction of schedules and a two-stage approach for duplication-based dag scheduling, IEEE Transactions on Parallel and Distributed Systems 20 (6) (2009) 857-871.
12. Y. Kwok, I. Ahmad, Static scheduling algorithms for allocating directed task graphs to multiprocessors, ACM Computing Surveys (CSUR) 31 (4) (1999) 406-471.
13. T. Braun, H. Siegel, N. Beck, L. Boloni, M. Maheswaran, A. Reuther, J. Robertson, M. Theys, B. Yao, D. Hensgen, et al., A comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems, Journal of Parallel and Distributed computing 61 (6) (2001) 810-837. (Pubitemid 33658407)
14. A. WebServices. 〈http://www.aws.amazon.com/ec2〉.
15. G. AppEngine. 〈http://code.google.com/appengine〉.
16. J. Li, S. Su, X. Cheng, Q. Huang, Z. Zhang, Cost-conscious scheduling for large graph processing in the cloud, in: 2011 IEEE 13th International Conference on High Performance Computing and Communications (HPCC), IEEE, 2011, pp. 808-813.
17. E. Deelman, G. Singh, M. Su, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, G. Berriman, J. Good, et al., Pegasus: a framework for mapping complex scientific workflows onto distributed systems, Scientific Programming 13 (3) (2005) 219-237. (Pubitemid 43022937)
18. E. Ulungu, J. Teghem, Multi-objective combinatorial optimization problems: a survey, Journal of Multi-Criteria Decision Analysis 3 (2) (1994) 83-104.
19. M. Wieczorek, A. Hoheisel, R. Prodan, Towards a general model of the multi-criteria workflow scheduling on the grid, Future Generation Computer Systems 25 (3) (2009) 237-256.
20. G. Sih, E. Lee, A compile-time scheduling heuristic for interconnection-constrained heterogeneous processor architectures, IEEE Transactions on Parallel and Distributed Systems 4 (2) (1993) 175-187. (Pubitemid 23639062)
21. D. Bozdag, U. Catalyurek, F. Ozguner, A task duplication based bottom-up scheduling algorithm for heterogeneous environments, in: 20th International Parallel and Distributed Processing Symposium, IPDPS 2006, IEEE, 2006, pp. 12-24.
22. T. Yang, A. Gerasoulis, Dsc: Scheduling parallel tasks on an unbounded number of processors, IEEE Transactions on Parallel and Distributed Systems 5 (9) (1994) 951-967.
23. M. Wieczorek, R. Prodan, T. Fahringer, Scheduling of scientific workflows in the askalon grid environment, ACM SIGMOD Record 34 (3) (2005) 56-62. (Pubitemid 43152533)
24. K. Kurowski, J. Nabrzyski, A. Oleksiak, J. Weglarz, Grid multicriteria job scheduling with resource reservation and prediction mechanisms, Perspectives in Modern Project Scheduling, 2006. pp. 345-373.
25. K. Kurowski, J. Nabrzyski, A. Oleksiak, J. Weglarz, Scheduling jobs on the grid-multicriteria approach, Computational methods in science and technology 12 (2) (2006) 123-138.
26. J. Yu, R. Buyya, C. Tham, Cost-based scheduling of scientific workflow applications on utility grids, in: First International Conference on e-Science and Grid Computing, 2005, IEEE, 2005, pp. 8-pp.
27. J. Yu, R. Buyya, A budget constrained scheduling of workflow applications on utility grids using genetic algorithms, in: Workshop on Workflows in Support of Large-Scale Science, 2006, WORKS'06, IEEE, 2006, pp. 1-10.
28. R. Sakellariou, H. Zhao, E. Tsiakkouri, M. Dikaiakos, Scheduling workflows with budget constraints, Integrated Research in Grid Computing (2007) 189-202.
29. G. Singh, C. Kesselman, E. Deelman, A provisioning model and its comparison with best-effort for performance-cost optimization in grids, in: Proceedings of the 16th international symposium on High performance distributed computing, ACM, 2007, pp. 117-126. (Pubitemid 47291198)
30. Q. Zhu, G. Agrawal, Resource provisioning with budget constraints for adaptive applications in cloud environments, in: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing, ACM, 2010, pp. 304-307.
31. S. Garg, R. Buyya, H. Siegel, Time and cost trade-off management for scheduling parallel applications on utility grids, Future Generation Computer Systems 26 (8) (2010) 1344-1355.
32. J. Barbosa, B. Moreira, Dynamic scheduling of a batch of parallel task jobs on heterogeneous clusters, Parallel Computing 37 (8) (2011) 428-438.
33. L. Bittencourt, R. Sakellariou, E. Madeira, Dag scheduling using a lookahead variant of the heterogeneous earliest finish time algorithm, in: 18th Euromicro International Conference on Parallel, Distributed and Network-Based Processing (PDP), 2010, IEEE, 2010, pp. 27-34.
34. D. generation program. 〈http://www.loria.fr/suter/dags/html〉.
35. P. Project. 〈https://confluence.pegasus.isi.edu/display/pegasus/ workflowgenerator〉.
36. S. Bharathi, A. Chervenak, E. Deelman, G. Mehta, M. Su, K. Vahi, Characterization of scientific workflows, in: Third Workshop on Workflows in Support of Large-Scale Science, 2008, WORKS 2008, IEEE, 2008, pp. 1-10.