A balanced scheduler with data reuse and replication for scientific workflows in cloud computing systems

Future Generation Computer Systems

Volumn 74, Issue , 2017, Pages 168-178

  Casas, Israel ^a,c   Taheri, Javid ^e   Ranjan, Rajiv ^b,c   Wang, Lizhe ^d   Zomaya, Albert Y ^a  

^a UNIVERSITY OF SYDNEY   (Australia)

^b NEWCASTLE UNIVERSITY   (United Kingdom)

^c CSIRO   (Australia)

^d CHINA UNIVERSITY OF GEOSCIENCES   (China)

^e KARLSTAD UNIVERSITY   (Sweden)

Author keywords

Big data; Cloud computing; Data intensive computing; Scheduling; Scientific workflow; Virtual machine

Indexed keywords

BIG DATA; CLOUD COMPUTING; ECONOMIC AND SOCIAL EFFECTS; SCHEDULING; VIRTUAL MACHINE; WEB SERVICES;

APPLICATION PROVIDERS; CLOUD COMPUTING ENVIRONMENTS; DATA-INTENSIVE COMPUTING; PERFORMANCE REQUIREMENTS; REPLICATION TECHNIQUES; SCIENTIFIC APPLICATIONS; SCIENTIFIC APPLICATIONS ON CLOUDS; SCIENTIFIC WORKFLOWS;

FACSIMILE;

EID: 84955245788     PISSN: 0167739X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.future.2015.12.005     Document Type: Article

References (41)

1. Etro, F., The economic impact of cloud computing on business creation, employment and output in Europe. Rev. Bus. Econ. 54 (2009), 179–208.
2. Millett, L.I., Fuller, S.H., The Future of Computing Performance:: Game Over or Next Level?. 2011, National Academies Press.
3. High Performance Computing—HPC Cloud Computing. Available:.
4. Armbrust, M., Fox, A., Griffith, R., Joseph, A.D., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., A view of cloud computing. Commun. ACM 53 (2010), 50–58.
5. P. Mell, T. Grance, The NIST definition of cloud computing, 2011.
6. Zhang, Q., Cheng, L., Boutaba, R., Cloud computing: state-of-the-art and research challenges. J. Internet Serv. Appl. 1 (2010), 7–18.
7. J.O. Gutierrez-Garcia, K.M. Sim, A family of heuristics for agent-based Cloud bag-of-tasks scheduling, in: 2011 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, CyberC, 2011, pp. 416–423.
8. Taheri, J., Zomaya, A.Y., Khan, S.U., Genetic algorithm in finding Pareto frontier of optimizing data transfer versus job execution in grids. Concurr. Comput.: Pract. Exper., 2012.
9. Turk, Ata, Oktay, Kerim Yasin, Aykanat, Cevdet, Query-log aware replicated declustering. IEEE Trans. Parallel Distrib. Syst. 24:5 (2013), 987–995.
10. Zaman, Sharrukh, Grosu, Daniel, A distributed algorithm for the replica placement problem. IEEE Trans. Parallel Distrib. Syst. 22:9 (2011), 1455–1468.
11. Allcock, Bill, et al. Secure, efficient data transport and replica management for high-performance data-intensive computing. 2001. MSS’01. Eighteenth IEEE Symposium on Mass Storage Systems and Technologies, 2001, IEEE.
12. Kloh, H., Schulze, B., Pinto, R., Mury, A., A bi-criteria scheduling process with CoS support on grids and clouds. Concurr. Comput.: Pract. Exper. 24:13 (2012), 1443–1460.
13. Meyer, L.A.V.C., Strategies for dynamic workflow scheduling on grids. (Ph.D. Thesis), 2007, COPPE, UFRJ (in Portuguese).
14. Wieczorek, M., Podlipnig, S., et al. Bi-criteria scheduling of scientific workflows for the grid. CCGRID’08: Pro-ceedings of the 2008 Eighth IEEE International Symposium on Cluster Computing and the Grid, 2008, IEEE Computer Society, Washington, DC, USA 978-0-7695-3156-4, 9–16.
15. Yu, J., Buyya, R., et al. Cost-based scheduling of scientific workflow application on utility grids. E-SCIENCE’05: Proceedings of the First International Conference on e-Science and Grid Computing, 2005, IEEE Computer Society, Washington, DC, USA 0-7695-2448-6, 140–147.
16. M. Xu, L. Cui, et al. A multiple QoS constrained scheduling strategy of multiple workflows for cloud computing, in: International Symposium on Parallel and Distributed Processing with Applications, Vol. 0, 2009, pp. 629–634.
17. Ramakrishnan, L., Reed, D.A., Performability modeling for scheduling and fault tolerance strategies for scientific work-flows. HPDC’08: Proceedings of the 17th International Symposium on High Performance Distributed Computing, 2008, ACM, New York, NY, USA 978-1-59593-997-5, 23–34.
18. M. Bandini, A.R. Mury, et al. A Grid-QoS decision support system using service level agreements, in: Congressoda Sociedade Brasileira, de Computacao de 2009.
19. Lin, H.-C., Raghavendra, C.S., An approximate analysis of the join the shortest queue (JSQ) policy. IEEE Trans. Parallel Distrib. Syst. 7:3 (1996), 301–307.
20. R. Achar, P. Thilagam, D. Shwetha, H. Pooja, Optimal scheduling of computational task in cloud using Virtual machine tree, in: 2012 Third International Conference on Paper Presented at the Emerging Applications of Information Technology, EAIT, 2012.
21. Zhang, F., Cao, J., Li, K., Khan, S.U., Hwang, K., Multi-objective scheduling of many tasks in cloud platforms. Future Gener. Comput. Syst. 37 (2014), 309–320.
22. Deng, L., Yu, Q., Peng, J., Adaptive scheduling strategies for cloud-based resource infrastructures. Secur. Commun. Netw. 5:10 (2012), 1102–1111.
23. M. Xu, L. Cui, H. Wang, Y. Bi, A multiple QoS constrained scheduling strategy of multiple workflows for cloud computing, in: 2009 IEEE International Symposium on Parallel and Distributed Processing with Applications, 2009, pp. 629–634.
24. S. Pandey, L. Wu, S.M. Guru, R. Buyya, A particle swarm optimization-based heuristic for scheduling workflow applications in cloud computing environments, in: 2010 24th IEEE International Conference on Advanced Information Networking and Applications, AINA, 2010, pp. 400–407.
25. Mezmaz, M., Melab, N., Kessaci, Y., Lee, Y.C., Talbi, E.-G., Zomaya, A.Y., et al. A parallel bi-objective hybrid metaheuristic for energy-aware scheduling for cloud computing systems. J. Parallel Distrib. Comput. 71 (2011), 1497–1508.
26. Moschakis, I.A., Karatza, H.D., Evaluation of gang scheduling performance and cost in a cloud computing system. J. Supercomput. 59:2 (2012), 975–992.
27. K. Tsakalozos, H. Kllapi, E. Sitaridi, M. Roussopoulos, D. Paparas, A. Delis, Flexible use of cloud resources through profit maximization and price discrimination, in: 2011 IEEE 27th International Conference on Paper Presented at the Data Engineering, ICDE, 2011.
28. de Oliveira, D., Ocaña, K.A., Baião, F., Mattoso, M., A provenance-based adaptive scheduling heuristic for parallel scientific workflows in clouds. J. Grid Comput. 10:3 (2012), 521–552.
29. Dean, J., Ghemawat, S., MapReduce: a flexible data processing tool. Commun. ACM 53:1 (2010), 72–77.
30. J. Yu, R. Buyya, C.K. Tham, Cost-based scheduling of scientific workflow applications on utility grids, in: 2005. First International Conference on e-Science and Grid Computing, 2005, pp. 8–147.
31. Garey, M.R., Johnson, D.S., Computers and Intractability: A Guide to the Theory of NP-Completeness, 1979. 1979, Freeman, San Francisco, LA.
32. R. Sakellariou, H. Zhao, A hybrid heuristic for DAG scheduling on heterogeneous systems, in: Parallel and Distributed Processing Symposium, 2004. Proceedings. 18th International, 2004, p. 111.
33. Sakellariou, R., Zhao, H., Tsiakkouri, E., Dikaiakos, M.D., Scheduling workflows with budget constraints. Integrated Research in GRID Computing, 2007, Springer, 189–202.
34. M.R. Hoseinyfarahabady, H.R. Samani, L.M. Leslie, Y.C. Lee, A.Y. Zomaya, Handling uncertainty: Pareto-efficient BoT scheduling on hybrid clouds, in: 2013 42nd International Conference on Paper Presented at the Parallel Processing, ICPP, 2013.
35. Bharathi, Shishir, et al. Characterization of scientific workflows. Workflows in Support of Large-Scale Science, WORKS 2008. Third Workshop on, 2008, IEEE.
36. Taheri, J., Zomaya, A.Y., Khan, S.U., Genetic algorithm in finding pareto frontier of optimizing data transfer versus job execution in grids. Concurr. Comput.: Pract. Exper., 2012.
37. Taheri, Javid, et al. Pareto frontier for job execution and data transfer time in hybrid clouds. Future Gener. Comput. Syst. 37 (2014), 321–334.
38. Taheri, Javid, et al. Hopfield neural network for simultaneous job scheduling and data replication in grids. Future Gener. Comput. Syst. 29:8 (2013), 1885–1900.
39. Pegasus: Workflow Management System [Online]. Available: http://pegasus.isi.edu/.
40. HTCondor: High Throughput Computing [Online]. Available: http://research.cs.wisc.edu/htcondor/.
41. AWS | Amazon Elastic Compute Cloud (EC2) [Online]. Available: http://aws.amazon.com/ec2/.