Failure-aware workflow scheduling in cluster environments

Cluster Computing

Volumn 13, Issue 4, 2010, Pages 421-434

  Yu, Zhifeng ^a   Wang, Chenjia ^a   Shi, Weisong ^a  

^a WAYNE STATE UNIVERSITY   (United States)

Author keywords

Cluster computing; Failure aware; Workflow scheduling

Indexed keywords

CLUSTER ENVIRONMENTS; COMPREHENSIVE EVALUATION; FAILURE PREDICTION; FAILURE-AWARE; HIGH PERFORMANCE CLUSTER; MAKESPAN; NP COMPLETE PROBLEMS; POTENTIAL RESOURCES; PREDICTION ACCURACY; SYSTEM AVAILABILITY; WORKFLOW APPLICATIONS; WORKFLOW SCHEDULING;

CLUSTER COMPUTING; COMPUTATIONAL COMPLEXITY; FORECASTING; MANAGEMENT;

SCHEDULING ALGORITHMS;

EID: 78249235311     PISSN: 13867857     EISSN: 15737543     Source Type: Journal    
DOI: 10.1007/s10586-010-0126-7     Document Type: Article

References (30)

1. Garey, M., Johnson, D.: Computers and Intractibility: A Guide to the Theory of NP-completeness. Freeman, San Francisco (1979).
2. Open science grid. [Online]. Available: http://www. opensciencegrid. org/.
3. Nsf taragrid. [Online]. Available: http://www. teragrid. org/.
4. Yang, L., Schopf, J., Foster, I.: Anomaly detection and diagnosis in grid environments. In: SC'07: Proceedings of the 2007 ACM/IEEE Conference on Supercomputing. IEEE Computer Society, Washington (2007).
5. Liang, Y., Sivasubramaniam, A., Moreira, J.: Filtering failure logs for a bluegene/l prototype. In: Proceedings of the 2005 International Conference on Dependable Systems and Networks (DSN'05), pp. 476-485. IEEE Computer Society, Washington (2005).
6. Fu, S., Xu, C.: Exploring event correlation for failure prediction in coalitions of clusters. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis (SC'07) (2007).
7. Oppenheimer, D., et al.: Service placement in shared wide-area platforms. In: Proceedings of the Twentieth ACM Symposium on Operating Systems Principles (SOSP'05), p. 1. ACM, New York (2005).
8. Zhang, Y., et al.: Performance implications of failures in large-scale cluster scheduling. In: Proceedings of 10th International WorkshopJob Scheduling Strategies for Parallel Processing (JSSPP'04), pp. 233-252 (2004).
9. Schroeder, B., Gibson, G.: A large-scale study of failures in high-performance computing systems. In: Proceedings of the International Conference on Dependable Systems and Networks (DSN'06), pp. 249-258. IEEE Computer Society, Washington (2006).
10. Yalagandula, P., et al.: Beyond availability: Towards a deeper understanding of machine failure characteristics in large distributed systems. In: Proceedings of the Workshop on Real, Large Distributed Systems (WORLDS'04) (2004).
11. Ren, X., et al.: Prediction of resource availability in fine-grained cycle sharing systems empirical evaluation. J. Grid Comput. 5(2), 173-195 (2007).
12. Salfner, F., Schieschke, M., Malek, M.: Predicting failures of computer systems: a case study for a telecommunication system. In: Proceedings of the 20th International Parallel and Distributed Processing Symposium (IPDPS 2006) (2006).
13. Li, Y., Lan, Z.: Exploit failure prediction for adaptive fault-tolerance in cluster computing. In: Proceedings of the Sixth IEEE International Symposium on Cluster Computing and the Grid (CCGRID'06), pp. 531-538. IEEE Computer Society, Washington (2006).
14. Li, Y., et al.: Fault-driven re-scheduling for improving system-level fault resilience. In: Proceedings of the 2007 International Conference on Parallel Processing (ICPP'07), p. 39. IEEE Computer Society, Washington (2007).
15. Oliner, A., et al.: Fault-aware job scheduling for bluegene/l systems. In: Proceedings of the 18th International Parallel and Distributed Processing Symposium (IPDPS'04). IEEE Computer Society, Washington (2004).
16. Hwang, S., Kesselman, C.: Gridworkflow: A flexible failure handling framework for the grid. In: Proceedings of the 12th IEEE International Symposium on High Performance Distributed Computing (HPDC'03), p. 126. IEEE Computer Society, Washington (2003).
17. Abawajy, J. H.: Fault-tolerant scheduling policy for grid computing systems. In: Proceedings of the 18th International Parallel and Distributed Processing Symposium (IPDPS'04). IEEE Computer Society, Washington (2004).
18. Dagman. [Online]. Available: http://www. cs. wisc. edu/condor/dagman/.
19. Dogan, A., Özgüner, F.: Matching and scheduling algorithms for minimizing execution time and failure probability of applications in heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13(3), 308-323 (2002).
20. Deelman, E., Blythe, J., Gil, Y., Kesselman, C.: Workflow management in griphyn. In: Grid Resource Management: State of the Art and Future Trends, pp. 99-116. Kluwer Academic, Norwell (2004).
21. Planet lab. [Online]. Available: http://www. planet-lab. org.
22. Yu, Z., Shi, W.: A planner-guided scheduling strategy for multiple grid workflow applications. In: Proceeding of Fourth International Workshop on Scheduling and Resource Management for Parallel and Distributed Systems (SRMPDS '08), Portland, Oregon, USA, September 2008.
23. Los Alamos National Laboratory. Operational data to support and enable computer science research (2006). [Online]. Available: http://institutes. lanl. gov/data/fdata/.
24. Salfner, F., Malek, M.: Proactive fault handling for system availability enhancement. In: Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05)-Workshop 16, p. 281. 1. IEEE Computer Society, Washington (2005).
25. Yu, Z., Shi, W.: An adaptive rescheduling strategy for grid workflow applications. In: Proceeding of 21st International Parallel and Distributed Processing Symposium (IPDPS'07), Long Beach, Florida, USA, March 2007.
26. Topcuouglu, H., Hariri, S., Wu, M.: Performance-effective and low-complexity task scheduling for heterogeneous computing. IEEE Trans. Parallel Distrib. Syst. 13(3), 260-274 (2002).
27. Oliner, A., Sahoo, R., Moreira, J., Gupta, M.: Performance implications of periodic checkpointing on large-scale cluster systems. In: Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (IPDPS'05), p. 299. 2. IEEE Computer Society, Washington (2005).
28. Schroeder, B., Gibson, G.: Understanding failures in petascale computers. J. Phys., Condens. Matter 19(45) (2007).
29. Hönig, U., Schiffmann, W.: A comprehensive test bench for the evaluation of scheduling heuristics. In: Proceedings of the 16th International Conference on Parallel and Distributed Computing and Systems (PDCS'04). IEEE, New York (2004).
30. Canon, L.-C., Jeannot, E., Sakellariou, R., Zheng, W.: Comparative evaluation of the robustness of dag scheduling heuristics. In: Integration Research in Grid Computing, CoreGRID Integration Workshop, pp. 63-74. Crete University Press, Heraklion (2008).