A physical and virtual compute cluster resource load balancing approach to data-parallel scientific workflow scheduling

Proceedings - 2011 IEEE World Congress on Services, SERVICES 2011

Volumn , Issue , 2011, Pages 212-215

  Wang, Jianwu ^a   Korambath, Prakashan ^b   Altintas, Ilkay ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Cluster resource manager comparison; Data parallel workflow scheduling; Load balancing; Virtual cluster

Indexed keywords

COMPUTATIONAL RESOURCE LOAD; COMPUTE RESOURCES; CPU-INTENSIVE; DATA PARALLEL; EXECUTION PERFORMANCE; LOAD-BALANCING; ORACLE GRID; RESOURCE MANAGERS; SCIENTIFIC WORKFLOWS; SMALL-SCALE CLUSTERS; VIRTUAL CLUSTERS; WORK-FLOWS; WORKFLOW ENGINES; WORKFLOW EXECUTION;

BALANCING; DISTRIBUTED COMPUTER SYSTEMS; MANAGERS; PARALLEL ARCHITECTURES; SCHEDULING;

MANAGEMENT;

EID: 80053417264     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SERVICES.2011.50     Document Type: Conference Paper

References (7)

1. C. Pautasso, G. Alonso. "Parallel Computing Patterns for Grid Workflows". Proc. of Workshop on Workflows in Support of Large-Scale Science (WORKS06), 2006
2. H. El-Rewini, T. Lewis, H. Ali. "Task Scheduling in Parallel and Distributed Systems". ISBN: 0-13-099235-6, PTR Prentice Hall, 1994.
3. J. Wang, P. Korambath, S. Kim, S. Johnson, K. Jin, D. Crawl, I. Altintas, S. Smallen, B. Labate, K. N. Houk. "Theoretical Enzyme Design Using the Kepler Scientific Workflows on the Grid". Proc. of the 5th Workshop on Computational Chemistry and Its Applications (5th CCA) at Int Conf. on Computational Science (ICCS 2010).
4. J. Dean and S. Ghemawat. "MapReduce: Simplified Data Processing on Large Clusters". Proc. of the 6th Symp. on Operating Systems Design and Implementation (OSDI 2004), pp. 137-150. USENIX Association, 2004.
5. J. Wang, D. Crawl, I. Altintas. "Kepler + Hadoop : A General Architecture Facilitating Data-Intensive Applications in Scientific Workflow Systems". Proc. of the 4th Workshop on Workflows in Support of Large-Scale Science (WORKS09) at Supercomputing 2009 (SC2009) Conference, 2009.
6. C. Zhang, H. D. Sterck. "CloudWF: A Computational Workflow System for Clouds Based on Hadoop". Proc. of the 1st Int. Conf. on Cloud Computing (CloudCom 2009).
7. X. Fei, S. Lu, C. Lin. "A MapReduce-Enabled Scientific Workflow Composition Framework". Proc. of 2009 IEEE Int. Conf. on Web Services (ICWS 2009), pp. 663-670, 2009.