What makes workflows work in an opportunistic environment?

Concurrency and Computation: Practice and Experience

Volumn 18, Issue 10, 2006, Pages 1187-1199

  Deelman, Ewa ^a   Kosar, Tevfik ^b   Kesselman, Carl ^a   Livny, Miron ^b  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^b University of Wisconsin Madison   (United States)

Author keywords

Advanced planning; Chimera; Condor; DAGMan; Data placement; Deferred planning; Grid; Just in time planning; Pegasus; Scheduling; Stork; Workflow delegation; Workflow management; Workflow mapping

Indexed keywords

COMPUTER APPLICATIONS; COMPUTER SYSTEMS; DATA REDUCTION; MAPPING; SCHEDULING;

ADVANCED PLANNING; CHIMERA; DATA PLACEMENT; DEFERRED PLANNING; GRID; JUST-IN-TIME PLANNING; PEGASUS; WORKFLOW DELEGATION; WORKFLOW MANAGEMENT; WORKFLOW MAPPING;

DATA STRUCTURES;

EID: 33747517185     PISSN: 15320626     EISSN: 15320634     Source Type: Journal    
DOI: 10.1002/cpe.1001     Document Type: Conference Paper

References (33)

1. CERN. The Compact Muon Solenoid Project, http://cmsinfo.cern.ch [2004].
2. CERN. A Toroidal LHC Apparatus Project. http://atlas.web.cern.ch/Atlas [2004].
3. Caltech. Laser Interferometer Gravitational Wave Observatory, http://www.ligo.caltech.edu [2004].
4. FNAL. Sloan Digital Sky Survey, http://tdpc01.fnal.gov/sdss/project.htm [2004].
5. Djorgovski SG, Gal RR, Odewahn SC, de Carvalho RR, Brunner R, Longo G, Scaramella R. The Palomar Digital Sky Survey (DPOSS). Wide Field Surveys in Cosmology, 14th IAP Meeting, 26-30 May 1988. Editions Frontieres: Paris, 1988.
6. Jacob JC, Katz DS, Prince T, Berriman GB, Good JC, Laity AC, Deelman E, Singh G, Su M-H. The montage architecture for Grid-enabled science processing of large, distributed datasets. Proceedings of the Earth Science Technology Conference (ESTC), June 2004.
7. Deelman E, et al. Mapping abstract complex workflows onto Grid environments. Journal of Grid Computing 2003; 1(1):25-29.
8. Deelman E, Blythe J. Gil Y, Kesselman C. Workflow management in GriPhyN. The Grid Resource Management. Kluwer: Dordrecht, 2003.
9. Deelman E, Blythe J, Gil Y, Kesselman C, Mehta G, Patil S. Su M, Vahi K. Pegasus: Mapping scientific workflows onto the Grid. Proceedings of Across Grid EU Conference (Lecture Notes in Computer Science, vol. 3165). Springer: Berlin, 2004.
10. Condor Team. The Directed Acyclic Graph Manager (DAGMan). http://www.cs.wisc.edu/condor/dagman [2002J.
11. Frey J. Tannenbaum T, Foster I. Livny M. Tuecke S. Condor-G: A computation management agent for multi-institutional Grids. Cluster Computing 2002; 5:237-246.
12. Kosar T. Livny M. Stork: Making data placement a first class citizen in the Grid. Proceedings of the 24th IEEE International Conference on Distributed Computing Systems (ICDCS 2004), Tokyo, Japan, March 2004. IEEE Computer Society Press: Los Alamitos, CA, 2004.
13. Griphyn. Grid Physics Network, http://www.griphyn.org [2004].
14. Litzkow MJ, Livny M, Mutka MW. Condor - a hunter of idle workstations. Proceedings of the 8th International Conference of Distributed Computing Systems. IEEE Computer Society Press: Los Alamitos, CA, 1988; 104-111.
15. Foster I, Kesselmann C. Globus: A toolkit-based Grid architecture. The Grid: Blueprints for a new Computing Infrastructure. Morgan Kaufmann: San Francisco, CA, 1999; 259-278.
16. Bayucan A, Henderson RL, Losiak C, Mann B, Proett T, Tweten D. Portable batch system: External reference specification. Technical Report, MRJ Technology Solutions, November 1999.
17. Zhou S. LSF: Load sharing in large-scale heterogeneous distributed systems. Proceedings of the Workshop on Cluster Computing, Tallahassee, FL, 1992.
18. IBM. Using and administering IBM load leveler. SC23-39S9, IBM Corporation, 1996.
19. Akarsu E, Fox G, Haupt T. WebFlow-high-level programming environment and visual authoring toolkit for high performance distributed computations. Proceedings of Supercomputing '98.
20. von Laszewski G, Foster I, Gawor J, Lane P. A Java commodity Grid kit. Concurrency and Computation: Practice and Experience 2001; 13(8-9):643-662.
21. Cao J, Jarvis SA. Saini S, Nudd GR. Grid-Flow: WorkFlow management for Grid computing. Proceedings of the 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid (CCGRID'03), Tokyo, Japan, 2003. ACM Press: New York. 2003; 198-205.
22. Globus Project. GridAnt: A Grid workflow system, http://www-unix.globus.org/cog/projects/gridant [2004].
23. Buyya R, Abramson D, Giddy J, Nimrod/G: An architecture of a resource management and scheduling system in a global computational Grid. HPC Asia 2000, Beijing, China, 14-17 May 2000. IEEE Computer Society Press: Los Alamitos, CA, 2000; 283-289.
24. Beiriger J, Johnson W, Bivens H, Humphreys S, Rhea R. Constructing the ASCI Grid. Proceedings of the 9th IEEE Symposium on High Performance Distributed Computing, 2000. IEEE Press: Piscataway, NJ, 2000; 193-200.
25. Globus Project. The Open Grid Services Architecture, http://ww.globus.org/ogsa [20041.
26. Fitzgerald S, Foster I, Kesselman C, von Laszewski G, Smith W, Tuecke S. A directory service for configuring high-performance distributed computations. Proceedings of the 6th IEEE Symposium on High-Performance Distributed Computing. IEEE Computer Society Press: Los Alamitos, CA, 1997; 365-375.
27. Czajkowski K, Foster I, Karonis N, Kesselman C, Martin S, Smith W, Tuecke S. A resource management architecture for metacomputing systems. Proceedings of the IPPS/SPDP '98 Workshop on Job Scheduling Strategies for Parallel Processing (Lecture Notes in Computer Science, vol. 1459). Springer: Berlin, 1998; 62-82.
28. Welch V, Siebenlist F, Foster I, Bresnahan J, Cajkowski K, Gawor J, Kesselman C, Meder S, Pearlman L, Tuecke S. Security for Grid services. Proceedings of the 12th IEEE Symposium on High Performance Distributed Computing (HPDC-12), Seattle, WA, June 2003. IEEE Press: Piscataway, NJ, 2003; 48-57.
29. Bent J, Thain D, Arpaci-Dusseau A, Arpaci-Dusseau R. Explicit control in a batch-aware distributed file system. Proceedings of the 1st USENIX/ACM Conference on Networked Systems Design and Implementation (NSDI 2004), San Francisco, CA, March 2004.
30. Berman F, Wolski R, Figueira S, Schopf J, Shao G. Application level scheduling on distributed heterogeneous networks. Proceedings ofSupercomputing'96, Pittsburgh, PA, November 1996.
31. Wolski R. Dynamically forecasting network performance to support dynamic scheduling using the Network Weather Service. Proceedings of the 6th IEEE Symposium on High Performance Distributed Computing (HPDC6), Portland, OR, August 1997. IEEE Computer Society Press: Los Alamitos, CA, 1997.
32. Beck M, Moore T, Plank J, Swany M. Logistical networking. Active Middleware Services, Hariri S, Lee C, Raghavendra C (eds.). Kluwer Academic: Dordrecht, 2000.
33. Thain D, Livny M. The ethemet approach to Grid computing. Proceedings of the 12th IEEE Symposium on High Performance Distributed Computing (HPDC-12), Seattle, WA, June 2003. IEEE Press: Piscataway, NJ, 2003.