Optimizing workflow data footprint

Scientific Programming

Volumn 15, Issue 4, 2007, Pages 249-268

  Singh, Gurmeet ^a   Vahi, Karan ^a   Ramakrishnan, Arun ^a   Mehta, Gaurang ^a   Deelman, Ewa ^a   Zhao, Henan ^b   Sakellariou, Rizos ^b   Blackburn, Kent ^c   Brown, Duncan ^c   Fairhurst, Stephen ^c,d   Meyers, David ^c   Berriman, G Bruce ^e   Good, John ^e   Katz, Daniel S ^f  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^b UNIVERSITY OF MANCHESTER   (United Kingdom)

^c CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^d UNIVERSITY OF WISCONSIN MILWAUKEE   (United States)

^e CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^f LOUISIANA STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COST EFFECTIVENESS; GRID COMPUTING; LARGE SCALE SYSTEMS; LASER INTERFEROMETRY; OPTIMIZATION;

CLEANUP TECHNIQUES; DATA STORAGE; DISTRIBUTED RESOURCES; WORKFLOW DATA;

DATA HANDLING;

EID: 37149000946     PISSN: 10589244     EISSN: None     Source Type: Journal    
DOI: 10.1155/2007/701609     Document Type: Conference Paper

References (44)

1. Condor DAGMan, http://www.cs.wisc.edu/condor/dagman.
2. Cray Fortran Reference Manual.
3. Montage, in http://montage.ipac.caltech.edu.
4. Open Science Grid.
5. The Two Micron All Sky Survey at IPAC (2MASS), in http://www.ipac. caltech.edu/2mass/.
6. Workflows in e-Science: Springer, 2006.
7. B. Abbott, Search for gravitational waves from binary inspirais in S3 and S4 LIGO data, arXiv:0704.3368 [gr-qc].
8. B. Abbott et al., Search for gravitational waves from binary black hole inspirais in LIGO data, Physical Review D (Particles, Fields, Gravitation, and Cosmology), (vol. 73), 2006, pp. 062001-17.
9. B. Abbott et al., Search for gravitational waves from primordial black hole binary coalescences in the galactic halo, Physical Review D (Particles, Fields, Gravitation, and Cosmology), (vol. 72), 2005, pp. 082002-8.
10. B.C. Barish and R. Weiss, LIGO and the Detection of Gravitational Waves, Physics Today 52 (1999), 44.
11. C. Baru et al., The SDSC Storage Resource Broker, presented at Proc. CASCON'98 Conference, 1998.
12. J. Bent et al., Flexibility, manageability, and performance in a Grid storage appliance, presented at 11th IEEE Internation al Symposium on High Performance Distributed Computing, 2002. HPDC-11 2002. Proceedings., 2002.
13. B. Berriman et al., Montage: A Grid-Enabled Image Mosaic Service for the NVO, presented at Astronomical Data Analysis Software & Systems (ADASS) XIII, 2003.
14. G.B. Berriman et al., Montage: A Grid Enabled Engine for Delivering Custom Science-Grade Mosaics On Demand, presented at SPIE Conference 5487: Astronomical Telescopes, 2004.
15. G. B. Berriman et al., Generating Complex Astronomy Work-flows, in: Workflows for e-Science, I.J. Taylor et al., eds., Springer, 2006.
16. J. Blythe et al., Task Scheduling Strategies for Workflow-based Applications in Grids, presented at CCGrid, Cardiff, UK, 2005.
17. D.A. Brown et al., A Case Study on the Use of Workflow Technologies for Scientific Analysis: Gravitational Wave Data Analysis, in: Workflows for e-Science, I.J. Taylor, et al., eds., Springer, 2006.
18. R. Buyya and M. Murshed, GridSim: A Toolkit for the Modeling and Simulation of Distributed Resource Management and Scheduling for Grid Computing, Concurrency and Computation: Practice and Experience 14 (2002), 1175-1220.
19. E. Deelman et al., GriPhyN and LIGO, Building a Virtual Data Grid for Gravitational Wave Scientists, presented at 11th Intl Symposium on High Performance Distributed Computing, 2002.
20. E. Deelman et al., Workflow Management in GriPhyN, in: Grid Resource Management, J. Nabrzyski, et al., eds., Kluwer, 2003.
21. E. Deelman et al., Pegasus: Mapping Scientific Workflows onto the Grid, presented at 2nd EUROPEAN ACROSS GRIDS CONFERENCE, Nicosia, Cyprus, 2004.
22. E. Deelman et al., Managing Large-Scale Workflow Execution from Resource Provisioning to Provenance tracking: The CyberShake Example, presented at 2nd IEEE International Conference on e-Science and Grid Computing, Amsterdam, The Netherlands, 2006.
23. E. Deelman et al., Pegasus: Mapping Large-Scale Workflows to Distributed Resources, in: Workflows for e-Science, D. Gannon et al., eds., Springer, 2006.
24. E. Deelman et al., Pegasus: a Framework for Mapping Complex Scientific Workflows onto Distributed Systems, Scientific Programming Journal 13 (2005), 219-237.
25. 1. Foster et al., The Grid2003 production grid: principles and practice, presented at 13th IEEE International Symposium on High Performance Distributed Computing (HPDC), 2004.
26. I. Foster and C. Kesselman, Globus: A Metacomputing Infrastructure Toolkit, International Journal of Supercomputer Applications 11 (1997), 115-128.
27. J. Frey et al., Condor-G: A Computation Management Agent for Multi-Institutional Grids, presented at 10th International Symposium on High Performance Distributed Computing, 2001.
28. Y-K. Kwok and I. Ahmad, Static scheduling algorithms for allocating directed task graphs to multiprocessors, ACM Comput Surv 31 (1999), 406-471.
29. A. Lathers et al., Enabling Parallel Scientific Applications with Workflow Tools, presented at Challenges of Large Applications in Distributed Environments (CLADE), Paris, 2006.
30. M. Litzkow et al., Condor - A Hunter of Idle Workstations, in: Proc. 8th Intl Conf. on Distributed Computing Systems, 1988, pp. 104-111.
31. P. Maechling et al., SCEC CyberShake Workflows - Automating Probabilistic Seismic Hazard Analysis Calculations, in: Workflows for e-Science, D. Gannon et al., eds., Springer, 2006.
32. A. Mandal et al., Scheduling Strategies for Mapping Application Workflows onto the Grid, presented at The 14th IEEE International Symposium on High Performance Distributed Computing (HPDC-14), 2005.
33. V. Nefedova et al., Automating Climate Science: Large Ensemble Simulations on the TeraGrid with the GriPhyN Virtual Data System, presented at e-Science and Grid Computing, 2006. e-Science '06. Second IEEE International Conference on, 2006.
34. V Nefedova et al., Automating Climate Science: Large Ensemble Simulations on the TeraGrid with the GriPhyN Virtual Data System, presented at e-Science, Amsterdam, The Netherlands, 2006.
35. A. Ramakrishnan et al., Scheduling Data-Intensive Workflows onto Storage-Constrained Distributed Resources, presented at 7th IEEE International Symposium on Cluster Computing and the Grid - CCGrid, 2007 (to appear).
36. K. Ranganathan and I. Foster, Decoupling Computation and Data Scheduling in Distributed Data Intensive Applications, presented at International Symposium for High Performance Distributed Computing (HPDC-11), Edinburgh, 2002.
37. K. Ranganathan and I. Foster, Identifying Dynamic Replication Strategies for a High Performance Data Grid, presented at International Workshop on Grid Computing, 2001.
38. R. Sakellariou and H. Zhao, A hybrid heuristic for DAG scheduling on heterogeneous systems, presented at 18th International Parallel and Distributed Processing Symposium, 2004.
39. A. Shoshani et al., Storage resource managers: Middleware components for grid storage, presented at Nineteenth IEEE Symposium on Mass Storage Systems, 2002.
40. D. Thain, Operating System Support for Space Allocation in Grid Storage Systems, presented at 7th IEEE/ACM International Conference on Grid Computing, Barcelona, Spain, 2006.
41. S. Venugopal and R. Buyya, A Set Coverage-based Mapping Heuristic for Scheduling Distributed Data-Intensive Applications on Global Grids, presented at 7th IEEE/ACM International Conference on Grid Computing, Barcelona, Spain, 2006.
42. S. Venugopal et al., A Grid Service Broker for Scheduling e-Science Applications on Global Data Grids, Concurrency and Computation: Practice and Experience 18 (2006), 685-699.
43. M. Wieczorek et al., Scheduling of scientific workflows in the ASKALON grid environment, SIGMOD Rec 34 (2005), 56-62.
44. J. Yu and R. Buyya, A Budget Constraint Scheduling of Workflow Applications on Utility Grids using Genetic Algorithms, presented at Workshop on Workflows in Support of LargeScale Science (WORKS06), Paris, France, 2006.