Data management challenges of data-intensive scientific workflows

Proceedings CCGRID 2008 - 8th IEEE International Symposium on Cluster Computing and the Grid

Volumn , Issue , 2008, Pages 687-692

  Deelman, Ewa ^a   Chervenak, Ann ^a  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CHLORINE COMPOUNDS; MANAGEMENT INFORMATION SYSTEMS; TECHNICAL PRESENTATIONS;

CLUSTER COMPUTING; DATA MANAGEMENT; INTERNATIONAL SYMPOSIUM; SCIENTIFIC WORKFLOWS;

MANAGEMENT;

EID: 50649089846     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CCGRID.2008.24     Document Type: Conference Paper

References (37)

1. "Open Science Grid." www.opensciencegrid.org
2. "Flexible Image Transport System." http://fits.gsfc.nasa.gov/
3. Workflows in e-Science. I. Taylor, E. Deelman, D. Gannon, and M. Shields, Eds.: Springer, 2006.
4. "Fast Data Transfer (FDT) service," 2007.
5. W. Allcock, et al., "Secure, Efficient Data Transport and Replica Management for High-Performance Data-Intensive Computing," Mass Storage Conference, 2001.
6. B. C. Barish, et al., "LIGO and the Detection of Gravitational Waves," Physics Today, vol. 52, 1999.
7. G. B. Berriman, et al., "Montage: A Grid Enabled Engine for Delivering Custom Science-Grade Mosaics On Demand," in SPIE Conference 5487: 2004.
8. D. A. Brown, et al., "A Case Study on the Use of Workflow Technologies for Scientific Analysis: Gravitational Wave Data Analysis," in Workflows for e-Science, I. Taylor, et al. Eds.: Springer, 2006.
9. A. Chervenak, et al. "Giggle: A Framework for Constructing Sclable Replica Location Services," in SC2002 Conference, Baltimore, MD, 2002.
10. A. Chervenak, et al. "Data Placement for Scientific Applications in Distributed Environments," in Grid 2007
11. A. L. Chervenak, et al. "Performance and Scalability of a Replica Location Service," in HPDC-13, 2004.
12. P. Couvares, et al., "Workflow Management in Condor," in Workflows in e-Science, I. Taylor, et al. Eds., Springer, 2006.
13. K. Czajkowski, et al., "Grid Information Services for Distributed Resource Sharing," in HPDC 2001
14. E. Deelman, et al., "Pegasus: a Framework for Mapping Complex Scientific Workflows onto Distributed Systems," Scientific Programming .l, vol. 13, 2005.
15. E. Deelman, et al., "Managing Large-Scale Workflow Execution from Resource Provisioning to Provenance Tracking: The CyberShake Example," e-Science 2006.
16. E. Deelman, et al., "Managing Large-Scale Scientific Workflows in Distributed Environments: Experiences and Challenges," in Workflows in e-Science, 2006.
17. E. Deelman, et al., "Pegasus: Mapping Large-Scale Workflows to Distributed Resources," in Workflows in e-Science, I. Taylor, et al. Eds.: Springer, 2006.
18. E. Deelman, et al., "Grid-Based Metadata Services," in 16th International Conference on Scientific and Statistical Database Management, 2004.
19. D. H. J. Epema, et al., "A Worldwide Flock of Condors: Load Sharing among Workstation Clusters," Future Generation Computer Systems, vol. 12, 1996.
20. I. Foster, et al., "The Anatomy of the Grid: Enabling Scalable Virtual Organizations," Int. J. of High Performance Computing Applications, vol. 15, 2001.
21. J. Freire, et al., "Managing Rapidly-Evolving Scientific Workflows.," IPAW, vol. 4145, pp. 10-18, 2006.
22. Y. Gil, et al., "Examining the Challenges of Scientific Workflows," IEEE Computer, December 2007.
23. C. A. Goble, et al., "myExperiment: social networking for workflow-using e-scientists," WORKS, pp. 1-2, 2007.
24. R. Henderson, et al., "Portable Batch System: External Reference Specification," 1996.
25. O. Lassila, et al., "Resource Description Framework (RDF) Model and Syntax Specification," 1999.
26. P. Maechling, et al., "Simplifying construction of complex workflows for non-expert users of the Southern California Earthquake Center Community Modeling Environment," SIGMOD Record, vol. 34, 2005.
27. D. E. Middleton, et al. "Enabling worldwide access to climate simulation data: the earth system grid (ESG)," Scientific Discovery Through Advanced Computing (SciDAC 2006), Journal of Physics: Conference Series, vol. 46, pp. 510-514, June 25-29, 2006
28. S. Miles, et al., "The Requirements of Using Provenance in e-Science Experiments," J. of Grid Computing, 2006.
29. S. Miles, et al., "Connecting Scientific Data to Scientific Experiments with Provenance," in e-Science, 2007
30. L. Moreau, et al., "The Open Provenance Model," University of Southampton 2007. http://eprints.ecs.soton.ac.uk/14979/1/opm.pdf
31. L. Moreau, et al., "The First Provenance Challenge," Concurrency and Computation: Practice and Experience, 2007.
32. L. Moreau, et al., "Concurrency and Computation:Practice and Experience, Special Issue on the First Provenance Challenge," 2007.
33. A. Rajasekar, et al., "Storage Resource Broker-Managing Distributed Data in a Grid," Computer Society of India Journal, Special Issue on SAN, vol. 33(4), 2003.
34. A. Ramakrishnan, et al., "Scheduling Data -Intensive Workflows onto Storage-Constrained Distributed Resources," in CCGrid 2007.
35. R. Sakellariou, et al., "Scheduling Workflows with Budget Constraints," Integrated Research in Grid Computing, CoreGrid series, Springer-Verlag, 2005.
36. G. Singh, et al., "Optimizing Workflow Data Footprint " Scientific Programming Journal, vol. 15, 2007
37. M. Wieczorek, et al., "Scheduling of Scientific Workflows in the ASKALON Grid Environment," SIGMOD Record, vol. 34, pp. 56-62, 2005