Scheduling data-intensive workflows onto storage-constrained distributed resources

Proceedings - Seventh IEEE International Symposium on Cluster Computing and the Grid, CCGrid 2007

Volumn , Issue , 2007, Pages 401-409

  Ramakrishnan, Arun ^a   Singh, Gurmeet ^b   Zhao, Henan ^c   Deelman, Ewa ^b   Sakellariou, Rizos ^c   Vahi, Karan ^b   Blackburn, Kent ^d   Meyers, David ^d,e   Samidi, Michael ^d  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^c UNIVERSITY OF MANCHESTER   (United Kingdom)

^d CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^e NORTHROP GRUMMAN CORPORATION   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; DATA STORAGE EQUIPMENT; DISTRIBUTED COMPUTER SYSTEMS; SCHEDULING;

DATA FILES; DISTRIBUTED RESOURCES;

DATA FLOW ANALYSIS;

EID: 34548309259     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CCGRID.2007.101     Document Type: Conference Paper

References (31)

1. E. Deelman and Y. Gil. Managing Large-Scale Scientific Workflows in Distributed Environments: Experiences and Challenges. In Proceedings of Workflows in e-Science, Amsterdam, 2006.
2. I. Taylor, E. Deelman, D. Gannon, and M. Shields (Eds). Workflows in e-Science, Springer-Verlag, 2006.
3. TeraGrid. http://www.teragrid.org/
4. Open Science Grid, http://www.opensciencegrid.org/
5. E. Deelman, G. Singh, M.-H. Su, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, G. B. Berriman, J. Good, A. Laity, J. C. Jacob, and D. S. Katz. Pegasus: a Framework for Mapping Complex Scientific Workflows onto Distributed Systems. Scientific Programming Journal, vol. 13, pp. 219-237, 2005.
6. Pegasus. http://pegasus.isi.edu/
7. E. Deelman, J. Blythe, Y. Gil, and C. Kesselman. Workflow Management in GriPhyN. In Grid Resource Management, J. Nabrzyski, J. Schopf, and J. Weglarz, Eds.: Kluwer, 2003.
8. D. S. Katz, N. Anagnostou, G. B. Berriman, E. Deelman, J. Good, J. C. Jacob, C. Kesselman, A. Laity, T. A. Prince, G. Singh, M.-H. Su, and R. Williams. Astronomical Image Mosaicking on a Grid: Initial Experiences. In Engineering the Grid - Status and Perspective, B. D. Martino, J. Dongarra, A. Hoisie, L. Yang, and H. Zima, Eds.: ASP, 2005.
9. E. Deelman, K. Blackburn, P. Ehrens, C. Kesselman, S. Koranda, A. Lazzarini, G. Mehta, L. Meshkat, L. Pearlman, K. Blackburn, and R. Williams. GriPhyN and LIGO, Building a Virtual Data Grid for Gravitational Wave Scientists. Proceedings of 11th Intl Symposium on High Performance Distributed Computing, 2002.
10. Duncan A. Brown et al. A Case Study on the Use of Workflow Technologies for Scientific Analysis: Gravitational Wave Data Analysis. In Workflows for e-Science: Scientific Workflows for Grids Springer-Verlag, 2006.
11. P. Maechling, H. Chalupsky, M. Dougherty, E. Deelman, Y. Gil, S. Gullapalli, V Gupta, C. Kesselman, J. Kim, G. Mehta, B. Mendenhall, T. A. Russ, G. Singh, M. Spraragen, G. Staples, and K. Vahi. Simplifying construction of complex workflows for non-expert users of the Southern California Earthquake Center Community Modeling Environment. SIGMOD Record, vol. 34, pp. 24-30, 2005.
12. A. Lathers, M.-H. Su, A. Kulungowski, A. W Lin, G. Mehta, S. T. Peltier, Ewa Deelman, and M. H. Ellisman. Enabling Parallel Scientific Applications with Workflow Tools. Proceedings of Challenges of Large Applications in Distributed Environments (CLADE), Paris, 2006.
13. J. Frey, T. Tannenbaum, I. Foster, M. Livny, and S. Tuecke. Condor-G: A Computation Management Agent for Multi-Institutional Grids. Cluster Computing, vol. 5, pp. 237-246, 2002.
14. I. Foster and C. Kesselman, Globus: A Metacomputing Infrastructure Toolkit. International Journal of Supercomputer Applications, vol. 11, pp. 115-128, 1997.
15. Michael Litzkow and Miron Livny and Matthew Mukta Condor - A Hunter of Idle Workstations In Proceedings of the 8th International Conference on Distributed Computing Systems, 1988, 104-111.
16. B. C. Barish and R. Weiss. LIGO and the Detection of Gravitational Waves. Physics Today, vol. 52, pp. 44, 1999.
17. B. Abbott et al. Search for gravitational waves from primordial black hole binary coalescences in the galactic halo In Physical Review D, Vol 72(08), 2005.
18. B. Abbott et al. Search for gravitational waves from binary black hole inspirals in LIGO data In Physical Review D, Vol 73(06), 2006.
19. R. Buyya and M. Murshed. Gridsim: A Toolkit for the Modeling and Simulation of Distributed Resource Management and Scheduling for Grid Computing. Concurrency and Computation: Practice and Experience, vol. 14, pp. 1175-1220, 2002.
20. B. Abbott et al. Search for gravitational waves from binary inspirals in S3 and S4 LIGO data in preparation 2007.
21. Y.-K. Kwok and I. Ahmad. Static Scheduling Algorithms for Allocating Directed Task Graphs. ACM Computing Surveys, 31(4):406-471, 1999.
22. J. Blythe, S. Jain, E. Deelman, Y. Gil, K. Vahi, A. Mandal, and K. Kennedy. Resource Allocation Strategies for Workflows in Grids. In IEEE International Symposium on Cluster Computing and the Grid (CCGrid), 2005.
23. A. Mandal, K. Kennedy, C. Koelbel, G. Marin, J. Mellor-Crummey, B. Liu and L. Johnsson. Scheduling Strategies for Mapping Application Workflows onto the Grid. In IEEE International Symposium on High Performance Distributed Computing (HPDC 2005), 2005.
24. R. Sakellariou, H. Zhao, E. Tsiakkouri, and M. D. Dikaiakos. Scheduling Workflows with Budget Constraints. In S. Gorlatch, M. Danelutto (Eds.), Integrated Research in Grid Computing, CoreGrid series, Springer-Verlag, to appear.
25. R. Sakellariou and H. Zhao. A Low-Cost Rescheduling Policy for efficient mapping of Workflows on Grid Systems. Scientific Programming, vol. 12, no. 4, pp. 253-262, December 2004.
26. M. Wieczorek, R. Prodan and T. Fahringer. Scheduling of Scientific Workflows in the ASKALON Grid Environment. SIGMOD Record, volume 34(3), September 2005.
27. J. Yu and R. Buyya. A Budget Constraint Scheduling of Workflow Applications on Utility Grids using Genetic Algorithms. Proceedings of the Workshop on Workflows in Support of Large-Scale Science (WORKS06), Paris, France, 2006.
28. H. Zhao and R. Sakellariou. Advance Reservation Policies for Workflows. 12th Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), Saint-Malo, France, June 2006.
29. H. Zhao and R. Sakellariou. Scheduling Multiple DAGs onto Heterogeneous Systems. Proceedings of the 15th Heterogeneous Computing Workshop (HCW), Rhodes, Greece, 2006.
30. K. Ranganathan and I. Foster. Design and Evaluation of Dynamic Replication Strategies for a High-Performance Data Grid. Proceedings of the International Conference on Computing in High Energy and Nuclear Physics, Beijing, September 2001.
31. K. Ranganathan and I. Foster. Decoupling Computation and Data Scheduling in Distributed Data-Intensive Applications. International Symposium for High Performance Distributed Computing (HPDC), Edinburgh, July 2002.