Scheduling data-intensive workflows on storage constrained resources

Proceedings of the 4th Workshop on Workflows in Support of Large-Scale Science, WORKS '09, in Conjunction with SC 2009

Volumn , Issue , 2009, Pages

  Bharathi, Shishir ^a   Chervenak, Ann ^a  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

Data management; Workflow execution

Indexed keywords

COMPUTE RESOURCES; CONSTRAINED RESOURCES; DATA MANAGEMENT WORKFLOW; DATA STAGING; LARGE AMOUNTS OF DATA; LOW COSTS; OVERALL COSTS; SIGNIFICANT IMPACTS; WORK-FLOWS;

GENETIC ALGORITHMS; HEURISTIC METHODS;

MANAGEMENT;

EID: 74049152497     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1645164.1645167     Document Type: Conference Paper

References (24)

1. E. Deelman et al., "Pegasus: A framework for mapping complex scientific workflows onto distributed systems," Scientific Programming, vol. 13, no. 3, pp. 219-237, 2005.
2. T. McPhillips et al., "Scientific workflow design for mere mortals," Future Generation Computer Systems, vol. 25, no. 5, pp. 541-551, 2009.
3. S. Bharathi and A. Chervenak, "Data staging strategies and their impact on the execution of scient ific workflows," to be presented at the Second Intl. Workshop on Data Aware Distribut ed Computing, in conjunction with HPDC 2009.
4. A. Ramakrishnan et al., "Scheduling data-intensive workflows onto storage-constrained distributed resources," in 7th IEEE Intl. Symp. on Cluster Computing and the Grid, 14-17 May 2007, Rio de Janeiro, Brazil., 2007, pp. 401-409.
5. R. Sethi, "Complete register allocation problems," SIAM J. Comput., vol. 4, no. 3, pp. 226-248, 1975.
6. W. Herroelen et al., "Resource-constrained project scheduling: A survey of recent developments," Computers & OR, vol. 25, no. 4, pp. 279-302, 1998.
7. "Dagman: Directed acyclic graph manager," http://www.cs.wisc. edu/condor/dagman.
8. J. H. Holland, Adaptation in Natural and Artificial Systems. The MIT Press, 1992.
9. D. E. Goldberg, Genetic Algorithms in Search Optimization and Machine Learning. Addison-Wesley, 1989.
10. M. Mitchell, "Handbook of genetic algorithms (l. d. davis)," Artif. Intell., vol. 100, no. 1-2, pp. 325-330, 1998.
11. E. S. H. Hou et al., "A genetic algorithm for multiprocessor scheduling," IEEE Trans. Parallel Distrib. Syst., vol. 5, no. 2, pp. 113-120, 1994.
12. Y.-K. Kwok and I. Ahmad, "Efficient scheduling of arbitrary task graphs to multiprocessors using a parallel genetic algorithm," J. Parallel Distrib. Comput., vol. 47, no. 1, pp. 58-77, 1997.
13. T. Phan et al., "Evolving toward the perfect schedule: Co-scheduling job assignments and data replication in wide-area systems using a genetic algorithm," in Job Scheduling Strategies for Parallel Processing, 11th Intl. Workshop, Cambridge, MA, USA, 2005, pp. 173-193.
14. M. L. Mauldin, "Maintaining diversity in genetic search," in Proceedings of the National Conference on Artificial Intelligence. Austin, TX, August 6-10, 1984, 1984, pp. 247-250.
15. E. Zitzler and L. Thiele, "Multiobjective evolutionary algorithms: a comparative case study and the strength pareto approach," IEEE Trans. Evolutionary Computation, vol. 3, no. 4, pp. 257-271, 1999.
16. E. Zitzler et al., "Comparison of multiobjective evolutionary algorithms: Empirical results," Evolutionary Computation, vol. 8, no. 2, pp. 173-195, 2000.
17. "Condor-G," http://www.cs.wisc.edu/condor/condorg.
18. "Amazon web services: http://aws.amazon.com."
19. "Amazon EC2 pricing: http://aws.amazon.com/ec2/#pricing."
20. "Amazon S3 pricing: http://aws.amazon.com/s3/#pricing."
21. E. Deelman et al., "The cost of doing science on the cloud: the Montage example," in Proceedings of the ACM/IEEE Conference on High Performance Computing, SC 2008, November 15-21, 2008, Austin, Texas, USA. IEEE/ACM 2008. IEEE/ACM, 2008, p. 50.
22. E. Santos-Neto et al., "Exploiting replication and data reuse to efficiently schedule data-intensive applications on grids," in Job Scheduling Strategies for Parallel Processing, 10th International Workshop, JSSPP 2004, New York, NY, USA, June 13, 2004, ser. Lecture Notes in Computer Science, D. G. Feitelson et al., Eds., vol. 3277. Springer, 2004, pp. 210-232.
23. T. Kosar and M. Livny, "Stork: Making data placement a first class citizen in the grid," in 24th Intl. Conf. on Distributed Computing Systems, 24-26 March 2004, Hachioji, Tokyo, Japan, 2004, pp. 342-349.
24. T. Kosar and M. Livny, "A framework for reliable and efficient data placement in distributed computing systems," J. Parallel Distrib. Comput., vol. 65, no. 10, pp. 1146-1157, 2005.