A cleanup algorithm for implementing storage constraints in scientific workflow executions

Proceedings of WORKS 2014: The 9th Workshop on Workflows in Support of Large-Scale Science - held in conjunction with SC 2014: The International Conference for High Performance Computing, Networking, Storage and Analysis

Volumn , Issue , 2014, Pages 41-49

  Srinivasan, Sudarshan ^a   Juve, Gideon ^b   Da Silva, Rafael Ferreira ^b   Vahi, Karan ^b   Deelman, Ewa ^b  

^a INDIAN INSTITUTE OF TECHNOLOGY HYDERABAD   (India)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; DISTRIBUTED COMPUTER SYSTEMS; WORK SIMPLIFICATION;

DATA INTENSIVE; LARGE-SCALE DATA ANALYSIS; SCIENTIFIC WORKFLOWS; SHARED RESOURCES; SHARED STORAGE; SIMPLE APPROACH; STORAGE CONSTRAINTS; WORKFLOW MANAGEMENT SYSTEMS;

DIGITAL STORAGE;

EID: 84988222800     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/WORKS.2014.8     Document Type: Conference Paper

References (20)

1. I. Taylor, E. Deelman, D. Gannon, and M. Shields, Workflows for e-Science. Springer, 2007.
2. A. Ramakrishnan, G. Singh, H. Zhao, E. Deelman, R. Sakellariou, K. Vahi, K. Blackburn, D. Meyers, and M. Samidi, "Scheduling dataintensive workflows onto storage-constrained distributed resources, " in Seventh IEEE International Symposium on Cluster Computing and the Grid, 2007.
3. G. Singh, K. Vahi, A. Ramakrishnan, G. Mehta, E. Deelman, H. Zhao, R. Sakellariou, K. Blackburn, D. Brown, S. Fairhurst, D. Meyers, G. B. Berriman, J. Good, and D. S. Katz, "Optimizing workflow data footprint, " Scientific Programming, vol. 15, no. 4, pp. 249-268, 2007.
4. S. Bharathi and A. Chervenak, "Scheduling data-intensive workflows on storage constrained resources, " in 4th Workshop on Workflows in Support of Large-Scale Science (WORKS), 2009.
5. W. Chen and E. Deelman, "Partitioning and scheduling workflows across multiple sites with storage constraints, " in Parallel Processing and Applied Mathematics, ser. Lecture Notes in Computer Science, 2012, vol. 7204, pp. 11-20.
6. R. Ferreira da Silva, G. Juve, E. Deelman, T. Glatard, F. Desprez, D. Thain, B. Tovar, and M. Livny, "Toward fine-grained online task characteristics estimation in scientific workflows, " in 8th Workshop on Workflows in Support of Large-Scale Science (WORKS), 2013.
7. E. Deelman, G. Singh, M. Su, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, G. B. Berriman, J. Good, A. Laity, J. C. Jacob, and D. S. Katz, "Pegasus: A framework for mapping complex scientific workflows onto distributed systems, " Scientific Programming, vol. 13, no. 3, pp. 219-237, 2005.
8. E. Deelman, K. Vahi, G. Juve, M. Rynge, S. Callaghan, P. Maechling, R. Mayani, W. Chen, R. Ferreira da Silva, M. Livny, and K. Wenger, "Pegasus, a workflow management system for science automation, " Future Generation Computer Systems, p. to appear, 2014.
9. T. Fahringer, A. Jugravu, S. Pllana, R. Prodan, C. Seragiotto, Jr., and H.-L. Truong, "Askalon: a tool set for cluster and grid computing: Research articles, " Concurr. Comput. : Pract. Exper., vol. 17, no. 2-4, pp. 143-169, 2005.
10. T. Oinn and et. al., "Taverna: lessons in creating a workflow environment for the life sciences: Research articles, " Concurr. Comput. : Pract. Exper., vol. 18, no. 10, pp. 1067-1100, 2006.
11. M. Abouelhoda, S. Issa, and M. Ghanem, "Tavaxy: Integrating taverna and galaxy workflows with cloud computing support, " BMC Bioinformatics, vol. 13, no. 1, p. 77, 2012.
12. W. Chen and E. Deelman, "Workflow overhead analysis and optimizations, " in 6th Workshop on Workflows in Support of Large-scale Science (WORKS), 2011.
13. P. Couvares, T. Kosar, A. Roy, J. Weber, and K. Wenger, "Workflow in condor, " in Workflows for e-Science, I. Taylor, E. Deelman, D. Gannon, and M. Shields, Eds. Springer Press, 2007.
14. G. B. Berriman, E. Deelman, J. C. Good, J. C. Jacob, D. S. Katz, C. Kesselman, A. C. Laity, T. A. Prince, G. Singh, and M. Su, "Montage: a grid-enabled engine for delivering custom science-grade mosaics on demand, " in SPIE Conference on Astronomical Telescopes and Instrumentation, vol. 5493, 2004, pp. 221-232.
15. R. Graves, T. Jordan, S. Callaghan, E. Deelman, E. Field, G. Juve, C. Kesselman, P. Maechling, G. Mehta, K. Milner, D. Okaya, P. Small, and K. Vahi, "CyberShake: A Physics-Based Seismic Hazard Model for Southern California, " Pure and Applied Geophysics, vol. 168, no. 3-4, pp. 367-381, 2011.
16. G. Juve, A. Chervenak, E. Deelman, S. Bharathi, G. Mehta, and K. Vahi, "Characterizing and profiling scientific workflows, " Future Generation Computer Systems, vol. 29, no. 3, pp. 682-692, 2013.
17. "Workflow Generator, " https://confluence.pegasus.isi.edu/display/pegasus/WorkflowGenerator.
18. "Workflow Archive, " http://www.workflowarchive.org.
19. R. Ferreira da Silva, W. Chen, G. Juve, K. Vahi, and E. Deelman, "Community resources for enabling and evaluating research on scientific workflows, " in 10th IEEE International Conference on e-Science, ser. eScience'14, 2014, p. to appear.
20. R. N. Calheiros, R. Ranjan, A. Beloglazov, C. A. F. De Rose, and R. Buyya, "CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms, " Software: Practice and Experience, vol. 41, no. 1, pp. 23-50, 2011.