Data parallelism in bioinformatics workflows using Hydra

HPDC 2010 - Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing

Volumn , Issue , 2010, Pages 507-515

  Coutinho, Fábio ^a   Ogasawara, Eduardo ^a   De Oliveira, Daniel ^a   Braganholo, Vanessa ^a   Lima, Alexandre A B ^a   Dávila, Alberto M R ^b   Mattoso, Marta ^a  

^a FEDERAL UNIVERSITY OF RIO DE JANEIRO   (Brazil)

^b INSTITUTO OSWALDO CRUZ   (Brazil)

Author keywords

Bioinformatics; Provenance; Scientific workflows

Indexed keywords

BIOINFORMATICS WORKFLOWS; CONTROL AND MONITOR; DATA FLOW; DATA PARALLELISM; DISTRIBUTED ENVIRONMENTS; HIGH-PERFORMANCE COMPUTING; LARGE DATASETS; OVERALL EXECUTION; PARALLEL EXECUTIONS; PERFORMANCE GAIN; PROVENANCE; SCIENTIFIC WORKFLOWS; WORK-FLOWS; WORKFLOW EXECUTION;

DISTRIBUTED COMPUTER SYSTEMS; MANAGEMENT; MIDDLEWARE; WORK SIMPLIFICATION;

BIOINFORMATICS;

EID: 78649981522     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1851476.1851550     Document Type: Conference Paper

References (36)

1. G.H. Travassos e M.O. Barros, 2003, Contributions of In Virtuo and In Silico Experiments for the Future of Empirical Studies in Software Engineering, In: Proc. of 2nd Workshop on Empirical Software Engineering the Future of Empirical Studies in Software Engineering, Roma
2. M. Zvelebil e J. Baum, 2007, Understanding Bioinformatics. 1 ed. Garland Science.
3. R.D. Stevens, H.J. Tipney, C.J. Wroe, T.M. Oinn, M. Senger, P.W. Lord, C.A. Goble, A. Brass, e M. Tassabehji, 2004, Exploring Williams-Beuren syndrome using myGrid, Bioinformatics, v. 20, n. suppl. 1 (Ago.), p. i303-310.
4. S.M.S.D. Cruz, V. Batista, A.M.R. Dávila, E. Silva, F. Tosta, C. Vilela, M.L.M. Campos, R. Cuadrat, D. Tschoeke, et al., 2008, OrthoSearch: a scientific workflow approach to detect distant homologies on protozoans, In: Proceedings of the 2008 ACM symposium on Applied computing, p. 1282-1286, Fortaleza, Ceara, Brazil.
5. A.M.R. Dávila, P.N. Mendes, G. Wagner, D.A. Tschoeke, R.R.C. Cuadrat, F. Liberman, L. Matos, T. Satake, K.A.C.S. Ocaña, et al., 2008, ProtozoaDB: dynamic visualization and exploration of protozoan genomes, Nucleic Acids Research, v. 36, n. Database issue (Jan.), p. D547-D552.
6. I. Altintas, C. Berkley, E. Jaeger, M. Jones, B. Ludascher, e S. Mock, 2004, Kepler: an extensible system for design and execution of scientific workflows, In: SSDBM, p. 423-424, Greece.
7. D. Hull, K. Wolstencroft, R. Stevens, C. Goble, M.R. Pocock, P. Li, e T. Oinn, 2006, Taverna: a tool for building and running workflows of services, Nucleic Acids Research, v. 34, n. Web Server issue, p. 729-732.
8. S.P. Callahan, J. Freire, E. Santos, C.E. Scheidegger, C.T. Silva, e H.T. Vo, 2006, VisTrails: visualization meets data management, In: Proc. SIGMOD 2006, p. 745-747, USA.
9. E. Deelman, G. Mehta, G. Singh, M. Su, e K. Vahi, 2007, "Pegasus: Mapping Large-Scale Workflows to Distributed Resources", Workflows for e-Science, Springer, p. 376-394.
10. I. Taylor, M. Shields, I. Wang, e A. Harrison, 2007, "The Triana Workflow Environment: Architecture and Applications", Workflows for e-Science, Springer, p. 320-339.
11. Y. Zhao, M. Hategan, B. Clifford, I. Foster, G. von Laszewski, V. Nefedova, I. Raicu, T. Stef-Praun, e M. Wilde, 2007, Swift: Fast, Reliable, Loosely Coupled Parallel Computation, In: Services 2007, p. 206, 199
12. C. Lin, S. Lu, X. Fei, D. Pai, e J. Hua, 2009, A Task Abstraction and Mapping Approach to the Shimming Problem in Scientific Workflows, In: Proc. Services 2009, p. 284-291
13. L.A. Meyer, S.C. Rössle, P.M. Bisch, e M. Mattoso, 2005, "Parallelism in Bioinformatics Workflows", High Performance Computing for Computational Science-VECPAR 2004, p. 583-597.
14. L. Meyer, D. Scheftner, J. Vöckler, M. Mattoso, M. Wilde, e I. Foster, 2007, "An Opportunistic Algorithm for Scheduling Workflows on Grids", High Performance Computing for Computational Science-VECPAR 2006, p. 1-12.
15. I. Raicu, I. Foster, e Yong Zhao, 2008, Many-task computing for grids and supercomputers, In: Workshop on Many-Task Computing on Grids and Supercomputers, p. 1-11
16. J. Freire, D. Koop, E. Santos, e C.T. Silva, 2008, Provenance for Computational Tasks: A Survey, Computing in Science and Engineering, v.10, n. 3, p. 11-21.
17. E. Ogasawara, D. Oliveira, F. Chirigati, C.E. Barbosa, R. Elias, V. Braganholo, A. Coutinho, e M. Mattoso, 2009, Exploring many task computing in scientific workflows, In: MTAGS 09, p. 1-10, Portland, Oregon.
18. A. Bayucan, R.L. Henderson, e J.P. Jones, 2000, Portable Batch System Administration Guide, Veridian System
19. G. Guerra, F. Rochinha, R. Elias, A. Coutinho, V. Braganholo, D.D. Oliveira, E. Ogasawara, F. Chirigati, e M. Mattoso, 2009, Scientific Workflow Management System Applied to Uncertainty Quantification in Large Eddy Simulation, In: Congresso Ibero Americano de Métodos Computacionais em Engenharia, Bú zios, Rio de Janeiro, Brazil.
20. C. Szyperski, 1997, Component Software: Beyond Object-Oriented Programming. Addison-Wesley Professional.
21. E. Walker e C. Guiang, 2007, Challenges in executing large parameter sweep studies across widely distributed computing environments, In: Workshop on Challenges of large applications in distributed environments, p. 11-18, Monterey, California, USA.
22. M. Wieczorek, R. Prodan, e T. Fahringer, 2005, Scheduling of scientific workflows in the ASKALON grid environment, SIGMOD Rec., v. 34, n. 3, p. 56-62.
23. J. Dean e S. Ghemawat, 2008, MapReduce: simplified data processing on large clusters, Commun. ACM, v. 51, n. 1, p. 107-113.
24. X. Fei, S. Lu, e C. Lin, 2009, A MapReduce-Enabled Scientific Workflow Composition Framework, In: ICWS, p. 663-670, Los Alamitos, CA, USA.
25. I. Raicu, Y. Zhao, C. Dumitrescu, I. Foster, e M. Wilde, 2007, Falkon: a Fast and Light-weight tasK executiON framework, In: SC07, p. 1-12, Reno, Nevada.
26. A. Matsunaga, M. Tsugawa, e J. Fortes, 2008, CloudBLAST: Combining MapReduce and Virtualization on Distributed Resources for Bioinformatics Applications, IEEE eScience 2008, v. 0, p. 229, 222.
27. D. Abramson, C. Enticott, e I. Altinas, 2008, Nimrod/K: towards massively parallel dynamic grid workflows, In: SC 08, p. 1-11, Austin, Texas.
28. C. Lin, S. Lu, Z. Lai, A. Chebotko, X. Fei, J. Hua, e F. Fotouhi, 2008, Service-Oriented Architecture for VIEW: A Visual Scientific Workflow Management System, In: Services, p. 335-342
29. D. Oliveira, F. Baião, e M. Mattoso, 2010, "Towards a Taxonomy for Cloud Computing from an e-Science Perspective", Cloud Computing: Principles, Systems and Applications (to be published), Heidelberg: Springer-Verlag
30. C. Yang, T. Han, e H. Kan, 2009, G-BLAST: a Grid-based solution for mpiBLAST on computational Grids, Concurr. Comput. : Pract. Exper., v. 21, n. 2, p. 225-255.
31. P. Balaji, W. Feng, J. Archuleta, H. Lin, R. Kettimuthu, R. Thakur, e X. Ma, 2008, Semantics-based distributed I/O for mpiBLAST, In: Proceedings of the 13th ACM SIGPLAN Symposium on Principles and practice of parallel programming, p. 293-294, Salt Lake City, UT, USA.
32. M. Cargnelli, G. Alleon, e F. Cappello, 2008, OpenWP: Combining annotation language and workflow environments for porting existing applications on grids, In: Proceedings of the 2008 9th IEEE/ACM International Conference on Grid Computing, p. 176-183
33. I. Foster e C. Kesselman, 2004, The Grid: Blueprint for a New Computing Infrastructure. Morgan Kaufmann.
34. P. Fusaro, G. Visaggio, e M. Tortorella, 1998, REP-ChaRacterizing and Exploiting Process Components: Results of Experimentation, In: Proceedings of the Working Conference on Reverse Engineering (WCRE'98), p. 20-29
35. A.A.B. Lima, M. Mattoso, e P. Valduriez, 2004, Adaptive Virtual Partitioning for OLAP Query Processing in a Database Cluster, Proc 19th SBBD, Brasília, Brazil, p. 92-105.
36. C. Furtado, A. Lima, E. Pacitti, P. Valduriez, e M. Mattoso, 2005, Physical and virtual partitioning in OLAP database clusters, In: 17th International Symposium on Computer Architecture and High Performance Computing, 2005. SBAC-PAD 2005., p. 143-150