Performance evaluation of parallel strategies in public clouds: A study with phylogenomic workflows

Future Generation Computer Systems

Volumn 29, Issue 7, 2013, Pages 1816-1825

  De Oliveira, Daniel ^a   Ocaña, Kary A C S ^a   Ogasawara, Eduardo ^a,b   Dias, Jonas ^a   Gonçalves, João ^a   Baião, Fernanda ^c   Mattoso, Marta ^a  

^a FEDERAL UNIVERSITY OF RIO DE JANEIRO   (Brazil)

^b CENTRO FEDERAL DE EDUCAÇÃO TECNOLÓGICA CELSO SUCKOW DA FONSECA   (Brazil)

^c STATE UNIVERSITY OF RIO DE JANEIRO   (Brazil)

Author keywords

Cloud computing; Hadoop; MapReduce; Phylogenomics; Scientific workflows

Indexed keywords

BIOLOGY; CLOUD COMPUTING; MULTIPROCESSING SYSTEMS;

EVOLUTIONARY RELATIONSHIPS; HADOOP; HIGH PERFORMANCE COMPUTING (HPC); MAP-REDUCE; PARALLEL PROCESSING; PERFORMANCE CAPABILITY; PHYLOGENOMICS; SCIENTIFIC WORKFLOWS;

TREES (MATHEMATICS);

EID: 84893817402     PISSN: 0167739X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.future.2012.12.019     Document Type: Article

References (63)

1. M. Zvelebil, J. Baum, Understanding Bioinformatics, first ed., Garland Science, 2007.
2. S. Matthews, T. Williams, MrsRF: an efficient MapReduce algorithm for analyzing large collections of evolutionary trees, BMC Bioinformatics 11 (2010) S15.
3. M. Mattoso, C. Werner, G.H. Travassos, V. Braganholo, L. Murta, E. Ogasawara, D. Oliveira, S.M.S. da Cruz, W. Martinho, Towards supporting the life cycle of large-scale scientific experiments, International Journal of Business Process Integration and Management 5 (1) (2010) 79-92.
4. E. Deelman, D. Gannon, M. Shields, I. Taylor, Workflows and e-Science: an overview of workflow system features and capabilities, Future Generation Computer Systems 25 (5) (2009) 528-540.
5. E. Walker, C. Guiang, Challenges in executing large parameter sweep studies across widely distributed computing environments, in: Workshop on Challengesof Large Applications in Distributed Environments, Monterey, California, USA, 2007, pp. 11-18.
6. S.N. Srirama, P. Jakovits, E. Vainikko, Adapting scientific computing problems to clouds using MapReduce, Future Generation Computer Systems 28 (1) (2012) 184-192.
7. Y. Gil, E. Deelman, M. Ellisman, T. Fahringer, G. Fox, D. Gannon, C. Goble, M. Livny, L. Moreau, et al., Examining the challenges of scientific workflows, Computer 40 (12) (2007) 24-32.
8. S.B. Davidson, J. Freire, Provenance and scientific workflows: challenges and opportunities, in: ACM SIGMOD International Conference on Management of Data, Vancouver, Canada. 2008, pp. 1345-1350.
9. J. Freire, D. Koop, E. Santos, C.T. Silva, Provenance for computational tasks: a survey, Computing in Science and Engineering 10 (3) (2008) 11-21. (Pubitemid 351559435)
10. P. Buneman, S. Khanna, W. Tan, Why and where: a characterization of data provenance, In: International Conference on Database Theory, 2001, pp. 316-330.
11. P. Buneman, W.-C. Tan, Provenance in databases, in: ACM SIGMOD International Conference on Management of Data, New York, NY, USA, 2007, pp. 1171-1173.
12. D. Oliveira, F. Baião, M. Mattoso, Towards a taxonomy for cloud computing from an e-Science perspective, in: Nick Antonopoulos, Lee Gillam (Eds.), Cloud Computing: Principles, Systems and Applications, Springer-Verlag, Heidelberg, 2010, pp. 47-62.
13. G. Mateescu, W. Gentzsch, C.J. Ribbens, Hybrid computing - where HPC meets grid and cloud computing, Future Generation Computer Systems 27 (5) (2011) 440-453.
14. V. Mauch, M. Kunze, M. Hillenbrand, High performance cloud computing, Future Generation Computer Systems (2012) http://dx.doi.org/10.1016/j.future. 2012.03.011.
15. K.R. Jackson, L. Ramakrishnan, K. Muriki, S. Canon, S. Cholia, J. Shalf, H.J. Wasserman, N.J. Wright, Performance analysis of high performance computing applications on the amazon web services cloud, in: IEEE Second International Conference on Cloud Computing Technology and Science, Washington, DC, USA, 2010, pp. 159-168.
16. T. Hey, S. Tansley, K. Tolle, The Fourth Paradigm: Data-Intensive Scientific Discovery, Microsoft Research, 2009.
17. Amazon EC2, Amazon Elastic Compute Cloud, (Amazon EC2) 2010. http://aws.amazon.com/ec2/.
18. D. Oliveira, E. Ogasawara, F. Baião, M. Mattoso, SciCumulus: a lightweight cloud middleware to explore many task computing paradigm in scientific workflows, in: 3rd International Conference on Cloud Computing, Washington, DC, USA, 2010, pp. 378-385.
19. Hadoop, Apache Hadoop Web Page, 2012. http://hadoop.apache.org/.
20. J. Dean, S. Ghemawat, MapReduce: a flexible data processing tool, Communications of the ACM 53 (2010) 72-77.
21. A. Matsunaga, M. Tsugawa, J. Fortes, CloudBLAST: combining MapReduce and virtualization on distributed resources for bioinformatics applications, in: IEEE eScience 2008, 2008, pp. 222-229.
22. A.L. Hartman, S. Riddle, T. McPhillips, B. Ludäscher, J.A. Eisen, Introducing W.A.T.E.R.S.: a workflow for the alignment, taxonomy, and ecology of ribosomal sequences, BMC Bioinformatics 11 (1) (2010) 317.
23. M. Wu, A.J. Scott, Phylogenomic analysis of bacterial and archaeal sequences with AMPHORA2, Bioinformatics 28 (7) (2012) 1033-1034.
24. E. Lord, M. Leclercq, A. Boc, A.B. Diallo, V. Makarenkov, Armadillo 1.1: an original workflow platform for designing and conducting phylogenetic analysis and simulations, PLoS One 7 (1) (2012) e29903.
25. A. Moustafa, D. Bhattacharya, A.E. Allen, iTree: a high-throughput phylogenomic pipeline, in: 5th Cairo International Biomedical Engineering Conference (CIBEC), Cairo, Egypt, 2010, pp. 103-107.
26. Z. Liu, S.B. Davidson, Y. Chen, Generating sound workflow views for correct provenance analysis, ACM Transactions on Database Systems 36 (1) (2011) 1-35.
27. O. Biton, S.B. Davidson, S. Khanna, S. Roy, Optimizing user views for workflows, in: 12th International Conference on Database Theory, Saint-Petersburg, Russia, 2009, p. 310.
28. O. Biton, S. Cohen-Boulakia, S.B. Davidson, C.S. Hara, Querying and managing provenance through user views in scientific workflows, in: IEEE 24th International Conference on Data Engineering, Cancun, Mexico, 2008, pp. 1072-1081.
29. X. Fei, S. Lu, C. Lin, A MapReduce-enabled scientific workflow composition framework, in: ICWS, Los Alamitos, CA, USA, 2009, pp. 663-670.
30. F. Coutinho, E. Ogasawara, D. de Oliveira, V. Braganholo, A.A.B. Lima, A.M.R. Dávila, M. Mattoso, Data parallelism in bioinformatics workflows using hydra, in: 19th ACM International Symposium on High Performance Distributed Computing, New York, NY, USA, 2010, pp. 507-515.
31. S.P. Callahan, J. Freire, E. Santos, C.E. Scheidegger, C.T. Silva, H.T. Vo, VisTrails: visualization meets data management, in: SIGMOD International Conference on Management of Data, Chicago, Illinois, USA, 2006, pp. 745-747.
32. I. Altintas, C. Berkley, E. Jaeger, M. Jones, B. Ludascher, S. Mock, Kepler: an extensible system for design and execution of scientific workflows, in: Scientific and Statistical Database Management, Greece, 2004, pp. 423-424.
33. D. Oliveira, E. Ogasawara, K. Ocana, F. Baiao, M. Mattoso, An adaptive parallel execution strategy for cloud-based scientific workflows, Concurrency and Computation: Practice and Experience 24 (2012) 1531-1550.
34. D. Oliveira, K. Ocaña, F. Baião, M. Mattoso, A provenance-based adaptive scheduling heuristic for parallel scientific workflows in clouds, Journal of Grid Computing 10 (3) (2012) 521-552.
35. F. Delsuc, H. Brinkmann, H. Philippe, Phylogenomics and the reconstruction of the tree of life, Nature Reviews Genetics 6 (5) (2005) 361-375. (Pubitemid 40577180)
36. J.A. Eisen, Phylogenomics: intersection of evolution and genomics, Science 300 (5626) (2003) 1706-1707. (Pubitemid 36712558)
37. K.M. Kocot, J.T. Cannon, C. Todt, M.R. Citarella, A.B. Kohn, A. Meyer, S.R. Santos, C. Schander, L.L. Moroz, et al., Phylogenomics reveals deep molluscan relationships, Nature 477 (7365) (2011) 452-456.
38. W. Miller, K.D. Makova, A. Nekrutenko, R.C. Hardison, Comparative genomics, Annual Review of Genomics and Human Genetics 5 (1) (2004) 15-56. (Pubitemid 39362740)
39. A.G. Clark, Genomics of the evolutionary process, Trends in Ecology and Evolution 21 (6) (2006) 316-321. (Pubitemid 43850061)
40. K.A.C.S. Ocaña, D. Oliveira, E. Ogasawara, A.M.R. Dávila, A.A.B. Lima, M. Mattoso, SciPhy: a cloud-based workflow for phylogenetic analysis of drug targets in protozoan genomes, in: O. Norberto de Souza, G.P. Telles, e M. Palakal (Eds.), Advances in Bioinformatics and Computational Biology, Springer, Berlin, Heidelberg, 2011, pp. 66-70.
41. K. Katoh, H. Toh, Parallelization of the MAFFT multiple sequence alignment program, Bioinformatics (Oxford, England) 26 (15) (2010) 1899-1900.
42. T. Lassmann, O. Frings, E.L.L. Sonnhammer, Kalign2: high-performance multiple alignment of protein and nucleotide sequences allowing external features, Nucleic Acids Research 37 (2008) 858-865.
43. M.A. Larkin, G. Blackshields, N.P. Brown, R. Chenna, P.A. McGettigan, H. McWilliam, F. Valentin, I.M. Wallace, A. Wilm, et al., Clustal W and clustal X version 2.0, Bioinformatics (Oxford, England) 23 (21) (2007) 2947-2948. (Pubitemid 350162903)
44. C.B. Do, M.S.P. Mahabhashyam, M. Brudno, S. Batzoglou, ProbCons: probabilistic consistency-based multiple sequence alignment, Genome Research 15 (2) (2005) 330-340. (Pubitemid 40309398)
45. D. Gilbert, Sequence file format conversion with command-line readseq, in: Andreas D. Baxevanis (Ed.), Current Protocols in Bioinformatics, Editoral Board, 2003, Appendix 1, p. 1E.
46. T.M. Keane, C.J. Creevey, M.M. Pentony, T.J. Naughton, J.O. Mclnerney, Assessment of methods for amino acid matrix selection and their use on empirical data shows that ad hoc assumptions for choice of matrix are not justified, BMC Evolutionary Biology 6 (2006) 29.
47. A. Stamatakis, RAxML-VI-HPC: maximum likelihood-based phylogenetic analyses with thousands of taxa and mixed models, Bioinformatics (Oxford, England) 22 (21) (2006) 2688-2690. (Pubitemid 44642609)
48. B.E. Dutilh, V. van Noort, R.T.J.M. van der Heijden, T. Boekhout, B. Snel, M.A. Huynen, Assessment of phylogenomic and orthology approaches for phylogenetic inference, Bioinformatics (Oxford, England) 23 (7) (2007) 815-824. (Pubitemid 47049915)
49. R.D.M. Page Andreas D. Baxevanis (Ed.), Visualizing phylogenetic trees using treeview, in: Current Protocols in Bioinformatics, Editoral Board, 2002, Chapter 6, p. Unit 6.2.
50. J. Felsenstein, PHYLIP - phylogeny inference package (version 3.2), Cladistics 5 (1989) 164-166.
51. E. Ogasawara, J. Dias, D. Oliveira, F. Porto, P. Valduriez, M. Mattoso, An algebraic approach for data-centric scientific workflows, Proceedings of the VLDB Endowment 4 (12) (2011) 1328-1339.
52. B. Carpenter, V. Getov, G. Judd, A. Skjellum, G. Fox, MPJ: MPI-like message passing for java, Concurrency: Practice and Experience 12 (11) (2000) 1019-1038. (Pubitemid 32032847)
53. K.D. Pruitt, T. Tatusova, W. Klimke, D.R. Maglott, NCBI reference sequences: current status, policy and new initiatives, Nucleic Acids Research 37 (Database issue) (2009) D32-D36.
54. D. Vega, D. Oliveira, E. Ogasawara, A.A.B. Lima, M. Mattoso, Modeling parallel bioinformatics workflows using MapReduce, in: International Workshop on Challenges in e-Science - SBAC, Petrópolis, RJ - Brazil, 2010, pp. 71-72.
55. M. Ding, L. Zheng, Y. Lu, L. Li, S. Guo, M. Guo, More convenient more overhead: the performance evaluation of Hadoop streaming, in: Proceedings of the 2011 ACM Symposium on Research in Applied Computation, New York, NY, USA, 2011, pp. 307-313.
56. L.M. Vaquero, L. Rodero-Merino, J. Caceres, M. Lindner, A break in the clouds: towards a cloud definition, SIGCOMM Computer Communication Review 39 (1) (2009) 50-55.
57. A. Iosup, M. Jan, O. Sonmez, D. Epema, The characteristics and performance of groups of jobs in grids, in: A.-M. Kermarrec, L. Bougé, e T. Priol (Eds.), Euro-Par 2007 Parallel Processing, Springer, Berlin, Heidelberg, 2007, pp. 382-393.
58. I. Raicu, Z. Zhang, M. Wilde, I. Foster, P. Beckman, K. Iskra, B. Clifford, Toward loosely coupled programming on petascale systems, in: ACM/IEEE Conference on Supercomputing, Austin, Texas, 2008, pp. 1-12.
59. I. Raicu, Y. Zhao, C. Dumitrescu, I. Foster, M. Wilde, Falkon: a fast and lightweight task execution framework, in: SC07, Reno, Nevada, 2007, pp. 1-12.
60. Q. He, S. Zhou, B. Kobler, D. Duffy, T. McGlynn, Case study for running HPC applications in public clouds, in: Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing, New York, NY, USA, 2010, pp. 395-401.
61. I. Raicu, I.T. Foster, Yong Zhao, Many-task computing for grids and supercomputers, in: Proceedings of the Workshop on Many-Task Computing on Grids and Supercomputers, Austin, Texas, USA, 2008, pp. 1-11.
62. E. Ogasawara, D. Oliveira, F. Chirigati, C.E. Barbosa, R. Elias, V. Braganholo, A. Coutinho, M. Mattoso, Exploring many task computing in scientific workflows, in: Proceedings of the 2nd Workshop on Many-Task Computing on Grids and Supercomputers, MTAGS'09, Portland, Oregon, USA, 2009, pp. 1-10.
63. R. Hasan, R. Sion, M. Winslett, Introducing secure provenance: problems and challenges, in: Proceedings of the 2007 ACM Workshop on Storage Security and Survivability, Alexandria, Virginia, USA, 2007, pp. 13-18.