Agile parallel bioinformatics workflow management using Pwrake

BMC Research Notes

Volumn 4, Issue , 2011, Pages

  Mishima, Hiroyuki ^a,b   Sasaki, Kensaku ^a,b   Tanaka, Masahiro ^c,d   Tatebe, Osamu ^c,d   Yoshiura, Koh Ichiro ^a  

^a NAGASAKI UNIVERSITY GRADUATE SCHOOL OF BIOMEDICAL SCIENCES   (Japan)

^b NAGASAKI UNIVERSITY   (Japan)

^c UNIVERSITY OF TSUKUBA   (Japan)

^d JAPAN SCIENCE AND TECHNOLOGY AGENCY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 80052593501     PISSN: None     EISSN: 17560500     Source Type: Journal    
DOI: 10.1186/1756-0500-4-331     Document Type: Article

References (46)

1. Workflow patterns. Van der Aalst WMP, Ter Hofstede AHM, Kiepuszewski B, Barros AP, Distrib Parallel Dat 2003 14 5 51 10.1023/A:1022883727209
2. Scientific Workflow Systems for 21st Century, New Bottle or New Wine? Zhao Y, Raicu I, Foster I, 2008 IEEE Congress on Services-Part I Honolulu, HI, USA 2008 467 471
3. Biopipe: A Flexible Framework for Protocol-Based Bioinformatics Analysis. Hoon S, Ratnapu KK, J-ming Chia, Kumarasamy B, Juguang X, Clamp M, Stabenau A, Potter S, Clarke L, Stupka E, Genome Res 2003 13 1904 1915 12869579 (Pubitemid 36987197)
4. Mapping complex scientific workflows onto distributed systems. Deelman E, Blythe J, Gil Y, Baker C, Mehta G, Vahi K, Blackburn K, Lazzarini A, Arbree A, Cavanaugh R, J Grid Comp 2003 1 25 39 10.1023/A:1024000426962
5. Taming heterogeneity-the Ptolemy approach. Eker J, Janneck JW, Lee EA, Liu J, Liu X, Lidvig J, Neuendorffer S, Sachs S, Xiong Y, Proc IEEE 2003 91 127 144 10.1109/JPROC.2002.805829
6. Taverna: a tool for the composition and enactment of bioinformatics workflows. Oinn T, Addis M, Ferris J, Marvin D, Senger M, Greenwood M, Carver T, Glover K, Pocock MR, Wipat A, Li P, Bioinformatics 2004 20 3045 3054 10.1093/bioinformatics/bth361 15201187 (Pubitemid 39619196)
7. Pegasys: software for executing and integrating analyses of biological sequences. Shah S, He D, Sawkins J, Druce J, Quon G, Lett D, Zheng G, Xu T, Ouellette BF, BMC Bioinformatics 2004 5 40 10.1186/1471-2105-5-40 15096276 (Pubitemid 38751296)
8. Scientific workflow management and the Kepler system. Ludäscher B, Altintas I, Berkley C, Higgins D, Jaeger E, Jones M, Lee EA, Tao J, Zhao Y, Concurrency Computat Pract Exper 2006 18 1039 1065 10.1002/cpe.994 (Pubitemid 44258497)
9. Programming scientific and distributed workflow with Triana services. Churches D, Gombas G, Harrison A, Maassen J, Robinson C, Shields M, Taylor I, Wang I, Concurrency Computat Pract Exper 2006 18 1021 1037 10.1002/cpe.992 (Pubitemid 44258496)
10. Biowep: a workflow enactment portal for bioinformatics applications. Romano P, Bartocci E, Bertolini G, De Paoli F, Marra D, Mauri G, Merelli E, Milanesi L, BMC Bioinformatics 2007 8 19 18047718
11. Swift: Fast, reliable, loosely coupled parallel computation. Zhao Y, Hategan M, Clifford B, Foster I, Von Laszewski G, Nefedova V, Raicu I, Stef-Praun T, Wilde M, Proceedings-2007 IEEE Congress on Services, SERVICES 2007 2007 199 206
12. BioWMS: a web-based Workflow Management System for bioinformatics. Bartocci E, Corradini F, Merelli E, Scortichini L, BMC Bioinformatics 2007 8 2 18269696
13. High-throughput bioinformatics with the Cyrille2 pipeline system. Fiers M, van der Burgt A, Datema E, de Groot J, van Ham R, BMC Bioinformatics 2008 9 96 10.1186/1471-2105-9-96 18269742 (Pubitemid 351420404)
14. KNIME-The Konstanz Information Miner. Berthold MR, Cebron N, Dill F, Gabriel TR, Kotter T, Meinl T, Thiel K, Wiswedel B, SIGKDD Explorations 2009 11 26 31 10.1145/1656274.1656280
15. Ergatis: a web interface and scalable software system for bioinformatics workflows. Orvis J, Crabtree J, Galens K, Gussman A, Inman JM, Lee E, Nampally S, Riley D, Sundaram JP, Felix V, Whitty B, Mahurkar A, Wortman J, White O, Angiuoli SV, Bioinformatics 2010 26 1488 1492 10.1093/bioinformatics/btq167 20413634
16. Galaxy: a comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life sciences. Goecks J, Nekrutenko A, Taylor J, Galaxy Team T, Genome Biol 2010 11 86 10.1186/gb-2010-11-8-r86 20738864
17. Adapters, shims, and glue - service interoperability for in silico experiments. Radetzki U, Leser U, Schulze-Rauschenbach SC, Zimmermann J, Lüssem J, Bode T, Cremers AB, Bioinformatics 2006 22 1137 1143 10.1093/bioinformatics/btl054 16481335
18. A Task Abstraction and Mapping Approach to the Shimming Problem in Scientific Workflows. Lin C, Lu S, Fei X, Pai D, Hua J, Services Computing, IEEE International Conference on Los Alamitos, CA, USA: IEEE Computer Society 2009 0 284 291
19. Agile methods in biomedical software development: a multi-site experience report. Kane D, Hohman M, Cerami E, McCormick M, Kuhlmman K, Byrd J, BMC Bioinformatics 2006 7 273 10.1186/1471-2105-7-273 16734914 (Pubitemid 46277087)
20. Grid Explorer: A Tool for Discovering, Selecting, and Using Distributed Resources Efficiently. Taura K, IPSJ SIG Technical Report 2004 2004-HPC-099 235 240
21. Python Programming Language. http://www.python.org/
22. Ruby Programming Language. http://www.ruby-lang.org/
23. Ruffus: a lightweight Python library for computational pipelines. Goodstadt L, Bioinformatics 2010 26 2778 2779 10.1093/bioinformatics/btq524 20847218
24. A lightweight, flow-based toolkit for parallel and distributed bioinformatics pipelines. Cieslik M, Mura C, BMC Bioinformatics 2011 12 61 10.1186/1471-2105-12-61 21352538
25. A little language for surveys: Constructing an internal DSL in Ruby. Cunningham HC, Proceedings of the 46th Annual Southeast Regional Conference on XX, ACM-SE 46 2008 282 287
26. Rake. http://rake.rubyforge.org/
27. Pwrake. https://github.com/masa16/pwrake
28. Pwrake: a parallel and distributed flexible workflow management tool for wide-area data intensive computing. Tanaka M, Tatebe O, Proceedings of the 19th ACM International Symposium on High Performance Distributed Computing New York, NY, USA: ACM 2010 356 359
29. Graphviz. http://graphviz.org/
30. Gfarm Grid File System. Tatebe O, Hiraga K, New Generat Comput 2010 28 257 275 10.1007/s00354-009-0089-5
31. The Genome Analysis Toolkit: A MapReduce framework for analyzing next-generation DNA sequencing data. McKenna A, Hanna M, Banks E, Sivachenko A, Cibulskis K, Kernytsky A, Garimella K, Altshuler D, Gabriel S, Daly M, DePristo MA, Genome Res 2010 20 1297 1303 10.1101/gr.107524.110 20644199
32. A framework for variation discovery and genotyping using next-generation DNA sequencing data. DePristo MA, Banks E, Poplin R, Garimella KV, Maguire JR, Hartl C, Philippakis AA, del Angel G, Rivas MA, Hanna M, McKenna A, Fennell TJ, Kernytsky AM, Sivachenko AY, Cibulskis K, Gabriel SB, Altshuler D, Daly MJ, Nat Genet 2011
33. Dindel: Accurate indel calls from short-read data. Albers CA, Lunter G, MacArthur DG, McVean G, Ouwehand WH, Durbin R, Genome Res 2010
34. A map of human genome variation from population-scale sequencing. The 1000 Genomes Project Consortium, Nature 2010 467 1061 1073 10.1038/nature09534 20981092
35. Whole-genome sequencing and comprehensive variant analysis of a Japanese individual using massively parallel sequencing. Fujimoto A, Nakagawa H, Hosono N, Nakano K, Abe T, Boroevich KA, Nagasaki M, Yamaguchi R, Shibuya T, Kubo M, Miyano S, Nakamura Y, Tsunoda T, Nat Genet 2010 42 931 936 10.1038/ng.691 20972442
36. A haplotype map of the human genome. The International HapMap Consortium, Nature 2005 437 1299 1320 10.1038/nature04226 16255080 (Pubitemid 41568671)
37. Fast and accurate short read alignment with Burrows-Wheeler transform. Li H, Durbin R, Bioinformatics 2009 25 1754 1760 10.1093/bioinformatics/btp324 19451168
38. The Sequence Alignment/Map format and SAMtools. Li H, Handsaker B, Wysoker A, Fennell T, Ruan J, Homer N, Marth G, Abecasis G, Durbin R, 1000 Genome Project Data Processing Subgroup, Bioinformatics 2009 25 2078 2079 10.1093/bioinformatics/btp352 19505943
39. Picard. http://picard.sourceforge.net/
40. A high-performance, portable implementation of the MPI message passing interface standard. Gropp W, Lusk E, Doss N, Skjellum A, Parallel Comput 1996 22 789 828 10.1016/0167-8191(96)00024-5 (Pubitemid 126364517)
41. Application of the Linux cluster for exhaustive window haplotype analysis using the FBAT and Unphased programs. Mishima H, Lidral AC, Ni J, BMC Bioinformatics 2008 9 Suppl 6 10 10.1186/1471-2105-9-S6-S10 19091009 (Pubitemid 351984732)
42. MapReduce: simplified data processing on large clusters. Dean J, Ghemawat S, Commun ACM 2008 51 107 113
43. An introduction to scripting in Ruby for biologists. Aerts J, Law A, BMC Bioinformatics 2009 10 221 10.1186/1471-2105-10-221 19607723
44. BioRuby: Bioinformatics software for the Ruby programming language. Goto N, Prins P, Nakao M, Bonnal R, Aerts J, Katayama T, Bioinformatics 2010 tq475
45. myExperiment: a repository and social network for the sharing of bioinformatics workflows. Goble CA, Bhagat J, Aleksejevs S, Cruickshank D, Michaelides D, Newman D, Borkum M, Bechhofer S, Roos M, Li P, De Roure D, Nucleic Acids Res 2010 38 677 W682 10.1093/nar/gkq429 20501605
46. Wf4ever. http://www.wf4ever-project.org/