A lightweight, flow-based toolkit for parallel and distributed bioinformatics pipelines

BMC Bioinformatics

Volumn 12, Issue , 2011, Pages

  Cieślik, Marcin ^a   Mura, Cameron ^a  

^a UNIVERSITY OF VIRGINIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BIOINFORMATIC ANALYSIS; BIOINFORMATICS WORKFLOWS; BIOMOLECULAR SEQUENCES; DATA TRANSFORMATION; DATAFLOW PROGRAMMING; DIRECTED ACYCLIC GRAPH (DAG); DISTRIBUTED BIOINFORMATICS; FUNCTIONAL TRANSFORMATION;

BIOINFORMATICS; COMPUTER SCIENCE; GRID COMPUTING; HIGH LEVEL LANGUAGES; MESSAGE PASSING; PIPELINE PROCESSING SYSTEMS; PIPELINES;

DATA PROCESSING;

ARTICLE; BIOLOGY; COMPUTER ANALYSIS; COMPUTER LANGUAGE; COMPUTER PROGRAM; METHODOLOGY; WORKFLOW;

COMPUTATIONAL BIOLOGY; COMPUTING METHODOLOGIES; PROGRAMMING LANGUAGES; SOFTWARE; WORKFLOW;

EID: 79951957782     PISSN: None     EISSN: 14712105     Source Type: Journal    
DOI: 10.1186/1471-2105-12-61     Document Type: Article

References (41)

1. Gil A, Deelman E, Ellisman M, Fahringer T, Fox G, Goble C, Livny M, Moreau L, Myers J. Examining the Challenges of Scientific Workflows. IEEE Computer vol 2007, 40:24-32.
2. Johnston WM, Hanna JRP, Millar RJ. Advances in dataflow programming languages. ACM Comput Surv 2004, 36:1-34.
3. Ludäscher B, Altintas I, Berkley C, Higgins D, Jaeger E, Jones M, Lee EA, Tao J, Zhao Y. Scientific workflow management and the Kepler system. Concurrency and Computation: Practice & Experience 2006, 18(10):1039-1065.
4. Halling-Brown M, Shepherd AJ. Constructing Computational Pipelines. Bioinformatics, Methods in Molecular Biology™ 2008, 453:451-470. Totflowa, NJ: Humana Press, Keith JM.
5. Deelman E, Gannon D, Shields M, Taylor I. Workflows and e-Science: An overview of workflow system features and capabilities. Future Gener Comput Syst 2009, 25(5):528-540.
6. Tiwari A, Sekhar AKT. Workflow based framework for life science informatics. Comput Biol Chem 2007, 31(5-6):305-319. 10.1016/j.compbiolchem.2007.08.009, 17931570.
7. Romano P. Automation of in-silico data analysis processes through workflow management systems. Brief Bioinform 2008, 9:57-68. 10.1093/bib/bbm056, 18056132.
8. Bartocci E, Corradini F, Merelli E, Scortichini L. BioWMS: a web-based Workflow Management System for bioinformatics. BMC Bioinformatics 2007, 8(Suppl 1):S2. 10.1186/1471-2105-8-S1-S2, 1885849, 17430564.
9. Orvis J, Crabtree J, Galens K, Gussman A, Inman JM, Lee E, Nampally S, Riley D, Sundaram JP, Felix V, Whitty B, Mahurkar A, Wortman J, White O, Angiuoli SV. Ergatis: a web interface and scalable software system for bioinformatics workflows. Bioinformatics 2010, 26(12):1488-1492. 10.1093/bioinformatics/btq167, 2881353, 20413634.
10. Fiers MWEJ, van der Burgt A, Datema E, de Groot JCW, van Ham RCHJ. High-throughput bioinformatics with the Cyrille2 pipeline system. BMC Bioinformatics 2008, 9:96. 10.1186/1471-2105-9-96, 2268656, 18269742.
11. Eker J, Janneck JW, Lee EA, Liu J, Liu X, Ludvig J, Neuendorffer S, Sachs S, Xiong Y. Taming heterogeneity - the Ptolemy approach. Proceedings of the IEEE 2003, 91:127-144.
12. Oinn T, Addis M, Ferris J, Marvin D, Senger M, Green-wood M, Carver T, Glover K, Pocock MR, Wipat A, Li P. Taverna: a tool for the composition and enactment of bioinformatics workflows. Bioinformatics 2004, 20(17):3045-3054. 10.1093/bioinformatics/bth361, 15201187.
13. Berthold MR, Cebron N, Dill F, Gabriel TR, Kötter T, Meinl T, Ohl P, Thiel K, Wiswedel B. KNIME - The Konstanz Information Miner. SIGKDD Explorations 2009, 11.
14. Deelman E, Singh G, hui Su M, Blythe J, Gil A, Kessel-man C, Mehta G, Vahi K, Berriman GB, Good J, Laity A, Jacob JC, Katz DS. Pegasus: a framework for mapping complex scientific workflows onto distributed systems. Scientific Programming Journal 2005, 13:219-237.
15. Romano P, Bartocci E, Bertolini G, Paoli FD, Marra D, Mauri G, Merelli E, Milanesi L. Biflowep: a workflow enactment portal for bioinformatics applications. BMC Bioinformatics 2007, 8(Suppl 1):S19. 10.1186/1471-2105-8-S1-S19, 1885848, 17430563.
16. Hoon S, Ratnapu KK, Chia JM, Kumarasamy B, Juguang X, Clamp M, Stabenau A, Potter S, Clarke L, Stupka E. Biopipe: a flexible framework for protocol-based bioinformatics analysis. Genome Res 2003, 13(8):1904-1915. 403782, 12869579.
17. Radetzki U, Leser U, Schulze-Rauschenbach SC, Zimmer-mann J, Lüssem J, Bode T, Cremers AB. Adapters, shims, and glue-service interoperability for in silico experiments. Bioinformatics 2006, 22(9):1137-1143. 10.1093/bioinformatics/btl054, 16481335.
18. Lin C, Lu S, Fei X, Pai D, Hua J. A Task Abstraction and Mapping Approach to the Shimming Problem in Scientific Workflows. SCC '09: Proceedings of the 2009 IEEE International Conference on Services Computing 2009, 284-291. Washington, DC, USA: IEEE Computer Society.
19. Morrison JP. Flow-Based Programming: A New Approach to Application Development 2010, CreateSpace.
20. Object-relational impedance mismatch. , http://en.wikipedia.org/wiki/Object-relational_impedance_mismatch
21. Van der Aalst W. The application of Petri nets to workflow management. Journal of Circuits Systems and Computers 1998, 8:21-66.
22. Python multiprocessing interface. , http://docs.python.org/library/multiprocessing.html
23. RPyC - Remote Python Calls. , http://rpyc.wikidot.com
24. Google Labs' WorkerPool API. , http://code.google.com/apis/gears/api_workerpool.html
25. Python decorators. , http://wiki.python.org/moin/PythonDecorators
26. NumPy's View casting. , http://docs.scipy.org/doc/numpy/user/basics.subclassing.html#view-casting
27. Schatz MC. CloudBurst: highly sensitive read mapping with MapReduce. Bioinformatics 2009, 25(11):1363-1369. 10.1093/bioinformatics/btp236, 2682523, 19357099.
28. Tu T, Rendleman CA, Borhani DW, Dror RO, Gullingsrud J, Jensen MO, Klepeis JL, Maragakis P, Miller P, Stafford KA, Shaw DE. A scalable parallel framework for analyzing terascale molecular dynamics simulation trajectories. SC '08: Proceedings of the 2008 ACM/IEEE conference on Supercomputing 2008, 1-12. Piscataway, NJ, USA: IEEE Press.
29. Earl D, Deem MW. Parallel tempering: Theory, applications, and new perspectives. Physical Chemistry Chemical Physics 2005, 7(23):3910-3916. 10.1039/b509983h, 19810318.
30. Luckow A, Jha S, Kim J, Merzky A, Schnor B. Adaptive distributed replica-exchange simulations. Philos Transact A Math Phys Eng Sci 2009, 367(1897):2595-2606. 10.1098/rsta.2009.0051, 19451113.
31. Misra J. A Discipline of Multiprogramming: Programming Theory for Distributed Applications 2001, Springer.
32. Jeffay K. The real-time producer/consumer paradigm: A paradigm for the construction of efficient, predictable real-time systems. SAC '93: Proceedings of the 1993 ACM/SIGAPP symposium on Applied computing 1993, 796-804. New York, NY, USA: ACM.
33. Dean J, Ghemawat S. MapReduce: Simplified data processing on large clusters. Communications of the ACM 2008, 51:107-113.
34. Yao Z, Barrick J, Weinberg Z, Neph S, Breaker R, Tompa M, Ruzzo WL. A computational pipeline for high-throughput discovery of cisregulatory noncoding RNA in prokaryotes. PLoS Comput Biol 2007, 3(7):e126. 10.1371/journal.pcbi.0030126, 1913097, 17616982.
35. Pierce BC. Types and programming languages 2002, Cambridge, MA, USA: MIT Press.
36. Vandervalk BP, McCarthy EL, Wilkinson MD. Moby and Moby 2: creatures of the deep (web). Brief Bioinform 2009, 10(2):114-128. 10.1093/bib/bbn051, 19151099.
37. Liu P, Wu JJ, Yang CH. Locality-Preserving Dynamic Load Balancing for Data-Parallel Applications on Distributed-Memory Multiprocessors. Journal of Information Science and Engineering 2002,
38. Soto CS, Fasnacht M, Zhu J, Forrest L, Honig B. Loop modeling: Sampling, filtering, and scoring. Proteins 2008, 70(3):834-843. 10.1002/prot.21612, 2553011,2553011, 17729286.
39. Kannan S, Zacharias M. Application of biasing-potential replica-exchange simulations for loop modeling and refinement of proteins in explicit solvent. Proteins 2010,
40. Frishman D, Argos P. Knflowledge-based protein secondary structure assignment. Proteins 1995, 23(4):566-579. 10.1002/prot.340230412, 8749853.
41. Hinsen K. The molecular modeling toolkit: A new approach to molecular simulations. Journal of Computational Chemistry 2000, 21(2):79-85.