A scalable data analysis platform for metagenomics

Proceedings - 2013 IEEE International Conference on Big Data, Big Data 2013

Volumn , Issue , 2013, Pages 21-26

  Tang, Wei ^a   Wilkening, Jared ^a   Desai, Narayan ^a,b   Gerlach, Wolfgang ^a,b   Wilke, Andreas ^a,b   Meyer, Folker ^a,b  

^a ARGONNE NATIONAL LABORATORY   (United States)

^b UNIVERSITY OF CHICAGO   (United States)

Author keywords

bioinformatics; cloud computing; data analysis platform; data management system; metagenomics; workflow

Indexed keywords

BIG DATA; BIOINFORMATICS; CLOUD COMPUTING; DATA HANDLING; DNA SEQUENCES; GENE ENCODING; INFORMATION ANALYSIS; WORK SIMPLIFICATION;

BIOLOGICAL SEQUENCE DATA; COMPUTATIONAL CHALLENGES; DATA MANAGEMENT SYSTEM; METAGENOMICS; SCALABLE DATA ANALYSIS; SCIENTIFIC COMMUNITY; WORKFLOW; WORKFLOW MANAGEMENT SYSTEMS;

INFORMATION MANAGEMENT;

EID: 84893281362     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/BigData.2013.6691723     Document Type: Conference Paper

References (33)

1. AWE project. https://github.com/MG-RAST/AWE
2. Earth Microbiome Project. http://www.earthmicrobiome.org
3. FASTA format. http://en.wikipedia.org/wiki/FASTA format
4. MG-RAST website. http://metagenomics.anl.gov
5. MongoDB. http://www.mongodb.org
6. NIH Human Microbiome Project. http://www.hmpdacc.org
7. Shock project. https://github.com/MG-RAST/Shock
8. Oracle Grid Engine. http://www.oracle.com/us/products/tools/oracle- gridengine-075549.html
9. A. Barker and J. Van Hemert, "Scientific workflow: A survey and research directions," Parallel Processing and Applied Mathematics, Lecture Notes in Computer Science 4967:746-753, 2008.
10. N. Blow, "Metagenomics: Exploring unseen communities," Nature, 453:687-690, 2008.
11. P. Cock, C. Fields, N. Goto, M. Heuer, and P. Rice,"The Sanger FASTQ file format for sequences with quality scores, and the Solexa/Illumina FASTQ variants," Nucl. Acids Res., 38(6):1767-1771, 2010.
12. J. Dean and S. Ghemawat, "MapReduce: simplified data processing on large clusters," in Proc. of Sixth Symposium on Operating System Design and Implementation, 2008.
13. E. Deelman, G. Singh, M. Su, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, K. Vahi, G. Berriman, J. Good, A. Laity, J. Jacob, and D. Katz, "Pegasus: A framework for mapping complex scientific workflows onto distributed systems," Scientific Programming, 13:219-237, 2005.
14. N. Desai, D. Antonopoulos, J. Gilbert, E. Glass, and F. Meyer, "From genomics to metagenomics," Current Opinion in Biotechnology, 23:72-76, 2012.
15. R. Fielding and R. Taylor, "Principled design of the modern web architecture," ACM Transactions on Internet Technology (TOIT), 2(2):115-150, 2002.
16. J. Goecks, A. Nekrutenko, J. Taylor, et al., "Galaxy: A comprehensive approach for supporting accessible, reproducible, and transparent computational research in the life science," Genome Biology, 11:R86, 2010.
17. W. J. Kent, "BLAT-The BLAST-like alignment tool," Genome Research, 12(4):656-664, 2002.
18. B. Ludascher, C. Berkley, M. Jones, and E. A. Lee, "Scientific workflow management and the Kepler system," Concurrency and Computation: Practice and Experience, 18(10):1039-1065, 2006.
19. A. McKenna, M. Hanna, E. Banks, A. Sivachenko, K. Cibulskis, A. Kernytsky, K. Garimella, D. Altshuler, S. Gabriel, M. Daly, and M. DePristo, "The Genome Analysis Toolkit: A MapReduce framework for analyzing next-generation DNA sequencing data," Genome Research, 20(9):1297-1303, 2010.
20. F. Meyer, D. Paarmann, M. D'Souza, R. Olson, E. M. Glass, M. Kubal, T. Paczian, A. Rodriguez, R. Stevens, A. Wilke, J. Wilkening, and R. Edwards, " The metagenomics RAST server-A public resource for the automatic phylogenetic and functional analysis of metagenomes," BMC Bioinformatics, 9(386):1-8, 2008.
21. P. Missier, S. Soiland-Reyes, S. Owen, W. Tan, A. Nenadic, I. Dunlop, A. Williams, T. Oinn, and C. Goble, "Taverna, reloaded," In Proc. of International Conference on Scientific and Statistical Database Management (SSDBM), 2010.
22. N. Mohamed, H. Lin, and W.-C. Feng, "Accelerating data-intensive genome analysis in the cloud," In Proc. of International Conference on Bioinformatics and Computational Biology (BICoB), 2013.
23. D. Pushkarev, N. Neff, and S. Quake, "Single-molecule sequencing of an individual human genome," Nature Biotechnoly, 27:847-850, 2009.
24. J. Qiu, J. Ekanayake, T. Gunarathne, J. Y. Choi, S.-H. Bae, H. Li, B. Zhang, T.-L.Wu, Y. Ruan, S. Ekanayake, A. Hughes, and G. Fox, "Hybrid cloud and cluster computing paradigms for life science applications," BMC Bioinformatics, 11(12):53, 2010.
25. L. Ramakrishnan, M. S. Reed, J. L. Tilson, and D. A. Reed, "Grid portals for bioinformatics," in International Workshop on Grid Computing Environments, 2006.
26. S. Sun, J. Chen, W. Li, I. Altinatas, A. Lin, S. Peltier, K. Stocks, E. Allen, M. Ellisman, J. Grethe, and J. Wooley, "Community cyberinfrastructure for advanced microbial ecology research and analysis: The CAMERA resource," Nucleic Acids Research, 2010. doi: 10.1093/nar/gkq1102
27. W. Tang, N. Desai, V. Vishwanath, and Z. Lan, "Multi-domain job coscheduling on leadership computing systems," The Journal of Supercomputing, 63(2):367-384, 2013.
28. D. Thain, T. Tannenbaum, and M. Livny, "Distributed computing in practice: the Condor experience,"Concurrency and Computation: Practice and Experience, 17(2-4): 323-356, 2005.
29. U.S. Department of Energy. The Magellan report on cloud computing. Available at: http://science.energy.gov//media/ascr/pdf/programdocuments/ docs/Magellan Final Report.pdf
30. M. Wilde, M. Hategan, J. M. Wozniak, B. Clifford, D. S. Katz, and I. Foster, "Swift: A language for distributed parallel scripting," Parallel Computing, 37(9):633-652, 2011.
31. A.Wilke, J.Wilkening, E. Glass, N. Desai, and F. Meyer, "An experience report: porting the MG-RAST rapid metagenomics analysis pipeline to the cloud," Concurrency and Computation: Practice and Experience, 23:2250-2257, 2011.
32. A. Wilke et al., "MG-RAST technical report and manual for version 3.3.6-rev.1," ftp://ftp.metagenomics.anl.gov/data/manual/mg-rasttech- report-v3 r1.pdf
33. J. Wilkening, A. Wilke, N. Desai, and F. Meyer, "Using clouds for metagenomics: A case study," in Proc. of IEEE International Conference on Cluster Computing, 2009.