A map-reduce framework for clustering metagenomes

Proceedings - IEEE 27th International Parallel and Distributed Processing Symposium Workshops and PhD Forum, IPDPSW 2013

Volumn , Issue , 2013, Pages 549-558

  Rasheed, Zeehasham ^a   Rangwala, Huzefa ^a  

^a George Mason University   (United States)

Author keywords

map reduce; metagenome clustering; minwise hashing

Indexed keywords

BENCHMARKING; GENES; GENETIC ENGINEERING;

AGGLOMERATIVE HIERARCHICAL CLUSTERING; CLUSTERING APPROACH; COMPUTATIONAL TOOLS; COMPUTATIONALLY EFFICIENT; MAP-REDUCE; METAGENOMES; MINWISE HASHING; SEQUENCE SIMILARITY;

CLUSTERING ALGORITHMS;

EID: 84899755646     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPSW.2013.100     Document Type: Conference Paper

References (32)

1. S.R. Gill, M. Pop, R.T. DeBoy, P.B. Eckburg, P.J. Turnbaugh, B.S. Samuel, J.I. Gordon, D.A. Relman, C.M. Fraser-Liggett, and K.E. Nelson. Metagenomic analysis of the human distal gut microbiome. science, 312(5778):1355-1359, 2006.
2. H.N. Poinar, C. Schwarz, J. Qi, B. Shapiro, R.D.E. MacPhee, B. Buigues, A. Tikhonov, D.H. Huson, L.P. Tomsho, A. Auch, et al. Metagenomics to paleogenomics: large-scale sequencing of mammoth dna. science, 311(5759):392-394, 2006.
3. J.C. Venter, K. Remington, J.F. Heidelberg, A.L. Halpern, D. Rusch, J.A. Eisen, D. Wu, I. Paulsen, K.E. Nelson, W. Nelson, et al. Environmental genome shotgun sequencing of the sargasso sea. science, 304(5667):66-74, 2004.
4. P. Hugenholtz and G.W. Tyson. Microbiology: metagenomics. Nature, 455(7212):481-483, 2008.
5. A. Charuvaka and H. Rangwala. Evaluation of short read metagenomic assembly. BMC genomics, 12(Suppl 2):S8, 2011.
6. J.F. Petrosino, S. Highlander, R.A. Luna, R.A. Gibbs, and J. Versalovic. Metagenomic pyrosequencing and microbial identification. Clinical chemistry, 55(5):856-866, 2009.
7. Zeehasham Rasheed and Huzefa Rangwala. Mc-minh: Metagenome clustering using minwise based hashing. In SIAM International Conference in Data Mining, Austin, TX, 05 2013. SIAM.
8. A.Z. Broder, M. Charikar, A.M. Frieze, and M. Mitzenmacher. Min-wise independent permutations. In Proceedings of the thirtieth annual ACM symposium on Theory of computing, pages 327-336. ACM, 1998.
9. M.L. Sogin, H.G. Morrison, J.A. Huber, D.M. Welch, S.M. Huse, P.R. Neal, J.M. Arrieta, and G.J. Herndl. Microbial diversity in the deep sea and the underexplored rare biosphere. Proceedings of the National Academy of Sciences, 103(32):12115-12120, 2006.
10. S. Chatterji, I. Yamazaki, Z. Bai, and J. Eisen. Compostbin: A dna composition-based algorithm for binning environmental shotgun reads. In Research in Computational Molecular Biology, pages 17-28. Springer, 2008.
11. M. Margulies, T.P. Jarvie, J.R. Knight, and J.F. Simons. The 454 life sciences picoliter sequencing system. Perspectives in Bioanalysis, 2:153-186, 2007.
12. W. Li and A. Godzik. Cd-hit: a fast program for clustering and comparing large sets of protein or nucleotide sequences. Bioinformatics, 22(13):1658-1659, 2006.
13. R.C. Edgar. Search and clustering orders of magnitude faster than blast. Bioinformatics, 26(19):2460-2461, 2010.
14. P.D. Schloss and J. Handelsman. Introducing dotur, a computer program for defining operational taxonomic units and estimating species richness. Applied and environmental microbiology, 71(3):1501-1506, 2005.
15. P.D. Schloss, S.L. Westcott, T. Ryabin, J.R. Hall, M. Hartmann, E.B. Hollister, R.A. Lesniewski, B.B. Oakley, D.H. Parks, C.J. Robinson, et al. Introducing mothur: opensource, platform-independent, community-supported software for describing and comparing microbial communities. Applied and environmental microbiology, 75(23):7537-7541, 2009.
16. Y. Sun, Y. Cai, L. Liu, F. Yu, M.L. Farrell, W. McKendree, and W. Farmerie. Esprit: estimating species richness using large collections of 16s rrna pyrosequences. Nucleic Acids Research, 37(10):e76-e76, 2009.
17. Zeehasham Rasheed, Huzefa Rangwala, and Daniel Barbara. LSH-Div:species diversity estimation using locality sensitive hashing. In IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Philadelphia, USA, 10 2012. IEEE.
18. Zeehasham Rasheed, Huzefa Rangwala, and Daniel Barbara. Efficient clustering of metagenomic sequences using locality sensitive hashing. In SIAM International Conference in Data Mining, pages 1023-1034, Anaheim, CA, 04 2012. SIAM.
19. B. Yang, Y. Peng, H. Leung, S.M. Yiu, J. Qin, R. Li, and F.Y.L. Chin. Metacluster: unsupervised binning of environmental genomic fragments and taxonomic annotation. In Proceedings of the First ACM International Conference on Bioinformatics and Computational Biology, pages 170-179. ACM, 2010.
20. M. Cameron, Y. Bernstein, and H.E. Williams. Clustered sequence representation for fast homology search. Journal of Computational Biology, 14(5):594-614, 2007.
21. A. Kalyanaraman, S. Aluru, S. Kothari, and V. Brendel. Efficient clustering of large est data sets on parallel computers. Nucleic Acids Research, 31(11):2963-2974, 2003.
22. J. Zola, X. Yang, S. Rospondek, and S. Aluru. Parallel tcoffee: A parallel multiple sequence aligner. Proceedings of ISCA PDCS-2007, pages 248-253, 2007.
23. X. Yang, J. Zola, and S. Aluru. Parallel metagenomic sequence clustering via sketching and maximal quasi-clique enumeration on map-reduce clouds. In Parallel & Distributed Processing Symposium (IPDPS), 2011 IEEE International, pages 1223-1233. IEEE, 2011.
24. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig latin: a not-so-foreign language for data processing. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1099-1110. ACM, 2008.
25. A. McKenna, M. Hanna, E. Banks, A. Sivachenko, K. Cibulskis, A. Kernytsky, K. Garimella, D. Altshuler, S. Gabriel, M. Daly, et al. The genome analysis toolkit: a mapreduce framework for analyzing next-generation dna sequencing data. Genome research, 20(9):1297-1303, 2010.
26. M.C. Schatz, B. Langmead, and S.L. Salzberg. Cloud computing and the dna data race. Nature biotechnology, 28(7):691, 2010.
27. D.P. Wall, P. Kudtarkar, V.A. Fusaro, R. Pivovarov, P. Patil, and P.J. Tonellato. Cloud computing for comparative genomics. BMC bioinformatics, 11(1):259, 2010.
28. P. Li, A. Shrivastava, and C.A. Konig. Gpu-based minwise hashing: Gpu-based minwise hashing. In Proceedings of the 21st international conference companion on World Wide Web, pages 565-566. ACM, 2012.
29. J.L. Carter and M.N.Wegman. Universal classes of hash functions. Journal of computer and system sciences, 18(2):143-154, 1979.
30. S.M. Huse, J.A. Huber, H.G. Morrison, M.L. Sogin, D.M. Welch, et al. Accuracy and quality of massively parallel dna pyrosequencing. Genome Biol, 8(7):R143, 2007.
31. M. Margulies, M. Egholm,W.E. Altman, S. Attiya, J.S. Bader, L.A. Bemben, J. Berka, M.S. Braverman, Y.J. Chen, Z. Chen, et al. Genome sequencing in microfabricated high-density picolitre reactors. Nature, 437(7057):376-380, 2005.
32. X. Huang. On global sequence alignment. Computer applications in the biosciences: CABIOS, 10(3):227-235, 1994.