Parallel metagenomic sequence clustering via sketching and maximal quasi-clique enumeration on map-reduce clouds

Proceedings - 25th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2011

Volumn , Issue , 2011, Pages 1223-1233

  Yang, Xiao ^a   Zola, Jaroslaw ^a   Aluru, Srinivas ^a,b  

^a Iowa State University of Science and Technology   (United States)

^b INDIAN INSTITUTE OF TECHNOLOGY BOMBAY   (India)

Author keywords

cloud computing; MapReduce; metagenomics; next generation sequencing; quasi clique enumeration; sequence clustering; sketching

Indexed keywords

MAP-REDUCE; METAGENOMICS; NEXT GENERATION SEQUENCING; QUASI CLIQUE ENUMERATION; SEQUENCE CLUSTERING; SKETCHING;

CLOUD COMPUTING; CLUSTERING ALGORITHMS; DISTRIBUTED PARAMETER NETWORKS; USER INTERFACES;

PROBLEM SOLVING;

EID: 80053243671     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPS.2011.116     Document Type: Conference Paper

References (48)

1. S. Gill et al., "Metagenomic analysis of the human distal gut microbiome." Science, vol. 312, no. 5778, pp. 1355-1359, 2006. (Pubitemid 43839542)
2. H. Poinar et al., "Metagenomics to paleogenomics: Large-scale sequencing of mammoth DNA." Science, vol. 311, no. 5759, pp. 392-394, 2006. (Pubitemid 43132666)
3. J. Venter et al., "Environmental genome shotgun sequencing of the sargasso sea." Science, vol. 304, no. 5667, pp. 66-74, 2004. (Pubitemid 38451457)
4. W. Ansorge, "Next-generation DNA sequencing techniques."Nat. Biotechnol., vol. 25, no. 4, pp. 195-203, 2009.
5. J. M. Perkel, "Sanger who? sequencing the next generation. "Science, vol. 10, pp. 275-279, 2009.
6. "454 Life Sciences," http://www.454.com/.
7. J. Gordon et al., "Extending our view of self: the human gut microbiome initiative (HGMI)."http://www.genome.gov/10002154, 2005.
8. C. Simon et al., "Phylogenetic diversity and metabolic potential revealed in a glacier ice metagenome." Appl. Environ. Microbiol., vol. 75, no. 23, pp. 7519-7526, 2009.
9. P. Turnbaugh et al., "A core gut microbiome in obese and lean twins." Nature, vol. 457, no. 7228, pp. 480-484, 2009.
10. A. Lakdawalla and H. Steenhouse, Illumina Genome Analyzer II System, 1st ed. Wiley-VCH, 2008, ch. 2, pp. 15-28.
11. Z. Liu et al., "Accurate taxonomy assignments from 16S rRNA sequences produced by highly parallel pyrosequencers."Nucleic Acids Res., vol. 36, no. 18, p. e120, 2008.
12. N. Verberkmoes et al., "Shotgun metaproteomics of the human distal gut microbiota." J. ISME, vol. 3, no. 2, pp. 179-189, 2009.
13. J. Cole et al., "The ribosomal database project: Improved alignments and new tools for rRNA analysis." Nucleic Acids Res., vol. 37, no. Database issue, pp. D141-D145, 2009.
14. T. DeSantis et al., "NAST: a multiple sequence alignment server for comparative analysis of 16S rRNA genes." Nucleic Acids Res., vol. 34, no. Web Server issue, pp. W394-W399, 2006.
15. -, "Greengenes, a chimera-checked 16S rRNA gene database and workbench compatible with ARB." Appl. Environ. Microbiol., vol. 72, no. 7, pp. 5069-5072, 2006.
16. N. Diaz et al., "TACOA: Taxonomic classification of environmental genomic fragments using a kernelized nearest neighbor approach." BMC Bioinf., vol. 10, p. 56, 2009.
17. D. Huson et al., "MEGAN analysis of metagenomic data."Genome Res., vol. 17, no. 3, pp. 377-386, 2007. (Pubitemid 46376747)
18. A. McHardy et al., "Accurate phylogenetic classification of variable-length DNA fragments." Nat. Methods, vol. 4, no. 1, pp. 63-72, 2007. (Pubitemid 46029478)
19. F. Meyer et al., "The metagenomics RAST server - a public resource for the automatic phylogenetic and functional analysis of metagenomes." BMC Bioinf., vol. 9, no. 1, p. 386, 2008.
20. Q. Wang et al., "Naive Bayesian classifier for rapid assignment of rRNA sequences into the new bacterial taxonomy."Appl. Environ. Microbiol., vol. 73, no. 16, pp. 5261-5267, 2007. (Pubitemid 47326658)
21. N. Blow, "Metagenomics: Exploring unseen communities."Nature, vol. 453, no. 7195, pp. 687-690, 2008.
22. W. Li, J. C. Wooley, and A. Godzik, "Probing metagenomics by rapid cluster analysis of very large datasets." PLoS One, vol. 3, no. 10, p. e3375, 2008.
23. "Computational methods and terabase metagenomics,"http://www. icis.anl.gov/programs/summer-1.
24. M. Cameron, Y. Bernstein, and H. Williams, "Clustered sequence representation for fast homology search." J. Comput. Biol., vol. 14, no. 5, pp. 594-614, 2007. (Pubitemid 47047816)
25. A. Kalyanaraman et al., "Efficient clustering of large EST data sets on parallel computers." Nucleic Acids Res., vol. 31, no. 11, pp. 2963-2974, 2003. (Pubitemid 37442139)
26. -, "Assembling genomes on large-scale parallel computers."J. of Parallel and Distrib. Comput., vol. 67, no. 12, pp. 1240 - 1255, 2007.
27. J. Zola, X. Yang, S. Rospondek, and S. Aluru, "Parallel-TCoffee: A parallel multiple sequence aligner," in In Proc. of ISCA PDCS-2007, 2007, pp. 248-253.
28. A. Broder et al., "Syntactic clustering of the web." Comput. Networks ISDN Systems, vol. 29, no. 8-13, pp. 1157-1166, 1997.
29. J. Dean and S. Ghemawat, "Mapreduce: simplified data processing on large clusters," in Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation, vol. 6, 2003, pp. 1-10.
30. A. McKenna et al., "The genome analysis toolkit: a mapreduce framework for analyzing next-generation DNA sequencing data." Genome Res., vol. 20, no. 9, pp. 1297-1303, 2010.
31. M. Schatz, B. Langmead, and S. Salzberg, "Cloud computing and the DNA data race." Nat. Biotechnol., vol. 28, no. 7, pp. 691-693, 2010.
32. D. Wall et al., "Cloud computing for comparative genomics."BMC Bioinf., vol. 11, p. 259, 2010.
33. J. Eid, A. Fehr, J. Gray, and K. L. et al., "Real-time dna sequencing from single polymerase molecules." Science, vol. 323, no. 5910, pp. 133-138, 2009.
34. R. Xu and D. Wunsch, "Survey of clustering algorithms."IEEE Trans. Neural Netw., vol. 16, no. 3, pp. 645-678, 2005. (Pubitemid 40718010)
35. V. Estivill-Castro, "Why so many clustering algorithms: a position paper." ACM SIGKDD Explor. Newsl., vol. 4, no. 1, pp. 65-75, 2002.
36. M. Charikar, "Similarity estimation techniques from rounding algorithms." in Proc. of ACM STOC, 2002, pp. 380-388.
37. G. Manku, A. Jain, and A. D. Sarma, "Detecting near-duplicates for web crawling." in Proc. of WWW, 2007, pp. 141-150.
38. A. Broder et al., "Min-wise independent permutations." J. Comput. System Sci., vol. 60, no. 3, pp. 630-659, 2000.
39. M. Henzinger, "Finding near-duplicate web pages: A large-scale evaluation of algorithms." in Proc. of ACM SIGIR, 2006, pp. 284-291.
40. N. Du, B. Wu, L. Xu, B. Wang, and X. Pei, "A parallel algorithm for enumerating all maximal cliques in complex network," in Data Mining Workshops, 2006. ICDM Workshops 2006. Sixth IEEE International Conference on, 2006, pp. 320-324.
41. M. C. Schmidt, N. F. Samatova, K. Thomas, and B.-H. Park, "A scalable, parallel algorithm for maximal clique enumeration," J. of Parallel and Distrib. Comput., vol. 69, no. 4, pp. 417-428, 2009.
42. B. Wu and X. Pei, "A parallel algorithm for enumerating all the maximal k-plexes," in Emerging technologies in knowledge discovery and data mining, ser. Lecture notes in computer science, vol. 4819, 2009, p. 476483.
43. B. Wu, S. Yang, H. Zhao, and B. Wang, "A distributed algorithm to enumerate all maximal cliques in mapreduce,"in Frontier of Computer Science and Technology, 2009. FCST '09. Fourth International Conference on, 2009, pp. 45-51.
44. "Amazon EC2," http://aws.amazon.com/ec2/.
45. "Boost C++ Libraries," http://www.boost.org/.
46. A.-L. Barabasi and R. Albert, "Emergence of scaling in random networks," Science, vol. 286, no. 5439, pp. 509-512, 1999.
47. S. Sharma, "Advanced Hadoop tuning and optimization."
48. G. Malewicz, M. H. Austern, A. J. Bik, and J. C. et al.. Dehnert, "Pregel: a system for large-scale graph processing,"in Proceedings of the 28th ACM symposium on Principles of distributed computing, ser. PODC '09, 2009, pp. 6-6.