Implementation of a scalable next generation sequencing business cloud platform - An experience report

Proceedings - 2011 IEEE 4th International Conference on Cloud Computing, CLOUD 2011

Volumn , Issue , 2011, Pages 598-605

  Doddavula, Shyam Kumar ^a   Rani, Madhavi ^a   Sarkar, Santonu ^a   Vachhani, Harsh Rajesh ^a   Jain, Akansha ^a   Kaushik, Mudit ^a   Ghosh, Anirban ^b  

^a INFOSYS TECHNOLOGIES LTD   (India)

^b Life Sciences Infosys Consulting Inc   (United States)

Author keywords

Amazon cloud; Application porting; BLAST; Genomics; Hadoop; Next generation sequencing; Scalability

Indexed keywords

ANALYSIS TECHNIQUES; BLAST; COLLABORATIVE PLATFORM; CONTROLLED EXPERIMENT; DATABASE PARTITIONING; EMPIRICAL RESULTS; EXPERIENCE REPORT; GENOME SEQUENCES; GENOMIC DATA; GENOMICS; HADOOP; JOB EXECUTION; LIFE-SCIENCES; NEXT GENERATION SEQUENCING;

COST EFFECTIVENESS; GENES; INNOVATION; LAKES; SCALABILITY;

CLOUD COMPUTING;

EID: 80053154066     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CLOUD.2011.60     Document Type: Conference Paper

References (42)

1. The 1000 Genomes Project Consortium, "A map of human genome variation from population-scale sequencing", Nature 467, 2010, pp 1061-1073.
2. The Apache Software Foundation. Hadoop. http://hadoop.apache.org/
3. J. Dean and S. Ghemawat, "MapReduce: Simplified DataProcessing on Large Clusters," in Sixth Symposium on Operating System Design and Implementation, Volume 6, 2004.
4. J. Dean and S. Ghemawat, "Mapreduce: a flexible data processing tool," in Comm of ACM, 53(1), 2010, pp. 72-77.
5. S. Ghemawat, H. Gobioff, and S.-T. Leung, "The Google File System," 19th Symposium on Operating Systems Principles, 2003.
6. Chang, F., Dean, J., et al., "Bigtable: A Distributed Storage System for Structured Data," Seventh Symposium on Operating System Design and Implementation, 2006.
7. Moretti, C., Steinhaeuser, K., Thain, D., Chawla, N.V, "Scaling Up Classifiers to Cloud Computers," IEEE ICDM, 2008.
8. Newman, A., Hunter, J., Li, Y.F., Bouton, C., Davis, M., "A Scale-Out RDF Molecule Store for Distributed Processing of Biomedical Data Semantic Web for Health Care and Life Sciences", Workshop WWW 2008.
9. Chu, C.-T., Kim, S.K., Lin, Y.-A., Yu, Y., Bradski, G., Ng, A.Y., Olukotun, K., "Map-Reduce for Machine Learning on Multi-core," Science, NIPS 2007.
10. J. Wang and Q. Mu, "Soap-HT-BLAST: high throughput BLAST based on Web services," Bioinformatics, vol. 19, no. 14, pp. 1863-1864, 2003. (Pubitemid 37220485)
11. F. Schmuck and R. Haskin, "The design, implementation, and evaluation of mpiblast," in Proc. ClusterWorld, 2003
12. A. Krishnan, "Gridblast: a globus-based high-throughput implementation of blast in a grid computing framework: Research articles," Concurr. Comput.: Pract. Exper., vol. 17, no. 13, pp. 1607-1623, 2005 (Pubitemid 41560624)
13. S. E. Dowd, J. Zaragoza, J. R. Rodriguez, M. J. Oliver, and P. R. Payton, "Windows.net network distributed basic local alignment search toolkit (w.nd-blast)," BMC Bioinformatics, vol. 6, no. 93, 2005.
14. P. C. Carvalho, R. V. Gloria, A. B. de Miranda, and W. M. Degrave, "Squid - a simple bioinformatics grid," BMC Bioinformatics, vol. 6, no. 197, 2005.
15. C. Oehmen and J. Nieplocha, "Scalablast: A scalable implementation of blast for high-performance data-intensive bioinformatics analysis," IEEE Transactions on Parallel and Distributed Systems, vol. 17, no. 8, pp. 740-749, 2006 (Pubitemid 44070144)
16. Y. Sun, S. Zhao, H. Yu, G. Gao, and J. Luo, "ABCGrid: Application for Bioinformatics Computing Grid," Bioinformatics, vol. 23, no. 9, pp. 1175-1177, 2007. (Pubitemid 47050575)
17. Massimo Gaggero, Simone Leo, Simone Manca, Federico Santoni, Omar Schiaratura, Gianluigi Zanetti. "Parallelizing bioinformatics applications with MapReduce", CRS4, Edificio 1, Sardegna Ricerche, Pula, Italy
18. MC Schatz, "BlastReduce: high performance short read mapping with MapReduce", University of Maryland, http://cgis.cs.umd.edu/Grad/ scholarlypapers/papers/MichaelSchatz.pdf
19. Michael C. Schatz "CloudBurst: Highly Sensitive Short Read Mapping with MapReduce", Bioinformatics, vol 25, pp.1363-1369, 2009.
20. J. Ekanayake, H. Li, B. Zhang, T. Gunarathne, S. Bae, J. Qiu, and G. Fox., Twister: "A Runtime for iterative MapReduce", in Proc. of the First International Workshop on MapReduce and its Applications of ACM HPDC 2010.
21. A. Matsunaga, M. Tsugawa, and J. Fortes. CloudBLAST: Combining MapReduce and Virtualization on Distributed Resources for Bioinformatics Applications. in IEEE Fourth International Conference on eScience (eScience '08). 2008
22. Wei Lu, Jared Jackson, and Roger Barga, AzureBlast: A Case Study of Developing Science Applications on the Cloud, in ScienceCloud: 1st Workshop on Scientific Cloud Computing co-located with HPDC 2010
23. Windows Azure Platform, Retrieved April 20, 2010, from Microsoft: http://www.microsoft.com/windowsazure/. http://www.microsoft.com/windowsazure/
24. Lavanya Ramakrishnan, "Hadoop for Scientific Workloads", Hadoop Summit, June 2010
25. R. K. Singh et al, "BioSCAN: A dynamically reconfigurable systolic array for biosequence analysis"
26. S. Leo, F. Santoni, and G. Zanetti. "Biodoop: Bioinformatics on Hadoop", ICPP Workshops, pages 415-422. IEEE Computer Society, 2009.
27. Basic Local Alignment Search Tool (NCBI) http://blast.ncbi.nlm.nih.gov/ Blast.cgi
28. Altschul, S.F., Gish, W., Miller, W., Myers, E.W. and Lipman, D.J. "Basic local alignment search tool". J. Mol. Biol. 215, 403-410, 1990.
29. Altschul, S.F., Madden, T.L., Schäffer, A.A., Zhang, J., Zhang, Z., Miller, W. and Lipman, D.J. "Gapped BLAST and PSI-BLAST: a new generation of protein database search programs". Nucleic Acids Res. 25, pp 3389-3402, 1997. (Pubitemid 27359211)
30. R. Braun, K. Pedretti, T. Casavant, T. Scheetz, C. Birkett, and C. Roberts. "Parallelization of local BLAST service on workstation clusters". Future Generation Computer Systems, 17(6):745-754, April 2001. (Pubitemid 32284779)
31. N. Camp, H. Cofer, and R. Gomperts. "High-throughput BLAST", SGI White Paper, September 1998.
32. E. Chi, E. Shoop, J. Carlis, E. Retzel, and J. Riedl. "Efficiency of shared-memory multiprocessors for a genetic sequence similarity search algorithm", 1997.
33. E. Glemet and J. Codani. "LASSAP, a LArge Scale Sequence compArison Package. Computer Applications In The Biosciences", 13(2):137-143, April 1997. (Pubitemid 27189998)
34. K. Pedretti, T. Casavant, R. Braun, T. Scheetz, C. Birkett, and C. Roberts. "Three complementary approaches to parallelization of local BLAST service on workstation clusters". Lecture Notes In Computer Science, 1662:271-282, 1999.
35. Stein D Lincoln. "The case for cloud computing in genome informatics". Genome Biology, 11:207-214, 2010.
36. Smith, Temple F.; and Waterman, Michael S. "Identification of Common Molecular Subsequences". Journal of Molecular Biology 147, 1981, pp 195-197.
37. Ghosh, A, Chakrabarti, A, "Streamlining Drug Discovery Research by Leveraging Grid Workflow Manager", Grid Computing in Life Sciences, World Scientific Publishing Company, 1210-133, 2006.
38. Dheepak RA, et. al., "Scalable enterprise level workflow manager for the Grid". IEEE Quality Software, Fifth International Conference on Quality Software. 341-348, 2005.
39. Simon Thornber, Cary O'Donnell, Claus Stie Kallesøe, John Wise. The Pistoia Alliance. The Sequence Service Project. G.I.T. Laboratory Journal, Trends in Drug Disocvery Business, 1-3, 2011.
40. D. L. Wheeler et al., "Database resources of the National Center for Biotechnology Information", Nucleic Acid Research, 35 (Database Issue), 2007 pp D5-D12. (Pubitemid 46056160)
41. Michael Armbrust et al., "A View of Cloud Computing", Comm of ACM, vol 53, no 4, pp 50-58, 2010.
42. Paul Flicek et al., "Ensembl 2011", Nucleic Acids Res. 39, pp 800-806, 2011.