Parallel genomic sequence-searching on an ad-hoc grid: Experiences, lessons learned, and implications

Proceedings of the 2006 ACM/IEEE Conference on Supercomputing, SC'06

Volumn , Issue , 2006, Pages

  Gardner, Mark K ^a   Feng, Wu Chun ^a   Archuleta, Jeremy ^b   Lin, Heshan ^c   Mal, Xiaosong ^c  

^a VIRGINIA POLYTECHNIC INSTITUTE AND STATE UNIVERSITY   (United States)

^b UNIVERSITY OF UTAH   (United States)

^c NORTH CAROLINA STATE UNIVERSITY   (United States)

Author keywords

Agile development; Bioinformatics; BLAST; Cluster computing; Fault tolerance; Grid computing; Optical networking; Scalability; Scheduling; Scripting; Sequence search

Indexed keywords

AGILE DEVELOPMENT; CLUSTER COMPUTING; SCRIPTING; SEQUENCE SEARCH;

BIOINFORMATICS; DATABASE SYSTEMS; FAULT TOLERANCE; FIBER OPTIC NETWORKS; GRID COMPUTING; SCALABILITY;

SEARCH ENGINES;

EID: 34548268181     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1188455.1188564     Document Type: Conference Paper

References (35)

1. ALTSCHUL, S., GISH, W., MILLER, W., MYERS, E., AND LIPMAN, D. 1990. Basic Local Alignment Search Tool. Journal of Molecular Biology 215, 403-410.
2. ALTSCHUL, S. F., MADDEN, T. L., SCHAFFER, A. A., ZHANG, J., ZHANG, Z., MILLER, W., AND LIPMAN, D.J. 1997. Gapped BLAST and PSIBLAST: A New Generation of Protein Database Search Programs. Nucleic Acids Research 25, 3389-3402.
3. BARROSO, L., DEAN, J., AND HÖLZLE, U. 2003. Web Search for a Planet: The Google Cluster Architecture. IEEE Micro 23, 2.
4. BJORNSON, R., SHERMAN, A., WESTON, S., WILLARD, N., AND WING, J. 2002. TurboBLAST(r): A parallel implementation of BLAST built on the TurboHub. In International Parallel and Distributed Processing Symposium.
5. The NCBI Handbook (BLAST Output: 1. The Traditional Report), http://www.ncbi.nlm.nili.gov/books/bv.fcgi?rid=handbook.section.615.
6. BRAUN, R., PEDRETTI, K., CASAVANT, T., SCHEETZ, T., BIRKETT, C., AND ROBERTS, C. 2001. Parallelization of Local BLAST Service on Workstation Clusters. Future Generation Computer Systems 17, 6 (Apr), 745-754.
7. CAMP, N., GOFER, H., AND GOMPERTS, R. 1998. High-throughput BLAST. Tech. rep., SGI, Sep.
8. CARNS, P., LIGON III, W., ROSS, R., AND THAKUR, R. 2000. PVFS: A Parallel File System For Linux Clusters. In Proceedings of the 4th Annual Linux Showcase and Conference.
9. CHI, E., SHOOP, E., CARLIS, J., RETZEL, E., AND RIEDL, J. 1997. Efficiency of Shared-Memory Multiprocessors for a Genetic Sequence Similarity Search Algorithm . Tech. rep., University of Minnesota.
10. DARLING, A., CAREY, L., AND FENG, W. 2003. The Design, Implementation, and Evaluation of mpiBLAST. In Cluster World Conference & Expo and the 4th International Conference on Linux Cluster: The HPC Revolution 2003.
11. DUMONTIER, M., AND HOGUE, C. W. 2002. NBLAST: A Cluster Variant of BLAST for NxN Comparisons. BMC Bioinformatics 3, 13 (May), 271-282.
12. EPCC Sun Data and Compute Grids Project Status Update. http://www.sun.com/products-n-solutions/edu/events/archive/hpc/ 2003presentations/heidelberg/GRID05_ Ratna_Abrol.pdf.
13. EPCC Using Sun Grid Engine and Globus to Schedule Jobs Across a Combination of Local and Remote Machines. http://www.epcc.ed.ac.uk/sungrid/ DISSEMIMTION/Regensburg-2002-04.ppt.
14. FENG, W. 2003. Green Destiny + mpiBLAST = Bioinfomagic. In 10th International Conference on Parallel Computing (ParCo). Folding@home. http://folding.stanford.edu/.
15. FOSTER, I., AND KESSELMAN, C. 1997. Globus: A Metacomputing Infrastructure Toolkit. International Journal of Supercomputer Applications 11, 2, 115-128.
16. GANS, J., 2005. Personal Communication.
17. Growth of GenBank: 1982-2004. http://www.ncbi. nim.hih.gov/Genbank/ genbankstats.html.
18. GHEMAWAT, S., GOBIOFF, H., AND LEUNG, S. 2003. The Google File System. In Proceedings of the 19th Symposium on Operating Systems Principles.
19. GROPP, W., LUSK, E., DOSS, N., AND SKJELLUM, A. 1996. High-performance, portable implementation of the MPI Message Passing Interface Standard. Parallel Computing 22, 6, 789-828.
20. INFINIBAND TRADE ASSOCIATION, 2000. InfiniBand Architecture Specification, Release 1.0, Oct.
21. KENT, W.J. 2002. Blat - The BL AST-Like Alignment Tool. Genome Research 12 (Apr), 656-664.
22. LIN, H., MA, X., CHANDRAMOHAN, P., GEIST, A., AND SAMATOVA, N. 2005. Efficient Data Access for Parallel BLAST. In International Parallel and Distributed Processing Symposium.
23. LIU, J., WU, J., KINI, S., WYCKOFF, P., AND PANDA, D., 2003. High Performance RDMA-Based MPI Implementation over InfiniBand.
24. LIU, J., WU, J., KINI, S. P., WYCKOFF, P., AND PANDA, D. K. 2003. High Performance RDMABased MPI Implementation over InfiniBand. In 17th Annual ACM International Conference on Supercomputing.
25. The Lustre File System, http://www.lustre.org/. mpiBLAST-G2. http://www.twgrid.org/News.Event/News/mpiblast.
26. MPICH2. http://www-unix.mes.ani.gov/mpi/mpich2/.
27. PAXSON, V. 1999. End-to-End Internet Packet Dynamics. IEEE/ACM Transactions on Networking 7, 3 (June), 277-292.
28. PEDRETTI, K., CASAVANT, T., BRAUN, R., SCHEETZ, T., BIRKETT, C., AND ROBERTS, C. 1999. Three Complementary Approaches to Parallelization of Local BLAST Service on Workstation Clusters. Lecture Notes in Computer Science 1662, 271-282.
29. RAPIER, C., AND STEVENS, M. Application Layer Network Window Management in the SSH Protocol. Tech. rep., Pittsburgh Supercomputing Center.
30. REED, D. A., 2004. High-End Computing: The Challenge of Scale. Director's Colloquium, Los Alamos National Laboratory, May.
31. SCHMUCK, F., AND HASKIN, R. 2002. GPFS: A Shared-Disk File System for Large Computing Clusters. In Proceedings of the First Conference on File and Storage Technologies.
32. SETI@home. http://setiathome.ssl.berkeley. edu/.
33. SINGH, R. K., DETTLOFF, W. D., CHI, V. L., HOFFMAN, D. L., TELL, S. G., WHITE, C. T., ALTSCHUL, S. F., AND ERICKSON, B. W., 1996. BioSCAN: A Dynamically Reconfigurable Systolic Array for Biosequence Analysis. http://www. dlhoffman.com/~hoffman/papers/nsf96.ps.gz.
34. SMITH, T. F., AND WATERMAN, M. S. 1981. Identification of Common Molecular Subsequences. Journal of Molecular Biology 147, 195-197.
35. StorCloud. http ://sc05.supercomputing.org/initiatives/storcloud.php.