Case studies in storage access by loosely coupled petascale applications

Proceedings of the 4th Annual Petascale Data Storage Workshop, PDSW '09, Held in conjunction with Supercomputing '09

Volumn , Issue , 2009, Pages 16-20

  Wozniak, Justin M ^a   Wilde, Michael ^a  

^a ARGONNE NATIONAL LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FILE ACCESS; I/O SUBSYSTEMS; I/O WORKLOAD; LABOR INTENSIVE PROCESS; PARALLEL JOBS; PETASCALE; PROGRAMMING TOOLS; REAL-WORLD; SCIENTIFIC APPLICATIONS; TIGHTLY-COUPLED;

DATA STORAGE EQUIPMENT;

TECHNICAL PRESENTATIONS;

EID: 77950510812     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1713072.1713078     Document Type: Conference Paper

References (31)

1. N. Ali, A. Devulapalli, D. Dalessandro, P.Wyckoff, and P. Sadayappan. An OSD-based approach to managing directory operations in parallel file systems. In Proc. CLUSTER, 2008.
2. S. F. Altschul, W. Gish, W. Miller, E. W. Myers, and D. J. Lipman. Basic local alignment search tool. J. Molecular Biology, 215(3), 1990.
3. J. Bent, D. Thain, A. C. Arpaci-Dusseau, R. H. Arpaci-Dusseau, and M. Livny. Explicit control in a batch-aware distributed file system. In Proc. USENIX Symposium on Networked Systems Design and Implementation, 2004.
4. P. Carns, S. Lang, R. Ross, M. Vilayannur, J. Kunkel, and T. Ludwig. Small-file access in parallel file systems. In Proc. International Parallel and Distributed Processing Symposium, 2009.
5. P. A. Cheeseman, M. W. Deem, D. J. Earl, , and W. I. Whitson. Adapting an application for use in a Condor based parameter sweep on TeraGrid. In Proc. TeraGrid 2007 Conference, 2007.
6. Y. Chen, W. Chen, M. H. Cobb, and Y. Zhao. PTMap - A sequence alignment software for unrestricted, accurate, and full-spectrum identification of post-translational modification sites. Proceedings of the National Academy of Sciences of the USA, 106(3), 2009.
7. A. Ching, K. Coloma, J. Li, W. keng Liao, and A. Choudhary. High-performance techniques for parallel I/O. In Handbook of Parallel Computing: Models, Algorithms and Applications, chapter 35. 2008.
8. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. In Proc. Operating Systems Design and Implementation, 2004.
9. J. DeBartolo, A. Colubri, A. K. Jha, J. Fitzgerald, and T. R. S. Karl F. Freed. Mimicking the folding pathway to improve homology-free protein structure prediction. Proc. National Academy of Sciences, 106(10), 2009.
10. J. J. Dongarra and D. W. Walker. The quest for petascale computing. Computing in Science and Engineering, 3(3), 2001.
11. M. P. I. Forum. MPI: A message-passing interface standard, 1994.
12. Y. Gil, P. A. González-Calero, and E. Deelman. On the black art of designing computational workflows. In Proc. Workshop on Workflows in Support of Large-Scale Science, 2007.
13. G. Grider, L. Ward, R. Ross, and G. Gibson. A business case for extensions to the POSIX I/O API for high end, clustered, and highly concurrent computing, 2006. Available at: http://www.opengroup.org/platform/hecewg/uploads/ 40/10891/ POSIXIO-API-Business-case-HEC-ggrider.pdf.
14. U. Hasson, J. I. Skipper, M. J. Wilde, H. C. Nusbaum, and S. L. Small. Improving the analysis, storage and sharing of neuroimaging data using relational databases and distributed computing. Neuroimage, 39(2), 2008.
15. D. M. Lorber and B. K. Shoichet. Hierarchical docking of databases of multiple ligand conformations. Current Topics in Medicinal Chemistry, 5(8), 2005.
16. G. Mackey, S. Sehrish, J. Bent, J. Lopez, S. Habib, and J. Wang. Introducing Map-Reduce to high end computing. In Proc. Petascale Data Storage Workshop, 2008.
17. H. M. Monti, A. R. Butt, and S. S. Vazhkudai. /Scratch as a cache: Rethinking HPC center scratch storage. In Proc. International Conference on Supercomputing, 2008.
18. C. Moretti, J. Bulosan, D. Thain, and P. J. Flynn. All-pairs: An abstraction for data-intensive cloud computing. In Proc. International Parallel and Distributed Processing Symposium, 2008.
19. E. B. Nightingale, P. M. Chen, and J. Flinn. Speculative execution in a distributed file system. ACM Transactions on Computer Systems, 24(4), 2006.
20. J. Ousterhout. Scripting: Higher-level programming for the 21st century. IEEE Computer, Mar. 1998.
21. I. Raicu, I. Foster, Y. Zhao, P. Little, C. Moretti, A. Chaudhary, and D. Thain. The quest for scalable support of data-intensive workloads in distributed systems. In Proc. High Performance Distributed Computing, 2009.
22. I. Raicu, Z. Zhang, M. Wilde, I. Foster, P. Beckman, K. Iskra, and B. Clifford. Towards loosely-coupled programming on petascale systems. In Proc. SC'08, 2008.
23. I. Raicu, Y. Zhao, C. Dumitrescu, I. Foster, and M. Wilde. Falkon: A Fast and Light-weight tasK executiON framework. In Proc SC'07, 2007.
24. F. Schmuck and R. Haskin. GPFS: A shared-disk file system for large computing clusters. In Proc. USENIX Conference on File and Storage Technologies, 2002.
25. S. Shepler, B. Callaghan, D. Robinson, R. Thurlow, C. Beame, M. Eisler, and D. Noveck. Network File System (NFS) version 4 protocol. RFC 3530, 2003.
26. I. Taylor, E. Deelman, D. Gannon, and M. Shields, editors. Workflows for e-Science. Springer, 2007.
27. D. Thain and C. Moretti. Efficient access to many small files in a filesystem for grid computing. In Proc. Conference on Grid Computing, 2007.
28. D. Thain, T. Tannenbaum, and M. Livny. Distributed computing in practice: The Condor experience. Concurrency and Computation: Practice and Experience, 17(2-4), 2005.
29. R. Thakur, W. Gropp, and E. Lusk. On implementing MPI-IO portably and with high performance. In Proc. of the Sixth Workshop on I/O in Parallel and Distributed Systems, May 1999.
30. Z. Zhang, A. Espinosa, K. Iskra, I. Raicu, I. Foster, and M. Wilde. Design and evaluation of a collective I/O model for loosely-coupled petascale programming. In Proc. MTAGS Workshop and SC'08, 2008.
31. Y. Zhao, M. Hategan, B. Clifford, I. Foster, G. von Laszewski, I. Raicu, T. Stef-Praun, and M. Wilde. Swift: Fast, reliable, loosely coupled parallel computation. In Proc. Workshop on Scientific Workflows, 2007.