Design and evaluation of a collective io model for loosely coupled petascale programming

2008 Workshop on Many-Task Computing on Grids and Supercomputers, MTAGS 2008

Volumn , Issue , 2008, Pages

  Zhang, Zhao ^a   Espinosa, Allan ^b   Iskra, Kamil ^c   Raicu, Ioan ^b   Foster, Ian ^a,b,c   Wilde, Michael ^a,c  

^a UNIVERSITY OF CHICAGO   (United States)

^b THE UNIVERSITY OF CHICAGO   (United States)

^c ARGONNE NATIONAL LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPLICATION PROGRAMS; CACHING TECHNIQUE; COLLECTIVE OPERATIONS; COMPUTING NODES; COUPLED MODELS; FILE SYSTEMS; INPUT AND OUTPUTS; INPUT DATA; LARGE-SCALE CLUSTERS; LARGE-SCALE MOLECULAR DYNAMICS; LOCAL FILE SYSTEMS; MANUAL TUNING; PETASCALE; PROTOTYPE MODELS; RECODING; SCIENTIFIC PROGRAMS; SCRIPTING LANGUAGES; SYNTHETIC BENCHMARK; TASK COMPUTING;

BENCHMARKING; DYNAMICS; FLIGHT DYNAMICS; MOLECULAR DYNAMICS; PARALLEL PROGRAMMING; SUPERCOMPUTERS; TUNING;

COMPUTER SYSTEMS PROGRAMMING;

EID: 67649231475     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/MTAGS.2008.4777908     Document Type: Conference Paper

References (24)

1. Argonne Leadership Computing Facility, http://www.alcf.anl.gov
2. S. Al-Kiswany, Ripeanu M., Vazhkudai S., "A Checkpoint Storage System for Desktop Grid Computing", Networked Systems Lab, U. of British Columbia, Tech Report NetSysLab-TR-2007-2104, 2007.
3. J. Bester, I. Foster, C. Kesselman, J. Tedesco, and S. Tuecke, "GASS: A data movement and access service for wide area computing systems", IOPADS 99: Proceedings of the Sixth Workshop on IO in Parallel and Distributed Systems, Atlanta, GA, pp 78-88, 1999.
4. IBM Blue Gene team, "Overview of the IBM Blue Gene/P Project". IBM Journal of Research and Development, vol.52, no. 1/2, pp. 199-220, Jan/Mar 2008.
5. Overview of DOCK, http://dock.compbio.ucsf.edu/Overview-of-DOCK/index.htm
6. FUSE: File System in Userspace. http://fuse.sourceforge.net/
7. K. Iskra, J. W. Romein, K. Yoshii, and P. Beckman. "ZOID: IO-forwarding infrastructure for petascale architectures". 13th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, pp. 153-162, Salt Lake City, UT, Feb. 2008.
8. G. Khanna, N. Vydyanathan, U. V. Catalyurek, T. M. Kurc, S. Krishnamoorthy, P. Sadayappan, J. H. Saltz, "Task Scheduling and File Replication for Data- Intensive Jobs with Batch-shared IO", Proceedings of the 15th IEEE International Symposium on High-Performance Distributed Computing (HPDC-15) pp. 241-252, June 2006. (Pubitemid 46016894)
9. G. Khanna, U. V. Catalyurek, T. M. Kurc, P. Sadayappan, J. H. Saltz, "Scheduling File Transfers for Data-Intensive Jobs on Heterogeneous Clusters", Proceedings of Euro-Par 2007 Parallel Processing, pp. 214- 223, August, 2007.
10. Message Passing Interface Forum, "MPI-2: Extensions to the Message-Passing Interface", http://www.mpiforum.org/docs/mpi-20-html/mpi2- report.html
11. K. Coloma, A. Ching, A. Choudhary, W. Liao R. Ross, R. Thakur, L. Ward, "A New Flexible MPI Collective IO Implementation", International Conference on Cluster Computing, 2006.
12. Network Block Device, http://nbd.sourceforge.net
13. J. Ousterhout, "Scripting: Higher-level programming for the 21st century", IEEE Computer Mar. 1998.
14. I. Raicu, Y. Zhao, C. Dumitrescu, I. Foster, M. Wilde. "Falkon: a Fast and Light-weight tasK executiON framework", IEEE/ACM Supercomputing 2007.
15. I. Raicu, Z. Zhang, M. Wilde, I. Foster, P. Beckman, K. Iskra, B. Clifford. "Toward Loosely Coupled Programming on Petascale Systems", to appear, IEEE/ACM Supercomputing 2008.
16. F. Schmuck, R. Haskin, GPFS: A Shared- Disk File System for Large Computing Clusters, Proceedings of the USENIX FAST02 Conference on File and Storage Technologies, Monterey, California, 2002.
17. D. Thain, T. Tannenbaum, and M. Livny, "Distributed Computing in Practice: The Condor Experience" Concurrency and Computation: Practice and Experience, vol.17, no. 2-4, pp. 323-356, Feb-Apr 2005. (Pubitemid 40285781)
18. D. Thain, C. Moretti, and J. Hemmes, Chirp: A Practical Global File system for Cluster and Grid Computing, Journal of Grid Computing, Springer, accepted for publication in 2008.
19. R. Thakur, W. Gropp, E. Lusk. Data Sieving and Collective IO in ROMIO, 7th Symposium on the Frontiers of Massively Parallel Computation, 1999.
20. Universal TUN/TAP Driver. http://vtun.sourceforge.net/tun
21. XAR - eXtensible ARchiver Project home page, http://code.google.com/p/ xar/
22. http://www.top500.org/system/9158
23. The ZeptoOS Project. http://www.zeptoos.org/
24. Y. Zhao, M. Hategan, B. Clifford, I. Foster, G. vonLaszewski, I. Raicu, T. Stef-Praun, M. Wilde, "Swift: Fast, Reliable, Loosely Coupled Parallel Computation" IEEE Workshop on Scientific Workflows 2007