Scalable massively parallel I/O to task-local files

Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, SC '09

Volumn , Issue , 2009, Pages

  Frings, Wolfgang ^a   Wolf, Felix ^a,b   Petkov, Ventsislav ^c  

^a Jülich Supercomputing Centre ^*  (Germany)

^b RWTH AACHEN UNIVERSITY   (Germany)

^c TECHNICAL UNIVERSITY OF MUNICH   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

APPLICATION SCENARIO; FILE HANDLING; FILE MANAGEMENT; FILE SYSTEMS; METADATA HANDLING; MULTIPLE TASKS; PARALLEL APPLICATION; PARALLEL I/O; PERFORMANCE DATA; PERFORMANCE IMPROVEMENTS; SOURCE CODE CHANGES;

METADATA;

EID: 74049099762     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1654059.1654077     Document Type: Conference Paper

References (24)

1. Advanced Simulation and Computing Program. The ASC SMG2000 benchmark code, 2001. https://asc.llnl.gov/computing-resources/ purple/archive/benchmarks/ smg/.
2. M. J. Brim and B. P. Miller. Group file operations for scalable tools and middleware. Technical Report No. TR1638, Computer Sciences Department, University of Wisconsin, 2008.
3. S. Fadden. An introduction to GPFS version 3.2.1, November 2008. IBM Corporation.
4. R. Fagin, J. Nievergelt, N. Pippenger, and H. R. Strong. Extendible hashing - a fast access method for dynamic files. ACM Transactions on Database Systems (TODS), 4(3):315 - 344, 1979.
5. J. Gailly and M. Adler. zlib general-purpose compression library, version 1.2.3. http://www.zlib.net, 2005.
6. M. Geimer, F. Wolf, B. J. N. Wylie, and B. Mohr. A scalable tool architecture for diagnosing wait states in massively-parallel applications. Parallel Computing, 35(7):375-388, 2009.
7. HDF5. http://www.hdfgroup.org/HDF5/.
8. IBM. General Parallel File System. http://www-03.ibm.com/systems/ clusters/ software/gpfs/index.html.
9. IOR Parallel File System Benchmark. http://sourceforge.net/projects/ior- sio/.
10. C. Jin, S. Klasky, S. Hodson, J. Lofstead, F. Zheng, M. Wolf, and R. Ross. ADIOS User's Manual. Oak Ridge National Laboratory, November 2008.
11. Jülich Supercomputing Centre. JUGENE. http://www.fz-juelich.de/jsc/ jugene.
12. W. Liao and A. Choudhary. Dynamically adapting file domain partitioning methods for collective I/O based on underlying parallel file system locking protocols. In Proc. of the ACM/IEEE SC08 Conference, Austin, TX, November 2008.
13. J. Lofstead, S. Klasky, K. Schwan, N. Podhorszki, and C. Jin. Flexible IO and integration for scientific codes through the adaptable IO system (ADIOS). In Proc. of the 6th International Workshop on Challenges of Large Applications in Distributed Environments (CLADE), pages 15-24, Boston, MA, USA, 2008.
14. J. M. May. Parallel I/O for High Performance Computing. Morgan Kaufmann Publishers, 2001.
15. S. Microsystems. Lustre file system. http:www.lustre.org.
16. S. Microsystems. Lustre file system - high performance storage architecture and scalable cluster file system (white paper), October 2008. http://www.sun.com/software/products/lustre/ docs/lustrefilesystem-wp.pdf.
17. MPI Forum. MPI: A message passing interface standard, version 2.1. Chapter 13, September 2008.
18. NetCFD. http://www.unidata.ucar.edu/software/netcdf/.
19. Oak Ridge National Laboratory. Jaguar. http: //www.nccs.gov/computing- resources/jaguar/.
20. Parallel Virtual File System. http://www.pvfs.org/.
21. Scalasca. http://www.scalasca.org/.
22. F. Schmuck and R. Haskin. GPFS: A shared-disk file system for large computing clusters. In FAST '02: Proceedings of the 1st USENIX Conference on File and Storage Technologies, pages 231-244, Berkeley, CA, USA, 2002. USENIX Association.
23. SIONlib. http://www.fz-juelich.de/jsc/sionlib/.
24. G. Sutmann, R. G. Winkler, and G. Gompper. Multi-particle collision dynamics coupled to molecular dynamics on massively parallel computers. (in preparation).