Dynamically adapting file domain partitioning methods for collective i/o based on underlying parallel file system locking protocols

2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2008

Volumn , Issue , 2008, Pages

  Liao, Wei Keng ^a   Choudhary, Alok ^a  

^a Northwestern University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COLLECTIVE I/O; DATA CONSISTENCY; DEGREE OF PARALLELISM; DOMAIN PARTITIONING; FILE SYSTEMS; I/O BENCHMARK; IMPLEMENTATION PLATFORMS; LOCKING MECHANISM; LOCKING PROTOCOLS; PARALLEL FILE SYSTEM; PARTITIONING METHODS; PROCESS COLLABORATION; SIGNIFICANT IMPACTS;

COMPUTER SCIENCE;

EID: 70350752443     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SC.2008.5222722     Document Type: Conference Paper

References (27)

1. H. Shan and J. Shalf, "Using IOR to Analyze the I/O Performance of XT3," in the Cray User Group Conference, May 2007.
2. Message Passing Interface Forum, MPI-2: Extensions to the Message Passing Interface, Jul. 1997, http://www.mpi-forum.org/docs/docs.html.
3. J. del Rosario, R. Brodawekar, and A. Choudhary, "Improved Parallel I/O via a Two-Phase Run-time Access Strategy," in the Workshop on I/O in Parallel Computer Systems at IPPS '93, Apr. 1993, pp. 56-70.
4. D. Kotz, "Disk-directed I/O for MIMD Multiprocessors," ACM Transactions on Computer Systems, vol. 15, no. 1, pp. 41-74, Feb. 1997.
5. R. Thakur, W. Gropp, and E. Lusk, Users Guide for ROMIO: A High-Performance, Portable MPI-IO Implementation, Technical Report ANL/MCS-TM-234, Mathematics and Computer Science Division, Argonne National Laboratory, Oct. 1997.
6. J. Squyres, A. Lumsdaine, W. George, J. Hagedorn, and J. Devaney, "The interoperable message passing interface (IMPI) extensions to LAM/MPI," in Proceedings, MPI Developers Conference (MPIDC), March 2000.
7. R. Thakur, W. Gropp, and E. Lusk, "An Abstract-Device Interface for Implementing Portable Parallel-I/O Interfaces," in the 6th Symposium on the Frontiers of Massively Parallel Computation, Oct. 1996.
8. -, "Data Sieving and Collective I/O in ROMIO," in the 7th Symposium on the Frontiers of Massively Parallel Computation, Feb. 1999.
9. B. Nitzberg and V. Lo, "Collective Buffering: Improving Parallel I/O Performance," in the Sixth IEEE International Symposium on High Performance Distributed Computing, August 1997, pp. 148-157.
10. R. Ross, R. Latham, W. Gropp, R. Thakur, and B. Toonen, "Implementing MPI-IO Atomic Mode Without File System Support," in the 5th IEEE/ACM International Symposium on Cluster Computing and the Grid, May 2005.
11. W. Liao, A. Ching, K. Coloma, A. Choudhary, and L. Ward, "An Implementation and Evaluation of Client-Side File Caching for MPI-IO," in the International Parallel and Distributed Processing Symposium, Mar. 2007.
12. IEEE/ANSI Std. 1003.1, Portable Operating System Interface (POSIX)- Part 1: System Application Program Interface (API) [C Language], 1996.
13. J. Prost, R. Treumann, R. Hedges, B. Jia, and A. Koniges, "MPIIO/ GPFS, an Optimized Implementation of MPI-IO on top of GPFS," in Supercomputing, Nov. 2001.
14. F. Schmuck and R. Haskin, "GPFS: A Shared-Disk File System for Large Computing Clusters," in the Conference on File and Storage Technologies (FAST'02), Jan. 2002, pp. 231-244.
15. Lustre: A Scalable, High-Performance File System, Whitepaper, Cluster File Systems, Inc., 2003.
16. J. Prost, R. Treumann, R. Hedges, A. Koniges, and A. White, "Towards a High-Performance Implementation of MPI-IO on top of GPFS," in the Sixth International Euro-Par Conference on Parallel Processing, Aug. 2000.
17. K. Seamons, Y. Chen, P. Jones, J. Jozwiak, and M. Winslett, "Serverdirected Collective I/O in Panda," in Supercomputing, Nov. 1995.
18. K. Coloma, A. Choudhary, W. Liao, W. Lee, E. Russell, and N. Pundit, "Scalable High-level Caching for Parallel I/O," in the International Parallel and Distributed Processing Symposium, Apr. 2004.
19. K. Coloma, A. Ching, A. Choudhary, W. Liao, R. Ross, R. Thakur, and L. Ward, "A new flexible MPI collective I/O implementation," in the IEEE Conference on Cluster Computing, Sep. 2006.
20. X. Ma, M. Winslett, J. Lee, and S. Yu, "Improving MPI-IO Output Performance with Active Buffering Plus Threads," in the International Parallel and Distributed Processing Symposium, Apr. 2003.
21. K. Coloma, A. Choudhary, W. Liao, W. Lee, and S. Tideman, "DAChe: Direct Access Cache System for Parallel I/O," in the 20th International Supercomputer Conference, Jun. 2005.
22. H. Yu, R. Sahoo, C. Howson, G. Almasi, J. Castanos, M. Gupta, J. Moreira, J. Parker, T. Engelsiepen, R. Ross, R. Thakur, R. Latham, and W. D. Gropp, "High Performance File I/O for the BlueGene/L Supercomputer," in the 12th International Symposium on High-Performance Computer Architecture (HPCA-12), Feb. 2006.
23. P. Wong and R. der Wijngaart, "NAS Parallel Benchmarks I/O Version 2.4," NASA Ames Research Center, Moffet Field, CA, Tech. Rep. NAS- 03-002, Jan. 2003.
24. M. Zingale, "FLASH I/O Benchmark Routine - Parallel HDF 5," Mar. 2001,http://flash.uchicago.edu/-zingale/flash benchmark io.
25. B. Fryxell, K. Olson, P. Ricker, F. X. Timmes, M. Zingale, D. Q. Lamb, P. MacNeice, R. Rosner, and H. Tufo, "FLASH: An Adaptive Mesh Hydrodynamics Code for Modelling Astrophysical Thermonuclear Flashes," Astrophysical Journal Suppliment, pp. 131-273, 2000.
26. HDF Group, Hierarchical Data Format, Version 5, The National Center for Supercomputing Applications, http://hdf.ncsa.uiuc.edu/HDF5.
27. R. Sankaran, E. Hawkes, J. Chen, T. Lu, and C. Law, "Direct Numerical Simulations of Turbulent Lean Premixed Combustion," Journal of Physics: conference series, vol. 46, pp. 38-42, 2006