Evaluating I/O characteristics and methods for storing structured scientific data

20th International Parallel and Distributed Processing Symposium, IPDPS 2006

Volumn 2006, Issue , 2006, Pages

  Ching, Avery ^a   Choudhary, Alok ^a   Liao, Wei Keng ^a   Ward, Lee ^b   Pundit, Neil ^b  

^a NORTHWESTERN UNIVERSITY   (United States)

^b SANDIA NATIONAL LABORATORIES   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER SIMULATION; DATA PROCESSING; DATA STORAGE EQUIPMENT; DATABASE SYSTEMS; INTERFACES (COMPUTER);

MULTI-DIMENSIONAL DATASETS; POST-PROCESSING; SCIENTIFIC APPLICATION; SCIENTIFIC DATA;

DATA STRUCTURES;

EID: 33847094265     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPS.2006.1639306     Document Type: Conference Paper

References (16)

1. S. J. Baylor and C. E. Wu. Parallel I/O workload characteristics using Vesta. In Proceedings of the IPPS '95 Workshop on Input/Output in Parallel and Distributed Systems, pages 16-29, April 1995.
2. P. H. Carns, W. B. Ligon III, R. B. Ross, and R. Thakur. PVFS: A parallel file system for Linux clusters. In Proceedings of the 4th Annual Linux Showcase and Conference, pages 317-327, Atlanta, GA, October 2000. USENIX Association.
3. A. Ching, A. Choudhary, K. Coloma, W. keng Liao, R. Ross, and W. Gropp. Noncontiguous access through MPI-IO. In Proceedings of the 3rd IEEE/A CM International Symposium on Cluster Computing and the Grid, May 2003.
4. A. Ching, A. Choudhary, W. keng Liao, R. Ross, and W. Gropp. Evaluating structured I/O methods for parallel file systems. In International Journal of High Performance Computing and Networking.
5. P. F. Corbett and D. G. Feitelson. The Vesta parallel file system. ACM Transactions on Computer Systems, 14(3):225-264, August 1996.
6. P. E. Crandall, R. A. Aydt, A. A. Chien, and D. A. Reed. Input/output characteristics of scalable parallel applications. In Proceedings of Supercomputing '95, San Diego, CA, December 1995. IEEE Computer Society Press.
7. B. Fryxell, K. Olson, P. Ricker, F. X. Timmes, M. Zingale, D. Q. Lamb, P. MacNeice, R. Rosner, and H. Tufo. FLASH: An adaptive mesh hydrodynamics code for modeling astrophysical thermonuclear flashes. Astrophysical Journal Supplement, 131:273, 2000.
8. HDF5 home page. http://hdf.ncsa.uiuc.edu/HDF5/.
9. J. Li, W. keng Liao, A. Choudhary, R. Ross, R. Thakur, W. Gropp, R. Latham, A. Sigel, B. Gallagher, and M. Zingale. Parallel netcdf: A high-performance scientific I/O interface. In Proceedings of Supercomputing 2003, November 2003.
10. N. Nieuwejaar and D. Kotz. The Galley parallel file system. In Proceedings of the 10th ACM International Conference on Supercomputing, pages 374-381, Philadelphia, PA, May 1996. ACM Press.
11. M. L. Norman, J. Shalf, S. Levy, and G. Daues. Diving deep: Data-management and visualization strategies for adaptive mesh refinement simulations. Computing in Science and Engg., 1(4):36-47, 1999.
12. J. Saltz, U. Catalyurek, T. Kurc, M. Gray, S. Hastings, S. Langella, S. Narayanan, R. Martino, S. Bryant, M. Peszynska, M. Wheeler, A. Sussman, M. Beynon, C. Hansen, D. Stredney, and D. Sessanna. Driving scientific applications by data in distributed environments. In Workshop on Dynamic Data-Driven Application Systems, June 2003.
13. F. Schmuck and R. Haskin. GPFS: A shared-disk file system for large computing clusters. In Proceedings of the FAST 2002 Conference on File and Storage Technologies, San Jose, CA, January 2002. IBM Almaden Research Center.
14. R. Thakur, W. Gropp, and E. Lusk. Data sieving and collective I/O in ROMIO. In Proceedings of the Seventh Symposium on the Frontiers of Massively Parallel Computation, pages 182-189. IEEE Computer Society Press, February 1999.
15. R. Thakur, W. Gropp, and E. Lusk. On implementing MPI-IO portably and with high performance. In Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems, pages 23-32, May 1999.
16. The parallel virtual file system 2 (PVFS2). http://www.pvfs.org/pvfs2/.