I/O performance challenges at leadership scale

Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, SC '09

Volumn , Issue , 2009, Pages

  Lang, Samuel ^a   Carns, Philip ^a   Latham, Robert ^a   Ross, Robert ^a   Harms, Kevin ^a   Allcock, William ^a  

^a ARGONNE NATIONAL LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL SCIENCE; COMPUTING FACILITIES; HIGH PERFORMANCE COMPUTING SYSTEMS; I/O PERFORMANCE; I/O SYSTEMS; SCALABLE I/O; STORAGE NODES; STORAGE SYSTEMS;

SUPERCOMPUTERS;

EID: 74049087130     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1654059.1654100     Document Type: Conference Paper

References (31)

1. W. Yu, J. Vetter, R. Canon, and S. Jiang, "Exploiting Lustre file joining for effective collective I/O," in Proceedings of the Seventh IEEE International Symposium on Cluster Computing and the Grid, 2007. CCGRID 2007, 2007, pp. 267-274.
2. W. Liao and A. Choudhary, "Dynamically adapting file domain partitioning methods for collective I/O based on underlying parallel file system locking protocols," in Proceedings of the 2008 ACM/IEEE conference on Supercomputing. IEEE Press, Piscataway, NJ, 2008.
3. F. Schmuck and R. Haskin, "GPFS: A shared-disk file system for large computing clusters," in Proceedings of the FAST 2002 Conference on File and Storage Technologies, January 2002.
4. B. Welch, M. Unangst, Z. Abbasi, G. Gibson, B. Mueller, J. Small, J. Zelenka, and B. Zhou, "Scalable performance of the Panasas parallel file system," in Proceedings of the 6th USENIX Conference on File and Storage Technologies, February 2008.
5. S. A. Weil, S. A. Brandt, E. L. Miller, D. D. E. Long, and C. Maltzahn, "Ceph: a scalable, high-performance distributed file system," in OSDI '06: Proceedings of the 7th symposium on Operating Systems Design and Implementation. Berkeley, CA: USENIX Association, 2006, pp. 307-320.
6. R. Hedges, B. Loewe, T. McLarty, and C. Morrone, "Parallel file system testing for the lunatic fringe: the care and feeding of restless I/O power users," in Proceedings of the 22nd IEEE/13th NASA Goddard Conference on Mass Storage Systems and Technologies, April 2005.
7. W. Yu, S. Oral, J. Vetter, and R. Barrett, "Efficiency evaluation of Cray XT parallel I/O stack," in Cray Users Group Meeting, 2007.
8. R. Oldfield, L. Ward, R. Riesen, A. Maccabe, P. Widener, and T. Kordenbrock, "Lightweight I/O for scientific applications," in Proceedings of the IEEE International Conference on Cluster Computing, 2006, pp. 1-11.
9. M. Oberg, H. Tufo, and M. Woitaszek, "Exploration of Parallel Storage Architectures for a Blue Gene/L on the TeraGrid," in 9th LCI International Conference on High-Performance Clustered Computing, March 2008.
10. J. Laros, L. Ward, R. Klundt, S. Kelly, J. Tomkins, and B. Kellogg, "Red Storm IO Performance Analysis," Cluster, Austin, TX, 2007.
11. W. Yu, J. S. Vetter, and H. S. Oral, "Performance characterization and optimization of parallel I/O on the Cray XT," in Proceedings of the 22nd IEEE International Parallel and Distributed Processing Symposium, April 2008.
12. M. Fahey, J. Larkin, and J. Adams, "I/O performance on a massively parallel Cray XT3/XT4," in International Symposium on Parallel and Distributed Processing, 2008, April 2008, pp. 1-12.
13. M. Kandaswamy, M. Kandemir, A. Choudhary, and D. Bernholdt, "Performance implications of architectural and software techniques on I/O-intensive applications," in International Conference on Parallel Processing, 1998, 1998, pp. 493-500.
14. J. Borrill, L. Oliker, J. Shalf, and H. Shan, "Investigation of leading HPC I/O performance using a scientific-application derived benchmark," in Proceedings of Supercomputing, November 2007.
15. H. Shan, K. Antypas, and J. Shalf, "Characterizing and predicting the I/O performance of HPC applications using a parameterized synthetic benchmark," in Proceedings of Supercomputing, November 2008.
16. R. T. Fisher, L. P. Kadanoff, D. Q. Lamb, A. Dubey, T. Plewa, A. Calder, F. Cattaneo, P. Constantin, I. Foster, M. E. Papka, S. I. Abarzhi, S. M. Asida, P. M. Rich, C. C. Glendening, K. Antypas, D. J. Sheeler, L. B. Reid, B. Gallagher, and S. G. Needham, "Terascale turbulence computation using the FLASH3 application framework on the IBM Blue Gene/L system," IBM J. Res. Dev., vol. 52, no. 1/2, pp. 127-136, 2008.
17. P. Carns, S. Lang, R. Ross, M. Vilayannur, J. Kunkel, and T. Ludwig, "Small-file access in parallel file systems," in Proceedings of the 23rd IEEE International Parallel and Distributed Processing Symposium, April 2009.
18. P. Carns, W. Ligon III, R. Ross, and R. Thakur, "PVFS: A parallel file system for Linux clusters," in Proceedings of the 4th Annual Linux Showcase & Conference, Atlanta-Volume 4. USENIX Association Berkeley, CA, 2000, pp. 28-28.
19. A. Ching, A. Choudhary, W. Liao, R. Ross, and W. Gropp, "Noncontiguous I/O through PVFS," in Proceedings of the 2002 IEEE International Conference on Cluster Computing, 2002, pp. 405-414.
20. D. Hildebrand and P. Honeyman, "Exporting storage systems in a scalable manner with pNFS," in Proceedings of the 22nd IEEE/13th NASA Goddard Conference on Mass Storage Systems and Technologies, 2005, pp. 18-27.
21. R. Thakur, W. Gropp, and E. Lusk, "Data sieving and collective I/O in ROMIO,"Mathematics and Computer Science Division, Argonne National Laboratory, Tech. Rep. ANL/MCS-P723-0898, August 1998. [Online]. Available: http://www.mcs.anl.gov/ thakur/papers/romio-coll.ps
22. R. Latham, R. Ross, and R. Thakur, "The impact of file systems on MPI-IO scalability," Lecture Notes in Computer Science, pp. 87-96, 2004.
23. A. Ching, A. Choudhary, W. K. Liao, R. Ross, and W. Gropp, "Evaluating structured I/O methods for parallel file systems," International Journal of High Performance Computing and Networking, vol. 2, pp. 133-145, 2004.
24. H. Yu, R. Sahoo, C. Howson, G. Almasi, J. Castanos, M. Gupta, J. Moreira, J. Parker, T. Engelsiepen, R. Ross et al., "High performance file I/O for the Blue Gene/L supercomputer," in Proceedings of the Twelfth International Symposium on High-Performance Computer Architecture, 2006, pp. 187-196.
25. IEEE, 1003.1-1988 INT/1992 Edition, IEEE Standard Interpretations of IEEE Standard Portable Operating System Interface for Computer Environments (IEEE Std 1003.1-1988). New York, NY: IEEE, 1988.
26. D. Kotz and N. Nieuwejaar, "Dynamic file-access characteristics of a production parallel scientific workload," in Proceedings of Supercomputing, 1994, pp. 640-649.
27. D. Koester, "What makes HPC applications challenging?" in Workshop on Patterns in High Performance Computing, May 2005. [Online]. Available: http://charm.cs.uiuc.edu/patHPC/papers/koester.ppt
28. A. Phanishayee, E. Krevat, V. Vasudevan, D. Andersen, G. Ganger, G. Gibson, and S. Seshan, "Measurement and analysis of TCP throughput collapse in cluster-based storage systems," Carnegie Mellon University, Tech. Rep. CMU-PDL-07-105, September 2007.
29. Philip Carns and Robert Latham and Robert Ross and Kamil Iskra and Samuel Lang and Katherine Riley, "24/7 Characterization of Petascale I/O Workloads," in Proceedings of the First Workshop on Interfaces and Abstractions for Scientific Data Storage, New Orleans, LA, September 2009.
30. P. Wong and R. der Wijngaart, "NAS parallel benchmarks I/O version 2.4," no. NAS-03-002, 2003.
31. J. Borrill, J. Carter, L. Oliker, D. Skinner, and R. Biswas, "Integrated performance monitoring of a cosmology application on leading HEC platforms," in Proceedings of the International Conference on Parallel Processing, June 20