24/7 characterization of petascale I/O workloads

Proceedings - IEEE International Conference on Cluster Computing, ICCC

Volumn , Issue , 2009, Pages

  Carns, Philip ^a   Latham, Robert ^a   Ross, Robert ^a   Iskra, Kamil ^a   Lang, Samuel ^a   Riley, Katherine ^a  

^a ARGONNE NATIONAL LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACCESS PATTERNS; APPLICATION DEVELOPMENT; COMPUTATIONAL SCIENCE; FUTURE APPLICATIONS; I/O WORKLOAD; MEMORY ACCESS; PETASCALE; RESEARCH COMMUNITIES; SCALE SYSTEM; SCIENTIFIC APPLICATIONS;

CLUSTER COMPUTING; COMPUTER SCIENCE; MESSAGE PASSING; TECHNICAL PRESENTATIONS;

TUNING;

EID: 72049121631     PISSN: 15525244     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CLUSTR.2009.5289150     Document Type: Conference Paper

References (26)

1. O. Zaki, E. Lusk, W. Gropp, and D. Swider, "Toward scalable performance visualization with Jumpshot," High Performance Computing Applications, vol.13, no.2, pp. 277-288, Fall 1999.
2. A. Chan, W. Gropp, and E. Lusk, "An efficient format for nearly constant-time access to arbitrary time intervals in large trace files," Scientific Programming, vol.16, no.2-3, pp. 155-165, 2008.
3. S. Shende and A. D. Malony, "The TAU parallel performance system," International Journal of High Performance Computing Applications, vol.20, no.2, pp. 287-331, Summer 2006.
4. D. Arnold, D. Ahn, B. de Supinski, G. Lee, B. Miller, and M. Schulz, "Stack trace analysis for large scale debugging," in Proceedings of the IEEE International Parallel and Distributed Processing Symposium, 2007, March 2007, pp. 1-10.
5. N. Nieuwejaar, D. Kotz, A. Purakayastha, C. S. Ellis, and M. Best, "File-access characteristics of parallel scientific workloads," IEEE Transactions on Parallel and Distributed Systems, vol.7, no.10, pp. 1075-1089, October 1996. [Online]. Available: http://www.computer.org/tpds/td1996/l1075abs. htm.
6. J. S. Vetter and M. O. McCracken, "Statistical scalability analysis of communication operations in distributed applications," SIGPLAN Notices, vol.36, no.7, pp. 123-132, 2001.
7. "mpiP: Lightweight, scalable MPI profiling," http://mpip.sourceforge.net/.
8. S. Byna, Y. Chen, X.-H. Sun, R. Thakur, and W. Gropp, "Parallel I/O prefetching using MPI file caching and I/O signatures," in SC '08: Proceedings of the 2008 ACM/IEEE Conference on Supercomputing. Piscataway, NJ, USA: IEEE Press, 2008, pp. 1-12.
9. M. Noeth, J. Marathe, F. Mueller, M. Schulz, and B. de Supinski, "Scalable compression and replay of communication traces in massively parallel environments," in SC '06: Proceedings of the 2006 ACM/IEEE Conference on Supercomputing. New York, NY, USA: ACM, 2006.
10. S. Seelam, I.-H. Chung, D.-Y. Hong, H.-F. Wen, and H. Yu, "Early experiences in application level I/O tracing on Blue Gene systems," in Proceedings of the 2008 IEEE International Parallel and Distributed Processing Symposium, 2008.
11. "HPC-5 open source software projects: LANL-Trace," http://institute.lanl.gov/data/software/#lanl-trace.
12. P. C. Roth, "Characterizing the I/O behavior of scientific applications on the Cray XT," in PDSW '07: Proceedings of the 2nd International Workshop on Petascale Data Storage. New York, NY, USA: ACM, 2007, pp. 50-55.
13. "FPMPI-2 fast profiling library for MPI," http://www.mcs.anl. gov/research/projects/fpmpi/WWW/.
14. J. J. Ritsko, I. Ames, S. I. Raider, and J. H. Robinson, "Blue Gene," IBM Journal of Research and Development, vol.49, March/May 2005.
15. "The Parallel Virtual File System," http://www.pvfs.org/.
16. R. Latham, N. Miller, R. B. Ross, and P. H. Carns, "A next-generation parallel file system for Linux clusters," LinuxWorld Magazine, January 2004. [Online]. Available: http://www.pvfs.org/documentation/ papers/linuxworld-JAN2004-PVFS2.pdf.
17. E. Lusk, N. Doss, and A. Skjellum, "A high-performance, portable implementation of the MPI message passing interface standard," Parallel Computing, vol.22, pp. 789-828, 1996.
18. R. Thakur and E. Lusk, "On implementing MPI-IO portably and with high performance," in Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems. ACM Press, 1999, pp. 23-32.
19. B. Jenkins, "Lookup8 hash algorithm," http://burtleburtle.net/ bob/c/lookup8.c.
20. J. Carter, J. Borrill, and L. Oliker, "Performance characteristics of a cosmology package on leading HPC architectures," in HiPC:International Conference on High Performance Computing. Springer, 2004, pp. 176-188.
21. "Chombo - infrastructure for adaptive mesh refinement," https://seesar.lbl.gov/ANAG/chombo/.
22. "Chombo I/O benchmark," http://www.nersc.gov/~ndk/ ChomboBenchmarks/chomboIOBenchmark.html.
23. "HDF5 home page," http://www.hdfgroup.org/HDF5/.
24. R. Sankaran, E. R. Hawkes, J. H. Chen, T. Lu, and C. K. Law, "Direct numerical simulations of turbulent lean premixed combustion," Journal of Physics: Conference Series, vol.46, pp. 38-42, 2006. [Online]. Available: http://stacks.iop.org/1742-6596/46/38
25. J. Li, W. keng Liao, A. Choudhary, R. Ross, R. Thakur, R. Latham, A. Siegel, B. Gallagher, and M. Zingale, "Parallel netCDF: A highperformance scientific I/O interface," in Proceedings of Supercomputing, 2003.
26. R. D. Nair and H. M. Tufo, "Petascale atmospheric general circulation models," Journal of Physics: Conference Series, vol.78, p. 012078 (5pp), 2007. [Online]. Available: http://stacks.iop.org/1742-6596/78/ 012078.