Understanding and improving computational science storage access through continuous characterization

ACM Transactions on Storage

Volumn 7, Issue 3, 2011, Pages

  Carns, Philip ^a   Harms, Kevin ^a   Allcock, William ^a   Bacon, Charles ^a   Lang, Samuel ^a   Latham, Robert ^a   Ross, Robert ^a  

^a ARGONNE NATIONAL LABORATORY   (United States)

Author keywords

I O characterization; Parallel file systems

Indexed keywords

BLUE GENE; COMPUTATIONAL SCIENCE; FILE SYSTEMS; NEW MECHANISMS; PARALLEL FILE SYSTEM; PETABYTES; PRODUCTION SYSTEM; SCIENCE AND ENGINEERING; SPECIFIC COMPONENT; STORAGE SYSTEMS;

FILE ORGANIZATION; SUPERCOMPUTERS; SYSTEMS ANALYSIS; VIRTUAL STORAGE;

CHARACTERIZATION;

EID: 80455127318     PISSN: 15533077     EISSN: 15533093     Source Type: Journal    
DOI: 10.1145/2027066.2027068     Document Type: Conference Paper

References (29)

1. AGRAWAL, N., ARPACI-DUSSEAU, A. C., AND ARPACI-DUSSEAU, R. H. 2008. Towards realistic file-system benchmarks with CodeMRI. SIGMETRICS Perform. Eval. Rev. 36, 2, 52-57
2. ANDERSON, E. 2009. Capture, conversion, and analysis of an intense NFS workload. In Proccedings of the 7th Conference on File and Storage Technologies (FAST'09). USENIX Association, Berkeley, CA, 139-152
3. CARNS, P., LATHAM, R., ROSS, R., ISKRA, K., LANG, S., AND RILEY, K. 2009. 24/7 characterization of petascale I/O workloads. In Proceedings of the Workshop on Interfaces and Architectures for Scientific Data Storage. Darshan 2010. Darshan. http://www.mcs.anl.gov/research/projects/darshan/
4. DAYAL, S. 2008. Characterizing HEC storage systems at rest. Tech. rep. CMU-PDL-08-109, Parallel Data Lab, Carnegie Mellon University
5. GANGER, G. R. 1995. Generating representative synthetic workloads: An unsolved problem. In Proceedings of the Computer Measurement Group (CMG) Conference. 1263-1269
6. GODARD, S. 2010. SYSSTAT utilities homepage. http://pagesperso-orange.fr/ sebastien.godard/
7. INCITE 2010. U.S. Department of Energy INCITE program. http://www.er.doe.gov/ascr/incite/
8. KIM, Y., GUNASEKARAN, R., SHIPMAN, G., DILLOW, D., ZHANG, Z., AND SETTLEMYER, B. 2010. Workload characterization of a leadership class storage cluster. In Proceedings of the 5th Petascale Data Storage Workshop (PDSW). 1-5
9. KLUNDT, R., WESTON, M., AND WARD, L. 2008. I/O tracing on Catamount. Tech. rep. SAND2008-3684, Sandia National Laboratory
10. KONWINSKI, A., BENT, J., NUNEZ, J., AND QUIST, M. 2007. Towards an I/O tracing framework taxonomy. In Proceedings of the 2nd International Workshop on Petascale Data Storage (PDSW'07). ACM, New York, NY, 56-62
11. LANG, S., CARNS, P., LATHAM, R., ROSS, R., HARMS, K., AND ALLCOCK, W. 2009. I/O performance challenges at leadership scale. In Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis (SC'09). ACM, New York, NY, 1-12
12. LANL-TRACE. 2010. HPC-5 open source software projects: LANL-Trace. http://institute.lanl.gov/data/software/#lanl-trace
13. LEUNG, A. W., PASUPATHY, S., GOODSON, G., AND MILLER, E. L. 2008. Measurement and analysis of large-scale network file system workloads. In Proceedings of the USENIX Technical Conference. USENIX Association, Berkeley, CA, 213-226
14. LIAO, W. AND CHOUDHARY, A. 2008. Dynamically adapting file domain partitioning methods for collective I/O based on underlying parallel file system locking protocols. In Proceedings of the ACM/IEEE Conference on Supercomputing. IEEE Press, Los Alamitos, CA
15. NIEUWEJAAR, N., KOTZ, D., PURAKAYASTHA, A., ELLIS, C. S., AND BEST, M. 1996. File-access characteristics of parallel scientific workloads. IEEE Trans. Paral. Distrib. Syst. 7, 10, 1075-1089 (Pubitemid 126780465)
16. NOETH, M., RATN, P., MUELLER, F., SCHULZ, M., AND DE SUPINSKI, B. R. 2009. Scalatrace: Scalable compression and replay of communication traces for high-performance computing. J. Paral. Distrib. Comput. 69, 696-710
17. REED, D. A., AYDT, R. A., NOE, R. J., ROTH, P. C., SHIELDS, K. A., SCHWARTZ, B.W., AND TAVERA, L. F. 1993. Scalable performance analysis: The Pablo performance analysis environment. In Proceedings of the Scalable Parallel Libraries Conference. IEEE Computer Society, 104-113
18. ROTH, P. C. 2007. Characterizing the I/O behavior of scientific applications on the Cray XT. In Proceedings of the 2nd International Workshop on Petascale Data Storage (PDSW'07). ACM, New York, NY, 50-55
19. SCHMUCK, F. AND HASKIN, R. 2002. GPFS: A shared-disk file system for large computing clusters. In Proceedings of the FAST Conference on File and Storage Technologies
20. SEELAM, S., CHUNG, I.-H., HONG, D.-Y.,WEN, H.-F., AND YU, H. 2008. Early experiences in application level I/O tracing on Blue Gene systems. In Proceedings of the IEEE International Parallel and Distributed Processing Symposium
21. SMIRNI, E. AND REED, D. 1997. Workload characterization of input/output intensive parallel applications. In Proceedings of the Conference on Modelling Techniques and Tools for Computer Performance Evaluation. Lecture Notes in Computer Science, vol. 1245. Springer-Verlag, 169-180 (Pubitemid 127075186)
22. TRAEGER, A., ZADOK, E., JOUKOV, N., AND WRIGHT, C. P. 2008. A nine year study of file system and storage benchmarking. ACM Trans. Stor. 4, 2, 1-56
23. USELTON, A., HAWISON, M., WRIGHT, N., SKINNER, D., SHALF, J., OLIKER, L., KEEN, N., AND KARAVANIC, K. 2010. Parallel I/O performance: From events to ensembles. In Proceedings of the 24th IEEE International Parallel and Distributed Processing Symposium
24. VETTER, J. S. AND MCCRACKEN, M. O. 2001. Statistical scalability analysis of communication operations in distributed applications. SIGPLAN Notices 36, 7, 123-132
25. VIJAYAKUMAR, K., MUELLER, F., MA, X., AND ROTH, P. C. 2009. Scalable I/O tracing and analysis. In Proceedings of the 4th Annual Workshop on Petascale Data Storage (PDSW'09). ACM, New York, NY, 26-31
26. WANG, F., XIN, Q., HONG, B., BRANDT, S. A.,MILLER, E. L., LONG, D. D. E., AND MCLARTY, T. T. 2004. File system workload analysis for large scale scientific computing applications. In Proceedings of the 21st IEEE/12th NASA Goddard Conference on Mass Storage Systems and Technologies. 139-152
27. WRIGHT, N. J., PFEIFFER, W., AND SNAVELY, A. 2009. Characterizing parallel scaling of scientific applications using IPM. In Proceedings of the 10th LCI International Conference on High-Performance Clustered Computing
28. WU, X., VIJAYAKUMAR, K.,MUELLER, F.,MA, X., AND ROTH, P. C. 2011. Probabilistic communication and I/O tracing with deterministic replay at scale. In Proceedings of the International Conference on Parallel Processing
29. YU, H., SAHOO, R. K., HOWSON, C., ALMASI, G., CASTANOS, J. G., GUPTA, M.,MOREIRA, J. E., PARKER, J. J., ENGELSIEPEN, T. E., ROSS, R., THAKUR, R., LATHAM, R., AND GROPP, W. D. 2006. High performance file I/O for the BlueGene/L supercomputer. In Proceedings of the 12th International Symposium on High-Performance Computer Architecture.