Understanding and improving computational science storage access through continuous characterization

IEEE Symposium on Mass Storage Systems and Technologies

Volumn , Issue , 2011, Pages

  Carns, Philip ^a   Harms, Kevin ^a   Allcock, William ^a   Bacon, Charles ^a   Lang, Samuel ^a   Latham, Robert ^a   Ross, Robert ^a  

^a ARGONNE NATIONAL LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPLICATION DOMAINS; BEHAVIOR CHANGE; BLUE GENE; BURSTINESS; COMPUTATIONAL SCIENCE; FILE SYSTEMS; I/O PERFORMANCE; JOB SIZE; NEW MECHANISMS; PETABYTES; PRODUCTION SYSTEM; SCIENCE AND ENGINEERING; SPECIFIC COMPONENT; STORAGE SYSTEMS;

CHARACTERIZATION; SUPERCOMPUTERS; SYSTEMS ANALYSIS; VIRTUAL STORAGE;

BEHAVIORAL RESEARCH;

EID: 79960911196     PISSN: 21601968     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/MSST.2011.5937212     Document Type: Conference Paper

References (27)

1. S. Lang, P. Carns, R. Latham, R. Ross, K. Harms, and W. Allcock, "I/O performance challenges at leadership scale," in SC '09: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis. New York, NY, USA: ACM, 2009, pp. 1-12.
2. G. R. Ganger, "Generating representative synthetic workloads: An unsolved problem," in Proceedings of the Computer Measurement Group (CMG) Conference, 1995, pp. 1263-1269.
3. A. Traeger, E. Zadok, N. Joukov, and C. P. Wright, "A nine year study of file system and storage benchmarking," Trans. Storage, vol. 4, no. 2, pp. 1-56, 2008.
4. N. Agrawal, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau, "Towards realistic file-system benchmarks with CodeMRI," SIGMETRICS Perform. Eval. Rev., vol. 36, no. 2, pp. 52-57, 2008.
5. E. Anderson, "Capture, conversion, and analysis of an intense NFS workload," in FAST '09: Proccedings of the 7th conference on File and storage technologies. Berkeley, CA, USA: USENIX Association, 2009, pp. 139-152.
6. A. W. Leung, S. Pasupathy, G. Goodson, and E. L. Miller, "Measurement and analysis of large-scale network file system workloads," in Proceedings of the 2008 USENIX Technical Conference. Berkeley, CA, USA: USENIX Association, 2008, pp. 213-226. [Online]. Available: http://portal.acm.org/citation.cfm?id=1404014.1404030
7. P. Carns, R. Latham, R. Ross, K. Iskra, S. Lang, and K. Riley, "24/7 characterization of petascale I/O workloads," in Proceedings of 2009 Workshop on Interfaces and Architectures for Scientific Data Storage, September 2009.
8. "U.S. Department of Energy INCITE program," http://www.er.doe.gov/ascr/incite/
9. F. Schmuck and R. Haskin, "GPFS: A shared-disk file system for large computing clusters," in Proceedings of the FAST 2002 Conference on File and Storage Technologies, January 2002.
10. S. Godard, "SYSSTAT utilities home page," http:// pagespersoorange.fr/sebastien.godard/.
11. S. Dayal, "Characterizing HEC storage systems at rest," Carnegie Mellon University Parallel Data Lab, Tech. Rep. CMU-PDL-08-109, 2008.
12. W. Liao and A. Choudhary, "Dynamically adapting file domain partitioning methods for collective I/O based on underlying parallel file system locking protocols," in Proceedings of the 2008 ACM/IEEE conference on Supercomputing. IEEE Press, Piscataway, NJ, 2008.
13. N. Nieuwejaar, D. Kotz, A. Purakayastha, C. S. Ellis, and M. Best, "File-access characteristics of parallel scientific workloads," IEEE Transactions on Parallel and Distributed Systems, vol. 7, no. 10, pp. 1075-1089, October 1996. [Online]. Available: http://www.computer.org/tpds/td1996/ l1075abs.htm. (Pubitemid 126780465)
14. Y. Kim, R. Gunasekaran, G. Shipman, D. Dillow, Z. Zhang, and B. Settlemyer, "Workload characterization of a leadership class storage cluster," in 5th Petascale Data Storage Workshop (PDSW), 2010, pp. 1-5.
15. H. Yu, R. K. Sahoo, C. Howson, G. Almasi, J. G. Castanos, M. Gupta, J. E. Moreira, J. J. Parker, T. E. Engelsiepen, R. Ross, R. Thakur, R. Latham, and W. D. Gropp, "High performance file I/O for the BlueGene/L supercomputer," in Proceedings of the 12th International Symposium on High-Performance Computer Architecture, 2006.
16. E. Smirni and D. Reed, "Workload characterization of input/output intensive parallel applications," in Proceedings of the Conference on Modelling Techniques and Tools for Computer Performance Evaluation, ser. Lecture Notes in Computer Science, vol. 1245. Springer-Verlag, June 1997, pp. 169-180. [Online]. Available: http://vibes.cs.uiuc.edu/Publications/Papers/Tools97.ps.gz (Pubitemid 127075186)
17. D. A. Reed, R. A. Aydt, R. J. Noe, P. C. Roth, K. A. Shields, B. W. Schwartz, and L. F. Tavera, "Scalable performance analysis: The Pablo performance analysis environment," in Proceedings of the Scalable parallel libraries conference. IEEE Computer Society, 1993, pp. 104-113.
18. F. Wang, Q. Xin, B. Hong, S. A. Brandt, E. L. Miller, D. D. E. Long, and T. T. Mclarty, "File system workload analysis for large scale scientific computing applications," in Proceedings of the 21st IEEE/12th NASA Goddard Conference on Mass Storage Systems and Technologies, 2004, pp. 139-152.
19. A. Uselton, M. Hawison, N. Wright, D. Skinner, J. Shalf, L. Oliker, N. Keen, and K. Karavanic, "Parallel I/O performance: From events to ensembles," in Proceedings of the 24th IEEE International Parallel and Distributed Processing Symposium, 2010.
20. N. J. Wright, W. Pfeiffer, and A. Snavely, "Characterizing parallel scaling of scientific applications using IPM," in 10th LCI International Conference on High-Performance Clustered Computing, 2009.
21. S. Seelam, I.-H. Chung, D.-Y. Hong, H.-F. Wen, and H. Yu, "Early experiences in application level I/O tracing on Blue Gene systems," in Proceedings of the 2008 IEEE International Parallel and Distributed Processing Symposium, 2008.
22. "HPC-5 open source software projects: LANL-Trace," http://institute.lanl.gov/data/software/#lanl-trace.
23. P. C. Roth, "Characterizing the I/O behavior of scientific applications on the Cray XT," in PDSW '07: Proceedings of the 2nd International Workshop on Petascale Data Storage. New York, NY, USA: ACM, 2007, pp. 50-55.
24. J. S. Vetter and M. O. McCracken, "Statistical scalability analysis of communication operations in distributed applications," SIGPLAN Notices, vol. 36, no. 7, pp. 123-132, 2001.
25. A. Konwinski, J. Bent, J. Nunez, and M. Quist, "Towards an I/O tracing framework taxonomy," in PDSW '07: Proceedings of the 2nd international workshop on Petascale data storage. New York, NY, USA: ACM, 2007, pp. 56-62.
26. R. Klundt, M. Weston, and L. Ward, "I/O tracing on Catamount," Sandia National Laboratory, Tech. Rep. SAND2008-3684, 2008.
27. "Darshan," http://www.mcs.anl.gov/research/projects/darshan/.