Performance characterization and optimization of parallel I/O on the cray XT

IPDPS Miami 2008 - Proceedings of the 22nd IEEE International Parallel and Distributed Processing Symposium, Program and CD-ROM

Volumn , Issue , 2008, Pages

  Yu, Weikuan ^a   Vetter, Jeffrey S ^a   Oral, H Sarp ^a  

^a OAK RIDGE NATIONAL LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BENCHMARKING; COMPUTER NETWORKS; DISTRIBUTED PARAMETER NETWORKS; METADATA; OPTIMIZATION; SUPERCOMPUTERS;

COLLECTIVE I/O; I/O PERFORMANCE; PARALLEL I/O;

DATA STORAGE EQUIPMENT;

EID: 51049089559     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPS.2008.4536277     Document Type: Conference Paper

References (39)

1. FLASH I/O Benchmark Routine - Parallel HDF 5,
2. IOR Benchmark, http://www.llnl.gov/asci/purple/benchmarks/limited/ior.
3. TOP 500 Supercomputers, http://www.top500.org/.
4. Simulation and Modeling at the Exascale for Energy, Ecological Sustainability and Global Security (E3SGS), Town Hall Meetings, http://hpcrd.lbl.gov/E3SGS/main.html, 2007.
5. N. R. Adiga, G. Almasi, G. S. Almasi, et al., An overview of the BlueGene/L Supercomputer, in Proceedings of the 2002 ACM/IEEE conference on Supercomputing Baltimore, Maryland: IEEE Computer Society Press, 2002.
6. S. R. Alam, R. F. Barrett, M. R. Fahey, J. A. Kuehn, J. M. Larkin, R. Sankaran, and P. H. Worley, Cray XT4: An Early Evaluation for PetaScale Scientific Simulation, in ACM/IEEE conference on High Performance Networking and Computing (SC07), Reno, NV, 2007.
7. N. J. Boden, D. Cohen, and others, Myrinet: A Gigabitper-Second Local Area Network, IEEE Micro, pp. 29-35, Feb 1995.
8. R. Brightwell, W. Camp, B. Cole, E. DeBenedictis, R. Leland, J. Tomkins, and A. B. MacCabe, Architectural specification for massively parallel computers: an experience and measurement-based approach: Research Articles, Concurr. Comput. : Pract. Exper., vol. 17, pp. 1271-1316, 2005.
9. R. Brightwell, D. Dourfler, and K. D. Underwood, A Comparison of 4X InfiniBand and Quadrics Elan-4 Technology, in Proceedings of Cluster Computing, '04, San Diego, California, 2004.
10. R. Brightwell, R. Riesen, B. Lawry, and A. B. Maccabe, Portals 3.0: Protocol Building Blocks for Low Overhead Communication, in Proceedings of the 2002 Workshop on Communication Architecture for Clusters (CAC), 2002.
11. Cluster File System, Lustre: A Scalable, High Performance File System,
12. DataDirect Network, S2A Nearline SATA Solution Overview, 2005.
13. DDN, Products: S2A9550,
14. M. Fahey, J. Larkin, and J. Adams, I/O Performance on a Massively Parallel Cray XT3/XT4, in 22nd IEEE International Parallel & Distributed Processing Symposium (IPDPS'08), Miami, FL, 2008.
15. D. Filliger, Personal Communication, 2007.
16. E. R. Hawkes, R. Sankaran, J. C. Sutherland, and J. H. Chen, Direct Numerical Simiulation of Turbulent Combustion: Fundamental Insights towards predictive Models, Journal of Physics: Conference Series, pp. 65-79, 2005.
17. High End Computing Extenstions Working Group (HECEWG), Manpage - openg (group open), http://www.opengroup.org/platform/hecewg/uploads/40/10899/openg. pdf.
18. HPCWire, NSB Approves Funds for Petascale Computing Systems, http://www.hpcwire.com/hpc/1715754.html, 2007.
19. J. Laros, L. Ward, R. Klundt, S. Kelly, J. Tomkins, and B. Kellogg, Red Storm IO Performance Analysis, in Cluster, Austin, TX, 2007.
20. Z. Lin, S. Ethier, T. S. Hahm, and W. M. Tang, Size Scaling of Turbulent Transport in Magnetically Confined Plasmas, Phys. Rev. Lett., vol. 88, p. 195004, Apr 2002.
21. J. Liu, B. Chandrasekaran, J. Wu, W. Jiang, S. P. Kini, W. Yu, D. Buntinas, P. Wyckoff, and D. K. Panda, Performance Comparison of MPI implementations over Infiniband, Myrinet and Quadrics, in Proceedings of Supercomputing '03 (SC '03), 2003.
22. J. Liu, B. Chandrasekaran, W. Yu, J. Wu, D. Buntinas, S. P. Kini, P. Wyckoff, and D. K. Panda, Micro-Benchmark Performance Comparison of High-Speed Cluster Interconnects, IEEE Micro, vol. 24, pp. 42-51, January-February 2004.
23. Los Alamos National Laboratory, High-Performance Computing: RoadRunner, http://www.lanl.gov/roadrunner/, 2006.
24. K. Pedretti, R. Brightwell, and J. Williams, Cplant" Runtime System Support for Multi-Processor and Heterogeneous Compute Nodes, in Proceedings of the IEEE International Conference on Cluster Computing: IEEE Computer Society, 2002.
25. F. Petrini, W.-c. Feng, A. Hoisie, S. Coll, and E. Frachtenberg, The Quadrics network: High-performance clustering technology, IEEE Micro, vol. 22, pp. 46-57, January/February 2002.
26. R. B. Ross, Parallel I/O Benchmarking Consortium,
27. H. Shan and J. Shalf, Using IOR to Analyze the I/O Performance of XT3, in Cray User Group (CUG), Seattle, WA, 2007.
28. R. Thakur and A. Choudhary, An Extended Two-Phase Method for Accessing Sections of Out-of-Core Arrays, Scientific Programming, vol. 5, pp. 301-317, Winter 1996.
29. R. Thakur, W. Gropp, and E. Lusk, Data Sieving and Collective I/O in ROMIO, in Proceedings of the Seventh Symposium on the Frontiers of Massively Parallel Computation, 1999, pp. 182-189.
30. R. Thakur, W. Gropp, and E. Lusk, On Implementing MPI-IO Portably and with High Performance, in Proceedings of the 6th Workshop on I/O in Parallel and Distributed Systems, 1999, pp. 23-32.
31. R. Thakur, W. Gropp, and E. Lusk, Optimizing noncontiguous accesses in MPI-IO, Parallel Computing, vol. 28, pp. 83-105, 2002.
32. The National Center for SuperComputing, HDF5 Home Page,
33. G. M. Timothy, S. David, and R. W. Stephen, A TeraFLOP Supercomputer in 1996: The ASCI TFLOP System, in Proceedings of the 10th International Parallel Processing Symposium: IEEE Computer Society, 1996.
34. J. S. Vetter, S. R. Alam, T. H. Dunigan, Jr.,, M. R. Fahey, P. C. Roth, and P. H. Worley, Early Evaluation of the Cray XT3, in IEEE International Parallel & Distributed Processing Symposium (IPDPS), Rhodes Island, Greece, 2006.
35. J. S. Vetter and F. Mueller, Communication Characteristics of Large-Scale Scientific Applications for Contemporary Cluster Architectures, in IPDPS, 2002.
36. K. Voruganti and P. Sarkar, An Analysis of Three Gigabit Networking Protocols for Stroage Area Networks, in Proceedings of International Conference on Performance, Computing, and Communications, 2001.
37. P. Wong and R. F. Van der Wijngaart, NAS Parallel Benchmarks I/O Version 2.4, NASA Advanced Supercomputing (NAS) Division NAS-03-002, 2002.
38. W. Yu, H. S. Oral, J. Vetter, and R. Barrett, Efficiency Evaluation of Cray XT Parallel IO Stack, in Cray User Group Meeting (CUG 2007, Seattle, Washington, 2007.
39. W. Yu, J. S. Vetter, and R. S. Canon, OPAL: An Open-Source MPI-IO Library over Cray XT, in International Workshop on Storage Network Architecture and Parallel I/O (SNAPI'07), San Diego, CA, 2007.