Improving parallel IO performance of cell-based AMR cosmology applications

Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium, IPDPS 2012

Volumn , Issue , 2012, Pages 933-944

  Yu, Yongen ^a   Rudd, Douglas H ^b   Lan, Zhiling ^a   Gnedin, Nickolay Y ^c   Kravtsov, Andrey ^d   Wu, Jingjin ^a  

^a Illinois Institute of Technology   (United States)

^b YALE UNIVERSITY   (United States)

^c NASA FERMILAB ASTROPHYSICS CENTER   (United States)

^d UNIVERSITY OF CHICAGO   (United States)

Author keywords

adaptive mesh refinement; cosmology simulations; data layout; high performance computing; parallel I O

Indexed keywords

ADAPTIVE MESH REFINEMENT; ADAPTIVE REFINEMENT; APPLICATION PROCESS; CELL-BASED; DATA FORMAT; DATA LAYOUTS; DISK SPACE; HIGH PERFORMANCE COMPUTING; LIMITED BANDWIDTH; LOCAL REGION; MAPPING MECHANISM; MULTIPLE APPLICATIONS; NUMERICAL APPROACHES; PARALLEL I/O; SPACE-FILLING CURVE;

CELLS; COMPUTER SIMULATION; CYTOLOGY; DESIGN; DISTRIBUTED PARAMETER NETWORKS; MAPPING; NUMERICAL ANALYSIS;

COSMOLOGY;

EID: 84866864503     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPS.2012.88     Document Type: Conference Paper

References (32)

1. M. Berger and J. Oliger. Adaptive Mesh Refinement for Hyperbolic Partial Differential Equations. Computational Physics, 1984.
2. R. Rosner, A. Calder, J. Dursi, B. Fryxell, D. Lamb, J. Niemeyer, K. Olson, P. Ricker, F. Timmes, J. Truran, H. Tueo, Y. Young, M. Zingale, E. Lusk, and R. Stevens. Flash code: Studying astrophysics thermonuclear flashes. Computing In Science and Engineering, 2000.
3. G. Bryan, T. Abel, and M. Norman. Achieving Extreme Resolution in Numerical Cosmology Using Adaptive Mesh Refinement: Resolving Primordial Star Formation. In Proc. of Supercomputing '01.
4. A. Razoumov, M. Norman, T. Abel, and D. Scott. Cosmological Hydrogen Reionization with Three-dimensional Radiative Transfer. Astrophysical Journal, 572:695-704, 2002.
5. A. Calder, B. Curtis, L. Dursi, B. Fryxell, G.Henry, P. MacNeice, K.Olson, P.Ricker, R. Rosner, F. Timmes, H. Tufo, J.Turan, and M.Zingale. High-performance reactive fluid flow simulations using adaptive mesh refinement on thousands of processors. In SC '00: Proceedings of the International Conference for High Performance Computing,Networking,Storage and Analysis.
6. B. W. O'Shea, G. Bryan, J. Bordner, M. L. Norman, T. Abel, R. Harkness, and A. Kritsuk. Introducing Enzo, an AMR Cosmology Application. ArXiv Astrophysics e-prints, March 2004.
7. Z. Lan, V. Taylor, and G. Bryan. A Novel Dynamic Load Balancing Scheme for Parallel Systems. Journal of Parallel and Distributed Computing (JPDC), 2002.
8. A. M. Khokhlov. Fully Threaded Tree Algorithms for Adaptive Refinement Fluid Dynamics Simulations. Journal of Computational Physics, 1998.
9. A. V. Kravtsov, A. A. Klypin, and A. M. Khokhlov. Adaptive Refinement Tree: A new High-resolution N-body code for Cosmological Simulations. Astrophys. J. Suppl, 111:73-94, 1997.
10. Z. Zheng, Z. Lan, R. Gupta, S. Coghlan, and P. Beckman. A Practical Failure Prediction with Location and Lead Time for Blue Gene/P. In Proc. DSNW '10, 2010.
11. Extreme Science and Engineering Discovery Environment. https://www.xsede.org/.
12. A.M. Khokhlov, E.S. Oran, and G.O. Thomas. Numerical Simulation of Deflagration-to-Detonation Transition: The Role of ShockFlame Interactions in Turbulent Flames. Combustion and Flame, 2180, 1999.
13. R. Teyssier. Cosmological hydrodynamics with adaptive mesh refinement A new high resolution code called RAMSES. Astronomy, 364:337-364, 2002.
14. Performance of Vector / Parallel Orientated Hydrodynamic Code, chapter Software. Springer, 2005.
15. P. Diener, N. Jansen, A. Khokhlov, and I. Novikov. Adaptive mesh refinement approach to construction of initial data for black hole collisions. Astrophysics, pages 1-13, 2008.
16. A. Kravtsov. Dark matter substructure and dwarf galactic satellites. Advances in Astronomy, 2010.
17. N. Gnedin and A. Kravtsov. Environmental Dependence of the Kennicutt-Schmidt Relation in Galaxies. The Astrophysical Journal, 2011.
18. D. Rudd, Andrew R. Zentner, and A. Kravtsov. Effect of Baryons and Dissipation on the Matter Power Spectrum. The Astrophysical Journal, 2008.
19. HDF5. www.hdfgroup.org/HDF5/.
20. Parallel NetCDF. cucis.ece.northwestern.edu/projects/PNETCDF.
21. J. Li, W. Liao, A. Choudhary, R. Ross, R. Thakur, W. Gropp, R. Latham, A. Siegel, B. Gallagher, and M. Zingale. Parallel netCDF: A high-performance scientific I/O interface. In Proc. of Supercomputing '03: High Performance Networking and Computing.
22. K. Gao, W. Liao, A. Choudhary, R. Ross, and R. Latham. Combining I/O Operations for Multiple Array Variables in Parallel NetCDF. In Proceedings of the Workshop on Interfaces and Architectures for Scientific Data Storage, held in conjunction with the the IEEE Cluster Conference, New Orleans, Louisiana, September 2009.
23. R.Thakur, W.Gropp, and E.Lusk. On Implementing MPI-IO Portably and with High Performance. In Proc. of IOPADS '99, 1999.
24. R. Latham, R. Ross, and R. Thakur. Implementing MPI-IO Atomic Mode and Shared File Pointers Using MPI One-Sided Communication. International Journal of High Performance Computing Applications, 21(2):132-143, 2007.
25. R. Ross, R. Latham, W. Gropp, R. Thakur, and B. Toonen. Implementing MPI-IO atomic mode without file system support. In Proceedings of CCGrid 2005, May 2005. Superseded by IJHPCA paper.
26. R.Thakur, W.Gropp, and E.Lusk. Data Sieving and Collective I/O in ROMIO. In Proc. of the 7th Symposium on the Frontiers of Massively parallel Computation, pages 182-189, 1999.
27. Lustre. http://wiki.lustre.org.
28. strace. http://en.wikipedia.org/wiki/Strace.
29. A. Ching, A. Choudhary, K.Coloma, and W.Liao. Noncontiguous I/O Accesses Through MPI-IO. In Proc. of IEEE/ACM International Symposium on Cluster Computing and the Grid, 2003.
30. X. Ma, M.Winslett, J. Lee, and S. Yu. Faster collective output through active buffering. In Proc. of Supercomputing '02, 2002.
31. W. Liao, K. Coloma, Ward A. Choudhary, and L, E. Russell, and S. Tideman. Collective Caching: Application-aware Client-side File Caching. In Proc. of HPDC '05, 2005.
32. X. Zhang, S. Jiang, and K. Davis. Making Resonance a Common Case: A High-Performance Implementation of Collective I/O on Parallel File Systems. In Proc. of IPDPS '09, 2009.