Efficient I/O and Storage of Adaptive-Resolution Data

International Conference for High Performance Computing, Networking, Storage and Analysis, SC

Volumn 2015-January, Issue January, 2014, Pages 413-423

  Kumar, Sidharth ^a   Edwards, John ^a   Bremer, Peer Timo ^a,c   Knoll, Aaron ^a   Christensen, Cameron ^a   Vishwanath, Venkatram ^b   Carns, Philip ^b   Schmidt, John A ^a   Pascucci, Valerio ^a  

^a UNIVERSITY OF UTAH   (United States)

^b ARGONNE NATIONAL LABORATORY   (United States)

^c LAWRENCE LIVERMORE NATIONAL LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

SUPERCOMPUTERS;

ADAPTIVE MESH REFINEMENT; ADAPTIVE RESOLUTION; FULL RESOLUTIONS; LARGE-SCALE COMBUSTIONS; LOWER RESOLUTION; MULTI RESOLUTION REPRESENTATION; REGIONS OF INTEREST; RESOLUTION LEVEL;

DIGITAL STORAGE;

EID: 84936953038     PISSN: 21674329     EISSN: 21674337     Source Type: Conference Proceeding    
DOI: 10.1109/SC.2014.39     Document Type: Conference Paper

References (28)

1. J. Li, W.-K. Liao, A. Choudhary, R. Ross, R. Thakur, W. Gropp, R. Latham, A. Siegel, B. Gallagher, and M. Zingale, "Parallel netCDF: A high-performance scientific I/O interface," in Proceedings of SC2003: High Performance Networking and Computing. Phoenix, AZ: IEEE Computer Society Press, November 2003.
2. "HDF5 home page," http://www.hdfgroup.org/HDF5/.
3. V. Pascucci and R. J. Frank, "Global static indexing for real-time exploration of very large regular grids," in Conference on High Performance Networking and Computing, archive proceedings of the ACM/IEEE Conference on Supercomputing, 2001.
4. V. Pascucci, D. E. Laney, R. J. Frank, F. Gygi, G. Scorzelli, L. Linsen, and B. Hamann, "Real-time monitoring of large scientific simulations," in ACM Symposium on Applied Computing, 2003, pp. 194-198.
5. S. Kumar, V. Vishwanath, P. Carns, J. Levine, R. Latham, G. Scorzelli, H. Kolla, R. Grout, R. Ross, M. Papka, J. Chen, and V. Pascucci, "Efficient data restructuring and aggregation for I/O acceleration in PIDX," in High Performance Computing, Networking, Storage and Analysis (SC), 2012 International Conference for, Nov 2012, pp. 1-11.
6. J. D. de St. Germain, J. McCorquodale, S. G. Parker, and C. R. Johnson, "Uintah: A massively parallel problem solving environment," in Ninth IEEE International Symposium on High Performance and Distributed Computing. IEEE, Piscataway, NJ, November 2000, pp. 33-41.
7. S. G. Parker, J. Guilkey, and T. Harman, "A component-based parallel infrastructure for the simulation of fluid-structure interaction," Engineering with Computers, vol. 22, pp. 277-292, 2006.
8. S. G. Parker, "A component-based architecture for parallel multi-physics PDE simulation." Future Generation Computer Systems, vol. 22, pp. 204-216, 2006.
9. J. H. Chen, A. Choudhary, B. de Supinski, M. DeVries, E. R. Hawkes, S. Klasky, W. K. Liao, K. L. Ma, J. M. Crummey, N. Podhorszki, R. Sankaran, S. Shende, and C. S. Yoo, "Terascale direct numerical simulations of turbulent combustion using s3d," in Computational Science and Discovery Volume 2, January 2009.
10. P. Colella, D. T. Graves, D. Modiano, D. B. Serani, and B. van Straalen, "Chombo software package for AMR applications," Lawrence Berkeley National Laboratory, Tech. Rep., 2000.
11. B. Fryxell, K. Olson, P. Ricker, F. X. Timmes, M. Zingale, D. Q. Lamb, P. MacNeice, R. Rosner, and H. Tufo, "FLASH: An adaptive mesh hydrodynamics code for modelling astrophysical thermonuclear flashes," Astrophysical Journal Supplement, vol. 131, pp. 273-334, 2000.
12. Y. Yu, D. H. Rudd, Z. Lan, N. Y. Gnedin, A. Kravtsov, and J. Wu, "Improving parallel I/O performance of cell-based AMR cosmology applications," in Parallel & Distributed Processing Symposium (IPDPS), 2012 IEEE 26th International. IEEE, 2012, pp. 933-944.
13. K. Gao, W.-K. Liao, A. Nisar, A. Choudhary, R. Ross, and R. Latham, "Using subfiling to improve programming flexibility and performance of parallel shared-file I/O," in International Conference on Parallel Processing, 2009. ICPP '09, September 2009, pp. 470-477.
14. J. Lofstead, S. Klasky, K. Schwan, N. Podhorszki, and C. Jin, "Flexible IO and integration for scientific codes through the adaptable IO system (ADIOS)," in Proceedings of the 6th International Workshop on Challenges of Large Applications in Distributed Environments, CLADE '08. New York: ACM, June 2008, pp. 15-24.
15. S. Kumar, V. Pascucci, V. Vishwanath, P. Carns, R. Latham, T. Peterka, M. Papka, and R. Ross, "Towards parallel access of multi-dimensional, multiresolution scientific data," in Proceedings of 2010 Petascale Data Storage Workshop, November 2010.
16. S. Kumar, V. Vishwanath, P. Carns, B. Summa, G. Scorzelli, V. Pascucci, R. Ross, J. Chen, H. Kolla, and R. Grout, "PIDX: Efficient parallel I/O for multi-resolution multi-dimensional scientific datasets," in IEEE International Conference on Cluster Computing, 2011.
17. S. Kumar, A. Saha, V. Vishwanath, P. Carns, J. A. Schmidt, G. Scorzelli, H. Kolla, R. Grout, R. Latham, R. Ross et al., "Characterization and modeling of pidx parallel I/O for performance optimization," in Proceedings of SC13: International Conference for High Performance Computing, Networking, Storage and Analysis. ACM, 2013, p. 67.
18. S. Kumar, C. Christensen, J. Schmidt, P.-T. Bremer, E. Brugger, V. Vishwanath, P. Carns, H. Kolla, R. Grout, J. Chen, M. Berzins, G. Scorzelli, and V. Pascucci, "Fast multiresolution reads of massive simulation datasets," in Supercomputing, ser. Lecture Notes in Computer Science, J. Kunkel, T. Ludwig, and H. Meuer, Eds. Springer International Publishing, 2014, vol. 8488, pp. 314-330.
19. B. Summa, G. Scorzelli, M. Jiang, P.-T. Bremer, and V. Pascucci, "Interactive editing of massive imagery made simple: Turning atlanta into atlantis," ACM Trans. Graph., vol. 30, pp. 7:1-7:13, April 2011.
20. G. M. Morton, A computer oriented geodetic data base and a new technique in file sequencing. International Business Machines Company, 1966.
21. V. Pascucci and R. J. Frank, "Hierarchical indexing for out-of-core access to multi-resolution data," in Hierarchical and Geometrical Methods in Scientific Visualization. Springer, 2003, pp. 225-241.
22. V. Pascucci, G. Scorzelli, B. Summa, P.-T. Bremer, A. Gyulassy, C. Christensen, S. Philip, and S. Kumar, "The ViSUS visualization framework," in High Performance Visualization: Enabling Extreme-Scale Scientific Insight, E. W. Bethel, H. Childs, and C. Hansen, Eds. CRC Press, 2012.
23. V. Pascucci, G. Scorzelli, B. Summa, P.-T. Bremer, A. Gyulassy, C. Christensen, and S. Kumar, "Scalable visualization and interactive analysis using massive data streams," Advances in Parallel Computing: Cloud Computing and Big Data, vol. 23, pp. 212-230, 2013.
24. "Visit home page," https://wci.llnl.gov/codes/visit/.
25. Q. Meng, A. Humphrey, J. Schmidt, and M. Berzins, "Investigating applications portability with the Uintah DAG-Based runtime system on PetScale supercomputers," in Proceedings of SC13: International Conference for High Performance Computing, Networking, Storage and Analysis. ACM, 2013, pp. 96:1-96:12.
26. G. Weber, P.-T. Bremer, M. Day, J. Bell, and V. Pascucci, "Feature tracking using reeb graphs," in Topological Methods in Data Analysis and Visualization. Springer, 2011, pp. 241-253.
27. C. Wang and H.-W. Shen, "Information theory in scientific visualization," Entropy, vol. 13, no. 1, pp. 254-273, 2011.
28. J. Woodring, S. Mniszewski, C. Brislawn, D. DeMarle, and J. Ahrens, "Revisiting wavelet compression for large-scale climate data using jpeg 2000 and ensuring data precision," in Large Data Analysis and Visualization (LDAV), 2011 IEEE Symposium on. IEEE, 2011, pp. 31-38.