Quality-aware data management for large scale scientific applications

Proceedings - 2012 SC Companion: High Performance Computing, Networking Storage and Analysis, SCC 2012

Volumn , Issue , 2012, Pages 816-820

  Zou, Hongbo ^a   Zheng, Fang ^a   Wolf, Matthew ^a   Eisenhauer, Greg ^a   Schwan, Karsten ^a   Abbasi, Hasan ^b   Liu, Qing ^b   Podhorszki, Norbert ^b   Klasky, Scott ^b  

^a Georgia Institute of Technology   (United States)

^b OAK RIDGE NATIONAL LABORATORY   (United States)

Author keywords

compression; Data management; HPC simulation; quality of information; visualization

Indexed keywords

CODE GENERATION; COMPRESSION TECHNIQUES; DATA MANAGEMENT SYSTEM; HPC SIMULATION; LOSSY COMPRESSION METHODS; QUALITY OF INFORMATION; QUALITY OF INFORMATIONS (QOI); SCIENTIFIC APPLICATIONS;

COMPACTION; DATA COMPRESSION; DATA HANDLING; FLOW VISUALIZATION; INFORMATION MANAGEMENT;

DIGITAL STORAGE;

EID: 84876552853     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SC.Companion.2012.114     Document Type: Conference Paper

References (18)

1. H. Abbasi, M. Wolf, G. Eisenhauer, S. Klasky, K. Schwan, and F. Zheng, "DataStager: Scalable Data Staging Services for Petascale Applications," ACM Symposium on High-Performance Parallel and Distributed Computing (HPDC'09), 2009, pp. 39-48.
2. H. Abbasi, M. Wolf, K. Schwan, G. Eisenhauer, A. Hilton, "XChange: Coupling Parallel Applications in a Dynamic Environment," Proc. of the 2004 IEEE International Conference on Cluster Computing (Cluster'04), 2004, pp. 471-480.
3. High-quality Data Compressor, bzip2, http://www.bzip.org.
4. Y. Chen, A. Ganapathi, and R. Katz, "To Compress or Not to Compress Compute vs. I/O Tradeoffs for MapReduce Energy Efficiency," Proc. The First ACM SIGCOMM Workshop on Green Networking, 2010, pp. 23-28.
5. G. Eisenhauer, M. Wolf, H. Abbasi, S. Klasky, and K. Schwan, "A Type System for High Performance Communication and Computation," Proc. The Workshop on D3Science associated with e-Science11, 2011, pp. 183-190.
6. G. Eisenhauer, M. Wolf, H. Abbasi, S. Klasky, and K. Schwan, "Event-based Systems: Opportunities and Challenges at Exascale," Proc. The 3rd ACM International Conference on Distributed Event-Based Systems, 2009, pp. 16-25.
7. N. Liu, C. Jason, C. Philip, C. Christopher, R. Robert, G. Gary, C. Adam, and M. Carlos, "On the Role of Burst Buffers in Leadership-class Storage Systems," Proc. The 28th IEEE MSST/SNAPI conference (MSST 2012), 2012, pp. 1-11.
8. Scientific Data Management, https://sdm.lbl.gov/sdmcenter/
9. E. R. Schendel, S. V. Pendse, J. Jenkins, D. A. Boyuka II, Z. Gong, S. Laksh., Q. Liu, H. Kolla, J. Chen, S. Klasky, R. B. Ross, N. F. Samatova, "ISOBAR hybrid compression-I/O interleaving for large-scale parallel I/O optimization," Proc. ACM Symposium on High-Performance Parallel and Distributed Computing (HPDC'12), 2012, pp. 61-72.
10. P. R. Woodward, D. H. Porter, and A. Iyer, "Initial Experientces with Grid-based Volume Visualization of Fluid Flow Simulations on PC Clusters," Proc. Visualization and Data Analysis 2005 (VDA2005), 2005, pp. 115-125.
11. T. Tu, H. Yu, L. Ramirez-Guzman, J. Bielak, O. Ghattas, K. Ma, and D. R. O'Hallaron, "Scalable Systems Software-From Mesh Generation to Scientific Visualization: an End-to-End Approach to Parallel Supercomputing," Proc. ACM/IEEE Conference on Supercomputing (SC06), 2006, pp. 1-15.
12. Compression Utility, Gzip, http://www.gzip.org.
13. Paraview VisIt, OpenSource Scientific Visualization and Graphical Anaysis Tool, https://wci.llnl.gov/codes/visit.
14. Y. Wiseman, K. Schwan, and P. Widener, "Efficient End-to-End Data Exchange Using Configurable Compression," Proc. IEEE International Conference on Distributed Computing Systems (ICDCS 2004), 2004, pp. 228-235.
15. B. Welton, D. Kimpe, J. Cope, C. M. Patrick, K. Iskra, R. Ross, "Improving I/O Forwarding Throughput with Data Compression," Proc. IEEE International Conference on Cluster Computing (Cluster'11), 2011, pp. 438-445.
16. M. Wolf, H. Abbasi, B. Collins, D. Spain, and K. Schwan, "Service Augmentation for High End Interactive Data Services," Proc. of the 2005 IEEE International Conference on Cluster Computing (Cluster'05), 2005, pp. 1-11.
17. H. Yu, C. Wang, R. W. Grout, J. H. Chen, and K. Ma, "In-situ Visualizaiton for Large-scale Combustion Simulations", Proc. IEEE Computer Graphics and Applications, 2010, pp. 45-57.
18. F. Zheng, H. Abbasi, C. Docan, J. F. Lofstead, Q. Liu, S. Klasky, M. Pu, N. Podhorszki, K. Schwan, and M. Wolf, "Predata-Preparatory Data Analytics on Peta-scale Machines", Proc. IEEE International Parallel and Distributed Processing Symp (IPDPS'10), 2010, pp 1-12.