HDF5-FastQuery: Accelerating complex queries on HDF datasets using fast bitmap indices

Proceedings of the International Conference on Scientific and Statistical Database Management, SSDBM

Volumn , Issue , 2006, Pages 149-158

  Gosink, Luke ^a   Shalf, John ^b   Stockinger, Kurt ^b   Wu, Kesheng ^b   Bethel, Wes ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b LAWRENCE BERKELEY NATIONAL LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADMINISTRATIVE DATA PROCESSING; DATABASE SYSTEMS; INFORMATION THEORY; MANAGEMENT INFORMATION SYSTEMS; STATISTICS; STORAGE (MATERIALS);

BITMAP INDEXES; COMPLEX QUERIES; DATA ANALYSIS; DATA-SETS; DATABASE MANAGEMENT; INTERNATIONAL CONFERENCES; LARGE SCALES; MULTI DIMENSIONAL; SCIENTIFIC DATA; SEMANTIC INDEXING; STORAGE AND RETRIEVAL (STORET); STORAGE FORMATS; SUPERNOVA (SN) EXPLOSIONS; USER COMMUNITIES;

STATISTICAL METHODS;

EID: 35948970859     PISSN: 10993371     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SSDBM.2006.27     Document Type: Conference Paper

References (21)

1. F. Alted, M. Fernandez-Alonso, PyTables: Processing and Analyzing Extremly Large Amounts of Data in Python In PyCon 2003, Washington D.C., USA March 2003.
2. J. B. Bell, M. S. Day, I. G. Shepherd, M. Johnson, R. K. Cheng, J. F. Grcar, V. E. Beckner, M. J. Lijewski. Numerical Simulation of a Laboratory-Scale Turbulent V-flame. In Proc. Natl. Acad. Sci. USA, 102(29), 10006-10011 2005.
3. M. Berger, P. Colella, Local Adaptive Mesh Refinement for Shock Hydrodynamics, In Journal of Computational Physics, May 1989
4. C.-Y. Chan and Y. E. Ioannidis. Bitmap Index Design and Evaluation. In SIGMOD, Seattle, Washington, USA, June 1998. ACM Press.
5. FITS - Flexible Image Transport System, http://heasarc.gsfc.nasa.gov/ docs/heasarc/fits.html
6. J. Gray, D. T. Liu, M. Nieto-Santisteban, A. Szalay, D. DeWitt and G. Heber, Scientific Data Management in the Coming Decade, In SIGMOD Record, 34(4), December 2005.
7. J. M. Hellerstein, E. Koutsoupias, C. H. Papadimitriou, On the Analysis of Indexing Schemes. In Symposium on Principles of Database Systems (PODS), Tucson, Arizona, USA, May, 1997.
8. B. Nam, A. Sussman, Improving Access to Multidimensional Self-describing Scientific Dataset. In International Symposium on Cluster Computing and the Grid (CCGrid), May 2003, Tokyo, Japan. IEEE Computer Society Press.
9. NetCDF - Network Common Data Form, http://www.unidata.ucar.edu/software/ netcdf/
10. H5IN: Indexing Interface, http://hdf.ncsa.uiuc.edu/RFC/H5IN/RM_H5IN.html, November 2005.
11. HDF - Hierarchical Data Format, http://hdf.ncsa.uiuc.edu/
12. FastBit: An Efficient Compressed Bitmap Index Technology, http:://sdm.lbl.gov/fastbit
13. T. Johnson. Performance Measurements of Compressed Bitmap Indices. In International Conference on Very Large Data Bases (VLDB), Edinburgh, Scotland, September 1999. Morgan Kaufmann.
14. P. O'Neil and D. Quass. Improved Query Performance with Variant Indexes. In International Conference on Management of Data (SIGMOD), Tucson, Arizona, USA, May 1997. ACM Press.
15. K. Stockinger, John Shalf, Wes Bethel, and K. Wu. DEX: Increasing the Capability of Scientific Data Analysis Pipelines by Using Efficient Bitmap Indices to Accelerate Scientific Visualization. In International Conference on Scientific and Statistical Database Management (SSDBM), Santa Barbara, California, USA, June 2005. IEEE Computer Society Press.
16. K. Stockinger, J. Shalf, W. Bethel, and K. Wu. Query-Driven Visualization of Large Data Sets. In IEEE Visualization 2005, Minneapolis, MN, October 23-25, 2005, IEEE Computer Society Press.
17. K. Stockinger, K. Wu, S. Campbell, S. Lau, M. Fisk, E. Gavrilov, A. Kent, C.E. Davis, R. Olinger, R. Young, J.E. Prewett, R Weber, T.P Caudell, E.W. Bethel, S. Smith. "Network Traffic Analysis With Query Driven Visualization SC 2005 HPC Analytics Results." In Supercomputing 2005, HPC Analytics Challenge, November 2005.
18. K. Stockinger, K. Wu, and A. Shoshani. Evaluation Strategies for Bitmap Indices with Binning. In International Conference on Database and Expert Systems Applications (DEXA), Zaragoza, Spain, September 2004. Springer-Verlag.
19. K. Wu, E. J. Otoo, and A. Shoshani. An Efficient Compression Scheme for Bitmap Indices. To appear in ACM Transcations on Database Systems, 2006. ACM Press.
20. K. Wu, E. J. Otoo, and A. Shoshani. On the Performance of Bitmap Indices for High Cardinality Attributes. In International Conference on Very Large Data Bases (VLDB), Toronto, Canada, August 31 -September 3, 2004, Morgan Kaufmann.
21. K. Wu, W. Koegler, J. Chen, and A. Shoshani. Using Bitmap Index for Interactive Exploration of Large Datasets. In International Conference on Scientific and Statistical Database Management (SSDBM), Cambridge, MA, 2003. IEEE Computer Society Press.