Supporting user-defined subsetting and aggregation over parallel NetCDF datasets

Proceedings - 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGrid 2012

Volumn , Issue , 2012, Pages 212-219

  Su, Yu ^a   Agrawal, Gagan ^a  

^a The Ohio State University   (United States)

Author keywords

Data intensive computing; Scientific databases

Indexed keywords

AGGREGATION QUERIES; DATA AGGREGATION; DATA MANAGEMENT TOOLS; DATA SET SIZE; DATA SETS; DATA-INTENSIVE COMPUTING; LIGHT WEIGHT; NOVEL ALGORITHM; PARALLELIZING; POST-FILTERING; PRE-FILTERING; RETRIEVAL SPEED; SCIENTIFIC DATA; SCIENTIFIC DATABASE; SCIENTIFIC DISCOVERY; WIDE AREA;

ALGORITHMS; GRID COMPUTING; INFORMATION MANAGEMENT;

DATA TRANSFER;

EID: 84863698152     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CCGrid.2012.45     Document Type: Conference Paper

References (16)

1. J. G. abd D. T. Liu, M. A. Nieto-Santisteban, A. Szalay, and D. J. DeWitt, "Scientific Data Management in the Coming Decade," in SIGMOD Record, 2005.
2. R. Music and T. Critchlow, "Practical lessons in supporting large-scale computational science," SIGMOD Record, vol. 4, no. 28, 1999.
3. M. Stonebraker, J. Becla, D. Dewitt, K.-T. Lim, D. Maier, O. Ratzesberger, and S. Zdonik, "Requirements for science data bases and scidb," in Conference on Innovative Data Systems Research (CIDR), January 2009. [Online]. Available: http://wwwdb.cs.wisc.edu/cidr/cidr2009/Paper 26.pdf
4. P. Cornillon, J. Callagher, and T. Sgouros, "OpeNDAP: Accessing data in a distributed, heterogeneous environment," Data Science Journal, vol. 2, pp. 164-174, Nov. 2003.
5. J. Li, W. keng Liao, A. N. Choudhary, R. B. Ross, R. Thakur, W. Gropp, R. Latham, A. Siegel, B. Gallagher, and M. Zingale, "Parallel netcdf: A high-performance scientific i/o interface," in SC. ACM, 2003, p. 39.
6. K. Gao, W. keng Liao, A. N. Choudhary, R. Ross, and R. Latham, "Combining i/o operations for multiple array variables in parallel netcdf," in CLUSTER. IEEE, 2009, pp. 1-10.
7. D. Wang, C. Zender, and S. Jenks, "Clustered workflow execution of retargeted data analysis scripts," in Cluster Computing and the Grid, 2008. CCGRID'08. 8th IEEE International Symposium on. IEEE, 2008, pp. 449-458.
8. J. Buck, N. Watkins, J. LeFevre, K. Ioannidou, C. Maltzahn, N. Polyzotis, and S. Brandt, "Scihadoop: Array-based query processing in hadoop," in SC, 2011.
9. J. Chou, K. Wu, O. Rübel, M. Prabhat, B. Austin, E. Bethel, R. Ryne, and A. Shoshani, "Parallel index and query for large scale data analysis,"in SC, 2011.
10. D. Bernholdt, S. Bharathi, D. Brown, K. Chanchio, M. Chen, A. Chervenak, L. Cinquini, B. Drach, I. Foster, P. Fox, J. Garcia, C. Kesselman, R. Markel, D. Middleton, V. Nefedova, L. Pouchard, A. Shoshani, A. Sim, G. Strand, and D.Williams, "The Earth System Grid: Supporting the Next Generation of Climate Modeling Research, " Proceedings of IEEE, vol. 93, no. 3, 2005.
11. M. Branco, D. Cameron, B. Gaidioz, V. Garonne, B. Koblitz, M. Lassnig, R. Rocha, P. Salgado, and T. Wenaus, "Managing ATLAS Data on Peta-Scale with DQ2," Journal of Physics: Conference Series, vol. 119, no. 6, 2008.
12. R. O. Sinnott, C. Bayliss, C. Davenhall, B. Harbulot, M. Jones, C. Millar, G. Roy, S. Roy, G. Stewart, J. Watt, and A. Aseno, "Secure Performance Oriented Data Management for Aseno," in Proceedings of E-Science, 2008.
13. J. No, R. Thakur, and A. Choudhary, "Integrating parallel file i/o and database support for high-performance scientific data management," in Supercomputing '00: Proceedings of the 2000 ACM/IEEE conference on Supercomputing (CDROM). Washington, DC, USA: IEEE Computer Society, 2000, p. 57.
14. A. S. Szalay, P. Z. Kunszt, A. Thakar, J. Gray, D. Slutz, and R. J. Brunner, "Designing and mining multi-terabyte astronomy archives: the Sloan Digital Sky Survey," in Proc. ACM SIGMOD. ACM Press, 2000, pp. 451-462. [Online]. Available: citeseer.ist.psu.edu/article/szalay00designing.html
15. "Large synoptic survey telescope, http://www.lsst.org."
16. L.Weng, G. Agrawal, U. Catalyurek, T. Kurc, S. Narayanan, and J. Saltz, "An Approach for Automatic Data Virtualization," in Proceedings of the Conference on High Performance Distributed Computing (HPDC), Jun. 2004.