Efficient metadata generation to enable interactive data discovery over large-scale scientific data collections

Proceedings - 2nd IEEE International Conference on Cloud Computing Technology and Science, CloudCom 2010

Volumn , Issue , 2010, Pages 573-580

  Pallickara, Sangmi Lee ^a   Pallickara, Shrideep ^a   Zupanski, Milija ^b   Sullivan, Stephen ^c  

^a Department of Computer Science ^*  (United States)

^b Department of Environmental and Radiological Health Sciences   (United States)

^c Univ Coop for Atmospheric Research   (United States)

Author keywords

Atmospheric sciences; Cloud computing; Data discovery; Large scale datasets; Metadata

Indexed keywords

ATMOSPHERIC SCIENCE; BINARY DATA FORMAT; BINARY FILES; DATA DISCOVERY; DATA SEARCHES; DATA SERVICES; DATA SETS; DECODING PROCESS; LARGE DATA; LARGE-SCALE DATASETS; MAP-REDUCE; METADATA GENERATION; MISSING DATA; PERFORMANCE EVALUATION; RUNTIMES; SCIENTIFIC COMPUTING; SCIENTIFIC DATA; WEB DOCUMENT;

COMPUTER ARCHITECTURE; COMPUTER SYSTEMS; DATA ACQUISITION; INTERNET; METADATA;

CLOUD COMPUTING;

EID: 79952411400     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CloudCom.2010.99     Document Type: Conference Paper

References (31)

1. G. K. Rutledge, et al., "NOMADS: A Climate and Weather Model Archive at the National Oceanic and Atmospheric Administration," Bull. Amer. Meteor. Soc., vol. 87, pp. 327-341, 2006.
2. The National Center for Atmospheric Resource. Available: http://www.ncar.ucar.edu/
3. R. L. Hurt, et al., "Astronomy Visuallization Metadata (AVM) in action!," American Astronomical Society, AAS Meeting vol. 213, 2009.
4. E. W. Sayers, et al., "Database resources of the National Center for Biotechnology Information," Nucleic Acids Research, vol. 38, 2010.
5. Public Data Sets on AWS. Available: http://aws.amazon.com/publicdatasets/
6. S. Brand-Gruwel and P. Gerjets, "Instructional support for enhancing students' information problem solving ability," Computers in Human Behavior, vol. 24, pp. 615-622, 2008.
7. G. Marchionini, Information seeking in electronic environments. Cambridge, UK: Cambridge University Press, 1995.
8. B. Plale, et al., "Cooperating Services for Managing Data Driven Computational Experimentation," Computing in Science and Engineering(CiSE) magazine, vol. 7, pp. 34-43, 2005.
9. K. Lin and A. K. Sinha, "Discovery and Semantic Integration of Geologic Data," Geoinformatics, pp. 2006-5201, 2006.
10. B. Domenico, et al., "Thematic Real-time Environmental Distributed Data Services (THREDDS): Incorporating Interactive Analysis Tools into NSDL," Journal of Interactivity in Digital Libraries, vol. 2, 2002.
11. R. K. Rew and G. P. Davis, "NetCDF: An Interface for Scientific Data Access," IEEE Computer Graphics and Applications, vol. 10, pp. 76-82, 1990.
12. B. CA, et al., "Standards for Microarray Data," Science, vol. 298, p. 539, 2002.
13. C. A. Ball, et al., "Submission of Microarray Data to Public Repositories," PLos Biology, vol. 2, pp. 1276-1277, 2004.
14. R. W. Moore, "Building Preservation Environments with Data Grid Technology," American Archivist, vol. 69, 2006.
15. P. C.-M. Kimura, et al., "A Demonstration of SciDB: A Science-Oriented DBMS," VLDB, vol. 2, pp. 1534-1537, 2009.
16. J. Dean and S. Ghemawat, "Mapreduce: Simplified data processing on large clusters," Communications of the ACM, vol. 51, pp. 107-113, 2008.
17. E. Deelman, et al., "Grid-Based Metadata Services," in International Conference on Scientific and Statistical Database Management Santorini Island Greece, 2004.
18. R. W. Moore, "Managing Large Distributed Data Sets Using the Storage Resource Broker," ITEA Journal of Test and Evaluation, 2007.
19. D. N. Williams, et al., "The Earth System Grid: Enabling Access to Multimodel Climate Simulation Data," Bull. Amer. Meteor. Soc., vol. 90, pp. 195-205, 2009.
20. L.-A. Dupigny-Giroux, et al., "NOAA's Climate Database Modernization Program: Rescuing, archiving, and digitizing history," Bulletin of the American Meteorological Society, vol. 88, pp. 1015-1017, 2007.
21. N. C. D. Center. Map Interface to Selected Online Data. Available: http://www.ncdc.noaa.gov/oa/dataaccesstools.html#climate
22. NOAA. NOAA Climate Services: Data Library. Available: http://www.climate.gov/#dataServices/dataLibrary
23. NCAR. Community Data Portal. Available: http://cdp.ucar.edu/
24. "Content Standard for Digital Geospatial Metadata," ed: Content Standard for Digital Geospatial Metadata, 1998.
25. "ISO 19115 Geographic Information-Metadata," ed: International Organization for Standardization, 2003.
26. S. Pallickara, et al., "An Overview of the Granules Runtime for Cloud Computing," in the IEEE eScience Conference, Indianapolis, USA, 2008.
27. S. Pallickara, et al., "Granules: A Lightweight, Streaming Runtime for Cloud Computing With Support for Map-Reduce.," in the IEEE International Conference on Cluster Computing, 2009.
28. M. Isard, et al., "Dryad: Distributed data-parallel programs from sequential building blocks," in European Conference on Computer Systems, Lisbon, Portugal, 2007.
29. S. J. Sullivan. WmoBufr. Available: http://sourceforge.net/projects/ wmobufr/
30. M. Zupanski, "Maximum Likelihood Ensemble Filter: Theoretical Aspects," Monthly Weather Review, vol. 133, pp. 1710-1726, 2005.
31. W. Thorpe, "A Guide to the WMO Code Form FM 94 BUFR.," ed. Monterey, CA.: Numerical and Oceanography Center, 1991.