LifeRaft: Data-Driven, batch processing for the exploration of scientific databases

CIDR 2009 - 4th Biennal Conference on Innovative Data Systems Research

Volumn , Issue , 2009, Pages

  Wang, Xiaodan ^a   Burns, Randal ^a   Malik, Tanu ^b  

^a JOHNS HOPKINS UNIVERSITY   (United States)

^b PURDUE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DATA INTENSIVE; DATA SHARING; DATA-DRIVEN; DISK DRIVE; MAXIMIZE THROUGHPUT; OVERLAPPING DATA; QUERY PROCESSING SYSTEM; QUERY RESPONSE TIME; SCIENTIFIC DATABASE;

SCHEDULING; THROUGHPUT;

BATCH DATA PROCESSING;

EID: 84863362137     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. P. Agrawal, D. Kifer, and C. Olston. Scheduling Shared Scans of Large Data Files. In VLDB, 2008.
2. R. S. Barga, D. Fay, D. Guo, S. Newhouse, Y. Simmhan, and A. Szalay. Efffcient Scheduling of Scientiffc Workffows in a High Performance Computing Cluster. In CLADE, 2008.
3. D. S. J. D. Couto, D. Aguayo, J. Bicket, and R. Morris. A High-Throughput Path Metric for Multi-Hop Wireless Routing. In MobiCom, 2003.
4. J. Dean and S. Ghemawat. MapReduce: Simpliffed Data Processing on Large Clusters. In OSDI, 2004.
5. The EcoliHub Web Service. http://www.ecolicommunity.org/.
6. M. R. Fornari and C. Iochpe. A Spatial Hash Join Algorithm Suited for Small Buffer Size. In GIS, 2004.
7. R. Geist and S. Daniel. A Continuum of Disk Scheduling Algorithms. ACM Transactions on Computer Systems, 5(1), 1987.
8. J. Gray, M. A. Nieto-Santisteban, and A. Szalay. The Zones Algorithm for Finding Points-Near-a-Point or Cross-Matching Spatial Datasets. Technical Report, Microsoft Research, 2006.
9. J. Gray and A. Szalay. Where the Rubber Meets the Sky: Bridging the Gap Between Databases and Science. IEEE Data Engineering Bulletin, 27(4):3.11, 2004.
10. N. Kaiser, H. Aussel, B. Burke, H. Boesgaard, K. Chambers, M. Chun, J. Heasley, K. Hodapp, B. Hunt, R. Jedicke, D. Jewitt, R. Kudritzki, G. Luppino, M. Maberry, E. Magnier, D. Monet, P. Onaka, A. Pickles, P. H. Rhoads, T. Simon, A. Szalay, I. Szapudi, D. Tholen, J. Tonry, M. Waterson, and J. Wick. Pan-STARRS: A Large Synoptic Survey Telescope Array. In SPIE, 2002.
11. A. Kementsietsidis, F. Neven, D. V. de Craen, and S. Vansummeren. Scalable MultiQuery Optimization for Exploratory Queries over Federated Scientiffc Databases. In VLDB, 2008.
12. P. Kunszt, A. Szalay, I. Csabai, and A. Thakar. The Indexing of the SDSS Science Archive. In ADASS, 2000.
13. Y. Li, E. Perlman, M. Wan, Y. Yang, C. Meneveau, R. Burns, S. Chen, A. Szalay, and G. Eyink. A Public Turbulence Database Cluster and Applications to Study Lagrangian Evolution of Velocity Increments in Turbulence. Journal of Turbulence, 9(31):1.29, 2008.
14. M.-L. Lo and C. V. Ravishankar. Spatial Hash-Joins. SIGMOD Record, 25(2):247.258, 1996.
15. G. Luo, J. F. Naughton, and C. J. Ellmann. A Non-Blocking Parallel Spatial Join Algorithm. In ICDE, 2002.
16. T. Malik, A. S. Szalay, A. S. Budavri, and A. R. Thakar. SkyQuery: A Web Service Approach to Federate Databases. In CIDR, 2003.
17. J. Myllymaki and M. Livny. Relational Joins for Data on Tertiary Storage. In ICDE, 1997.
18. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig Latin: A Not-So-Foreign Language for Data Processing. In SIGMOD, 2008.
19. W. O'Mullane, N. Li, M. Nieto-Santisteban, A. Szalay, and A. Thakar. Batch is Back: CasJobs, Serving Multi-TB Data on the Web. In ICWS, 2005.
20. J. M. Patel and D. J. DeWitt. Partition Based Spatial-Merge Join. In SIGMOD, 1996.
21. S. Sarawagi. Query Processing in Tertiary Memory Databases. In VLDB, 1995.
22. The Sloan Digital Sky Survey. http://www.sdss.org.
23. T. Sterling, D. J. Becker, D. Savarese, J. E. Dorband, U. A. Ranawake, and C. V. Packer. BEOWULF: A Parallel Workstation for Scientiffc Computation. In ICPP, 1995.
24. A. Szalay, J. Gray, A. Thakar, P. Kuntz, T. Malik, J. Raddick, C. Stoughton, and J. Vandenberg. The SDSS SkyServer - Public Access to the Sloan Digital Sky Server Data. In SIGMOD, 2002.
25. B. L. Worthington, G. R. Ganger, and Y. N. Patt. Scheduling Algorithms for Modern Disk Drives. In SIGMETRICS, 1994.
26. H. C. Yang, A. Dasdan, R.-L. Hsiao, and D. S. Parker. Map-Reduce-Merge: Simpli-ed Relational Data Processing on Large Clusters. In SIGMOD, 2007.
27. J.-B. Yu and D. J. DeWitt. Query Pre-Execution and Batching in Paradise: A Two-Pronged Approach to the Efffcient Processing of Queries on Tape-Resident Raster Images. In SSDBM, 1997.