CoScan: Cooperative scan sharing in the cloud

Proceedings of the 2nd ACM Symposium on Cloud Computing, SOCC 2011

Volumn , Issue , 2011, Pages

  Wang, Xiaodan ^a   Sarma, Anish Das ^b   Olston, Christopher ^c   Burns, Randal ^a  

^a Johns Hopkins University   (United States)

^b GOOGLE INC   (United States)

^c YAHOO RESEARCH   (United States)

Author keywords

Algorithms

Indexed keywords

COMPUTING ENVIRONMENTS; DATA ANALYTICS; MAP-REDUCE; MULTIPLE DATA SOURCES; MULTIPLE USER; ORDERS OF MAGNITUDE; RESOURCE CONTENTION; RUNTIMES; SCHEDULING FRAMEWORKS; SOFT DEADLINES; WORK-FLOWS;

ALGORITHMS; COMPUTER SYSTEMS;

CLOUD COMPUTING;

EID: 82155168655     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2038916.2038927     Document Type: Conference Paper

References (38)

1. R. Abbott and H. Garcia-Molina. Scheduling Real-time Transactions. SIGMOD Rec., 17:71.81, March 1988.
2. P. Agrawal, D. Kifer, and C. Olston. Scheduling Shared Scans of Large Data Files. In VLDB, 2008.
3. H. Andrade, T. Kurc, A. Sussman, and J. Saltz. Efficient Execution of Multiple Query Workloads in Data Analysis Applications. In SC, 2001.
4. P. Brucker. Scheduling Algorithms (4th Ed.). Springer, 2004.
5. G. Candea, N. Polyzotis, and R. Vingralek. A Scalable, Predictable Join Operator for Highly Concurrent Data Warehouses. In VLDB, 2009.
6. F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. E. Gruber. Bigtable: A Distributed Storage System for Structured Data. In OSDI, 2006.
7. S. Chaudhuri, V. Narasayya, and R. Ramamurthy. Estimating Progress of Execution for SQL Queries. In SIGMOD, 2004.
8. J. Dean and S. Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. In OSDI, 2004.
9. G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels. Dynamo: Amazon's Highly Available Key-Value Store. In SOSP, 2007.
10. A. Dua and N. Bambos. Scheduling with Soft Deadlines for Input Queued Switches. In Allerton, 2006.
11. Amazon EC2. http://aws.amazon.com/ec2.
12. P. M. Fernandez. Red Brick Warehouse: A Read-mostly RDBMS for Open SMP Platforms. SIGMOD Rec., 23:492.502, May 1994.
13. S. Ganguly, W. Hasan, and R. Krishnamurthy. Query Optimization for Parallel Execution. SIGMOD Rec., 21:9.18, June 1992.
14. A. Gates, O. Natkovich, S. Chopra, P. Kamath, S. Narayanam, C. Olston, B. Reed, S. Srinivasan, and U. Srivastava. Building a High-Level Datafiow System on top of MapReduce: The Pig Experience. PVLDB, 2(2):1414.1425, 2009.
15. A. Gupta, S. Sudarshan, and S. Vishwanathan. Query Scheduling in Multiquery Optimization. In IDEAS, 2001.
16. Apache. Hadoop: Open-Source Implementation of MapReduce. http://hadoop.apache.org.
17. S. Harizopoulos, V. Shkapenyuk, and A. Ailamaki. QPipe: A Simultaneously Pipelined Relational Query Engine. In SIGMOD, 2005.
18. H. Hoogeveen. Multicriteria Scheduling. European Journal of Operational Research, 167:592.623, 2005. (Pubitemid 40220081)
19. D. Karger, C. Stein, and J. Wein. Scheduling Algorithms. In M. J. Atallah, editor, Handbook of Algorithms and Theory of Computation. CRC Press, 1997.
20. R. M. Karp. Reducibility Among Combinatorial Problems. Complexity of Computer Computations, pages 85.103, 1972.
21. K. Lai, L. Rasmusson, E. Adar, L. Zhang, and B. A. Huberman. Tycoon: An Implementation of a Distributed, Market-based Resource Allocation System. Multiagent Grid Syst., 1:169.182, August 2005.
22. A. Lakshman and P. Malik. Cassandra: A Decentralized Structured Storage System. SIGOPS Oper. Sys. Rev., 44(2):35.40, 2010.
23. J. Lenstra, A. R. Kan, and P. Brucker. Complexity of Machine Scheduling Problems. Annals of Discrete Mathematics, 1:343.362, 1977.
24. K. Morton, A. Friesen, M. Balazinska, and D. Grossman. Estimating the Progress of MapReduce Pipelines. In ICDE, 2010.
25. J. Myllymaki and M. Livny. Relational Joins for Data on Tertiary Storage. In ICDE, 1997.
26. T. Nykiel, M. Potamias, C. Mishra, G. Kollios, and N. Koudas. MRShare: Sharing Across Multiple Queries in MapReduce. Proc. VLDB Endow., 3:494.505, September 2010.
27. C. Olston, G. Chiou, L. Chitnis, F. Liu, Y. Han, M. Larsson, A. Neumann, V. B. N. Rao, V. Sankarasubramanian, S. Seth, C. Tian, T. ZiCornell, and X. Wang. Nova: Continuous Pig/Hadoop Workfiows. In SIGMOD, 2011.
28. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig Latin: A Not-So-Foreign Language for Data Processing. In SIGMOD, 2008.
29. E. Otoo, D. Rotem, and A. Romosan. Optimal File-Bundle Caching Algorithms for Data-Grids. In SC, 2004.
30. Pig Performance Benchmark. https://issues.apache.org/jira/browse/PIG-200.
31. S. Sarawagi. Query Processing in Tertiary Memory Databases. In VLDB, 1995.
32. T. K. Sellis. Multiple-Query Optimization. ACM Trans. Database Syst., 13(1):23.52, March 1988.
33. A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, H. Liu, P. Wyckoff, and R. Murthy. Hive - A Warehousing Solution Over a Map-Reduce Framework. In VLDB, 2009.
34. P. Unterbrunner, G. Giannikis, G. Alonso, D. Fauser, and D. Kossmann. Predictable Performance for Unpredictable Workloads. In VLDB, 2009.
35. X. Wang, R. Burns, and T. Malik. LifeRaft: Data-Driven, Batch Processing for the Exploration of Scientific Databases. In CIDR, 2009.
36. X. Wang, E. Perlman, R. Burns, T. Malik, T. Budavári, C. Meneveau, and A. Szalay. JAWS: Job-Aware Workload Scheduling for the Exploration of Turbulence Simulations. In SC, 2010.
37. J.-B. Yu and D. J. DeWitt. Query Pre-Execution and Batching in Paradise: A Two-Pronged Approach to the Efficient Processing of Queries on Tape-Resident Raster Images. In SSDBM, 1997.
38. M. Zukowski, S. Héman, N. Nes, and P. Boncz. Cooperative Scans: Dynamic Bandwidth Sharing in a DBMS. In VLDB, 2007.