MRShare: Sharing across multiple queries in mapreduce

Proceedings of the VLDB Endowment

Volumn 3, Issue 1, 2010, Pages 494-505

  Nykiel, Tomasz ^a   Potamias, Michalis ^b   Mishra, Chaitanya ^a,c   Kollios, George ^b   Koudas, Nick ^a  

^a UNIVERSITY OF TORONTO   (Canada)

^b BOSTON UNIVERSITY   (United States)

^c FACEBOOK   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLOUD COMPUTING; DISTRIBUTED COMPUTER SYSTEMS; OPTIMIZATION; QUERY PROCESSING;

ANALYTICAL QUERIES; LARGE-SCALE DATA ANALYSIS; MONETARY VALUE; MULTIPLE QUERIES; OPTIMIZATION PROBLEMS; OVERALL EFFECTIVENESS; PROCESSING INFRASTRUCTURES; SCIENTIFIC RESEARCHES;

NETWORK FUNCTION VIRTUALIZATION;

EID: 84859260019     PISSN: None     EISSN: 21508097     Source Type: Conference Proceeding    
DOI: 10.14778/1920841.1920906     Document Type: Article

References (26)

1. Amazon EC2. http://aws.amazon.com/ec2/.
2. Blogscope. http://www.blogscope.net/.
3. Hadoop project. http://hadoop.apache.org/.
4. Saving energy in datacenters. http://www1.eere.energy.gov/industry/datacenters/.
5. A. Abouzeid, K. Bajda-Pawlikowski, D. Abadi, A. Rasin, and A. Silberschatz. HadoopDB: An architectural hybrid of mapreduce and dbms technologies for analytical workloads. In VLDB, 2009.
6. P. Agrawal, D. Kifer, and C. Olston. Scheduling shared scans of large data files. Proc. VLDB Endow., 1(1):958-969, 2008.
7. G. Candea, N. Polyzotis, and R. Vingralek. A scalable, predictable join operator for highly concurrent data warehouses. In VLDB, 2009.
8. R. Chaiken, B. Jenkins, P.-A. Larson, B. Ramsey, D. Shakib, S. Weaver, and J. Zhou. Scope: easy and efficient parallel processing of massive data sets. Proc. VLDB Endow., 1(2):1265-1276, 2008.
9. S. Chaudhuri and K. Shim. Optimization of queries with user-defined predicates. ACM Trans. Database Syst., 24(2):177-228, 1999.
10. J. Cohen, B. Dolan, M. Dunlap, J. M. Hellerstein, and C. Welton. Mad skills: New analysis practices for big data. PVLDB, 2(2):1481-1492, 2009.
11. J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. In OSDI '04, pages 137-150.
12. S. Finkelstein. Common expression analysis in database applications. In SIGMOD '82, pages 235-245, 1982.
13. E. Friedman, P. Pawlowski, and J. Cieslewicz. Sql/mapreduce: A practical approach to self-describing, polymorphic, and parallelizable user-defined functions. In VLDB, 2009.
14. A. Gates, O. Natkovich, S. Chopra, P. Kamath, S. Narayanam, C. Olston, B. Reed, S. Srinivasan, and U. Srivastava. Building a highlevel dataflow system on top of mapreduce: The pig experience. PVLDB, 2(2):1414-1425, 2009.
15. S. Harizopoulos, V. Shkapenyuk, and A. Ailamaki. Qpipe: a simultaneously pipelined relational query engine. In SIGMOD '05, pages 383-394, 2005.
16. C. Olston, B. Reed, A. Silberstein, and U. Srivastava. Automatic optimization of parallel dataflow programs. In USENIX Annual Tech. Conf., pages 267-273, 2008.
17. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig latin: a not-so-foreign language for data processing. In SIGMOD, pages 1099-1110, 2008.
18. A. Pavlo, E. Paulson, A. Rasin, D. J. Abadi, D. J. DeWitt, S. Madden, and M. Stonebraker. A comparison of approaches to large-scale data analysis. In SIGMOD '09, pages 165-178, 2009.
19. R. Pike, S. Dorward, R. Griesemer, and S. Quinlan. Interpreting the data: Parallel analysis with sawzall. Scientific Programming, 13(4):277-298, 2005.
20. L. Qiao, V. Raman, F. Reiss, P. J. Haas, and G. M. Lohman. Main-memory scan sharing for multi-core cpus. Proc. VLDB Endow., 1(1):610-621, 2008.
21. T. K. Sellis. Multiple-query optimization. ACM Trans. Database Syst., 13(1):23-52, 1988.
22. A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, H. Liu, P. Wyckoff, and R. Murthy. Hive - a warehousing solution over a map-reduce framework. In VLDB, 2009.
23. H.-c. Yang, A. Dasdan, R.-L. Hsiao, and D. S. Parker. Map-reduce-merge: simplified relational data processing on large clusters. In SIGMOD, pages 1029-1040, 2007.
24. Y. Yu, M. Isard, D. Fetterly, M. Budiu, Ú. Erlingsson, P. K. Gunda, and J. Currey. Dryadlinq: A system for general-purpose distributed data-parallel computing using a high-level language. In OSDI, pages 1-14, 2008.
25. J. Zhou, P.-A. Larson, J.-C. Freytag, and W. Lehner. Efficient exploitation of similar subexpressions for query processing. In SIGMOD '07, pages 533-544, 2007.
26. M. Zukowski, S. Héman, N. Nes, and P. Boncz. Cooperative scans: dynamic bandwidth sharing in a dbms. In VLDB '07, pages 723-734, 2007.