Processing theta-joins using MapReduce

Proceedings of the ACM SIGMOD International Conference on Management of Data

Volumn , Issue , 2011, Pages 949-960

  Okcan, Alper ^a   Riedewald, Mirek ^a  

^a NORTHEASTERN UNIVERSITY   (United States)

Author keywords

MapReduce; skew; theta join processing

Indexed keywords

DATA FLOW ANALYSIS;

CONDITION; CONTROL-DATA FLOW; FLOW BASED; JOIN ALGORITHM; JOIN PROCESSING; MAP-REDUCE; RANDOMIZED ALGORITHMS; SIMPLE++; SKEW; THETA JOIN PROCESSING;

MAPREDUCE;

EID: 79960020260     PISSN: 07308078     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1989323.1989423     Document Type: Conference Paper

References (20)

1. Apache hadoop. http://hadoop.apache.org.
2. Apache hive. http://hadoop.apache.org/hive.
3. Apache pig. http://pig.apache.org/.
4. F. N. Afrati and J. D. Ullman. Optimizing joins in a map-reduce environment. In EDBT, pages 99-110, 2010.
5. S. Blanas, J. M. Patel, V. Ercegovac, J. Rao, E. J. Shekita, and Y. Tian. A comparison of join algorithms for log processing in mapreduce. In SIGMOD, pages 975-986, 2010.
6. J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. In OSDI, 2004.
7. D. J. DeWitt and J. Gray. Parallel database systems: The future of high performance database systems. Commun. ACM, 35(6):85-98, 1992. (Pubitemid 23642225)
8. D. J. DeWitt, J. F. Naughton, and D. A. Schneider. An evaluation of non-equijoin algorithms. In VLDB, pages 443-452, 1991.
9. D. J. DeWitt, J. F. Naughton, D. A. Schneider, and S. Seshadri. Practical skew handling in parallel joins. In VLDB, pages 27-40, 1992.
10. G. Graefe. Query evaluation techniques for large databases. ACM Comput. Surv., 25, 1993.
11. C. Hahn and S. Warren. Extended edited synoptic cloud reports from ships and land stations over the globe, 1952-1996. http://cdiac.ornl.gov/ftp/ndp026c/ ndp026c.pdf.
12. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: Distributed data-parallel programs from sequential building blocks. In EuroSys, pages 59-72, 2007.
13. Y. Kwon, M. Balazinska, B. Howe, and J. Rolia. Skew-resistant parallel processing of feature-extracting scientific user-defined functions. In SoCC, 2010.
14. C. Olston, B. Reed, A. Silberstein, and U. Srivastava. Automatic optimization of parallel dataflow programs. In USENIX, pages 267-273, 2008.
15. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig latin: A not-so-foreign language for data processing. In SIGMOD, pages 1099-1110, 2008.
16. A. Pavlo, E. Paulson, A. Rasin, D. J. Abadi, D. J. DeWitt, S. Madden, and M. Stonebraker. A comparison of approaches to large-scale data analysis. In SIGMOD, pages 165-178, 2009.
17. R. Pike, S. Dorward, R. Griesemer, and S. Quinlan. Interpreting the data: Parallel analysis with sawzall. Sci. Program., 13(4):277-298, 2005.
18. J. W. Stamos and H. C. Young. A symmetric fragment and replicate algorithm for distributed joins. IEEE Trans. Parallel Distrib. Syst., 4:1345-1354, 1993.
19. R. Vernica, M. J. Carey, and C. Li. Efficient parallel set-similarity joins using mapreduce. In SIGMOD, pages 495-506, 2010.
20. H.-C. Yang, A. Dasdan, R.-L. Hsiao, and D. S. Parker. Map-reduce-merge: Simplified relational data processing on large clusters. In SIGMOD, pages 1029-1040, 2007.