Advanced join strategies for large-scale distributed computation

Proceedings of the VLDB Endowment

Volumn 7, Issue 13, 2014, Pages 1484-1495

  Bruno, Nicolas ^a   Kwon, YongChul ^a   Wu, Ming Chuan ^a  

^a MICROSOFT   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

HIGH LEVEL LANGUAGES;

DISTRIBUTED COMPUTATIONS; EXECUTION STRATEGIES; EXPERIMENTAL VALIDATIONS; JOIN OPERATION; JOIN STRATEGIES; LARGE CLUSTERS; MASSIVE DATA SETS; SCRIPTING LANGUAGES;

DATA STREAMS;

EID: 84905855910     PISSN: None     EISSN: 21508097     Source Type: Conference Proceeding    
DOI: 10.14778/2733004.2733020     Document Type: Conference Paper

References (25)

1. F. Afrati and J. Ullman. Optimizing multiway joins in a map-reduce environment. IEEE Transactions on Knowledge and Data Engineering, 23(9):1282-1298, 2011.
2. Apache. Hadoop. http://hadoop.apache.org/.
3. S. Blanas, J. M. Patel, V. Ercegovac, J. Rao, E. J. Shekita, and Y. Tian. A comparison of join algorithms for log processing in MapReduce. In Proc. of the SIGMOD Conf., pages 975-986, 2010.
4. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. In Proceedings of OSDI Conference, pages 10-10, 2004.
5. D. DeWitt, J. Naughton, D. Schneider, and S. S. Seshadri. Practical skew handling in parallel joins. In Proc. of the 18th VLDB Conf., pages 27-40, 1992.
6. A. F. Gates, O. Natkovich, S. Chopra, P. Kamath, S. M. Narayanamurthy, C. Olston, B. Reed, S. Srinivasan, and U. Srivastava. Building a high-level dataflow system on top of map-reduce: the pig experience. Proc. of the VLDB Endowment, 2:1414-1425, August 2009.
7. G. Graefe. Query evaluation techniques for large databases. ACM Computing Surveys, 25(2):73-169, 1993.
8. He Yongqiang. handle skewed keys for a join in a separate job. https://issues.apache.org/jira/browse/HIVE-964.
9. K. Hua and C. Lee. Handling data skew in multiprocessor database computers using partition tuning. In Proc. of the 17th VLDB Conf., pages 525-535, 1991.
10. K. Hua, C. Lee, and C. Hua. Dynamic load balancing in multicomputer database systems using partition tuning. IEEE TKDE, 7(6):968-983, 1995.
11. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: Distributed data-parallel programs from sequential building blocks. In Proc. of EuroSys Conference, pages 59-72, 2007.
12. M. Kitsuregawa and Y. Ogawa. Bucket spreading parallel hash: a new, robust, parallel hash join method for data skew in the super database computer (sdc). In Proc. of the 16th VLDB Conf., pages 210-221, 1990.
13. W. Li, D. Gao, and R. Snodgrass. Skew handling techniques in sort-merge join. In Proc. of the SIGMOD Conf., pages 169-180, 2002.
14. Namit Jain. Skewed Join Optimization. https://issues.apache.org/jira/browse/HIVE-3086.
15. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig latin: A not-so-foreign language for data processing. In Proceedings of SIGMOD Conference, pages 1099-1110, 2008.
16. S. Schelter, C. Boden, M. Schenck, A. Alexandrov, and V. Markl. Distributed matrix factorization with mapreduce using a series of broadcast-joins. In Proc. of the 7th ACM conf. on Recommender Systems, pages 281-284, 2013.
17. A. Shatdal and J. F. Naughton. Using shared virtual memory for parallel join processing. In Proc. of the SIGMOD Conf., pages 119-128, 1993.
18. Sriranjan Manjunath. support for skewed outer join. https://issues.apache.org/jira/browse/PIG-1035.
19. A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, N. Zhang, S. Antony, H. Liu, and R. Murthy. Hive-a petabyte scale data warehouse using Hadoop. In Proceedings of ICDE Conference, pages 996-1005, 2010.
20. J. L. Wolf, D. M. Dias, and P. S. Yu. An effective algorithm for parallelizing sort merge joins in the presence of data skew. In Proceedings of the second international symposium on databases in parallel and distributed systems, DPDS '90, pages 103-115, 1990.
21. J. L. Wolf, D. M. Dias, P. S. Yu, and J. Turek. An effective algorithm for parallelizing hash joins in the presence of data skew. In Proc. of the 7th ICDE Conf., pages 200-209, 1991.
22. Y. Xu and P. Kostamaa. Efficient outer join data skew handling in parallel dbms. Proc. of the VLDB Endowment, 2(2):1390-1396, 2009.
23. Y. Xu, P. Kostamaa, X. Zhou, and L. Chen. Handling data skew in parallel joins in shared-nothing systems. In Proc. of the SIGMOD Conf., pages 1043-1052, 2008.
24. J. Zhou, N. Bruno, M.-C. Wu, P.-Å. Larson, R. Chaiken, and D. Shakib. SCOPE: Parallel databases meet mapreduce. The VLDB Journal, 21(5):611-636, 2012.
25. J. Zhou, P.-Å. Larson, and R. Chaiken. Incorporating partitioning and parallel plans into the SCOPE optimizer. In Proceedings of ICDE Conference, pages 1060-1071, 2010.