Query optimization for massively parallel data processing

Proceedings of the 2nd ACM Symposium on Cloud Computing, SOCC 2011

Volumn , Issue , 2011, Pages

  Wu, Sai ^a   Li, Feng ^a   Mehrotra, Sharad ^b   Ooi, Beng Chin ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^b Ringgold Standard Institution Informatics   (United States)

Author keywords

Hive; MapReduce; Multi way join; Query optimization

Indexed keywords

CONVENTIONAL DATABASE; COST MODELS; DATA WAREHOUSE SYSTEMS; HIVE; MAP-REDUCE; MULTI-WAY JOIN; PARALLEL DATA PROCESSING; PLAN-BASED; PROCESSING NODES; PROCESSING SYSTEMS; QUERY OPTIMIZATION; QUERY OPTIMIZER; QUERY PROCESSING SYSTEM; SQL QUERY;

CLOUD COMPUTING; DATA WAREHOUSES; OPTIMIZATION; QUERY PROCESSING;

DATA HANDLING;

EID: 82155187171     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2038916.2038928     Document Type: Conference Paper

References (30)

1. http://hadoop.apache.org.
2. http://wiki.apache.org/hadoop/hive/languagemanual/joins.
3. http://www.aster.com.
4. http://www.greenplum.com.
5. http://www.tpc.org/tpch/.
6. F. N. Afrati and J. D. Ullman. Optimizing joins in a map-reduce environment. EDBT, 2009.
7. P. A. Bernstein, N. Goodman, E. Wong, C. L. Reeve, and J. B. Rothnie, Jr. Query processing in a system for distributed databases (sdd-1). ACM Trans. Database Syst., 6(4):602-625, 1981. (Pubitemid 12503981)
8. Y. Cao, C. Chen, F. Guo, D. Jiang, Y. Lin, B. C. Ooi, H. T. Vo, S. Wu, and Q. Xu. Es2: A cloud data storage systemfor supporting both oltp and olap. In ICDE, pages 291-302, 2011.
9. S. Chaudhuri. An overview of query optimization in relational systems. In PODS, pages 34-43, 1998.
10. C. Chen, G. Chen, D. Jiang, B. C. Ooi, L. Shi, H. T. Vo, and S. Wu. E3: an elastic execution engine for scalable data processing. Technical Report, National University of Singapore, School of Computing. TRA07/11, 2011.
11. C. Chen, G. Chen, D. Jiang, B. C. Ooi, H. T. Vo, S. Wu, and Q. Xu. Providing scalable database services on the cloud. In WISE, pages 1-19, 2010.
12. G. Chen, H. T. Vo, S. Wu, B. C. Ooi, and M. T. Özsu. A framework for supporting dbms-like indexes in the cloud. In VLDB, 2011.
13. M.-S. Chen, P. S. Yu, and K.-L. Wu. Optimization of parallel execution for multi-join queries. IEEE Trans. on Knowl. And Data Eng., 8(3):416-428, 1996. (Pubitemid 126777347)
14. T. Condie, N. Conway, P. Alvaro, J. M. Hellerstein, K. Elmeleegy, and R. Sears. Mapreduce online. Technical report, EECS Department, University of California, Berkeley, Oct 2009.
15. J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. pages 137-150.
16. M. J. Franklin, B. T. Jónsson, and D. Kossmann. Performance tradeoffs for client-server query processing. SIGMOD Rec., 25(2):149-160, 1996. (Pubitemid 126440764)
17. E. Friedman, P. Pawlowski, and J. Cieslewicz. Sql/mapreduce: a practical approach to self-describing, polymorphic, and parallelizable user-defined functions. VLDB, 2009.
18. S. Ganguly, W. Hasan, and R. Krishnamurthy. Query optimization for parallel execution. SIGMOD Rec., 21(2), 1992.
19. M. Jarke and J. Koch. Query optimization in database systems. ACM Comput. Surv., 16(2):111-152, 1984.
20. Y. Jia. Running tpc-h queries on hive. In http://issues.apache.org/jira/ browse/HIVE-600, 2009.
21. D. Jiang, B. C. Ooi, L. Shi, and S. Wu. The performance of mapreduce: An in-depth study. PVLDB, 3(1):472-483, 2010.
22. Y. Lin, D. Agrawal, C. Chen, B. C. Ooi, and S.Wu. Llama: leveraging columnar storage for scalable join processing in the mapreduce framework. In SIGMOD Conference, pages 961-972, 2011.
23. T. Nykiel, M. Potamias, C. Mishra, G. Kollios, and N. Koudas. Mrshare: Sharing across multiple queries in mapreduce. In VLDB, 2010.
24. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig latin: a not-so-foreign language for data processing. In SIGMOD, 2008.
25. K. Ono and G. M. Lohman. Measuring the complexity of join enumeration in query optimization. In VLDB, pages 314-325, 1990.
26. V. Poosala, P. J. Haas, Y. E. Ioannidis, and E. J. Shekita. Improved histograms for selectivity estimation of range predicates. SIGMOD Rec., 25(2), 1996.
27. R. Stewart. Performance and programmability comparison mapreduce query languages. In Master Thesis, Heriot-Watt University, 2010.
28. A. Thusoo, R. Murthy, J. S. Sarma, Z. Shao, N. Jain, P. Chakka, S. Anthony, H. Liu, and N. Zhang. Hive - a petabyte scale data warehousing using hadoop. In ICDE, 2010.
29. A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, H. Liu, P. Wychoff, and R. Murthy. Hive - a warehousing solution over a map-reduce framework. In VLDB, 2009.
30. M. Zaharia, D. Borthakur, J. S. Sarma, K. Elmeleegy, S. Shenker, and I. Stoica. Job scheduling for multi-user mapreduce clusters. In Technical Report, UCB/EECS-2009-55, University of California at Berkeley, 2009.