Advanced partitioning techniques for massively distributed computation

Proceedings of the ACM SIGMOD International Conference on Management of Data

Volumn , Issue , 2012, Pages 13-24

  Zhou, Jingren ^a   Bruno, Nicolas ^a   Lin, Wei ^a  

^a MICROSOFT   (United States)

Author keywords

distributed computation; partitioning; query optimization; scope

Indexed keywords

CONCEPTUAL DATA MODELS; CRITICAL BUSINESS; DISTRIBUTED COMPUTATIONS; DISTRIBUTED DATA; EXECUTION STRATEGIES; HIGHER-LEVEL PROGRAMMING LANGUAGES; LARGE CLUSTERS; MACHINE PROGRAMMING; MAP-REDUCE; MICROSOFT; PARTITIONING; PARTITIONING STRATEGIES; PARTITIONING TECHNIQUES; PERFORMANCE BOTTLENECKS; QUERY OPTIMIZATION; SCOPE;

COMPUTER PROGRAMMING; OPTIMIZATION;

DISTRIBUTED COMPUTER SYSTEMS;

EID: 84862647304     PISSN: 07308078     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2213836.2213839     Document Type: Conference Paper

References (24)

1. Apache. Hadoop. http://hadoop.apache.org/.
2. K. S. Beyer, V. Ercegovac, R. Gemulla, A. Balmin, M. Eltabakh, C.-C. Kanne, F. Ozcan, and E. J. Shekita. Jaql: A scripting language for large scale semistructured data analysis. In Proceedings of VLDB Conference, 2011.
3. R. Chaiken, B. Jenkins, P.-Å. Larson, B. Ramsey, D. Shakib, S. Weaver, and J. Zhou. SCOPE: Easy and efficient parallel processing of massive data sets. In Proceedings of VLDB Conference, 2008.
4. B. Chattopadhyay, L. Lin, W. Liu, S. Mittal, P. Aragonda, V. Lychagina, Y. Kwon, and M. Wong. Tenzing: A SQL implementation on the mapreduce framework. In Proceedings of VLDB Conference, 2011.
5. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. In Proceedings of OSDI Conference, 2004.
6. D. DeWitt and J. Gray. Parallel database systems: The future of high performance database processing. Communications of the ACM, 36(6), 1992.
7. S. Ghemawat, H. Gobioff, and S.-T. Leung. The Google file system. In Proceedings of SOSP Conference, 2003.
8. G. Graefe. Encapsulation of parallelism in the Volcano query processing system. In Proceeding of SIGMOD Conference, 1990.
9. G. Graefe. The Cascades framework for query optimization. Data Engineering Bulletin, 18(3), 1995.
10. G. Graefe, A. Linville, and L. Shapiro. Sort versus hash revisited. IEEE Transactions on Knowledge and Data Engineering, 6(6), 1994.
11. G. Graefe and W. J. McKenna. The Volcano optimizer generator: Extensibility and efficient search. In Proceeding of ICDE Conference, 1993.
12. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: Distributed data-parallel programs from sequential building blocks. In Proc. of EuroSys Conference, 2007.
13. A. Jhingran, T. Malkemus, and S. Padmanabhan. Query optimization in DB2 parallel edition. Data Engineering Bulletin, 20(2), 1997.
14. H. Lu. Query Processing in Parallel Relational Database Systems. IEEE Computer Society Press, 1994.
15. S. Melnik, A. Gubarev, J. J. Long, G. Romer, S. Shivakumar, M. Tolton, and T. Vassilakis. Dremel: Interactive analysis of webscale datasets. In Proceedings of VLDB Conference, 2010.
16. T. Neumann and G. Moerkotte. A combined framework for grouping and order optimization. In Proceedings of VLDB Conference, 2004.
17. T. Neumann and G. Moerkotte. An efficient framework for order optimization. In Proceedings of ICDE, 2004.
18. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig latin: A not-so-foreign language for data processing. In Proceedings of SIGMOD Conference, 2008.
19. P. G. Selinger, M. M. Astrahan, D. D. Chamberlin, R. A. Lorie, and T. G. Price. Access path selection in a relational database management system. In Proceedings of SIGMOD Conference, 1979.
20. D. Simmen, E. Shekita, and T. Malkenus. Fundamental techniques for order optimization. In Proceedings of SIGMOD Conference, 1996.
21. A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, N. Zhang, S. Antony, H. Liu, and R. Murthy. Hive - a petabyte scale data warehouse using Hadoop. In Proceedings of ICDE Conference, 2010.
22. X. Wang and M. Cherniack. Avoiding sorting and grouping in processing queries. In Proc. of VLDB Conference, 2003.
23. Y. Yu, M. Isard, D. Fetterly, M. Budiu, U. Erlingsson, P. K. Gunda, and J. Currey. DryadLINQ: A system for general-purpose distributed data-parallel computing using a high-level language. In Proc. of OSDI Conference, 2008.
24. J. Zhou, P.-Å. Larson, and R. Chaiken. Incorporating partitioning and parallel plans into the SCOPE optimizer. In Proceedings of ICDE Conference, 2010.