Incorporating partitioning and parallel plans into the SCOPE optimizer

Proceedings - International Conference on Data Engineering

Volumn , Issue , 2010, Pages 1060-1071

  Zhou, Jingren ^a   Larson, Per Ake ^a   Chaiken, Ronnie ^a  

^a MICROSOFT   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COST-BASED OPTIMIZATION; DATA PARTITIONING; DISTRIBUTED COMPUTING PLATFORM; LARGE CLUSTERS; MASSIVE DATA; MICROSOFT; NEW OPPORTUNITIES; OPTIMIZERS; PERFORMANCE IMPROVEMENTS; QUERY OPTIMIZATION; RELATIONAL OPERATOR; SCRIPTING LANGUAGES;

OPTIMIZATION;

SORTING;

EID: 77952771965     PISSN: 10844627     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDE.2010.5447802     Document Type: Conference Paper

References (18)

1. Apache. Hadoop. http://hadoop.apache.org/.
2. R. Chaiken, B. Jenkins, P.-Å. Larson, B. Ramsey, D. Shakib, S. Weaver, and J. Zhou. Scope: Easy and efficient parallel processing of massive data sets. In Proceedings of VLDB Conference, 2008.
3. F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. Gruber. Bigtable: A distributed storage system for structured data. In Proceedings of OSDI Conference, 2006.
4. H. Darwen and C. Date. The role of functional dependencies in query decomposition. Relational Database Writings 1989-1991, 1992.
5. J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. In Proceedings of OSDI Conference, 2004.
6. S. Ghemawat, H. Gobioff, and S.-T. Leung. The Google file system. In Proceedings of SOSP Conference, 2003.
7. G. Graefe. Encapsulation of parallelism in the Volcano query processing system. In Proceeding of SIGMOD Conference, 1990.
8. G. Graefe. The Cascades framework for query optimization. Data Engineering Bulletin, 18(3), 1995.
9. G. Graefe and W. J. McKenna. The Volcano optimizer generator: Extensibility and efficient search. In Proceeding of ICDE Conference, 1993.
10. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: Distributed data-parallel programs from sequential building blocks. In Proceedings of EuroSys Conference, 2007.
11. H. Lu. Query Processing in Parallel Relational Database Systems. IEEE Computer Society Press, 1994.
12. T. Neumann and G. Moerkotte. A combined framework for grouping and order optimization. In Proceedings of VLDB Conference, 2004.
13. T. Neumann and G. Moerkotte. An efficient framework for order optimization. In Proceedings of ICDE Conference, 2004.
14. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig latin: A not-so-foreign language for data processing. In Proceedings of SIGMOD Conference, 2008.
15. P. G. Selinger, M. M. Astrahan, D. D. Chamberlin, R. A. Lorie, and T. G. Price. Access path selection in a relational database management system. In Proceedings of SIGMOD Conference, 1979.
16. D. Simmen, E. Shekita, and T. Malkenus. Fundamental techniques for order optimization. In Proceedings of SIGMOD Conference, 1996.
17. X. Wang and M. Cherniack. Avoiding sorting and grouping in processing queries. In Proceedings of VLDB Conference, 2003.
18. Y. Yu, M. Isard, D. Fetterly, M. Budiu, U. Erlingsson, P. K. Gunda, and J. Currey. DryadLINQ: A system for general-purpose distributed data-parallel computing using a high-level language. In Proceedings of OSDI Conference, 2008.