Distributed aggregation for data-parallel computing: Interfaces and implementations

SOSP'09 - Proceedings of the 22nd ACM SIGOPS Symposium on Operating Systems Principles

Volumn , Issue , 2009, Pages 247-260

  Yu, Yuan ^a   Gunda, Pradeep Kumar ^a   Isard, Michael ^a  

^a MICROSOFT RESEARCH   (United States)

Author keywords

Cloud computing; Concurrency; Distributed programming

Indexed keywords

CLOUD COMPUTING; DATA PARALLEL; DATA-INTENSIVE APPLICATION; DISTRIBUTED COMPUTING SYSTEMS; DISTRIBUTED PROGRAMMING; DISTRIBUTED PROGRAMMING MODEL; HIGH-LEVEL QUERIES; LANGUAGE INTEGRATION; LARGE-COMPUTING; MATERIAL EFFECT; MATRIX MULTIPLICATION; PARALLEL SERVERS; PROGRAMMING INTERFACE; STATE OF THE ART; USER DEFINED FUNCTIONS;

COMPUTATIONAL EFFICIENCY; COMPUTER OPERATING SYSTEMS; COMPUTER SCIENCE; HIGH LEVEL LANGUAGES; LINGUISTICS; PARALLEL ARCHITECTURES; QUERY LANGUAGES;

ARTS COMPUTING;

EID: 72249089011     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1629575.1629600     Document Type: Conference Paper

References (28)

1. The HIVE project. http://hadoop.apache.org/hive/.
2. Database Languages - SQL, ISO/IEC 9075-*:2003, 2003.
3. Hadoop wiki. http://wiki.apache.org/hadoop/, April 2008.
4. C. Baru and G. Fecteau. An overview of DB2 parallel edition. In International Conference on Management of Data (SIGMOD), pages 460-462, New York, NY, USA, 1995. ACM Press.
5. G. E. Blelloch. Programming parallel algorithms. Communications of the ACM (CACM), 39(3):85-97, 1996.
6. R. Chaiken, B. Jenkins, P.-Å. Larson, B. Ramsey, D. Shakib, S. Weaver, and J. Zhou. SCOPE: Easy and efficient parallel processing of massive data sets. In International Conference of Very Large Data Bases (VLDB), August 2008.
7. S. Chaudhuri. An overview of query optimization in relational systems. In PODS '98: Proceedings of the seventeenth ACM SIGACT-SIGMOD-SIGART symposium on Principles of database systems, pages 34-43, 1998.
8. M. Cole. Algorithmic skeletons: structured management of parallel computation. MIT Press, Cambridge, MA, USA, 1991.
9. T. Cruanes, B. Dageville, and B. Ghosh. Parallel SQL execution in Oracle 10g. In International Conference on Management of Data (SIGMOD), pages 850-854, Paris, France, 2004. ACM.
10. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. In Proceedings of the 6th Symposium on Operating Systems Design and Implementation (OSDI), pages 137-150, Dec. 2004.
11. D. DeWitt, S. Ghandeharizadeh, D. Schneider, H. Hsiao, A. Bricker, and R. Rasmussen. The Gamma database machine project. IEEE Transactions on Knowledge and Data Engineering, 2(1), 1990.
12. D. DeWitt and J. Gray. Parallel database systems: The future of high performance database processing. Communications of the ACM, 36(6), 1992.
13. A. Eisenberg, J. Melton, K. Kulkarni, J.-E. Michels, and F. Zemke. Sql:2003 has been published. SIGMOD Rec., 33(1):119-126, 2004.
14. G. Graefe. Encapsulation of parallelism in the Volcano query processing system. In International Conference on Management of data (SIGMOD), pages 102-111, New York, NY, USA, 1990. ACM Press.
15. G. Graefe. Query evaluation techniques for large databases. ACM Computing Surveys, 25(2):73-169, 1993.
16. J. Gray, S. Chaudhuri, A. Bosworth, A. Layman, D. Reichart, M. Venkatrao, F. Pellow, and H. Pirahesh. Data cube: A relational aggregation operator generalizing group-by, cross-tab, and sub-totals. Data Mining and Knowledge Discovery, 1(1), 1997.
17. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: Distributed data-parallel programs from sequential building blocks. In Proceedings of European Conference on Computer Systems (EuroSys), pages 59-72, March 2007.
18. M. Isard and Y. Yu. Distributed data-parallel computing using a high-level programming language. In International Conference on Management of Data (SIGMOD), June 29-July 2 2009.
19. R. Lämmel. Google's mapreduce programming model - revisited. Science of Computer Programming, 70(1):1-30, 2008.
20. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig Latin: A not-so-foreign language for data processing. In International Conference on Management of Data (Industrial Track) (SIGMOD), Vancouver, Canada, June 2008.
21. R. Pike, S. Dorward, R. Griesemer, and S. Quinlan. Interpreting the data: Parallel analysis with Sawzall. Scientific Programming, 13(4):277-298, 2005.
22. F. Rabhi and S. Gorlatch. Patterns and Skeletons for Parallel and Distributed Computing. Springer, 2003.
23. C. Ranger, R. Raghuraman, A. Penmetsa, G. Bradski, and C. Kozyrakis. Evaluating mapreduce for multi-core and multiprocessor systems. In HPCA '07: Proceedings of the 2007 IEEE 13th International Symposium on High Performance Computer Architecture, pages 13-24, 2007.
24. L. A. Rowe and M. R. Stonebraker. The postgres data model. In International Conference of Very Large Data Bases (VLDB), pages 83-96. Society Press, 1987.
25. J. Russell. Oracle9i Application Developer's Guide - Fundamentals. Oracle Corporation, 2002.
26. P. Trinder, H.-W. Loidl, and R. Pointon. Parallel and distributed Haskells. Journal of Functional Programming, 12((4&5)):469-510, 2002.
27. Y. Yu, M. Isard, D. Fetterly, M. Budiu, Ú. Erlingsson, P. K. Gunda, and J. Currey. DryadLINQ: A system for general-purpose distributed data-parallel computing using a high-level language. In Proceedings of the 8th Symposium on Operating Systems Design and Implementation (OSDI), December 8-10 2008.
28. Y. Yu, M. Isard, D. Fetterly, M. Budiu, Ú. Erlingsson, P. K. Gunda, J. Currey, F. McSherry, and K. Achan. Some sample programs written in DryadLINQ. Technical Report MSR-TR-2008-74, Microsoft Research, May 2008.