Distributed data-parallel computing using a high-level programming language

SIGMOD-PODS'09 - Proceedings of the International Conference on Management of Data and 28th Symposium on Principles of Database Systems

Volumn , Issue , 2009, Pages 987-994

  Isard, Michael ^a   Yu, Yuan ^a  

^a MICROSOFT RESEARCH   (United States)

Author keywords

Cloud computing; Concurrency; Distributed programming

Indexed keywords

CLOUD COMPUTING; CONTINUOUS OPERATION; DATA MODELS; DATA PARALLEL; DATA SETS; DEVELOPMENT TOOLS; DISTRIBUTED DATA; DISTRIBUTED DATABASE; DISTRIBUTED EXECUTION; DISTRIBUTED PROGRAMMING; EXECUTION ENVIRONMENTS; HIGH-LEVEL PROGRAMMING LANGUAGE; LARGE SCALE DATA; PROGRAMMING MODELS; RELIABLE EXECUTION; SEQUENTIAL PROGRAMS; SIDE EFFECT;

C (PROGRAMMING LANGUAGE); COMPUTER SCIENCE; COMPUTER SOFTWARE; LINGUISTICS; PARALLEL ARCHITECTURES; PARALLEL PROGRAMMING; PROGRAM DEBUGGING;

DATABASE SYSTEMS;

EID: 70849091519     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1559845.1559962     Document Type: Conference Paper

References (16)

1. The DryadLINQ project. http://research.microsoft.com/projects/dryadLINQ/.
2. The LINQ project. http://msdn.microsoft.com/netframework/future/linq/.
3. Hadoop wiki. http://wiki.apache.org/hadoop/, April 2008.
4. H. Boral, W. Alexander, L. Clay, G. Copeland, S. Danforth, M. Franklin, B. Hart, M. Smith, and P. Valduriez. Prototyping Bubba, a highly parallel database system. IEEE Trans. on Knowl. and Data Eng., 2(1):4-24, 1990.
5. R. Chaiken, B. Jenkins, P,-A°. Larson, B. Ramsey, D. Shakib, S. Weaver, and J. Zhou. SCOPE: Easy and efficient parallel processing of massive data sets. In International Conference of Very Large Data Bases (VLDB), August 2008.
6. T. Cruanes, B. Dageville, and B. Ghosh. Parallel SQL execution in Oracle 10g. In ACM SIGMOD, pages 850-854, Paris, France, 2004. ACM.
7. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters, In Proceedings of the 6th Symposium on Operating Systems Design and Implementation (OSDI), pages 137-150, Dec. 2004.
8. D. DeWitt, S. Ghandeharizadeh, D. Schneider, H. Hsiao, A. Bricker, and R. Rasmussen. The Gamma database machine project. IEEE Transactions on Knowledge and Data Engineering, 2(1), 1990.
9. D. DeWitt and J. Gray. Parallel database systems: The future of high performance database processing. Communications of the ACM, 36(6), 1992.
10. J. Duffy. A query language for data parallel programming. In Proceedings of the 2007 workshop on Declarative aspects of multicore programming, January 2007.
11. G. Graefe. Encapsulation of parallelism in the Volcano query processing system. In SIGMOD International Conference on Management of data, pages 102-111, New York, NY, USA, 1990. ACM Press.
12. J. M. Hellerstein, M. Stonebraker, and J. Hamilton. Architecture of a database system. Foundations and Trends in Databases, 1(2):141-259, 2007.
13. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: Distributed data-parallel programs from sequential building blocks. In Proceedings of European Conference on Computer Systems (EuroSys), pages 59-72, March 2007.
14. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig Latin: A not-so-foreign language for data processing. In International Conference on Management of Data (Industrial Track) (SIGMOD), Vancouver, Canada, June 2008.
15. Y. Yu, M. Isard, D. Fetterly, M. Budiu,Ú. Erlingsson, P. K. Gunda, and J. Currey. DryadLINQ: A system for general-purpose distributed data-parallel computing using a high-level language. In Proceedings of the 8th Symposium on Operating Systems Design and Implementation (OSDI), December 8-10 2008.
16. Y. Yu, M. Isard, D. Fetterly, M. Budiu,U.́ Erlingsson, P. K. Gunda, J. Currey, F. McSherry, and K. Achan. Some sample programs written in DryadLINQ. Technical Report MSR-TR-2008-74, Microsoft Research, May 2008.