Dryadinc: Reusing work in large-scale computations

Workshop on Hot Topics in Cloud Computing, HotCloud 2009

Volumn , Issue , 2009, Pages

  Popa, Lucian ^a   Budiu, Mihai ^b   Yu, Yuan ^b   Isard, Michael ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLOUD COMPUTING;

DATA PARTITION; INCREMENTAL COMPUTATION; LARGE SCALE COMPUTATION;

LARGE DATASET;

EID: 85047363692     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (19)

1. ACAR, U. A., BLELLOCH, G. E., AND HARPER, R. Adaptive functional programming. In ACM POPL (2002).
2. CHAIKEN, R., BOB JENKINS, P.-ØA. L., RAMSEY, B., SHAKIB, D., WEAVER, S., AND ZHOU, J. SCOPE: Easy and efficient parallel processing of massive data sets. In VLDB (2008).
3. DEAN, J., AND GHEMAWAT, S. Mapreduce: Simplified data processing on large clusters. In OSDI (2004).
4. DEMERS, A., REPS, T., AND TEITELBAUM, T. Incremental evaluation for attribute grammars with application to syntax-directed editors. In ACM POPL (1981).
5. DONG, G. Incremental maintenance of recursive views: a survey. Materialized views: techniques, implementations, and applications (1999).
6. GHEMAWAT, S., GOBIOFF, H., AND LEUNG, L. The Google file system. In SOSP (2003).
7. GRAY, J., AND SZALAY, A. Science in an exponential world. Nature 440, 23 (March 23 2006).
8. GUPTA, A., AND MUMICK, I. S. Maintenance of materialized views: Problems, techniques, and applications. IEEE Data Engineering Bulletin (1995).
9. HEYDON, A., LEVIN, R., AND YU, Y. Caching function calls using precise dependencies. In ACMSIGPLANPLDI (2000).
10. HOOVER, R. Incremental graph evaluation. PhD Thesis (1987).
11. ISARD, M., BUDIU, M., YU, Y., BIRRELL, A., AND FETTERLY, D. Dryad: Distributed data-parallel programs from sequential building blocks. In EuroSys (2007).
12. LIU, Y. A., STOLLER, S. D., AND TEITELBAUM, T. Static caching for incremental computation. ACM Trans. Program. Lang. Syst (1998).
13. LOGOTHETIS, D., AND YOCUM, K. Ad-hoc data processing in the cloud. Proc. VLDB Endow. (2008).
14. MICHIE, D. Memo functions and machine learning. Nature, 218 (1968).
15. OLSTON, C., REED, B., SRIVASTAVA, U., KUMAR, R., AND TOMKINS, A. Pig Latin: A not-so-foreign language for data processing. In ACM SIGMOD (2008).
16. PIKE, R., DORWARD, S., GRIESEMER, R., AND QUIN-LAN, S. Interpreting the data: Parallel analysis with Sawzall. Scientific Programming 13, 4 (2005), 277-298.
17. PUGH, W., AND TEITELBAUM, T. Incremental computation via function caching. In ACM POPL (1989).
18. YAHOO! Hadoop. http://hadoop.apache.org/, 2007.
19. YU, Y., ISARD, M., FETTERLY, D., BUDIU, M., ERLINGSSON, Ú., GUNDA, P. K., AND CURREY, J. DryadLINQ: A system for general-purpose distributed data-parallel computing using a high-level language. In OSDI (2008).