Streaming big data with self-adjusting computation

DDFP 2013 - Proceedings of the 2013 ACM SIGPLAN Workshop on Data Driven Functional Programming, Co-located with POPL 2013

Volumn , Issue , 2013, Pages 15-18

  Acar, Umut A ^a   Chen, Yan ^b  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b MAX PLANCK INSTITUTE FOR SOFTWARE SYSTEMS   (Germany)

Author keywords

Incremental mapreduce; Self adjusting computation

Indexed keywords

BIG DATUM; DATA SETS; MAP-REDUCE; SELF-ADJUSTING COMPUTATIONS; WIKIPEDIA;

COMPUTER PROGRAMMING LANGUAGES; DATA HANDLING; FUNCTIONAL PROGRAMMING;

DIGITAL STORAGE;

EID: 84873481186     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2429376.2429382     Document Type: Conference Paper

References (20)

1. U. A. Acar. Self-Adjusting Computation. PhD thesis, Department of Computer Science, Carnegie Mellon University, May 2005.
2. U. A. Acar, G. E. Blelloch, and R. Harper. Adaptive functional programming. ACM Trans. Prog. Lang. Sys., 28(6):990-1034, 2006.
3. U. A. Acar, A. Ahmed, and M. Blume. Imperative self-adjusting computation. In Proceedings of the 25th Annual ACM Symposium on Principles of Programming Languages, 2008.
4. U. A. Acar, G. E. Blelloch, M. Blume, R. Harper, and K. Tangwongsan. An experimental analysis of self-adjusting computation. ACM Trans. Prog. Lang. Sys., 32(1):3:1-53, 2009.
5. P. Bhatotia, A. Wieder, R. Rodrigues, U. A. Acar, and R. Pasquin. Incoop: MapReduce for incremental computations. In Proceedings of the 2nd ACM Symposium on Cloud Computing, SOCC '11, pages 7:1-7:14.
6. S. Burckhardt, D. Leijen, C. Sadowski, J. Yi, and T. Ball. Two for the price of one: A model for parallel and incremental computation. In ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, 2011.
7. Y. Chen, J. Dunfield, M. A. Hammer, and U. A. Acar. Implicit self-adjusting computation for purely functional programs. In Int'l Conference on Functional Programming (ICFP '11), pages 129-141, Sept. 2011.
8. Y. Chen, J. Dunfield, and U. A. Acar. Type-directed automatic incrementalization. In ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), Jun 2012.
9. J. Dean and S. Ghemawat. MapReduce: simplified data processing on large clusters. In Proc. 6th Symposium on Operating Systems Design and Implementation (OSDI'04).
10. C. Demetrescu, I. Finocchi, and G. Italiano. Handbook on Data Structures and Applications, chapter 36: Dynamic Graphs. CRC Press, 2005.
11. P. K. Gunda, L. Ravindranath, C. A. Thekkath, Y. Yu, and L. Zhuang. Nectar: Automatic management of data and computation in data centers. In Proc. 9th Symp. Operating Systems Design and Implementation (OSDI'10).
12. M. Hammer, U. A. Acar, M. Rajagopalan, and A. Ghuloum. A proposal for parallel self-adjusting computation. In DAMP '07: Declarative Aspects of Multicore Programming, 2007.
13. M. A. Hammer, U. A. Acar, and Y. Chen. CEAL: a C-based language for self-adjusting computation. In ACM SIGPLAN Conference on Programming Language Design and Implementation, 2009.
14. B. He, M. Yang, Z. Guo, R. Chen, B. Su, W. Lin, and L. Zhou. Comet: batched stream processing for data intensive distributed computing. In Proc. 1st Symposium on Cloud computing (SoCC'10).
15. R. Ley-Wild, U. A. Acar, and M. Fluet. A cost semantics for self-adjusting computation. In Proceedings of the 26th Annual ACM Symposium on Principles of Programming Languages, 2009.
16. D. Logothetis, C. Olston, B. Reed, K. C. Webb, and K. Yocum. Stateful bulk processing for incremental analytics. In 1st Symp. on Cloud computing(SoCC'10). MLton. MLton web site. http://www.mlton.org.
17. C. Olston and et al. Nova: continuous Pig/Hadoop workflows. In Proceed- ings of the International Conference on Management of Data, 2011.
18. D. Peng and F. Dabek. Large-scale incremental processing using distributed transactions and notifications. In Proc. 9th Symposium on Operating Systems Design and Implementation (OSDI'10).
19. L. Popa, M. Budiu, Y. Yu, and M. Isard. DryadInc: Reusing work in largescale computations. In Worksh. on Hot Topics in Cloud Computing (HotCloud'09).
20. G. Ramalingam and T. Reps. A categorized bibliography on incremental computation. In Principles of Programming Languages, pages 502-510, 1993.