ELF: Efficient lightweight fast stream processing at scale

Proceedings of the 2014 USENIX Annual Technical Conference, USENIX ATC 2014

Volumn , Issue , 2014, Pages 25-36

  Hu, Liting ^a   Schwan, Karsten ^a   Amur, Hrishikesh ^a   Chen, Xin ^a  

^a GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BUFFER STORAGE; FORESTRY; TREES (MATHEMATICS);

DATA CHARACTERISTICS; DATACENTER; DECENTRALISED; DIFFERENT TIME WINDOWS; JOB FUNCTIONS; MEMORY EFFICIENT; STREAM PROCESSING; STREAM PROCESSING SYSTEMS; WEB SERVERS; WORKERS';

SCALABILITY;

EID: 85077465475     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (29)

1. Flume. http://flume.apache.org/, 2013.
2. Sparsehash. http://code.google.com/p/sparsehash/.
3. Twitter streaming apis. https://dev.twitter.com/docs/ streaming-apis, 2012.
4. Storm. https://github.com/nathanmarz/storm.git.
5. Zeromq. http://zeromq.org/, 2012.
6. D. J. Abadi, Y. Ahmad, M. Balazinska, M. Cherniack, J. hyon Hwang, W. Lindner, A. S. Maskey, E. Rasin, E. Ryvkina, N. Tatbul, Y. Xing, and S. Zdonik. The design of the borealis stream processing engine. In CIDR, pages 277-289, 2005.
7. T. Akidau, A. Balikov, K. Bekiroglu, S. Chernyak, J. Haberman, R. Lax, S. McVeety, D. Mills, P. Nordstrom, and S. Whittle. Mill-wheel: Fault-tolerant stream processing at internet scale. In VLDB, 2013.
8. M. H. Ali, C. Gerea, B. S. Raman, B. Sezgin, and E. Tarnavski. Microsoft cep server and online behavioral targeting. Proc. VLDB Endow., 2(2):1558-1561, Aug. 2009.
9. H. Amur, W. Richter, D. G. Andersen, M. Kaminsky, K. Schwan, A. Balachandran, and E. Zawadzki. Memory-efficient groupby-aggregate using compressed buffer trees. In SOCC. 2013.
10. B. Babcock, S. Babu, M. Datar, R. Motwani, and J. Widom. Models and issues in data stream systems. In PODS, 2002.
11. P. Bhatotia, A. Wieder, R. Rodrigues, U. A. Acar, and R. Pasquin. Incoop: Mapreduce for incremental computations. In SOCC, pages 7:1-7:14. 2011.
12. T. Condie, N. Conway, P. Alvaro, J. M. Hellerstein, K. Elmeleegy, and R. Sears. Mapreduce online. In NSDI. 2010.
13. H. Gao, J. Hu, C. Wilson, Z. Li, Y. Chen, and B. Y. Zhao. Detecting and characterizing social spam campaigns. In IMC. 2010.
14. B. Gedik, H. Andrade, K.-L. Wu, P. S. Yu, and M. Doo. Spade: the system s declarative stream processing engine. In SIGMOD, pages 1123-1134. 2008.
15. B. He, M. Yang, Z. Guo, R. Chen, B. Su, W. Lin, and L. Zhou. Comet: batched stream processing for data intensive distributed computing. In SOCC, pages 63-74. 2010.
16. W. Lam, L. Liu, S. Prasad, A. Rajaraman, Z. Vacheri, and A. Doan. Muppet: Mapreduce-style processing of fast data. Proc. VLDB Endow., 5(12):1814-1825, Aug. 2012.
17. B. Li, E. Mazur, Y. Diao, A. McGregor, and P. Shenoy. A platform for scalable one-pass analytics using mapreduce. In SIGMOD, pages 985-996. 2011.
18. D. Logothetis, C. Olston, B. Reed, K. C. Webb, and K. Yocum. Stateful bulk processing for incremental analytics. In SOCC, pages 51-62. 2010.
19. D. Logothetis, C. Trezzo, K. C. Webb, and K. Yocum. In-situ mapreduce for log processing. In USENIXATC. 2011.
20. D. G. Murray, F. McSherry, R. Isaacs, M. Isard, P. Barham, and M. Abadi. Naiad: a timely dataflow system. In SOSP. 2013.
21. L. Neumeyer, B. Robbins, A. Nair, and A. Kesari. S4: Distributed stream computing platform. In ICDMW, pages 170-177. 2010.
22. C. Olston, G. Chiou, L. Chitnis, F. Liu, Y. Han, M. Larsson, A. Neumann, V. B. Rao, V. Sankarasubramanian, S. Seth, C. Tian, T. ZiCornell, and X. Wang. Nova: continuous pig/hadoop work-flows. In SIGMOD, pages 1081-1090. 2011.
23. D. Peng and F. Dabek. Large-scale incremental processing using distributed transactions and notifications. In OSDI. 2010.
24. Z. Qian, Y. He, C. Su, Z. Wu, H. Zhu, T. Zhang, L. Zhou, Y. Yu, and Z. Zhang. Timestream: reliable stream computation in the cloud. In Eurosys, pages 1-14. 2013.
25. A. I. T. Rowstron and P. Druschel. Pastry: Scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In Middleware, pages 329-350. 2001.
26. A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, H. Liu, P. Wyckoff, and R. Murthy. Hive: a warehousing solution over a map-reduce framework. Proc. VLDB Endow., 2(2):1626-1629, 2009.
27. M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. McCauley, M. J. Franklin, S. Shenker, and I. Stoica. Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. In NSDI, pages 2-2. 2012.
28. M. Zaharia, T. Das, H. Li, T. Hunter, S. Shenker, and I. Stoica. Discretized streams: Fault-tolerant streaming computation at scale. In SOSP. ACM, Sept. 2013.
29. S. Zdonik, M. Stonebraker, M. Cherniack, U. Çetintemel, M. Balazinska, and H. Balakrishnan. The aurora and medusa projects. Data Engineering, 51:3, 2003.