Benchmarking streaming computation engines: Storm, flink and spark streaming

Proceedings - 2016 IEEE 30th International Parallel and Distributed Processing Symposium, IPDPS 2016

Volumn , Issue , 2016, Pages 1789-1792

  Chintapalli, Sanket ^a   Dagit, Derek ^a   Evans, Bobby ^a   Farivar, Reza ^a   Graves, Thomas ^a   Holderbaugh, Mark ^a   Liu, Zhuo ^a   Nusbaum, Kyle ^a   Patil, Kishorkumar ^a   Peng, Boyang Jerry ^a   Poulosky, Paul ^a  

^a YAHOO INC   (United States)

Author keywords

Benchmark; Flink; Low Latency; Spark; Storm; Streaming processing

Indexed keywords

BENCHMARKING; DATA HANDLING; DATA TRANSFER; ELECTRIC SPARKS; ENGINES; STORMS;

COMPUTATION ENGINE; FLINK; LOW LATENCY; PERFORMANCE COMPARISON; REAL TIME STREAMING; STREAMING COMPUTATIONS; STREAMING DATA PROCESSING; STREAMING PROCESSING;

PIPELINE PROCESSING SYSTEMS;

EID: 84991585535     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPSW.2016.138     Document Type: Conference Paper

References (16)

1. "Apache Storm Project." http://storm.apache.org/.
2. "Apache Flink Project." http://flink.apache.org/.
3. "Apache Spark Project." http://spark.apache.org/.
4. M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica, "Spark: cluster computing with working sets," in HotCloud, 2010.
5. "Apache Samza Project." http://samza.apache.org/.
6. "Apache Apex Project." http://apex.incubator.apache.org/.
7. T. Akidau, R. Bradshaw, C. Chambers, S. Chernyak, R. J. Fernández-Moctezuma, R. Lax, S. McVeety, D. Mills, F. Perry, E. Schmidt, et al., "The dataflow model: a practical approach to balancing correctness, latency, and cost in massive-scale, unbounded, out-of-order data processing," Proceedings of the VLDB Endowment, vol. 8, no. 12, pp. 1792-1803, 2015.
8. L. Neumeyer, B. Robbins, A. Nair, and A. Kesari, "S4: Distributed stream computing platform," in Data Mining Workshops (ICDMW), 2010 IEEE International Conference on, pp. 170-177, IEEE, 2010.
9. "JStorm Project." http://github.com/alibaba/jstorm/.
10. S. Kulkarni, N. Bhagat, M. Fu, V. Kedigehalli, C. Kellogg, S. Mittal, J. M. Patel, K. Ramasamy, and S. Taneja, "Twitter heron: Stream processing at scale," in Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, pp. 239-250, ACM, 2015.
11. "Which Stream Processing Engine: Storm, Spark, Samza, or Flink?." http://www.altaterra.net/blogpost/288668/225612/Which-Stream-Processing-Engine-Storm-Spark-Samza-or-Flink/.
12. "High-throughput, low-latency, and exactly-once stream processing with Apache Flink." http://data-artisans.com/high-throughput-low-latencyand-exactly-once-stream-processing-with-apache-flink/.
13. "Streaming Big Data: Storm, Spark and Samza." https://tsicilian.wordpress.com/2015/02/16/streaming-big-data-stormspark-and-samza/.
14. "Yahoo Streaming Benchmark." https://github.com/yahoo/streamingbenchmarks.
15. "Apache Kafka Project." http://kafka.apache.org/.
16. "Redis Project." http://redis.io