Stela: Enabling stream processing systems to scale-in and scale-out on-demand

Proceedings - 2016 IEEE International Conference on Cloud Engineering, IC2E 2016: Co-located with the 1st IEEE International Conference on Internet-of-Things Design and Implementation, IoTDI 2016

Volumn , Issue , 2016, Pages 22-31

  Xu, Le ^a   Peng, Boyang ^b   Gupta, Indranil ^a  

^a University of Illinois at Urbana Champaign   (United States)

^b YAHOO INC   (United States)

Author keywords

Distributed Systems; Elasticity; Scalability; Stream Processing

Indexed keywords

BENCHMARKING; DISTRIBUTED PARAMETER CONTROL SYSTEMS; ELASTICITY; SCALABILITY; STORMS; THROUGHPUT;

DISTRIBUTED STREAM PROCESSING; DISTRIBUTED SYSTEMS; MICRO-BENCHMARK; ON DEMANDS; REAL TIME; STREAM PROCESSING; STREAM PROCESSING SYSTEMS;

BIG DATA;

EID: 84978062188     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IC2E.2016.38     Document Type: Conference Paper

References (27)

1. Apache Hadoop, " http://hadoop.apache.org/, 2016, ONLINE
2. Apache Hive, " https://hive.apache.org/, 2016, ONLINE
3. Apache Pig, " http://pig.apache.org/, 2016, ONLINE
4. Apache Spark, " https://spark.apache.org/, 2016, ONLINE
5. Apache Storm, " http://storm.incubator.apache.org/, 2016, ONLINE
6. Apache Zookeeper, " http://zookeeper.apache.org/, 2016, ONLINE
7. Emulab, " http://emulab.net/, 2016, ONLINE
8. D. Abadi, D. Carney, U. Cetintemel, M. Cherniack, C. Convey, C. Erwin, E. Galvez, M. Hatoun, A. Maskey, A. Rasin, et al., Aurora: A data stream management system, " in Proceedings of the 2003 ACM SIGMOD International Conference on Management of Data. ACM, 2003, pp. 666-666.
9. D. J. Abadi, Y. Ahmad, M. Balazinska, U. Cetintemel, M. Cherniack, J.-H. Hwang, W. Lindner, A. Maskey, A. Rasin, E. Ryvkina, et al., The design of the Borealis stream processing engine." in The Conference on Innovative Data Systems Research (CIDR), vol. 5, 2005, pp. 277-289.
10. L. Amini, H. Andrade, R. Bhagwan, F. Eskesen, R. King, P. Selo, Y. Park, and C. Venkatramani, SPC: A distributed, scalable platform for data mining, " in Proceedings of the 4th International Workshop on Data Mining Standards, Services, and Platforms. ACM, 2006, pp. 27-37.
11. L. Aniello, R. Baldoni, and L. Querzoni, Adaptive online scheduling in Storm, " in Proceedings of the 7th ACM International Conference on Distributed Event-based Systems. ACM, 2013, pp. 207-218.
12. M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, et al., A view of cloud computing, " Communications of the ACM, vol. 53, no. 4, pp. 50-58, 2010.
13. R. Castro Fernandez, M. Migliavacca, E. Kalyvianaki, and P. Pietzuch, Integrating scale out, and fault tolerance in stream processing using operator state management, " in Proceedings of the 2013 ACM SIGMOD international conference on Management of data. ACM, 2013, pp. 725-736.
14. B. Gedik, H. Andrade, K.-L. Wu, P. S. Yu, and M. Doo, SPADE: The System S declarative stream processing engine, " in Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. ACM, 2008, pp. 1123-1134.
15. B. Gedik, S. Schneider, M. Hirzel, and K.-L. Wu, Elastic scaling for data stream processing, " IEEE Transactions on Parallel, and Distributed Systems., vol. 25, no. 6, pp. 1447-1463, 2014.
16. V. Gulisano, R. Jimenez-Peris, M. Patino-Martinez, C. Soriente, and P. Valduriez, StreamCloud: An elastic, and scalable data streaming system, " IEEE Transactions on Parallel, and Distributed Systems., vol. 23, no. 12, pp. 2351-2365, 2012.
17. T. Heinze, L. Roediger, A. Meister, Y. Ji, Z. Jerzak, and C. Fetzer, Online parameter optimization for elastic data stream processing, " in Proceedings of the Sixth ACM Symposium on Cloud Computing. ACM, 2015, pp. 276-287.
18. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly, Dryad: Distributed data-parallel programs from sequential building blocks, " in ACM SIGOPS Operating Systems Review, vol. 41, no. 3. ACM, 2007, pp. 59-72.
19. N. Jain, L. Amini, H. Andrade, R. King, Y. Park, P. Selo, and C. Venkatramani, Design, implementation, and evaluation of the linear road benchmark on the stream processing core, " in Proceedings of the 2006 ACM SIGMOD International Conference on Management of Data. ACM, 2006, pp. 431-442.
20. S. Kulkarni, N. Bhagat, M. Fu, V. Kedigehalli, C. Kellogg, S. Mittal, J. M. Patel, K. Ramasamy, and S. Taneja, Twitter Heron: Stream processing at scale, " in Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data. ACM, 2015, pp. 239-250.
21. S. Loesing, M. Hentschel, T. Kraska, and D. Kossmann, Stormy: An elastic, and highly available streaming service in the cloud, " in Proceedings of the 2012 Joint EDBT/ICDT Workshops. ACM, 2012, pp. 55-60.
22. B. Lohrmann, P. Janacik, and O. Kao, Elastic stream processing with latency guarantees, " in Distributed Computing Systems (ICDCS), 2015 IEEE 35th International Conference on, June 2015, pp. 399-410.
23. S. Schneider, H. Andrade, B. Gedik, A. Biem, and K.-L. Wu, Elastic scaling of data parallel operators in stream processing, " in IEEE International Symposium on Parallel & Distributed Processing, 2009. IPDPS 2009. IEEE, 2009, pp. 1-12.
24. N. Tatbul, Y. Ahmad, U. Çetintemel, J.-H. Hwang, Y. Xing, and S. Zdonik, Load management, and high availability in the Borealis distributed stream processing engine, " in GeoSensor Networks. Springer, 2008, pp. 66-85.
25. M. Wall, Big Data: Are you ready for blast-off?" http://www.bbc.com/.news/business-26383058, 2016, ONLINE
26. K.-L. Wu, K. W. Hildrum, W. Fan, P. S. Yu, C. C. Aggarwal, D. A. George, B. Gedik, E. Bouillet, X. Gu, G. Luo, et al., Challenges, and experience in prototyping a multi-modal stream analytic, and monitoring application on System S, " in Proceedings of the 33rd International Conference on Very Large Databases. VLDB Endowment, 2007, pp. 1185-1196.
27. M. Zaharia, T. Das, H. Li, S. Shenker, and I. Stoica, Discretized streams: An efficient, and fault-Tolerant model for stream processing on large clusters, " in Proceedings of the 4th USENIX conference on Hot Topics in Cloud Computing. USENIX Association, 2012, pp. 10-10.