Toward reliable and rapid elasticity for streaming dataflows on clouds

Proceedings - International Conference on Distributed Computing Systems

Volumn 2018-July, Issue , 2018, Pages 1096-1106

  Shukla, Anshu ^a   Simmhan, Yogesh ^b  

^a ERICSSON RESEARCH   (Sweden)

^b INDIAN INSTITUTE OF SCIENCE   (India)

Author keywords

[No Author keywords available]

Indexed keywords

STORMS;

APACHE STORMS; DATA FLOW; DATA PLATFORM; INPUT RATE; LOW-LATENCY STREAMING; STREAMING DATA; TASK MIGRATION;

DISTRIBUTED COMPUTER SYSTEMS;

EID: 85050982703     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDCS.2018.00109     Document Type: Conference Paper

References (28)

1. Y. Simmhan, S. Aman, A. Kumbhare, R. Liu, S. Stevens, Q. Zhou, and V. Prasanna, "Cloud-based software platform for big data analytics in smart grids, " Computing in Science & Engineering (CiSE), vol. 15, no. 4, 2013.
2. D. Laney, "3d data management: Controlling data volume, velocity and variety, " META Group Research Note, vol. 6, p. 70, 2001.
3. B. Peng, M. Hosseini, Z. Hong, R. Farivar, and R. Campbell, "Rstorm: Resource-aware scheduling in storm, " in ACM International Middleware Conference (Middleware), 2015.
4. A. Shukla and Y. Simmhan, "Model-driven scheduling for distributed stream processing systems, " Journal of Parallel and Distributed Computing, vol. 117, no. 98-114, 2017.
5. A. Ishii and T. Suzumura, "Elastic stream computing with clouds, " in IEEE International Conference on Cloud Computing (CLOUD), 2011.
6. L. Xu, B. Peng, and I. Gupta, "Stela: Enabling stream processing systems to scale-in and scale-out on-demand, " in IEEE International Conference on Cloud Engineering (IC2E), 2016.
7. B. Gedik, S. Schneider, M. Hirzel, and K.-L. Wu, "Elastic scaling for data stream processing, " IEEE Transactions on Parallel and Distributed Systems (TPDS), vol. 25, no. 6, pp. 1447-1463, 2014.
8. "Guaranteeing message processing, " http://storm. apache. org/releases/1. 1. 2/Guaranteeing-messageprocessing. html, Apache Storm, Tech. Rep., 2017.
9. "Storm state management, " http://storm. apache. org/releases/1. 1. 2/Statecheckpointing. html, Apache Storm, Tech. Rep., 2017.
10. S. Chintapalli, D. Dagit, B. Evans, R. Farivar, T. Graves, M. Holderbaugh, Z. Liu, K. Nusbaum, K. Patil, B. J. Peng et al., "Benchmarking streaming computation engines: Storm, flink and spark streaming, " in IEEE IPDPS Workshops, 2016.
11. L. Fischer and A. Bernstein, "Workload scheduling in distributed stream processors using graph partitioning, " in IEEE International Conference on Big Data (Big Data), 2015.
12. T. Akidau, A. Balikov, K. Bekiro?glu, S. Chernyak, J. Haberman, R. Lax, S. McVeety, D. Mills, P. Nordstrom, and S. Whittle, "Millwheel: Fault-tolerant stream processing at internet scale, " The Proceedings of the VLDB Endowment (PVLDB), vol. 6, no. 11, 2013.
13. A. Biem, E. Bouillet, H. Feng, A. Ranganathan, A. Riabov, O. Verscheure, H. Koutsopoulos, and C. Moran, "Ibm infosphere streams for scalable, real-time, intelligent transportation services, " in ACM International Conference on Management of Data (SIGMOD), 2010.
14. I. Lujic, V. De Maio, and I. Brandic, "Efficient edge storage management based on near real-time forecasts, " in IEEE International conference on Fog and Edge Computing (ICFEC), 2017.
15. X. Liu, A. Harwood, S. Karunasekera, B. Rubinstein, and R. Buyya, "E-storm: Replication-based state management in distributed stream processing systems, " in IEEE International conference on Parallel Processing (ICPP), 2017.
16. Y. Wu and K.-L. Tan, "Chronostream: Elastic stateful stream computation in the cloud, " in IEEE International Conference on Data Engineering (ICDE), 2015.
17. B. Satzger, W. Hummer, P. Leitner, and S. Dustdar, "Esc: Towards an elastic stream computing platform for the cloud, " in IEEE International Conference on Cloud Computing (CLOUD), 2011.
18. B. Lohrmann, P. Janacik, and O. Kao, "Elastic stream processing with latency guarantees, " in IEEE International Conference on Distributed Computing Systems (ICDCS), 2015.
19. K. G. S. Madsen, Y. Zhou, and J. Cao, "Integrative dynamic reconfiguration in a parallel stream processing engine, " in IEEE International Conference on Data Engineering (ICDE), 2017.
20. R. Khandekar, K. Hildrum, S. Parekh, D. Rajan, J. Wolf, K.-L. Wu, H. Andrade, and B. Gedik, "Cola: Optimizing stream processing applications via graph partitioning, " in ACM International Middleware Conference (Middleware), 2009.
21. A. Kumbhare, Y. Simmhan, and V. K. Prasanna, "Exploiting application dynamism and cloud elasticity for continuous dataflows, " in IEEE/ACM International Conference for High Performance Computing, Networking, Storage, and Analysis (Supercomputing), 2013.
22. X. Zhang, Z.-Y. Shae, S. Zheng, and H. Jamjoom, "Virtual machine migration in an over-committed cloud, " in IEEE Network Operations and Management Symposium (NOMS), 2012.
23. W. Voorsluys, J. Broberg, S. Venugopal, and R. Buyya, "Cost of virtual machine live migration in clouds: A performance evaluation, " in IEEE International Conference on Cloud Computing Technology and Science (CloudCom), 2009.
24. M. Zhao and R. J. Figueiredo, "Experimental study of virtual machine migration in support of reservation of cluster resources, " in IEEE Virtualization Technology in Distributed Computing, 2007.
25. Z. Shen, S. Subbiah, X. Gu, and J. Wilkes, "Cloudscale: Elastic resource scaling for multi-tenant cloud systems, " in ACM Symposium on Cloud Computing (SoCC), 2011.
26. E. Casalicchio, D. A. Menascé, and A. Aldhalaan, "Autonomic resource provisioning in cloud systems with availability goals, " in ACM International Conference on Autonomic Computing, 2013.
27. M. S. Bouguerra, A. Gainaru, L. B. Gomez, F. Cappello, S. Matsuoka, and N. Maruyam, "Improving the computing efficiency of hpc systems using a combination of proactive and preventive checkpointing, " in IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2013.
28. N. Naksinehaboon, Y. Liu, C. Leangsuksun, R. Nassar, M. Paun, and S. L. Scott, "Reliability-aware approach: An incremental checkpoint/ restart model in hpc environments, " in IEEE International Symposium on Cluster, Cloud and Grid Computing (CCGRID), 2008.