Latency-aware elastic scaling for distributed data stream processing systems

DEBS 2014 - Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems

Volumn , Issue , 2014, Pages 13-22

  Heinze, Thomas ^a   Jerzak, Zbigniew ^a   Hackenbroich, Gregor ^a   Fetzer, Christof ^b  

^a SAP RESEARCH   (Germany)

^b DRESDEN UNIVERSITY OF TECHNOLOGY   (Germany)

Author keywords

distributed data stream processing; elasticity; load balancing

Indexed keywords

ELASTICITY; RESOURCE ALLOCATION; SOFTWARE ARCHITECTURE;

DATA STREAM PROCESSING; DISTRIBUTED DATA STREAM PROCESSING; ELASTIC SYSTEMS; END TO END LATENCIES; LATENCY CONSTRAINTS; OPERATOR PLACEMENTS; SERVICE LEVEL AGREEMENTS; SYSTEM UTILIZATION;

DATA COMMUNICATION SYSTEMS;

EID: 84903161696     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2611286.2611294     Document Type: Conference Paper

References (23)

1. D. J. Abadi, Y. Ahmad, M. Balazinska, U. Cetintemel, M. Cherniack, J.-H. Hwang, W. Lindner, A. S. Maskey, A. Rasin, E. Ryvkina, N. Tatbul, Y. Xing, and S. Zdonik. The Design of the Borealis Stream Processing Engine. In CIDR, pages 277-289, 2005.
2. Y. Ahmad and U. Çetintemel. Network-aware query processing for stream-based applications. In VLDB, pages 456-467, 2004.
3. M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, and M. Zaharia. A view of cloud computing. Communications of the ACM, pages 50-58, 2010.
4. M. Arrington. AOL proudly releases massive amounts of private data. TechCrunch: http://www.techcrunch.com/2006/08/06/aol-proudly-releasesmassive- amounts-of-user-search-data, 2006.
5. B. Chandramouli, J. Goldstein, R. Barga, M. Riedewald, and I. Santos. Accurate latency estimation in a distributed event processing system. In ICDE, pages 255-266, 2011.
6. E. Coffman Jr, M. Garey, and D. Johnson. Approximation algorithms for bin packing: A survey. In Approximation algorithms for NP-hard problems, pages 46-93, 1996.
7. S. Das, D. Agrawal, and A. El Abbadi. Elastras: An elastic transactional data store in the cloud. In USENIX HotCloud, 2009.
8. A. J. Elmore, S. Das, D. Agrawal, and A. El Abbadi. Zephyr: live migration in shared nothing databases for elastic cloud platforms. In SIGMOD, pages 301-312, 2011.
9. R. C. Fernandez, M. Migliavacca, E. Kalyvianaki, and P. Pietzuch. Integrating scale out and fault tolerance in stream processing using operator state management. In SIGMOD, pages 725-736, 2013.
10. B. Gedik, S. Schneider, M. Hirzel, and K. Wu. Elastic scaling for data stream processing. IEEE TPDS, 2013.
11. V. Gulisano, R. Jimenez-Peris, M. Patino-Martinez, C. Soriente, and P. Valduriez. Streamcloud: An elastic and scalable data streaming system. IEEE TPDS, pages 2351-2365, 2012.
12. T. Heinze, V. Pappalardo, Z. Jerzak, and C. Fetzer. Auto-scaling techniques for elastic data stream processing. In ICDEW, 2014.
13. T. Lorido-Botrán, J. Miguel-Alonso, and J. A. Lozano. Auto-scaling techniques for elastic applications in cloud environments. Department of Computer Architecture and Technology, University of Basque Country, Tech. Rep. EHU-KAT-IK-09-12, 2012.
14. S. Martello and P. Toth. Algorithms for knapsack problems. Surveys in combinatorial optimization, pages 213-258, 1987.
15. P. Mell and T. Grance. The NIST definition of cloud computing. NIST Special Publication 800-145, 2011.
16. P. Pietzuch, J. Ledlie, J. Shneidman, M. Roussopoulos, M. Welsh, and M. Seltzer. Network-aware operator placement for stream-processing systems. In ICDE, pages 49-49, 2006.
17. E. A. Rundensteiner, L. Ding, T. Sutherland, Y. Zhu, B. Pielech, and N. Mehta. CAPE: Continuous query engine with heterogeneous-grained adaptivity. In VLDB, pages 1353-1356, 2004.
18. T. K. Sellis. Multiple-query optimization. ACM TODS, pages 23-52, 1988.
19. M. A. Shah, J. M. Hellerstein, S. Chandrasekaran, and M. J. Franklin. Flux: An adaptive partitioning operator for continuous query systems. In ICDE, pages 25-36, 2003.
20. M. Stonebraker, U. Çetintemel, and S. Zdonik. The 8 requirements of real-time stream processing. ACM SIGMOD Record, pages 42-47, 2005. (Pubitemid 43152547)
21. E. Wu, Y. Diao, and S. Rizvi. High-performance complex event processing over streams. In SIGMOD, pages 407-418, 2006.
22. Y. Yang, J. Kramer, D. Papadias, and B. Seeger. Hybmig: A hybrid approach to dynamic plan migration for continuous queries. IEEE TKDE, pages 398-411, 2007.
23. Y. Zhu, E. A. Rundensteiner, and G. T. Heineman. Dynamic plan migration for continuous queries over data streams. In SIGMOD, pages 431-442, 2004.