Optimal operator placement for distributed stream processing applications

DEBS 2016 - Proceedings of the 10th ACM International Conference on Distributed and Event-Based Systems

Volumn , Issue , 2016, Pages 69-80

  Cardellini, Valeria ^a   Grassi, Vincenzo ^a   Lo Presti, Francesco ^a   Nardelli, Matteo ^a  

^a UNIVERSITY OF ROME TOR VERGATA   (Italy)

Author keywords

Distributed data stream processing; Optimal placement; QoS

Indexed keywords

DATA COMMUNICATION SYSTEMS; DATA MINING; DISTRIBUTED PARAMETER CONTROL SYSTEMS; INTEGER PROGRAMMING; QUALITY OF SERVICE; SOFTWARE ARCHITECTURE;

COMPUTATIONAL RESOURCES; DATA STREAM PROCESSING; DISTRIBUTED DATA SOURCES; DISTRIBUTED DATA STREAM PROCESSING; DISTRIBUTED STREAM PROCESSING; EXTRACT INFORMATIONS; INTEGER LINEAR PROGRAMMING; OPTIMAL PLACEMENTS;

DATA HANDLING;

EID: 84978708916     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2933267.2933312     Document Type: Conference Paper

References (29)

1. L. Aniello, R. Baldoni, and L. Querzoni. Adaptive online scheduling in Storm. In Proc. of ACM DEBS '13, pages 207-218, 2013.
2. N. Backman, R. Fonseca, and U. Cetintemel. Managing parallelism for stream processing in the cloud. In Proc. of HotCDP '12. ACM, 2012.
3. A. Benoit, H. Casanova, V. Rehn-Sonigo, and Y. Robert. Resource allocation for multiple concurrent in-network stream-processing applications. Parallel Computing, 37(8):331-348, 2011.
4. F. Bonomi, R. Milito, J. Zhu, and S. Addepalli. Fog computing and its role in the Internet of Things. In Proc. of MCC '12, pages 13-16. ACM, 2012.
5. A. Brook. Low-latency distributed applications in finance. ACM Commun., 58(7):42-50, 2015.
6. V. Cardellini, V. Grassi, F. Lo Presti, and M. Nardelli. Distributed QoS-aware scheduling in Storm. In Proc. of ACM DEBS '15, pages 344-347, 2015.
7. A. Chatzistergiou and S. D. Viglas. Fast heuristics for near-optimal task allocation in data stream processing over clusters. In Proc. of ACM CIKM '14, 2014.
8. L. Cui, F. R. Yu, and Q. Yan. When big data meets software-defined networking: SDN for big data and big data for SDN. IEEE Network, 30(1):58-65, 2016.
9. F. Dabek, R. Cox, F. Kaashoek, and R. Morris. Vivaldi: A decentralized network coordinate system. SIGCOMM Comput. Commun. Rev., 34(4), 2004.
10. R. Eidenbenz and T. Locher. Task allocation for distributed stream processing. In Proc. of IEEE INFOCOM '16, 2016.
11. G. Falcone, G. Nicosia, and A. Pacifici. Minimizing part transfer costs in flexible manufacturing systems: A computational study on different lower bounds. In Proc. of UKSim '13, pages 359-364, 2013.
12. L. Fischer, T. Scharrenbach, and A. Bernstein. Scalable linked data stream processing via network-aware workload scheduling. In Proc. of SSWS '13, 2013.
13. M. R. Garey and D. S. Johnson. Computers and Intractability. W. H. Freeman, 1979.
14. T. Heinze, L. Aniello, L. Querzoni, and Z. Jerzak. Cloud-based data stream processing. In Proc. of ACM DEBS '14, pages 238-245, 2014.
15. J. Huang, G. Liu, Q. Duan, and Y. Yan. QoS-aware service composition for converged network-cloud service provisioning. In Proc. of IEEE SCC '14, 2014.
16. Y. Huang, Z. Luan, R. He, and D. Qian. Operator placement with QoS constraints for distributed stream processing. In Proc. of 7th Int'l Conf. on Network and Service Management, CNSM '11, 2011.
17. G. T. Lakshmanan, Y. Li, and R. Strom. Placement strategies for Internet-scale data stream systems. IEEE Internet Computing, 12(6):50-60, 2008.
18. T. Li, J. Tang, and J. Xu. A predictive scheduling framework for fast and distributed stream data processing. In Proc. of IEEE Big Data '15, 2015.
19. P. Pietzuch, J. Ledlie, J. Shneidman, M. Roussopoulos, et al. Network-aware operator placement for stream-processing systems. In Proc. of IEEE ICDE '06, 2006.
20. R. Prasad, C. Dovrolis, M. Murray, and K. Claffy. Bandwidth estimation: metrics, measurement techniques, and tools. IEEE Network, 17(6), 2003.
21. S. Rizou, F. Durr, and K. Rothermel. Solving the multi-operator placement problem in large-scale operator networks. In Proc. of ICCCN '10, 2010.
22. I. Stanoi, G. Mihaila, T. Palpanas, and C. Lang. WhiteWater: distributed processing of fast streams. IEEE Trans. Softw. Eng., 19(9):1214-1226, 2007.
23. C. Thoma, A. Labrinidis, and A. Lee. Automated operator placement in distributed data stream management systems subject to user constraints. In Proc. of IEEE ICDEW '14, pages 310-316, 2014.
24. A. Toshniwal, S. Taneja, A. Shukla, K. Ramasamy, et al. Storm@Twitter. In Proc. of ACM SIGMOD '14, pages 147-156, 2014.
25. J. Xu, Z. Chen, J. Tang, and S. Su. T-Storm: Traffic-aware online scheduling in Storm. In Proc. of IEEE ICDCS '14, pages 535-544, 2014.
26. K. P. Yoon and C.-L. Hwang. Multiple Attribute Decision Making: An Introduction. Sage Pubns, 1995.
27. M. Zaharia, M. Chowdhury, T. Das, A. Dave, et al. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. In Proc. of USENIX NSDI '12, 2012.
28. Y. Zhou, B. C. Ooi, K.-L. Tan, and J. Wu. Efficient dynamic operator placement in a locally distributed continuous query system. In Proc. of OTM '06, volume 4275 of LNCS, pages 54-71. Springer, 2006.
29. Q. Zhu and G. Agrawal. Resource allocation for distributed streaming applications. In Proc. of IEEE ICPP '08, pages 414-421, 2008.