Task allocation for distributed stream processing

Proceedings - IEEE INFOCOM

Volumn 2016-July, Issue , 2016, Pages

  Eidenbenz, Raphael ^a   Locher, Thomas ^a  

^a ABB CORPORATE RESEARCH   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION ALGORITHMS; COMPUTATION THEORY; DATA HANDLING; DISTRIBUTED PARAMETER CONTROL SYSTEMS;

COMPUTATIONAL COSTS; COMPUTATIONAL TASK; CONSTANT FACTOR APPROXIMATION; DISTRIBUTED STREAM PROCESSING; LARGE AMOUNTS OF DATA; PHYSICAL RESOURCES; STREAM PROCESSING; THEORETICAL FOUNDATIONS;

COMPUTER ARCHITECTURE;

EID: 84983354600     PISSN: 0743166X     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/INFOCOM.2016.7524433     Document Type: Conference Paper

References (26)

1. T. Akidau, A. Balikov, K. Bekirǒglu, S. Chernyak, J. Haberman, R. Lax, S. McVeety, D. Mills et al., "MillWheel: Fault-Tolerant Stream Processing at Internet Scale," Proc. VLDB Endowment, vol. 6, no. 11, pp. 1033-1044, 2013
2. A. Arasu, B. Babcock, S. Babu, M. Datar, K. Ito, I. Nishizawa, J. Rosenstein, and J. Widom, "STREAM: the Stanford Stream Data Manager (Demonstration Description)," in Proc. ACM International Conference on Management of Data, 2003, pp. 665-665
3. S. Chandrasekaran, O. Cooper, A. Deshpande, M. J. Franklin, J. M. Hellerstein, W. Hong, S. Krishnamurthy, S. R. Madden et al., "TelegraphCQ: Continuous Dataflow Processing," in Proc. 2003 ACM International Conference on Management of Data, 2003, pp. 668-668
4. B. Gedik, H. Andrade, K.-L. Wu, P. S. Yu, and M. Doo, "SPADE: The System S Declarative Stream Processing Engine," in Proc. ACM International Conference on Management of Data, 2008, pp. 1123-1134
5. L. Neumeyer, B. Robbins, A. Nair, and A. Kesari, "S4: Distributed Stream Computing Platform," in Proc. IEEE International Conference on Data Mining Workshops (ICDMW), 2010, pp. 170-177
6. J. Dean and S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," in Proc. 6th Conference on Symposium on Opearting Systems Design &Implementation (OSDI), 2004, pp. 137-150
7. K. Takamizawa, T. Nishizeki, and N. Saito, "Linear-time Computability of Combinatorial Problems on Series-parallel Graphs," Journal of the ACM (JACM), vol. 29, no. 3, pp. 623-641, 1982
8. M. Cherniack, H. Balakrishnan, M. Balazinska, D. Carney, U. Cetintemel, Y. Xing, and S. B. Zdonik, "Scalable Distributed Stream Processing," in Proc. 1st Biennial Conference on Innovative Data Systems Research (CIDR), vol. 3, 2003, pp. 257-268
9. B. Babcock, S. Babu, M. Datar, R. Motwani, and J. Widom, "Models and Issues in Data Stream Systems," in Proc. 21st ACM Symposium on Principles of Database Systems (PODS), 2002, pp. 1-16
10. D. J. Abadi, Y. Ahmad, M. Balazinska, U. Çetintemel, M. Cherniack, J.-H. Hwang, W. Lindner, A. Maskey et al., "The Design of the Borealis Stream Processing Engine," in Proc. 2nd Biennial Conference on Innovative Data Systems Research (CIDR), vol. 5, 2005, pp. 277-289
11. D. J. Abadi, D. Carney, U. Çetintemel, M. Cherniack, C. Convey, S. Lee, M. Stonebraker, N. Tatbul, and S. Zdonik, "Aurora: A New Model and Architecture for Data Stream Management," The International Journal on Very Large Data Bases, vol. 12, no. 2, pp. 120-139, 2003
12. D. Carney, U. Çetintemel, M. Cherniack, C. Convey, S. Lee, G. Seidman, M. Stonebraker, N. Tatbul, and S. Zdonik, "Monitoring Streams: A New Class of Data Management Applications," in Proc. 28th International Conference on Very Large Data Bases (VLDB), 2002, pp. 215-226
13. L. Amini, N. Jain, A. Sehgal, J. Silber, and O. Verscheure, "Adaptive of Control Extreme-scale Stream Processing Systems," in Proc. 26th IEEE International Conference on Distributed Computing Systems (ICDCS), 2006, pp. 71-71
14. Y. Xing, S. Zdonik, and J.-H. Hwang, "Dynamic Load Distribution in the Borealis Stream Processor," in Proc. 21st International Conference on Data Engineering (ICDE), 2005, pp. 791-802
15. N. Tatbul, U. Çetintemel, and S. Zdonik, "Staying Fit: Efficient Load Shedding Techniques for Distributed Stream Processing," in Proc. 33rd international Conference on Very Large Data Bases (VLDB), 2007, pp. 159-170
16. A. Chatzistergiou and S. D. Viglas, "Fast Heuristics for Near-Optimal Task Allocation in Data Stream Processing over Clusters," in Proc. 23rd ACM International Conference on Conference on Information and Knowledge Management, 2014, pp. 1579-1588
17. R. Khandekar, K. Hildrum, S. Parekh, D. Rajan, J. Wolf, K.-L. Wu, H. Andrade, and B. Gedik, "COLA: Optimizing Stream Processing Applications via Graph Partitioning," in Proc. 10th International Middleware Conference (Middleware), 2009, pp. 308-327
18. A. Riabov and Z. Liu, "Scalable Planning for Distributed Stream Processing Systems," in Proc. International Conference on Automated Planning and Scheduling (ICAPS), 2006, pp. 31-41
19. J. Wolf, N. Bansal, K. Hildrum, S. Parekh, D. Rajan, R. Wagle, K.-L. Wu, and L. Fleischer, "SODA: An Optimizing Scheduler for Large-Scale Stream-Based Distributed Computer Systems," in Proc. 9th International Middleware Conference (Middleware), 2008, pp. 306-325
20. Y. Xing, J.-H. Hwang, U. Çetintemel, and S. Zdonik, "Providing Resiliency to Load Variations in Distributed Stream Processing," in Proc. 32nd International Conference on Very Large Data Bases (VLDB), 2006, pp. 775-786
21. P. Pietzuch, J. Ledlie, J. Shneidman, M. Roussopoulos, M. Welsh, and M. Seltzer, "Network-aware Operator Placement for Stream-processing Systems," in Proc. 22nd International Conference on Data Engineering (ICDE), 2006, pp. 49-60
22. S. Mattheis, T. Schuele, A. Raabe, T. Henties, and U. Gleim, "Work Stealing Strategies for Parallel Stream Processing in Soft Real-Time Systems," in Proc. International Conference on Architecture of Computing Systems (ARCS), 2012, pp. 172-183
23. S. Hartmann and D. Briskorn, "A Survey of Variants and Extensions of the Resource-Constrained Project Scheduling Problem," European Journal of Operational Research, vol. 207, no. 1, pp. 1-14, 2010
24. J.-B. Wang, C. Ng, and T. E. Cheng, "Single-Machine Scheduling with Deteriorating Jobs under a Series-parallel Graph Constraint," Computers &Operations Research, vol. 35, no. 8, pp. 2684-2693, 2008
25. V. J. Rayward-Smith, "UET Scheduling with Unit Interprocessor Communication Delays," Discrete Applied Mathematics, vol. 18, pp. 55-71, 1987
26. L. Finta, Z. Liu, I. Mills, and E. Bampis, "Scheduling UET-UCT Series-Parallel Graphs on Two Processors," Theoretical Computer Science (TCS), vol. 162, no. 2, pp. 323-340, 1996