Meeting predictable buffer limits in the parallel execution of event processing operators

Proceedings - 2014 IEEE International Conference on Big Data, IEEE Big Data 2014

Volumn , Issue , 2014, Pages 402-411

  Mayer, Ruben ^a   Koldehofe, Boris ^a   Rothermel, Kurt ^a  

^a UNIVERSITY OF STUTTGART   (Germany)

Author keywords

Complex Event Processing; Data Parallelization; Quality of Service; Self Adaptation; Stream Processing

Indexed keywords

DATA HANDLING;

COMPLEX EVENT PROCESSING; COMPLEX EVENTS; DATA PARALLELIZATION; EVENT PATTERN; EVENT PROCESSING; EVENTS DETECTION; PARALLELIZATIONS; QUALITY-OF-SERVICE; SELF- ADAPTATIONS; STREAM PROCESSING;

QUALITY OF SERVICE;

EID: 84921822331     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/BigData.2014.7004257     Document Type: Conference Paper

References (37)

1. C. Balkesen, N. Dindar, M. Wetter, and N. Tatbul, "Rip: run-based intra-query parallelism for scalable complex event processing," in Proceedings of the 7th ACM international conference on Distributed event-based systems, ser. DEBS '13, 2013, pp. 3-14.
2. B. Ottenwälder, B. Koldehofe, K. Rothermel, K. Hong, D. Lillethun, and U. Ramachandran, "Mcep: A mobility-aware complex event processing system," ACM Trans. Internet Technol., vol. 14, no. 1, pp. 6: 1-6: 24 Aug. 2014.
3. B. Ottenwälder, B. Koldehofe, K. Rothermel, and U. Ramachandran "Migcep: Operator migration for mobility driven distributed complex event processing," in Proceedings of the 7th ACM International Conference on Distributed Event-based Systems, ser. DEBS '13, 2013, pp. 183-194.
4. B. Ottenwälder, B. Koldehofe, K. Rothermel, K. Hong, and U. Ramachandran "Recep: Selection-based reuse for distributed complex event processing," in Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems, ser. DEBS '14, 2014, pp. 59-70.
5. B. Koldehofe, B. Ottenwälder, K. Rothermel, and U. Ramachandran "Moving range queries in distributed complex event processing," in Proceedings of the 6th ACM International Conference on Distributed Event-Based Systems, ser. DEBS '12, 2012, pp. 201-212.
6. S. Srinivasagopalan, S. Mukhopadhyay, and R. Bharadwaj, "A complexevent-processing framework for smart-grid management," in 2012 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), March 2012 pp. 272-278.
7. G. G. Koch, B. Koldehofe, and K. Rothermel, "Cordies: Expressive event correlation in distributed systems," in Proceedings of the Fourth ACM International Conference on Distributed Event-Based Systems, ser. DEBS '10, 2010, pp. 26-37.
8. G. Cugola and A. Margara, "Processing flows of information: From data stream to complex event processing," ACM Comput. Surv., vol. 44 no. 3, pp. 15: 1-15: 62, Jun. 2012.
9. C. Clark. (2014) Improving speed and transparency of market data. Http: //exchanges.nyx.com/cclark/improving-speed-and-transparencymarket-data.
10. A. Martin, A. Brito, and C. Fetzer, "Scalable and elastic realtime click stream analysis using streammine3g," in Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems, ser. DEBS '14, 2014, pp. 198-205.
11. T. Condie, N. Conway, P. Alvaro, J. M. Hellerstein, K. Elmeleegy and R. Sears, "Mapreduce online," in Proceedings of the 7th USENIX conference on Networked systems design and implementation, ser. NSDI'10, 2010, pp. 21-21.
12. A. Brito, A. Martin, T. Knauth, S. Creutz, D. Becker, S. Weigert, and C. Fetzer, "Scalable and low-latency data processing with stream mapreduce," in Proceedings of the 2011 IEEE Third International Conference on Cloud Computing Technology and Science, ser. CloudCom '11, 2011 pp. 48-58.
13. D. Battré, S. Ewen, F. Hueske, O. Kao, V. Markl, and D. Warneke "Nephele/pacts: a programming model and execution framework for web-scale analytical processing," in Proceedings of the 1st ACM symposium on Cloud computing, ser. SoCC '10, 2010, pp. 119-130.
14. (2014, September) Storm. Http: //storm-project.net/.
15. L. Neumeyer, B. Robbins, A. Nair, and A. Kesari, "S4: Distributed stream computing platform," in Proceedings of the 2010 IEEE International Conference on Data Mining Workshops, ser. ICDMW '10, Dec. pp. 170-177.
16. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly, "Dryad: distributed data-parallel programs from sequential building blocks," in Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007, ser. EuroSys '07, 2007, pp. 59-72.
17. S. Schneider, M. Hirzel, B. Gedik, and K.-L. Wu, "Auto-parallelizing stateful distributed streaming applications," in Proceedings of the 21st international conference on Parallel architectures and compilation techniques, ser. PACT '12. New York, NY, USA: ACM, 2012, pp. 53-64.
18. M. Hirzel, R. Soulé, S. Schneider, B. Gedik, and R. Grimm, "A catalog of stream processing optimizations," ACM Comput. Surv., vol. 46, no. 4 pp. 46: 1-46: 34, Mar. 2014.
19. S. Chakravarthy and D. Mishra, "Snoop: An expressive event specification language for active databases," Data Knowl. Eng., vol. 14, no. 1 pp. 1-26, 1994.
20. R. Castro Fernandez, M. Migliavacca, E. Kalyvianaki, and P. Pietzuch "Integrating scale out and fault tolerance in stream processing using operator state management," in Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data, ser. SIGMOD '13 2013, pp. 725-736.
21. B. Gedik, S. Schneider, M. Hirzel, and K.-L. Wu, "Elastic scaling for data stream processing," IEEE Transactions on Parallel and Distributed Systems, vol. 25, no. 6, pp. 1447-1463, June 2014.
22. B. Koldehofe, R. Mayer, U. Ramachandran, K. Rothermel, and M. Völz "Rollback-recovery without checkpoints in distributed event processing systems," in Proceedings of the 7th ACM international conference on Distributed event-based systems, ser. DEBS '13, 2013, pp. 27-38.
23. A. Arasu, S. Babu, and J. Widom, "The cql continuous query language: Semantic foundations and query execution," The VLDB Journal, vol. 15 no. 2, pp. 121-142, Jun. 2006.
24. R. Adaikkalavan and S. Chakravarthy, "Seamless event and data stream processing: Reconciling windows and consumption modes," in Proceedings of the 16th International Conference on Database Systems for Advanced Applications-Volume Part I, ser. DASFAA'11. Berlin Heidelberg: Springer-Verlag, 2011, pp. 341-356.
25. D. Gross, J. Shortle, J. Thompson, and C. Harris, Fundamentals of Queueing Theory, ser. Wiley Series in Probability and Statistics. Wiley 2011.
26. N. R. Herbst, N. Huber, S. Kounev, and E. Amrehn, "Self-adaptive workload classification and forecasting for proactive resource provisioning," Concurrency and Computation: Practice and Experience, vol. 26 no. 12, pp. 2053-2078, 2014.
27. D. G. Feitelson, "Workload modeling for computer systems performance evaluation," book draft, to be published by Cambridge University Press in 2015. Version 1.0.1, typeset on April 8, 2014.
28. M. A. Stephens, "Edf statistics for goodness of fit and some comparisons," Journal of the American statistical Association, vol. 69, no. 347 pp. 730-737, 1974.
29. T. M. Franke, T. Ho, and C. A. Christie, "The chi-square test: Often used and more often misinterpreted," American Journal of Evaluation vol. 33, no. 3, pp. 448-458, 2012.
30. S. K. Bose. (2014) M/m/m/inf queue. Http: //nptel.ac.in/courses/117103017/6.
31. H. Tijms, "New and old results for the m/d/c queue," fAEUg-International Journal of Electronics and Communications, vol. 60 no. 2, pp. 125-130, 2006.
32. V. Guerriero, "Power law distribution: Method of multi-scale inferential statistics," Journal of Modern Mathematics Frontier (JMMF), vol. 1, pp. 21-28, 2012.
33. R. Mao and G. Mao, "Road traffic density estimation in vehicular networks," in Proceedings of the 2013 IEEE Wireless Communications and Networking Conference, ser. WCNC '13, April 2013, pp. 4653-4658.
34. C. Balkesen, N. Tatbul, and M. T. Ö zsu, "Adaptive input admission and management for parallel stream processing," in Proceedings of the 7th ACM International Conference on Distributed Event-based Systems, ser. DEBS '13, 2013, pp. 15-26.
35. S. Kounev, "Performance modeling and evaluation of distributed component-based systems using queueing petri nets," IEEE Transactions on Software Engineering, vol. 32, no. 7, pp. 486-502, 2006.
36. B. Chandramouli, J. Goldstein, R. Barga, M. Riedewald, and I. Santos "Accurate latency estimation in a distributed event processing system," in Proceedings of the 2011 IEEE 27th International Conference on Data Engineering, ser. ICDE '11, 2011, pp. 255-266.
37. T. Heinze, Z. Jerzak, G. Hackenbroich, and C. Fetzer, "Latencyaware elastic scaling for distributed data stream processing systems," in Proceedings of the 8th ACM International Conference on Distributed Event-Based Systems, ser. DEBS '14, 2014, pp. 13-22