Massively-parallel stream processing under QoS constraints with nephele

HPDC '12 - Proceedings of the 21st ACM Symposium on High-Performance Parallel and Distributed Computing

Volumn , Issue , 2012, Pages 271-282

  Lohrmann, Björn ^a   Warneke, Daniel ^a   Kao, Odej ^a  

^a TECHNISCHE UNIVERSITÄT BERLIN   (Germany)

Author keywords

Large Scale Data Processing; MapReduce; Quality of Service; Stream Processing

Indexed keywords

CONTINUOUS DATA; DATA PROCESSING SYSTEMS; DATA THROUGHPUT; DATA VOLUME; DESIGN PRINCIPLES; JOB EXECUTION; LATENCY CONSTRAINTS; MANUAL INTERACTION; MAP-REDUCE; MULTIMEDIA STREAMING APPLICATIONS; PARALLEL DATA PROCESSING; PROOF OF CONCEPT; QOS CONSTRAINTS; QOS REQUIREMENTS; SMART METERS; STREAM PROCESSING; STREAM PROCESSING SYSTEMS; STREAM PROCESSOR;

DATA TRANSFER;

QUALITY OF SERVICE;

EID: 84863894558     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2287076.2287117     Document Type: Conference Paper

References (18)

1. D. Abadi, Y. Ahmad, M. Balazinska, U. Cetintemel, M. Cherniack, J. Hwang, W. Lindner, A. Maskey, A. Rasin, E. Ryvkina, et al. The design of the Borealis stream processing engine. In Second Biennial Conference on Innovative Data Systems Research, CIDR '05, pages 277-289, 2005.
2. D. Abadi, D. Carney, U. Çetintemel, M. Cherniack, C. Convey, S. Lee, M. Stonebraker, N. Tatbul, and S. Zdonik. Aurora: A new model and architecture for data stream management. The VLDB Journal, 12(2):120-139, 2003.
3. M. Aldinucci and M. Danelutto. Stream parallel skeleton optimization. In Proc. of the 11th IASTED International Conference on Parallel and Distributed Computing and Systems, PDCS '99, pages 955-962. IASTED/ACTA, 1999.
4. A. Alexandrov, S. Ewen, M. Heimel, F. Hueske, O. Kao, V. Markl, E. Nijkamp, and D. Warneke. MapReduce and PACT - comparing data parallel programming models. In Proc. of the 14th Conference on Database Systems for Business, Technology, and Web, BTW '11, pages 25-44. GI, 2011.
5. S. Babu and J. Widom. Continuous queries over data streams. SIGMOD Rec., 30:109-120, Sept. 2001.
6. D. Battré, S. Ewen, F. Hueske, O. Kao, V. Markl, and D. Warneke. Nephele/PACTs: A programming model and execution framework for web-scale analytical processing. In Proc. of the 1st ACM symposium on Cloud computing, SoCC '10, pages 119-130. ACM, 2010.
7. D. Battré, M. Hovestadt, B. Lohrmann, A. Stanik, and D. Warneke. Detecting bottlenecks in parallel DAG-based data flow programs. In Proc. of the 2010 IEEE Workshop on Many-Task Computing on Grids and Supercomputers, MTAGS '10, pages 1-10. IEEE, 2010.
8. V. Borkar, M. Carey, R. Grover, N. Onose, and R. Vernica. Hyracks: A flexible and extensible foundation for data-intensive computing. In Proc. of the 2011 IEEE 27th International Conference on Data Engineering, ICDE '11, pages 1151-1162. IEEE, 2011.
9. Q. Chen, M. Hsu, and H. Zeller. Experience in Continuous analytics as a Service (CaaaS). In Proc. of the 14th International Conference on Extending Database Technology, EDBT/ICDT '11, pages 509-514. ACM, 2011.
10. M. Cherniack, H. Balakrishnan, M. Balazinska, D. Carney, U. Cetintemel, Y. Xing, and S. Zdonik. Scalable distributed stream processing. In Proc. of the First Biennial Conference on Innovative Data Systems Research, CIDR '03, pages 257-268, 2003.
11. T. Condie, N. Conway, P. Alvaro, J. M. Hellerstein, K. Elmeleegy, and R. Sears. MapReduce Online. In Proc. of the 7th USENIX conference on Networked systems design and implementation, NSDI '10, pages 21-21. USENIX Association, 2010.
12. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. Communications of the ACM, 51(1):107-113, Jan. 2008.
13. E. N. M. Elnozahy, L. Alvisi, Y.-M. Wang, and D. B. Johnson. A survey of rollback-recovery protocols in message-passing systems. ACM Comput. Surv., 34(3):375-408, Sept. 2002.
14. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: Distributed data-parallel programs from sequential building blocks. ACM SIGOPS Operating Systems Review, 41(3):59-72, Mar. 2007.
15. R. Motwani, J. Widom, A. Arasu, B. Babcock, S. Babu, M. Datar, G. Manku, C. Olston, J. Rosenstein, and R. Varma. Query processing, approximation, and resource management in a data stream management system. In First Biennial Conference on Innovative Data Systems Research, CIDR '03, pages 245-256, 2003.
16. D. Murray, M. Schwarzkopf, C. Smowton, S. Smith, A. Madhavapeddy, and S. Hand. CIEL: A universal execution engine for distributed data-flow computing. In Proc. of the 8th USENIX conference on Networked systems design and implementation, NSDI '11, pages 9-9. USENIX Association, 2011.
17. L. Neumeyer, B. Robbins, A. Nair, and A. Kesari. S4: Distributed stream computing platform. In 2010 IEEE International Conference on Data Mining Workshops, ICDMW '10, pages 170-177. IEEE, 2010.
18. D. Warneke and O. Kao. Exploiting dynamic resource allocation for efficient parallel data processing in the cloud. IEEE Transactions on Parallel and Distributed Systems, 22(6):985-997, June 2011.