Hirundo: A mechanism for automated production of optimized data stream graphs

ICPE'12 - Proceedings of the 3rd Joint WOSP/SIPEW International Conference on Performance Engineering

Volumn , Issue , 2012, Pages 335-346

  Dayarathna, Miyuru ^a   Suzumura, Toyotaro ^a  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

Author keywords

Data intensive computing; Fault tolerance; Performance optimization; Scalability; Stream processing

Indexed keywords

AUTOMATED PRODUCTIONS; DATA STREAM; DATA-INTENSIVE COMPUTING; EMPIRICAL OPTIMIZATION; FINE TUNING; PERFORMANCE GAIN; PERFORMANCE OPTIMIZATIONS; PROGRAM PERFORMANCE; RUNTIME ENVIRONMENTS; RUNTIMES; SHORT PERIODS; STREAM PROCESSING;

FAULT TOLERANCE; OPTIMIZATION; SCALABILITY; SEMANTICS; SOFTWARE PROTOTYPING;

COMPUTER SOFTWARE;

EID: 84861081466     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2188286.2188347     Document Type: Conference Paper

References (32)

1. R. Ahmed and et al. Cost-based query transformation in oracle. VLDB'06, pages 1026-1036, 2006.
2. A. W. Appel. Modern compiler implementation in Java. Cambridge University Press, 2002.
3. A. Arasu, S. Babu, and J. Widom. The cql continuous query language: semantic foundations and query execution. The VLDB Journal, 15:121-142, June 2006. (Pubitemid 43938935)
4. S. Babu. Towards automatic optimization of mapreduce programs. SoCC'10, pages 137-142, 2010.
5. C. Ballard and et al. IBM Infosphere Streams: Harnessing Data in Motion. IBM, 2010.
6. P. Banerjee and et al. The paradigm compiler for distributed-memory multicomputers. Computer, 28:37-47, Oct 1995.
7. S. Bellamkonda and et al. Enhanced subquery optimizations in oracle. Proc. VLDB Endow., 2:1366-1377, August 2009.
8. B. Chapman, H. Herbeck, and H. Zima. Automatic support for data distribution. In DMCC, pages 51 -58, May 1991.
9. D. Cook. Gold parsing system. URL:http://www.goldparser.org/Dec. 2011.
10. M. Dayarathna, S. Takeno, and T. Suzumura. A performance study on operator-based stream processing systems. In IEEE IISWC, 2011.
11. B. Gedik, H. Andrade, and K.-L. Wu. A code generation approach to optimizing high-performance distributed data stream processing. In CIKM'09, pages 847-856, 2009.
12. B. Gedik and et al. Spade: the system s declarative stream processing engine. In SIGMOD'08, pages 1123-1134, 2008.
13. M. Hall and et al. Loop transformation recipes for code generation and auto-tuning. In Languages and Compilers for Parallel Computing, pages 50-64. 2010.
14. H. Herodotou and et al. Query optimization techniques for partitioned tables. SIGMOD'11, pages 49-60, 2011.
15. M. Hirzel and et al. Spl stream processing language specification. Nov 2009.
16. IBM. Ibm infosphere streams version 1.2: Programming model and language reference. Feb 2010.
17. N. Kabra, and D. J. DeWitt. Efficient mid-query re-optimization of sub-optimal query execution plans. SIGMOD'98, pages 106-117, 1998. (Pubitemid 128655961)
18. R. Khandekar and et al. Cola: Optimizing stream processing applications via graph partitioning. In Middleware 2009, pages 308-327. 2009.
19. C. S. Liew and et al. Towards optimising distributed data streaming graphs using parallel streams. In HPDC'10, pages 725-736, 2010.
20. S. Marsland. Machine Learning : An Algorithmic Perspective. Chapman Hall/CRC, 2009.
21. L. Neumeyer, B. Robbins, A. Nair, and A. Kesari. S4: Distributed stream computing platform. In KDCloud 2010, December 2010.
22. D. Palermo, E. Hodges, and P. Banerjee. Compiler optimization of dynamic data distributions for distributed-memory multicomputers. In Compiler Optimizations for Scalable Parallel Systems, volume 1808, pages 445-484. 2001. (Pubitemid 33274130)
23. J. Qin and et al. A novel graph based approach for automatic composition of high quality grid workflows. In HPDC'09, pages 167-176, 2009.
24. R. Rea and K. Mamidipaka. Ibm infosphere streams: Enabling complex analytics with ultra-low latencies on data in motion. May 2009.
25. Scipy. Scientific tools for python. URL:http://www.scipy.org/2011.
26. S. Sodhi, J. Subhlok, and Q. Xu. Performance prediction with skeletons. Cluster Computing, 11:151-165, 2008. (Pubitemid 351411541)
27. T. Suzumura, T. Yasue, and T. Onodera. Scalable performance of system s for extract-transform-load processing. In SYSTOR'10, pages 7:1-7:14, 2010.
28. Z. Wang and M. F. O'Boyle. Partitioning streaming parallelism for multi-cores: a machine learning based approach. In PACT'10, pages 307-318, 2010.
29. G. Yaikhom and et al. Federated enactment of workflow patterns. In Euro-Par 2010 - Parallel Processing, volume 6271, pages 317-328. 2010.
30. L. T. Yang, X. Ma, and F. Mueller. Cross-platform performance prediction of parallel applications using partial execution. In SC'05, Washington, DC, USA, 2005.
31. D. F. Yuan Yu, Michael Isard and M. Budiu. Dryadlinq: A system for general-purpose distributed data-parallel computing using a high-level language. OSDI'08, pages 1-14, 2008.
32. X. J. Zhang and et al. Workload characterization for operator-based distributed stream processing applications. In DEBS'10, pages 235-247, 2010.