Elastic scaling of data parallel operators in stream processing

IPDPS 2009 - Proceedings of the 2009 IEEE International Parallel and Distributed Processing Symposium

Volumn , Issue , 2009, Pages

  Schneider, Scott ^a,b   Andrade, Henrique ^b   Gedik, Buǧra ^b   Biem, Alain ^b   Wu, Kun Lung ^b  

^a State University   (United States)

^b IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CRITICAL SECTIONS; DATA ANALYTICS; DATA PARALLEL; DYNAMIC ASPECTS; IMAGING APPLICATIONS; MULTI-CORE PROCESSOR; OPERATING POINTS; PROCESSING CYCLES; RECENT TRENDS; STREAM PROCESSING; STREAMING APPLICATIONS; SYNTHETIC WORKLOADS;

DISTRIBUTED PARAMETER NETWORKS; RADIO ASTRONOMY;

HYDRAULICS;

EID: 70449877411     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPS.2009.5161036     Document Type: Conference Paper

References (24)

1. A. Arasu, B. Babcock, S. Babu, M. Datar, K. Ito, R. Motwani, I. Nishizawa, U. Srivastava, D. Thomas, R. Varma, and J. Widom, "STREAM: The Stanford stream data manager," IEEE Data Engineering Bulletin, vol. 26, 2003.
2. D. J. Abadi, Y. Ahmad, M. Balazinska, U. Cetintemel, M. Cherniack, J.-H. Hwang, W. Lindner, A. S. Maskey, A. Rasin, E. Ryvkina, N. Tatbul, Y. Xing, and S. Zdonik, "The design of the Borealis stream processing engine," in Proceedings of Conference on Innovative Data Systems Research (CIDR 05), 2005.
3. "StreamBase Systems," http://www.streambase.com.
4. S. Chandrasekaran, O. Cooper, A. Deshpande, M. J. Franklin, J. M. Hellerstein, W. Hong, S. Krishnamurthy, S. R. Madden, V. Raman, F. Reiss, and M. A. Shah, "TelegraphCQ: Continuous dataflow processing for an uncertain world," in Proceedings of Conference on Innovative Data Systems Research (CIDR 03), 2003.
5. W. Thies, M. Karczmarek, and S. Amarasinghe, "StreamIt: A language for streaming applications," in Proceedings of the International Conference on Compiler Construction (CC 2002), April 2002.
6. G. Upadhyaya, V. S. Pai, and S. P. Midkiff, "Expressing and exploiting concurrency in networked applications with aspen," in Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP 2007), 2007.
7. H. Andrade, B. Gedik, K.-L. Wu, and P. S. Yu, "Scale-up strategies for processing high-rate data streams in System S," in Proceedings of the International Conference on Data Engineering (ICDE 2009) - to appear, 2009.
8. D. A. Menasce and V. A. F. Almeida, Ćapacity Planning For Web Performance, 1998.
9. D. A. Menasce, V. A. F. Almeida, and L. W. Dowdy, Ćapacity Planning and Performance Modeling, 2000. [10] B. Gedik, H. Andrade, and K.-L. Wu, "A code generation approach for optimizing high performance distributed data stream processing," in Proceedings of the USENIX Annual Technical Conference (USENIX 2009) - submitted for publication, 2009.
10. B. Gedik, H. Andrade, K.-L. Wu, P. S. Yu, and M. Doo, "SPADE: The System S declarative stream processing engine," in Proceedings of the ACM International Conference on Management of Data (SIGMOD 2008), 2008. [12] P. Gepner and M. F. Kowalik, "Multi-core processors: New way to achieve high system performance," in Proceedings of the International Conference on Parallel Computing in Electrical Engineering (PARELEC 06), 2006, pp. 9-13. [13] L. Amini, H. Andrade, R. Bhagwan, F. Eskesen, R. King, P. Selo, Y Park, and C. Venkatramani, "SPC: A distributed, scalable platform for data mining," in Workshop on Data Mining Standards, Services and Platforms (DM-SSP 06), Philadelphia, PA, 2006. [14] N. Jain, L. Amini, H. Andrade, R. King, Y Park, P. Selo, and C. Venkatramani, "Design, implementation, and evaluation of the linear road benchmark on the Stream Processing Core," in Proceedings of the ACM International Conference on Management of Data (SIGMOD 2006), 2006.
11. K.-L. Wu, P. S. Yu, B. Gedik, K. W. Hildrum, C. C. Aggarwal, E. Bouillet, W. Fan, D. A. George, X. Gu, G. Luo, and H. Wang, "Challenges and experience in prototyping a multi-modal stream analytic and monitoring application on System S," in Proceedings of the International Conference on Very Large Data Bases Conference (VLDB 2007), 2007.
12. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins, "Pig Latin: A not-so-foreign language for data processing," in Proceedings of the ACM International Conference on Management of Data (SIGMOD 2008), 2008.
13. J. D. Ullman, Database and Knowledge-Base Systems. Computer Science Press, 1988.
14. G. Hulten and P. Domingos, "VFML - a toolkit for mining high-speed time-changing data streams," http://www.cs. washington.edu/dm/vfml, October 2003.
15. D. S. Turaga, O. Verscheure, J. Wong, L. Amini, G. Yocum, E. Begle, and B. Pfeifer, "Online FDC control limit tuning with yield prediction using incremental decision tree learning," in Sematech AEC/APC, 2007, 2007.
16. H. Wang, H. Andrade, B. Gedik, and K.-L. Wu, "Using vectorization through code generation for high throughput stream processing applications," in Proceedings of the ACM International Conference on Supercomputing (ICS 2009) -submitted for publication, 2009.
17. M. Frigo, C. E. Leiserson, and K. H. Randall, "The implementation of the Cilk-5 multithreaded language," in Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI1998), 1998, pp. 212-223.
18. "The Australian Square Kilometre Array Pathfinder," http://www.atnf.csiro.au/projects/askap.
19. T. Cornwell, K. Golap, and S. Bhatnagar, "W projection: A new algorithm for wide field imaging with radio synthesis arrays," in Proceedings of the Astronomical Data Analysis Software and Systems XIV ASP Conference Series, vol. 347, December 2005, p. 86.
20. M. Beynon, R. Ferreira, T. Kurc, A. Sussman, and J. Saltz, "DataCutter: Middleware for filtering very large scientific datasets on archival storage systems," in Proceedings of the 8th Goddard Conference on Mass Storage Systems and Tech-nologies/17th IEEE Symposium on Mass Storage Systems, College Park, MD, Mar. 2000.
21. L. Amini, N. Jain, A. Sehgal, J. Silber, and O. Verscheure, "Adaptive control of extreme-scale stream processing systems," Lisbon, Portugal, July 2006.
22. R. C. Whaley, A. Petitet, and J. Dongarra, "Automated empirical optimizations of software and the atlas project," Parallel Computing, vol. 27, no. 1-2, pp. 3-35, 2001.
23. L. Renganarayanan and S. Rajopadhye, "Positivity, posyn-omials and tile size selection," in Proceedings of the ACM/IEEE SC Conference (SC 2008), Austin, TX, November 2008.
24. "Hadoop," http://hadoop.apache.org.