A highly efficient consolidated platform for stream computing and Hadoop

Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2012

Volumn , Issue , 2012, Pages 2026-2034

  Matsuura, Hiroya ^a   Ganse, Masaru ^a   Suzumura, Toyotaro ^b  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

^b IBM RESEARCH TOKYO   (Japan)

Author keywords

hadoop; stream computing

Indexed keywords

COMPUTE RESOURCES; COMPUTING PARADIGM; DATA STREAM PROCESSING; DISTRIBUTED DATA STREAM PROCESSING; HADOOP; LOAD-BALANCING MECHANISMS; SECONDARY STORAGE; STREAM COMPUTING; STREAM PROCESSING; STREAMING DATA;

DIGITAL STORAGE; DISTRIBUTED PARAMETER NETWORKS; NETWORK MANAGEMENT; PARALLEL ARCHITECTURES;

DATA COMMUNICATION SYSTEMS;

EID: 84867431906     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IPDPSW.2012.252     Document Type: Conference Paper

References (21)

1. Daniel J Abadi, Yanif Ahmad, Magdalena Balazinska, Ugur Cetintemel, Mitch Cherniack, Jeong-Hyon Hwang, Wolfgang Lindner, Anurag S Maskey, Alexander Rasin, Esther Ryvkina, Nesime Tatbul, Ying Xing and Stan Zdonik, The Design of the Borealis Stream Processing Engine, CIDR 2005
2. Chandrasekaran, Sirish and Cooper, Owen and Deshpande, Amol and Franklin, Michael J. and Hellerstein, Joseph M. and Hong, Wei and Krishnamurthy, Sailesh and Madden, Samuel R. and Reiss, Fred and Shah, Mehul A., TelegraphCQ: Continuous dataflow processing for an uncertain world, SIGMOD 2003
3. Yousuke Watanabe and Hiroyuki Kitagawa, a Sustain-able Query Processing Method in. Distributed Stream Environment, the Database Society of Japan 2007
4. Bugra Gedik, Henrique Andrade and Kun-Lung Wu, A Code Generation Approach to Optimizing High-Performance Distributed Data Stream Processing, ACM 2008
5. Bugra Gedik, Henrique Andrade, Kun-Lung Wu, Philip S. Yu and Myung Cheol Doo, SPADE : the System S declarative Stream Procesing Engine, SIGMOD 2008
6. Lisa Amini, Henrique Andrade , Ranjita Bhagwan , Frank Eskesen , Richard King , Yoonho Park and Chitra Venkatramani, SPC : a distributed, scalable platform for data mining, DMSSP 2006
7. Joel Wolf, Nikhil Bansal, Kirsten Hildrum, Sujay Parekh, Deepak Rajan, Rohit Wagle, Kun-Lung Wu and Lisa Fleischer, SODA : An optimizing scheduler for large-scale stream-based distributed computer systems, Middleware 2008
8. Yahoo, Apache Hadoop, http://hadoop.apache.org/
9. Jeffrey Dean and Sanjay Ghemawat, Mapreduce: simplified data processing on large clusters, OSDI 2008.
10. Yahoo Hadoop Tutorial, http://developer.yahoo.com/hadoop/tutorial/ module4.html
11. Kenji Yamanishi and Jun-ichi Takeuchi, A unifying framework for detecting outliers and change points from non-stationary time series data, SIGKDD 2002
12. Karve, A. and Kimbrel, T. and Pacifici, G. and Spreitzer, M. and Steinder, M. and Sviridenko, M. and Tantawi, A., Dynamic placement for clustered web applications, WWW 2006
13. Carrera, David and Steinder, Malgorzata and Whalley, Ian and Torres, Jordi and Ayguade, Eduard, Enabling resource sharing between transactional and batch workloads using dynamic application placement, Middleware 2008
14. Michael Isard, Mihai Budiu, Yuan Yu, Andrew Birrell, Dennis Fetterly: Dryad: distributed data-parallel programs from sequential building blocks. EuroSys 2007: 59-72
15. Yuan Yu, Michael Isard, Dennis Fetterly, Mihai Budiu, Úlfar Erlingsson, Pradeep Kumar Gunda, Jon Currey: DryadLINQ: A System for General-Purpose Distributed Data-Parallel Computing Using a High-Level Language. OSDI 2008: 1-14
16. Matei Zaharia, Andy Konwinski, Anthony D. Joseph, Randy H. Katz, Ion Stoica: Improving MapReduce Performance in Heterogeneous Environments. OSDI 2008: 29-42
17. Matei Zaharia, Dhruba Borthakur, Joydeep Sen Sarma, Khaled Elmeleegy, Scott Shenker, Ion Stoica: Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling. EuroSys 2010: 265-278
18. Yong Zhao, Mihael Hategan, Ben Clifford, Ian T. Foster, Gregor von Laszewski, Veronika Nefedova, Ioan Raicu, Tiberiu Stef-Praun, Michael Wilde: Swift: Fast, Reliable, Loosely Coupled Parallel Computation. IEEE SCW 2007: 199-206
19. Omer Ozan Sonmez, Nezih Yigitbasi, Alexandru Iosup, Dick H. J. Epema: Trace-based evaluation of job runtime and queue wait time predictions in grids. HPDC 2009: 111-120
20. Joao Pedro Costa, Pedro Martins, Jose Cecilio, and Pedro Furtado. 2010. StreamNetFlux: birth of transparent integrated CEP-DBs. In Proceedings of the Fourth ACM International Conference on Distributed Event-Based Systems (DEBS '10). ACM, New York, NY, USA, 101-102.
21. L. Neumeyer, B. Robbins, A. Nair, and A. Kesari. S4: Distributed stream computing platform. In International Workshop on Knowledge Discovery Using Cloud and Distributed Computing Platforms (KDCloud, 2010) Proceedings. IEEE, December 2010.