On streaming consistency of big data stream processing in heterogenous clutsers

Proceedings - 2015 18th International Conference on Network-Based Information Systems, NBiS 2015

Volumn , Issue , 2015, Pages 476-482

  Xhafa, Fatos ^a   Naranjo, Victor ^a   Barolli, Leonard ^b   Takizawa, Makoto ^c  

^a UNIVERSITAT POLITÈCNICA DE CATALUNYA   (Spain)

^b FUKUOKA INSTITUTE OF TECHNOLOGY   (Japan)

^c HOSEI UNIVERSITY   (Japan)

Author keywords

Big data stream; Consistency of streaming; FlightRadar24; Generic log adapter; Global flight monitoring system; Parallel data processing; Real time analytics; Yahoo!S4

Indexed keywords

DATA COMMUNICATION SYSTEMS; DATA HANDLING; DATA MINING; DECISION MAKING; INFORMATION SYSTEMS; INTERNET; MONITORING; REAL TIME SYSTEMS; THROUGHPUT;

DATA STREAM; FLIGHTRADAR24; GENERIC LOG ADAPTER; MONITORING SYSTEM; PARALLEL DATA PROCESSING; REAL-TIME ANALYTICS; YAHOO!S4;

BIG DATA;

EID: 84964944621     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/NBiS.2015.122     Document Type: Conference Paper

References (19)

1. Acar, U.A., and Chen, Y. Streaming big data with selfadjusting computation. In Proceedings of the 2013 workshop on Data driven functional programming (DDFP '13). ACM, New York, NY, USA, 15-18. DOI=10.1145/2429376.2429382 http://doi.acm.org/10.1145/2429376.2429382
2. Balazinska, M., Balakrishnan, H., Madden, S.R. and Stonebraker, M. Fault-tolerance in the borealis distributed stream processing system. ACM Trans. Database Syst. 33, 1, Article 3 (2008), 44 pages. DOI=10.1145/1331904.1331907 http://doi.acm.org/10.1145/1331904.1331907
3. Barga, R.S, Goldstein, J., Ali, M. and Hong, M. Consistent Streaming Through Time: A Vision for Event Stream Processing. In Proceedings of the International Conference on Innovative Data Systems Research (CIDR), CA, 2007.
4. Caballé, S. and Xhafa, F. Distributed-based massive processing of activity logs for efficient user modeling in a Virtual Campus. Cluster Computing 16(4): 829-844 (2013)
5. Chauhan, J., Chowdhury, S.A. and Makaroff, D. Performance Evaluation of Yahoo! S4: A First Look. Seventh International Conference on P2P, Parallel, Grid, Cloud and Internet Computing (3PGCIC-2012), 12-14 Nov. 2012 Page(s): 58-65, Victoria, BC DOI: 10.1109/3PGCIC.2012.55, IEEE, 2012
6. Dean, J. and Ghemawat, S. MapReduce: Simplified Data Processing on Large Clusters. CACM, 51(1), 2008.
7. Fernandez, R. C. and Pietzuch, P. Towards Low-Latency and In-Memory Large-Scale Data Processing. Doctoral Workshop of the 7th ACM International Conference on Distributed Event Based Systems (DEBS), 06/2013, USA, 2013.
8. Grabarnik, G., Salahshour, A., Subramanian, B, and Ma, Sh. Generic Logging Adapter Toolkit. International Conference on Autonomic Computing (ICAC'04), 2004.
9. M. Hirzel, M., Andrade, H., Gedik, B., Jacques-Silva, G.,Khandekar, R., Kumar, V., Mendell, M., Nasgaard, H., Schneider, S., Soulé, R. and Wu, K.-L. IBM Streams Processing Language: Analyzing Big Data in motion, IBM J. RES. & DEV. VOL. 57 NO. 3/4 PAPER 7 MAY/JULY 2013
10. Hoeksema, J., Kotoulas, S. High-performance Distributed Stream Reasoning using S4. In Ordring Workshop at ISWC, 2011.
11. Kryo ObjectBuffer http://kryo.googlecode.com/svnhistory/ r107/api/com/esotericsoftware/kryo/ObjectBuffer.html
12. Mishne, G., Dalton, J., Li, Z., Sharma, A. and Lin, J. Fast data in the era of big data: Twitter's real-time related query suggestion architecture. Proc. ACM SIGMOD Int. Conf. on Management of Data, Page(s) 1147-1158, 2013.
13. Neumeyer, L., Robbins, B., Nair, A., and A. Kesari. S4: Distributed Stream Computing Platform. Proc. 2010 IEEE International Conference on Data Mining Workshops, Page(s) 170-177, 2010.
14. Salfner, F., Tschirpke, S. and Malek, M. Comprehensive Logfiles for Autonomic Systems. 18th International Parallel and Distributed Processing Symposium (IPDPS'04)-Workshop 11, 2004
15. Sun, D., Zhang, G., Zheng, W. and Li,K. Key technologies for big data stream computing, In Big Data: Algorithms, Analytics, and Applications, Kuan-Ching Li, Hai Jiang, Laurence T. Yang, and Alfredo Cuzzocrea, eds., CRC Press, 2015.
16. Topol, B., Ogle, D., Pierson, D., Thoensen, J., Sweitzer, J., Chow, M., Hoffmann,M.A., Durham, P., Telford, R., Sheth, S. and Studwell, T. Autonomic problem determination: A first step towards self-healing computing systems, IBM Autonomic Computing, 2003.
17. Xhafa, F., Claudi Paniagua, C., Barolli, L., and Caballé, S. Using Grid services to parallelize IBM's Generic Log Adapter. Journal of Systems and Software 84(1): 55-62 (2011)
18. Xhafa, F., Naranjo, V. and Caballé, S. Processing and Analytics of Big Data Streams with Yahoo!S4. In Proc. of 29th AINA-2015, Gwangju, Korea, 2015.
19. Zaharia, M., Das, T., Li, H., Shenker, S., and Stoica, I. Discretized streams: an efficient and fault-tolerant model for stream processing on large clusters. In Proceedings of the 4th USENIX conference on Hot Topics in Cloud Ccomputing (HotCloud'12). USENIX Association, Berkeley, CA, USA, 2012.