Survey of real-time processing systems for big data

ACM International Conference Proceeding Series

Volumn , Issue , 2014, Pages 356-361

  Liu, Xiufeng ^a   Lftikhar, Nadeem ^b   Xie, Xike ^c  

^a UNIVERSITY OF WATERLOO   (Canada)

^b Department of Hospitality and Experience Management University College Northern   (Denmark)

^c AALBORG UNIVERSITY   (Denmark)

Author keywords

Architectures; Big data; Real time; Survey; Systems

Indexed keywords

ARCHITECTURE; COMPUTER SYSTEMS; DATABASE SYSTEMS; OPEN SYSTEMS; SURVEYING; SURVEYS;

BUSINESS ENVIRONMENTS; DISTRIBUTED COMPUTING TECHNOLOGY; HIGH SCALABILITIES; OPEN SOURCE IMPLEMENTATION; OPEN-SOURCE TECHNOLOGY; REAL-TIME; REALTIME PROCESSING; SYSTEM ARCHITECTURES;

BIG DATA;

EID: 84906817792     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2628194.2628251     Document Type: Conference Paper

References (34)

1. A Quick Message Queue Benchmark: ActiveMQ, RabbitMQ, HornetQ, QPID, Apollo, URL: atx-aeon.com/wp/2013/ 04/10/a-quick-message-queue-benchmark- activemqrabbitmq-hornetq-qpid-apollo as of 2014/5/1.
2. D. J. Abadi et al. Column-stores vs. row-stores: how different are they really? In Proc, of the SIGMOD, pp. 967-980, 2008.
3. Apache Flume. URL: flume.apache.org as of 2014/5/1.
4. Apache Hadoop NextGen MapReduce (YARN). URL: hadoop.apache.org as of 2014/5/1.
5. Apache Kafka - A high-throughput distributed messaging system. URL: kafka.apache.org as of 2014/5/1.
6. Big Data Survey Europe. URL: www.pmone.com/ fileadmin/user-upload/doc/ study/BARC-BIG-DATA- SURVEY-EN-final.pdf as of 2014/5/1.
7. T. Condie et al. MapReduce Online. NSDI 10(4):20, 2010.
8. J. Dean, and S. Ghemawat. Mapreduce: simplified data processing on large clusters. CACM, 1(51): 107-113, 2008.
9. J. Dittrich et al. Hadoop++: making a yellow elephant run like a cheetah. PVLDB, 3(1):518-529, 2010.
10. R. Gemulla et al. CoHadoop: flexible data placement and its exploitation in Hadoop. PVLDB, 4(9):575-585, 2011.
11. K. Goodhope et al. Building Linkedln's real-time activity data pipeline. IEEE Data Eng. Bull, 35(2):33-45, 2012.
12. Hadoop. URL: hadoop.apache.org as of 2014/5/1.
13. C. Hewitt et al. A universal modular actor formalism for artificial intelligence. IJCAI, 1973.
14. Y. He et al. RCFile: A fast and space-efficient data placement structure in MapReduce-based warehouse systems. In Proc. of ICDE, pp. 1199-1208, 2011.
15. HStreaming. URL: www.adello.com as of 2014/5/1.
16. Impala. URL: www.cloudera.com as of 2014/5/1.
17. J. Kreps et al. Kafka: A distributed messaging system for log processing. In Proc. of NetDB, 2011.
18. K. Lee et al. Parallel data processing with MapReduce: a survey. SIGMOD Record 40(4):11-20, 2012.
19. X. Liu. Data warehousing technologies for large-scale and right-time data. Dissertation, Aalborg University, DK, 2012.
20. X. Liu, C. Thomsen, and T. B. Pedersen. ETLMR: a highly scalable dimensional ETL framework based on mapreduce. In Proc. of DaWak, pp. 96-11, 2011.
21. X. Liu, C. Thomsen, and T. B. Pedersen. Mapreduce-based dimensional etl made easy. PVLDB, 5(12): 1882-1885, 2012.
22. X. Liu, C. Thomsen, and T. B. Pedersen. CloudETL: Scalable dimensional for Hive. In Proc. of IDEAS, 2014.
23. N. Marz and J. Warren. Big data: principles and best practices of scalable realtime data systems. Manning, 2013.
24. S. Melnik et al. Dremel: interactive analysis of web-scale datasets. PVLDB 3(1-2):330-339, 2010.
25. L. Neumeyer et al. S4: distributed stream computing platform. In: Proc. ofICDMW, pp. 170-177, 2010.
26. S4 distributed stream computing platform. URL: incubator.apache.org/s4 as of 2014/5/1.
27. Scribe. URL: github.com/facebook/scribe as of 2014/5/1.
28. Spark. URL: spark.incubator.apache.org as of 2014/5/1.
29. M. Stonebraker et al. MapReduce and parallel DBMSs: friends or foes? Commun. ACM, 53(1):64-71, 2010.
30. Storm. URL: storm-project.net as of 2014/5/1.
31. C. Thomsen, T. B. Pedersen, and W. Lehner. RiTE: Providing on-demand data for right-time data warehousing. In Proc. of ICDE, pp. 456-465, 2008.
32. M. Zaharia et al. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. NSDI, pp. 2-2, 2012.
33. M. Zaharia et al. Discretized streams: an efficient programming model for large-scale stream processing. USENIX HotCloud, pp. 10-10, 2012.
34. M. Zaharia et al. Discretized streams: fault-tolerant streaming computation at scale. In Proc. of SOSP, pp. 423-138, 2013.