Gobblin: Unifying data ingestion for hadoop

Proceedings of the VLDB Endowment

Volumn 8, Issue 12 12, 2015, Pages 1764-1760

  Qiao, Lin ^a   Li, Yinan ^a   Takiar, Sahil ^a   Liu, Ziyang ^a   Veeramreddy, Narasimha ^a   Tu, Min ^a   Dai, Ying ^a   Buenrostro, Issac ^a   Surlaker, Kapil ^a   Das, Shirshanka ^a   Botev, Chavdar ^a  

^a LINKEDIN CORPORATION   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CENTRALISED; DATA INGESTIONS; DATA-SOURCE; FILESYSTEM; GENERIC DATA; LARGE VOLUMES; LINKEDIN; OPEN SOURCE PRODUCTS; OPERATIONAL PROBLEMS; STREAMING SYSTEMS;

EID: 84953852658     PISSN: None     EISSN: 21508097     Source Type: Journal    
DOI: None     Document Type: Chapter

References (25)

1. Aegisthus. https://github.com/Netix/aegisthus.
2. Apache Flink. https://ink.apache.org/.
3. Apache Flume. https://ume.apache.org/.
4. Apache Giraph. http://giraph.apache.org/.
5. Apache Hadoop. https://hadoop.apache.org/.
6. Apache Kafka. http://kafka.apache.org/.
7. Apache Oozie. http://oozie.apache.org/.
8. Apache Spark. https://spark.apache.org/.
9. Apache Sqoop. http://sqoop.apache.org/.
10. Azkaban: Open-source Workow Manager. http://azkaban.github.io/.
11. Camus. https://github.com/linkedin/camus.
12. Chronos. http://nerds.airbnb.com/introducing-chronos/.
13. Morphlines. http://cloudera.github.io/cdk/docs/current/cdk- morphlines/index.html.
14. Pinot. https://github.com/linkedin/pinot.
15. Presto. https://prestodb.io/.
16. Quartz Scheduler. http://quartz-scheduler.org/.
17. RocksDB. http://rocksdb.org/.
18. YARN. http://hadoop.apache.org/docs/current/hadoop- yarn/hadoop-yarn-site/YARN.html.
19. S. Chen. Cheetah: A High Performance, Custom Data Warehouse on Top of MapReduce. PVLDB, 3(2):1459-1468, 2010.
20. K. Gopalakrishna, S. Lu, Z. Zhang, A. Silberstein, K. Surlaker, R. Subramonian, and B. Schulman. Untangling Cluster Management with Helix. In SoCC, 2012.
21. P. Hunt, M. Konar, F. P. Junqueira, and B. Reed. ZooKeeper: Wait-free Coordination for Internet-scale Systems. In USENIX, 2010.
22. M. Kornacker, A. Behm, V. Bittorf, T. Bobrovytsky, C. Ching, A. Choi, J. Erickson, M. Grund, D. Hecht, M. Jacobs, I. Joshi, L. Kú, D. Kumar, A. Leblang, N. Li, I. Pandis, H. Robinson, D. Rorke, S. Rus, J. Russell, D. Tsirogiannis, S. Wanderman-Milne, and M. Yoder. Impala: A Modern, Open-Source SQL Engine for Hadoop. In CIDR, 2015.
23. L. Qiao, K. Surlaker, S. Das, T. Quiggle, B. Schulman, B. Ghosh, A. Curtis, O. Seeliger, Z. Zhang, A. Auradkar, C. Beaver, G. Brandt, M. Gandhi, K. Gopalakrishna, W. Ip, S. Jagadish, S. Lu, A. Pachev, A. Ramesh, A. Sebastian, R. Shanbhag, S. Subramaniam, Y. Sun, S. Topiwala, C. Tran, J. Westerman, and D. Zhang. On Brewing Fresh Espresso: LinkedIn's Distributed Data Serving Platform. In SIGMOD Conference, pages 1135-1146, 2013.
24. V. Raman, G. Swart, L. Qiao, F. Reiss, V. Dialani, D. Kossmann, I. Narang, and R. Sidle. Constant-Time Query Processing. In ICDE, pages 60-69, 2008.
25. R. Sumbaly, J. Kreps, L. Gao, A. Feinberg, C. Soman, and S. Shah. Serving Large-scale Batch Computed Data with Project Voldemort. In FAST, 2012.