Data warehousing and analytics infrastructure at facebook

Proceedings of the ACM SIGMOD International Conference on Management of Data

Volumn , Issue , 2010, Pages 1013-1020

  Thusoo, Ashish ^a   Shao, Zheng ^a   Anthony, Suresh ^a   Borthakur, Dhruba ^a   Jain, Namit ^a   Sen Sarma, Joydeep ^a   Murthy, Raghotham ^a   Liu, Hao ^a  

^a FACEBOOK   (United States)

Author keywords

analytics; data discovery; data warehouse; distributed file system; distributed systems; facebook; hadoop; hive; log aggregation; map reduce; resource sharing; scalability; scribe

Indexed keywords

ANALYTICS; DATA DISCOVERY; DISTRIBUTED FILE SYSTEMS; DISTRIBUTED SYSTEMS; FACEBOOK; HADOOP; RESOURCE SHARING;

DATA HANDLING; SCALABILITY; SOCIAL NETWORKING (ONLINE); TERBIUM ALLOYS;

DATA WAREHOUSES;

EID: 77954709174     PISSN: 07308078     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1807167.1807278     Document Type: Conference Paper

References (11)

1. Apache Hadoop wiki. Available at http://wiki.apache.org/hadoop.
2. Apache Hadoop Hive wiki. Available at http://wiki.apache.org/hadoop/Hive.
3. Scribe wiki. Available at http://wiki.github.com/facebook/scribe.
4. Ghemawat, S., Gobioff, H. and Leung, S. 2003. The Google File System. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (Lake George, NY, Oct. 2003).
5. Dean, J. and Ghemawat S. 2004. MapReduce: Simplified Data Processing on Large Clusters. In Proceedings of the 6th Symposium on Operating System Design and Implementation (San Francisco, CA, Dec. 2004). OSDI'04.
6. HDFS Architecture. Available at http://hadoop.apache.org/common/docs/ current/hdfs-design.pd f.
7. Hive DDL wiki. Available at http://wiki.apache.org/hadoop/Hive/ LanguageManual/DDL.
8. Thusoo, A., Murthy, R., Sen Sarma, J., Shao, Z., Jain, N., Chakka, P., Anthony, A., Liu, H., Zhang, N. 2010. Hive - A Petabyte Scale Data Warehouse Using Hadoop. In Proceedings of 26th IEEE International Conference on Data Engineering (Long Beach, California, Mar. 2010). ICDE'10.
9. Ailamaki, A., DeWitt, D.J., Hill, M.D., Skounakis, M. 2001. Weaving Relations for Cache Performance. In Proceedings of 27th Very Large Data Base Conference (Roma, Italy, 2001). VLDB'01.
10. Zaharia, M., Borthakur, D., Sen Sarma, J., Elmeleegy, K., Shenker, S., Stoica, I. 2009. Job Scheduling for Multi-User MapReduce Clusters. UC Berkeley Technical Report UCB/EECS-2009-55 (Apr. 2009).
11. Fan, B., Tantisiriroj, W., Xiao, Lin, Gibson, G. 2009. DiskReduce: RAID for Data-Intensive Scalable Computing. In Proceedings of 4th Petascale Data Storage Workshop Supercomputing Conference (Portland, Oregon, Nov. 2009). Supercomputing PDSW'09.