RCFile: A fast and space-efficient data placement structure in MapReduce-based warehouse systems

Proceedings - International Conference on Data Engineering

Volumn , Issue , 2011, Pages 1199-1208

  He, Yongqiang ^a   Lee, Rubao ^b   Huai, Yin ^b   Shao, Zheng ^a   Jain, Namit ^a   Zhang, Xiaodong ^b   Xu, Zhiwei ^c  

^a OHIO STATE UNIVERSITY   (United States)

^b Ohio State University   (United States)

^c INSTITUTE OF GEOLOGY AND GEOPHYSICS   (China)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVITY; CONVENTIONAL DATABASE; CRITICAL FACTORS; DATA ANALYSIS SYSTEM; DATA ANALYTICS; DATA PLACEMENT; DATA WAREHOUSE SYSTEMS; DISTRIBUTED SYSTEMS; FACEBOOK; LARGE DATA; MAP-REDUCE; PRODUCTION SYSTEM; SOCIAL NETWORKS; STORAGE SPACES; USER BEHAVIORS; WAREHOUSE PERFORMANCE; WAREHOUSE SYSTEMS; WEB SERVICE PROVIDERS; WORKLOAD PATTERNS;

BEHAVIORAL RESEARCH; DATA PROCESSING; DATA REDUCTION; DATA WAREHOUSES; MAMMALS; SOCIAL NETWORKING (ONLINE); USER INTERFACES; WEB SERVICES;

SEARCH ENGINES;

EID: 79957794587     PISSN: 10844627     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDE.2011.5767933     Document Type: Conference Paper

References (27)

1. http://www.facebook.com/press/info.php?statistics.
2. J. Dean and S. Ghemawat, "MapReduce: Simplified data processing on large clusters," in OSDI, 2004, pp. 137-150.
3. http://hadoop.apache.org/.
4. A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, H. Liu, P. Wyckoff, and R. Murthy, "Hive-a warehousing solution over a Map-Reduce framework," PVLDB, vol. 2, no. 2, pp. 1626-1629, 2009.
5. A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, N. Zhang, S. Anthony, H. Liu, and R. Murthy, "Hive-a petabyte scale data warehouse using hadoop," in ICDE, 2010, pp. 996-1005.
6. A. Gates, O. Natkovich, S. Chopra, P. Kamath, S. Narayanam, C. Olston, B. Reed, S. Srinivasan, and U. Srivastava, "Building a highlevel dataflow system on top of MapReduce: The Pig experience," PVLDB, vol. 2, no. 2, pp. 1414-1425, 2009.
7. A. Thusoo, Z. Shao, S. Anthony, D. Borthakur, N. Jain, J. S. Sarma, R. Murthy, and H. Liu, "Data warehousing and analytics infrastructure at facebook," in SIGMOD Conference, 2010, pp. 1013-1020.
8. R. Ramakrishnan and J. Gehrke, "Database management systems," McGraw-Hill, 2003.
9. G. P. Copeland and S. Khoshafian, "A decomposition storage model," in SIGMOD Conference, 1985, pp. 268-279.
10. M. Stonebraker, D. J. Abadi, A. Batkin, X. Chen, M. Cherniack, M. Ferreira, E. Lau, A. Lin, S. Madden, E. J. O'Neil, P. E. O'Neil, A. Rasin, N. Tran, and S. B. Zdonik, "C-store: A column-oriented dbms," in VLDB, 2005, pp. 553-564. (Pubitemid 43991033)
11. D. J. Abadi, S. Madden, and N. Hachem, "Column-stores vs. row-stores: how different are they really?" in SIGMOD Conference, 2008.
12. A. L. Holloway and D. J. DeWitt, "Read-optimized databases, in depth," PVLDB, vol. 1, no. 1, pp. 502-513, 2008.
13. A. Ailamaki, D. J. DeWitt, M. D. Hill, and M. Skounakis, "Weaving relations for cache performance," in VLDB, 2001, pp. 169-180.
14. D. Tsirogiannis, S. Harizopoulos, M. A. Shah, J. L. Wiener, and G. Graefe, "Query processing techniques for solid state drives," in SIGMOD Conference, 2009, pp. 59-72.
15. S. Harizopoulos, V. Liang, D. J. Abadi, and S. Madden, "Performance tradeoffs in read-optimized databases," in VLDB, 2006, pp. 487-498.
16. M. Stonebraker and U. Ç etintemel, "One size fits all: An idea whose time has come and gone (abstract)," in ICDE, 2005, pp. 2-11. (Pubitemid 41731109)
17. V. Raman and G. Swart, "How to wring a table dry: Entropy compression of relations and querying of compressed relations," in VLDB, 2006.
18. V. Raman, G. Swart, L. Qiao, F. Reiss, V. Dialani, D. Kossmann, I. Narang, and R. Sidle, "Constant-time query processing," in ICDE, 2008, pp. 60-69.
19. P. A. Boncz, S. Manegold, and M. L. Kersten, "Database architecture optimized for the new bottleneck: Memory access," in VLDB, 1999.
20. http://wiki.apache.org/pig/zebra.
21. R. A. Hankins and J. M. Patel, "Data morphing: An adaptive, cacheconscious storage technique," in VLDB, 2003, pp. 417-428.
22. A. Pavlo, E. Paulson, A. Rasin, D. J. Abadi, D. J. DeWitt, S. Madden, and M. Stonebraker, "A comparison of approaches to large-scale data analysis," in SIGMOD Conference, 2009, pp. 165-178.
23. A. Abouzeid, K. Bajda-Pawlikowski, D. J. Abadi, A. Rasin, and A. Silberschatz, "HadoopDB: An architectural hybrid of MapReduce and DBMS technologies for analytical workloads," PVLDB, vol. 2, no. 1, pp. 922-933, 2009.
24. A. L. Holloway, V. Raman, G. Swart, and D. J. DeWitt, "How to barter bits for chronons: compression and bandwidth trade offs for database scans," in SIGMOD Conference, 2007, pp. 389-400. (Pubitemid 47630823)
25. S. Chen, "Cheetah: A high performance, custom data warehouse on top of mapreduce," PVLDB, vol. 3, no. 2, pp. 1459-1468, 2010.
26. F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. Gruber, "Bigtable: A distributed storage system for structured data," in OSDI, 2006, pp. 205-218.
27. http://hbase.apache.org.