Full-text indexing for optimizing selection operations in large-scale data analytics

MapReduce'11 - Proceedings of the 2nd International Workshop on MapReduce and Its Applications

Volumn , Issue , 2011, Pages 59-66

  Lin, Jimmy ^a   Ryaboy, Dmitriy ^a   Weil, Kevin ^a  

^a TWITTER INC   (United States)

Author keywords

database optimization; hadoop; MapReduce; pig

Indexed keywords

ANALYTICAL MODEL; DATA ANALYTICS; DATA BLOCKS; DATA SETS; DATABASE OPTIMIZATION; EXECUTION ENGINE; FULL-TEXT INDEX; HADOOP; MAP-REDUCE; OPEN SOURCE IMPLEMENTATION; OPTIMIZING SELECTION; PIG; PROCESSING TIME; PROOF OF CONCEPT; QUERY TERMS; RUNNING TIME; SOCIAL MEDIA; SUPPORT OPERATIONS; TEXT FIELDS; UNSTRUCTURED DATA; WEB-BASED APPLICATIONS; WORKER NODES;

DISTRIBUTED COMPUTER SYSTEMS; MATHEMATICAL MODELS; OPTIMIZATION; USER INTERFACES;

SEARCH ENGINES;

EID: 79961034447     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1996092.1996105     Document Type: Conference Paper

References (21)

1. A. Abouzeid, K. Bajda-Pawlikowski, D. Abadi, A. Silberschatz, and A. Rasin. HadoopDB: An architectural hybrid of MapReduce and DBMS technologies for analytical workloads. VLDB, 2009.
2. S. Agrawal, S. Chaudhuri, and G. Das. DBXplorer: A system for keyword-based search over relational databases. ICDE, 2002.
3. M. Cafarella and C. Ré. Manimal: Relational optimization for data-intensive programs. WebDB, 2010.
4. F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. Gruber. Bigtable: A distributed storage system for structured data. OSDI, 2006.
5. J. Dean. Challenges in building large-scale information retrieval systems. WSDM Keynote, 2009.
6. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. OSDI, 2004.
7. J. Dean and S. Ghemawat. MapReduce: A flexible data processing tool. CACM, 53(1):72-77, 2010.
8. S. Dessloch and N. Mattos. Integrating SQL databases with content-specific search engines. VLDB, 1997.
9. J. Dittrich, J.-A. Quiané-Ruiz, A. Jindal, Y. Kargin, V. Setty, and J. Schad. Hadoop++: Making a yellow elephant run like a cheetah (without it even noticing). VLDB, 2010.
10. A. Doan, J. Naughton, A. Baid, X. Chai, F. Chen, T. Chen, E. Chu, P. DeRose, B. Gao, C. Gokhale, J. Huang, W. Shen, and B.-Q. Vuong. The case for a structured approach to managing unstructured data. CIDR, 2009.
11. A. Gates, O. Natkovich, S. Chopra, P. Kamath, S. Narayanamurthy, C. Olston, B. Reed, S. Srinivasan, and U. Srivastava. Building a high-level dataflow system on top of MapReduce: The Pig experience. VLDB, 2009.
12. S. Ghemawat, H. Gobioff, and S.-T. Leung. The Google File System. SOSP, 2003.
13. S. Melnik, A. Gubarev, J. Long, G. Romer, S. Shivakumar, M. Tolton, and T. Vassilakis. Dremel: Interactive analysis of web-scale datasets. VLDB, 2010.
14. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig Latin: A not-so-foreign language for data processing. SIGMOD, 2008.
15. A. Pavlo, E. Paulson, A. Rasin, D. Abadi, D. DeWitt, S. Madden, and M. Stonebraker. A comparison of approaches to large-scale data analysis. SIGMOD, 2009.
16. D. Peng and F. Dabek. Large-scale incremental processing using distributed transactions and notifications. OSDI, 2010.
17. M. Stonebraker, D. Abadi, D. DeWitt, S. Madden, E. Paulson, A. Pavlo, and A. Rasin. MapReduce and parallel DBMSs: Friends or foes? CACM, 53(1):64-71, 2010.
18. A. Thusoo, J. Sarma, N. Jain, Z. Shao, P. Chakka, N. Zhang, S. Anthony, H. Liu, and R. Murthy. Hive - a petabyte scale data warehouse using Hadoop. ICDE, 2010.
19. G. Weikum. DB & IR: Both sides now. SIGMOD Keynote, 2007.
20. H. Yang, A. Dasdan, R.-L. Hsiao, and D. S. Parker. Map-Reduce-Merge: Simplified relational data processing on large clusters. SIGMOD, 2007.
21. J. Yu, L. Qin, and L. Chang. Keyword search in relational databases: A survey. Data Engineering, 33(1):67-78, 2010.