Batch text similarity search with mapreduce

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 6612 LNCS, Issue , 2011, Pages 412-423

  Li, Rui ^a,b,c   Ju, Li ^d   Peng, Zhuo ^a   Yu, Zhiwei ^a   Wang, Chaokun ^a,b,c  

^a TSINGHUA UNIVERSITY   (China)

^b TSINGHUA NATIONAL LABORATORY FOR INFORMATION SCIENCE AND TECHNOLOGY   (China)

^c BEIJING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS   (China)

^d Henan College of Finance and Taxation   (China)

Author keywords

Batch Text Similarity Search; MapReduce

Indexed keywords

INVERTED INDEX STRUCTURES; MAP-REDUCE; REAL DATA SETS; TEXT QUERY; TEXT SIMILARITY; TWO-STAGE ALGORITHM;

EID: 79955119726     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-20291-9_46     Document Type: Conference Paper

References (15)

1. Web-1t: Web 1t 5-gram version 1, http://www.ldc.upenn.edu/Catalog/ CatalogEntry.jsp?catalogId=LDC2006T13
2. Dean, J., Ghemawat, S.: Mapreduce: Simplified data processing on large clusters. In: OSDI (2004)
3. Vernica, R., Carey, M.J., Li, C.: Efficient parallel set-similarity joins using mapreduce. In: SIGMOD (2010)
4. Bayardo, R.J., Ma, Y., Srikant, R.: Scaling up all pairs similarity search. In: WWW, pp. 131-140. ACM, New York (2007) (Pubitemid 47582245)
5. Lewis, J., Ossowski, S., Hicks, J., Errami, M., Garner, H.R.: Text similarity: an alternative way to search medline. Bioinformatics 22(18) (2006)
6. Berchtold, S., Christian, G., Braunmüller, B., Keim, D.A., Kriegel, H.P.: Fast parallel similarity search in multimedia databases. In: SIGMOD, pp. 1-12 (1997) (Pubitemid 127456769)
7. Dong, X., Halevy, A.Y., Madhavan, J., Nemes, E., Zhang, J.: Similarity search for web services. In: VLDB, pp. 372-383. Morgan Kaufmann, San Francisco (2004)
8. Gravano, L., Ipeirotis, P.G., Jagadish, H.V., Koudas, N., Muthukrishnan, S., Srivastava, D.: Approximate string joins in a database (almost) for free. In: VLDB, pp. 491-500 (2001)
9. Jin, L., Li, C., Mehrotra, S.: Efficient similarity string joins in large data sets. In: VLDB (2002)
10. Arasu, A., Ganti, V., Kaushik, R.: Efficient exact set-similarity joins. In: VLDB, pp. 918-929. ACM, New York (2006)
11. Sarawagi, S., Kirpal, A.: Efficient set joins on similarity predicates. In: SIGMOD, pp. 743-754. ACM, New York (2004)
12. Chaudhuri, S., Ganti, V., Kaushik, R.: A primitive operator for similarity joins in data cleaning. In: ICDE, p. 5. IEEE Computer Society, Los Alamitos (2006) (Pubitemid 44539797)
13. Xiao, C., Wang, W., Lin, X., Yu, J.X.: Efficient similarity joins for near duplicate detection. In: WWW, pp. 131-140. ACM, New York (2008)
14. Lin, J.: Brute force and indexed approaches to pairwise document similarity comparisons with mapreduce. In: SIGIR, pp. 155-162. ACM, New York (2009)
15. Hadoop: Apache Hadoop, http://hadoop.apache.org/