Trie-join: A trie-based method for efficient string similarity joins

VLDB Journal

Volumn 21, Issue 4, 2012, Pages 437-461

  Feng, Jianhua ^a   Wang, Jiannan ^a   Li, Guoliang ^a  

^a TSINGHUA UNIVERSITY   (China)

Author keywords

Data integration and cleaning; Edit distance; String similarity joins; Subtrie pruning; Trie index

Indexed keywords

DATA INTEGRATION; EDIT DISTANCE; STRING SIMILARITY; SUBTRIE PRUNING; TRIE INDEX;

HARDWARE; INFORMATION SYSTEMS;

ALGORITHMS;

EID: 84864286297     PISSN: 10668888     EISSN: 0949877X     Source Type: Journal    
DOI: 10.1007/s00778-011-0252-8     Document Type: Article

References (59)

1. http://secondstring. sourceforge. net/.
2. http://www. dcs. shef. ac. uk/~sam/simmetrics. html.
3. Agrawal S., Chakrabarti K., Chaudhuri S., Ganti V.: Scalable ad-hoc entity extraction from text collections. PVLDB 1(1), 945-957 (2008).
4. Arasu, A., Chaudhuri, S., Kaushik, R.: Transformation-based framework for record matching. In: ICDE, pp. 40-49 (2008).
5. Arasu, A., Ganti, V., Kaushik, R.: Efficient exact set-similarity joins. In: VLDB, pp. 918-929 (2006).
6. Augsten, N., Böhlen, M. H., Dyreson, C. E., Gamper, J.: Approximate joins for data-centric xml. In: ICDE, pp. 814-823 (2008).
7. Bayardo, R. J., Ma, Y., Srikant, R.: Scaling up all pairs similarity search. In: WWW, pp. 131-140 (2007).
8. Bryan, B., Eberhardt, F., Faloutsos, C.: Compact similarity joins. In: ICDE, pp. 346-355 (2008).
9. Celikik, M., Bast, H.: Fast error-tolerant search on very large texts. In: SAC, pp. 1724-1731 (2009).
10. Chakrabarti, K., Chaudhuri, S., Ganti, V., Xin, D.: An efficient filter for approximate membership checking. In: SIGMOD Conference, pp. 805-818 (2008).
11. Chaudhuri, S., Ganjam, K., Ganti, V., Motwani, R.: Robust and efficient fuzzy match for online data cleaning. In: SIGMOD Conference, pp. 313-324 (2003).
12. Chaudhuri S., Ganti V., Kaushik R.: Data debugger: An operator-centric approach for data quality solutions. IEEE Data Eng. Bull. 29(2), 60-66 (2006).
13. Chaudhuri, S., Ganti, V., Kaushik, R.: A primitive operator for similarity joins in data cleaning. In: ICDE, pp. 5-16 (2006).
14. Chaudhuri, S., Kaushik, R.: Extending autocompletion to tolerate errors. In: SIGMOD Conference, pp. 707-718 (2009).
15. Ciaccia, P., Patella, M., Zezula, P.: M-tree: An efficient access method for similarity search in metric spaces. In: VLDB, pp. 426-435 (1997).
16. Cole, R., Gottlieb, L.-A., Lewenstein, M.: Dictionary matching and indexing with errors and don't cares. In: STOC, pp. 91-100 (2004).
17. Fredkin E.: Trie memory. Commun. ACM 3(9), 490-499 (1960).
18. Gonnet G. H.: Handbook of Algorithms and Data structures. Addison-Wesley, Reading (1984).
19. Gravano, L., Ipeirotis, P. G., Jagadish, H. V., Koudas, N., Muthukrishnan, S., Srivastava, D.: Approximate string joins in a database (almost) for free. In: VLDB, pp. 491-500 (2001).
20. Guha, S., Koudas, N., Srivastava, D., Yu, T.: Index-based approximate xml joins. In: ICDE, pp. 708-710 (2003).
21. Hadjieleftheriou, M., Chandel, A., Koudas, N., Srivastava, D.: Fast indexes and algorithms for set similarity selection queries. In: ICDE, pp. 267-276 (2008).
22. Hadjieleftheriou, M., Koudas, N., Srivastava, D.: Incremental maintenance of length normalized indexes for approximate string matching. In: SIGMOD Conference, pp. 429-440 (2009).
23. Hadjieleftheriou M., Srivastava D.: Weighted set-based string similarity. IEEE Data Eng. Bull. 33(1), 25-36 (2010).
24. Hadjieleftheriou M., Yu X., Koudas N., Srivastava D.: Hashed samples: selectivity estimators for set similarity selection queries. PVLDB 1(1), 201-212 (2008).
25. Heinz S., Zobel J., Williams H. E.: Burst tries: a fast, efficient data structure for string keys. ACM Trans. Inf. Syst. 20(2), 192-223 (2002).
26. Jaro, M. A. Unimatch: A record linkage system: User's manual. Technical report, U. S. Bureau of the Census, Washington, D. C., (1976).
27. Jestes, J., Li, F., Yan, Z., Yi, K.: Probabilistic string similarity joins. In: SIGMOD Conference, pp. 327-338 (2010).
28. Ji, S., Li, G., Li, C., Feng, J.: Efficient interactive fuzzy keyword search. In WWW, pp. 433-439 (2009).
29. Kahveci, T., Singh, A. K.: Efficient index structures for string databases. In: VLDB, pp. 351-360 (2001).
30. Kim, M.-S., Whang, K.-Y., Lee, J.-G., Lee, M.-J. n-Gram/2L: A space and time efficient two-level n-gram inverted index structure. In: VLDB, pp. 325-336 (2005).
31. Knuth D. E.: The Art of Computer Programming, Volume 1: Fundamental algorithms. Addison-Wesley, Reading (1968).
32. Lee, H., Ng, R. T., Shim, K.: Extending q-grams to estimate selectivity of string matching with low edit distance. In: VLDB, pp. 195-206 (2007).
33. Lee H., Ng R. T., Shim K.: Power-law based estimation of set similarity join size. PVLDB 2(1), 658-669 (2009).
34. Li, C., Lu, J., Lu, Y.: Efficient merging and filtering algorithms for approximate string searches. In: ICDE, pp. 257-266 (2008).
35. Li, C., Wang, B., Yang, X. VGRAM: Improving performance of approximate queries on string collections using variable-length grams. In: VLDB, pp. 303-314 (2007).
36. Li, G., Deng, D., Feng, J. Faerie: efficient filtering algorithms for approximate dictionary-based entity extraction. In: SIGMOD Conference, pp. 529-540 (2011).
37. Li G., Ji S., Li C., Feng J.: Efficient fuzzy full-text type-ahead search. VLDB J. 20(4), 617-640 (2011).
38. Lian X., Chen L.: Set similarity join on probabilistic data. PVLDB 3(1), 650-659 (2010).
39. Lu, J., Han, J., Meng, X.: Efficient algorithms for approximate member extraction using signature-based inverted lists. In: CIKM, pp. 315-324 (2009).
40. Morrison D. R.: Patricia: practical algorithm to retrieve information coded in alphanumeric. J. ACM 15, 514-534 (1968).
41. Navarro G.: A guided tour to approximate string matching. ACM Comput. Surv. 33(1), 31-88 (2001).
42. Nilsson S., Karlsson G.: Ip-address lookup using lc-tries. IEEE J. Selected Areas Commun. 17, 1083-1092 (1999).
43. Peterson J. L.: Computer programs for detecting and correcting spelling errors. Commun. ACM 23(12), 676-687 (1980).
44. Russell, R. C.: Available at http://patft. uspto. gov/netacgi/nph-Parser?patentnumber=1261167 (1918).
45. Sahinalp, S. C., Tasan, M., Macker, J., Özsoyoglu, Z. M.: Distance based indexing for string proximity search. In: ICDE, pp. 125-136 (2003).
46. Sakoe H., Chiba S.: Dynamic programming algorithm optimization for spoken word recognition. IEEE Trans. Acoust Speech Signal Process 26, 43-49 (1978).
47. Salton G.: Introduction to Modern Information Retrieval. McGraw Hill, NY (1987).
48. Sarawagi, S., Kirpal, A.: Efficient set joins on similarity predicates. In: SIGMOD Conference, pp. 743-754 (2004).
49. Schulz K. U., Mihov S.: Fast string correction with levenshtein automata. Intl J Doc Anal Recognit 5(1), 67-85 (2002).
50. Sussenguth E. H.: Use of tree structures for processing files. Commun. ACM 6, 272-279 (1963).
51. Vernica, R., Carey, M. J., Li, C.: Efficient parallel set-similarity joins using mapreduce. In: SIGMOD Conference, pp. 495-506 (2010).
52. Wang J., Li G., Feng J.: Trie-join: Efficient trie-based string similarity joins with edit-distance constraints. PVLDB 3(1), 1219-1230 (2010).
53. Wang, J., Li, G., Feng, J.: Fast-join: An efficient method for fuzzy token matching based string similarity join. In: ICDE pp. 458-469 (2011).
54. Wang J., Li G., Yu J. X., Feng J.: Entity matching: how similar is similar. PVLDB 4(10), 622-633 (2011).
55. Wang, W., Xiao, C., Lin, X., Zhang, C.: Efficient approximate entity extraction with edit distance constraints. In: SIGMOD Conference, pp. 759-770 (2009).
56. Xiao C., Wang W., Lin X.: Ed-join: an efficient algorithm for similarity joins with edit distance constraints. PVLDB 1(1), 933-944 (2008).
57. Xiao, C., Wang, W., Lin, X., Shang, H.: Top-k set similarity joins. In: ICDE, pp. 916-927 (2009).
58. Xiao, C., Wang, W., Lin, X., Yu, J. X.: Efficient similarity joins for near duplicate detection. In: WWW, pp. 131-140 (2008).
59. Yang, X., Wang, B., Li, C.: Cost-based variable-length-gram selection for string collections to support approximate queries efficiently. In: SIGMOD Conference, pp. 353-364 (2008).