Simple and efficient algorithm for approximate dictionary matching

Coling 2010 - 23rd International Conference on Computational Linguistics, Proceedings of the Conference

Volumn 2, Issue , 2010, Pages 851-859

  Okazaki, Naoaki ^a   Tsujii, Jun'ichi ^b  

^a UNIVERSITY OF TOKYO   (Japan)

^b UNIVERSITY OF MANCHESTER   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

COSINE SIMILARITY; DATA SETS; DICTIONARY MATCHING; ENGLISH WORD; INVERTED LIST; LARGE-SCALE DATASETS; SCALABLE PERFORMANCE; SIMILARITY MEASURE; SIMPLE AND EFFICIENT ALGORITHMS;

ALGORITHMS; COMPUTATIONAL LINGUISTICS;

COMPUTATIONAL EFFICIENCY;

EID: 80053414333     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (25)

1. Andoni, Alexandr and Piotr Indyk. 2008. Nearoptimal hashing algorithms for approximate nearest neighbor in high dimensions. Communications of the ACM, 51(1):117-122.
2. Arasu, Arvind, Venkatesh Ganti, and Raghav Kaushik. 2006. Efficient exact set-similarity joins. In VLDB '06: Proceedings of the 32nd International Conference on Very Large Data Bases, pages 918-929.
3. Behm, Alexander, Shengyue Ji, Chen Li, and Jiaheng Lu. 2009. Space-constrained gram-based indexing for efficient approximate string search. In ICDE '09: Proceedings of the 2009 IEEE International Conference on Data Engineering, pages 604-615.
4. Bergsma, Shane and Grzegorz Kondrak. 2007. Alignment-based discriminative string similarity. In ACL '07: Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 656-663.
5. Bocek, Thomas, Ela Hunt, and Burkhard Stiller. 2007. Fast similarity search in large dictionaries. Technical Report ifi-2007.02, Department of Informatics (IFI), University of Zurich.
6. Chandel, Amit, P. C. Nagesh, and Sunita Sarawagi. 2006. Efficient batch top-k search for dictionarybased entity recognition. In ICDE '06: Proceedings of the 22nd International Conference on Data Engineering.
7. Charikar, Moses S. 2002. Similarity estimation techniques from rounding algorithms. In STOC '02: Proceedings of the thiry-fourth annual ACM symposium on Theory of computing, pages 380-388. (Pubitemid 35009398)
8. Chaudhuri, Surajit, Venkatesh Ganti, and Raghav Kaushik. 2006. A primitive operator for similarity joins in data cleaning. In ICDE '06: Proceedings of the 22nd International Conference on Data Engineering.
9. Cohen, William W., Pradeep Ravikumar, and Stephen E. Fienberg. 2003. A comparison of string distance metrics for name-matching tasks. In Proceedings of the IJCAI-2003 Workshop on Information Integration on the Web (IIWeb-03), pages 73-78.
10. Davis, Jason V., Brian Kulis, Prateek Jain, Suvrit Sra, and Inderjit S. Dhillon. 2007. Information-theoretic metric learning. In ICML '07: Proceedings of the 24th International Conference on Machine Learning, pages 209-216.
11. Gravano, Luis, Panagiotis G. Ipeirotis, H. V. Jagadish, Nick Koudas, S. Muthukrishnan, and Divesh Srivastava. 2001. Approximate string joins in a database (almost) for free. In VLDB '01: Proceedings of the 27th International Conference on Very Large Data Bases, pages 491-500.
12. Henzinger, Monika. 2006. Finding near-duplicate web pages: a large-scale evaluation of algorithms. In SIGIR '06: Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 284- 291.
13. Huynh, Trinh N. D., Wing-Kai Hon, Tak-Wah Lam, and Wing-Kin Sung. 2006. Approximate string matching using compressed suffix arrays. Theoretical Computer Science, 352(1-3):240-249. (Pubitemid 43243293)
14. Kim, Min-Soo, Kyu-Young Whang, Jae-Gil Lee, and Min-Jae Lee. 2005. n-Gram/2L: a space and time efficient two-level n-gram inverted index structure. In VLDB '05: Proceedings of the 31st International Conference on Very Large Data Bases, pages 325- 336. (Pubitemid 43991014)
15. Lee, Hongrae, Raymond T. Ng, and Kyuseok Shim. 2007. Extending q-grams to estimate selectivity of string matching with low edit distance. In VLDB '07: Proceedings of the 33rd International Conference on Very Large Data Bases, pages 195-206.
16. Li, Chen, Bin Wang, and Xiaochun Yang. 2007. Vgram: improving performance of approximate queries on string collections using variable-length grams. In VLDB '07: Proceedings of the 33rd International Conference on Very Large Data Bases, pages 303-314.
17. Li, Chen, Jiaheng Lu, and Yiming Lu. 2008. Efficient merging and filtering algorithms for approximate string searches. In ICDE '08: Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, pages 257-266.
18. Liu, Xuhui, Guoliang Li, Jianhua Feng, and Lizhu Zhou. 2008. Effective indices for efficient approximate string search and similarity join. InWAIM '08: Proceedings of the 2008 The Ninth International Conference on Web-Age Information Management, pages 127-134.
19. Manku, Gurmeet Singh, Arvind Jain, and Anish Das Sarma. 2007. Detecting near-duplicates for web crawling. In WWW '07: Proceedings of the 16th International Conference on World Wide Web, pages 141-150. (Pubitemid 47582246)
20. Navarro, Gonzalo and Ricardo Baeza-Yates. 1998. A practical q-gram index for text retrieval allowing errors. CLEI Electronic Journal, 1(2).
21. Ravichandran, Deepak, Patrick Pantel, and Eduard Hovy. 2005. Randomized algorithms and nlp: using locality sensitive hash function for high speed noun clustering. In ACL '05: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pages 622-629.
22. Sarawagi, Sunita and Alok Kirpal. 2004. Efficient set joins on similarity predicates. In SIGMOD '04: Proceedings of the 2004 ACM SIGMOD international conference on Management of data, pages 743-754.
23. Wang, Wei, Chuan Xiao, Xuemin Lin, and Chengqi Zhang. 2009. Efficient approximate entity extraction with edit distance constraints. In SIGMOD '09: Proceedings of the 35th SIGMOD International Conference on Management of Data, pages 759-770.
24. Winkler, William E. 1999. The state of record linkage and current research problems. Technical Report R99/04, Statistics of Income Division, Internal Revenue Service Publication.
25. Xiao, Chuan, Wei Wang, and Xuemin Lin. 2008. Ed- Join: an efficient algorithm for similarity joins with edit distance constraints. In VLDB '08: Proceedings of the 34th International Conference on Very Large Data Bases, pages 933-944.