Towards scalable real-time entity resolution using a similarity-aware inverted index approach

Conferences in Research and Practice in Information Technology Series

Volumn 87, Issue , 2008, Pages 51-60

  Christen, Peter ^a   Gayler, Ross ^b  

^a AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

^b Veda Advantage   (Australia)

Author keywords

Approximate string comparisons; Data matching; Record linkage; Scalability; Similarity measures.

Indexed keywords

APPROXIMATE MATCHING; DATA MATCHING; DATA SETS; DOCUMENT COLLECTION; INVERTED INDEXING; INVERTED INDICES; MATCHING SPEED; PERSONAL INFORMATION; REAL WORLD DATA; RECORD LINKAGE; RESOLUTION TECHNIQUES; SIMILARITY MEASURE; STANDARD BLOCKING; STRING COMPARISON; STRUCTURED DATABASE; VERY LARGE DATABASE;

DATA HANDLING; DATABASE SYSTEMS; INFORMATION RETRIEVAL; INFORMATION TECHNOLOGY; RESEARCH; SCALABILITY; SEARCH ENGINES; VIRTUAL REALITY; WEBSITES;

QUERY PROCESSING;

EID: 67650216370     PISSN: 14451336     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Article

References (27)

1. Aggarwal, C.C. & Yu, P.S. (2000), The IGrid index: Reversing the dimensionality curse for similarity in-dexing in high dimensional space, in 'ACM Inter-national Conference on Knowledge Discovery and Data Mining' (SIGKDD'00), Boston, pp. 119-129.
2. Aizawa, A. & Oyama, K. (2005), A fast linkage de-tection scheme for multi-source information inte-gration, in 'Web Information Retrieval and Inte-gration' (WIRI'05), Tokyo, pp. 30-39.
3. Baxter, R., Christen, P. & Churches, T. (2003), A comparison of fast blocking methods for record linkage, in 'ACM SIGKDD'03 Workshop on Data Cleaning, Record Linkage and Object Consolida-tion', Washington DC, pp. 25-27.
4. Bayardo, R.J., Ma, Y. & Srikant, R. (2007), Scaling up all pairs similarity search, in 'International Con-ference on World Wide Web' (WWW'07), Banff, Canada, pp. 131-140.
5. Bhattacharya, I. & Getoor, L. (2007), 'Query-time entity resolution', Journal of Artificial Intelligence Research, 30, 621-657.
6. Bilenko, M., Kamath, B, & Mooney, R.J. (2006), Adaptive blocking: Learning to scale up record linkage, in 'IEEE International Conference on Data Mining' (ICDM'06), Hong Kong, pp. 87-96.
7. Christen, P. (2006), A comparison of personal name matching: Techniques and practical issues, in 'Workshop on Mining Complex Data' (MCD'06), held at IEEE ICDM'06, Hong Kong.
8. Christen, P. (2008), Febrl - An open source data cleaning, deduplication and record linkage system with a graphical user interface, in 'ACM Inter-national Conference on Knowledge Discovery and Data Mining' (SIGKDD'08), Las Vegas, pp. 1065- 1068.
9. Christen, P. & Goiser, K. (2007), Quality and com-plexity measures for data linkage and deduplica-tion, in F. Guillet & H. Hamilton, eds, 'Qual-ity Measures in Data Mining', Springer Studies in Computational Intelligence, Vol. 43, pp. 127-151.
10. Cohen W.W., Ravikumar P. & Fienberg S.E. (2003), A comparison of string distance metrics for name-matching tasks, in 'IJCAI'03 Workshop on Infor-mation Integration on the Web' (IIWeb'03), Aca-pulco, pp. 73-78.
11. Cohen, W.W. & Richman, J. (2002), Learning to match and cluster large high-dimensional data sets for data integration, in 'ACM International Con-ference on Knowledge Discovery and Data Mining' (SIGKDD'02), Edmonton, pp. 475-480.
12. Conrad, J.G., Guo, X.S. & Schriber, C.P. (2003), On-line duplicate detection: Signature reliability in a dynamic retrieval environment, in 'ACM Confer-ence on Information and Knowledge Management' (CIKM'03), New Orleans, pp. 443-452.
13. Dong, X., Halevy, A., & Madhavan, J. (2005), Refer-ence reconciliation in complex information spaces, in 'ACM International Conference on Management of Data' (SIGMOD'05), Baltimore, pp. 85-96.
14. Elfeky, M.G., Verykios, V.S. & Elmagarmid, A.K. (2002), TAILOR: A record linkage toolbox, in 'International Conference on Data Engineering' (ICDE'02), San Jose, pp. 17-28.
15. Elmagarmid, A.K., Ipeirotis, P.G. & Verykios, V.S. (2007), 'Duplicate record detection: A survey', IEEE Transactions on Knowledge and Data Engi-neering (TKDE), 19(1), 1-16.
16. Fellegi, I.P. & Sunter, A.B. (1969), 'A theory for record linkage', Journal of the American Statisti-cal Society, 64(328), 1183-1210.
17. Hernandez, M.A. & Stolfo, S.J. (1995), The merge/purge problem for large databases, in 'ACM International Conference on Management of Data' (SIGMOD'95), San Jose, pp. 127-138.
18. Gu, L. & Baxter, R. (2006), Decision models for record linkage, in 'Selected Papers from AusDM', Springer LNCS 3755, pp. 146-160.
19. Jin, L., Li, C. & Mehrotra, S. (2003), Efficient record linkage in large data sets, in 'International Confer-ence on Database Systems for Advanced Applica-tions' (DASFAA'03), Tokyo, pp. 137-146.
20. Kalashnikov, D.V. & Mehrotra, S. (2006), 'Domain-independent data cleaning via analysis of entity-relationship graph', ACM Transactions on Database Systems (TODS), 31(2), 716-767.
21. Michelson, M. & Knoblock, C.A. (2006), Learning blocking schemes for record linkage, in 'National Conference on Artificial Intelligence' (AAAI'06), Boston.
22. Weis, M. & Naumann, F. (2007), 'Space and time scalability of duplicate detection in graph data', Technical report, University of Potsdam, Germany.
23. Winkler, W.E. (2006), 'Overview of record linkage and current research directions', Technical Report RR2006/02, US Bureau of the Census.
24. Witten, I.H, Moffat, A & Bell, T.C (1999), Managing Gigabytes: Compressing and Indexing Documents and Images, 2nd Ed. Morgan Kaufmann.
25. Yan, S., Lee, D., Kan, M.Y., & Giles, L.C. (2007), Adaptive sorted neighborhood methods for efficient record linkage, in 'ACM/IEEE-CS Joint Confer-ence on Digital Libraries' (JCDL'07'), Vancouver, pp. 185-194.
26. Yin, X., Han, J. & Yu, P.S. (2006), LinkClus: Effi-cient clustering via heterogeneous semantic links, in 'International Conference on Very Large Data Bases' (VLDB'06), Seoul, pp. 427-438.
27. Zobel, J. & Moffat, A. (2006), 'Inverted files for text search engines', ACM Computing Surveys (CSUR), 38(2).