Scalable iterative graph duplicate detection

IEEE Transactions on Knowledge and Data Engineering

Volumn 24, Issue 11, 2012, Pages 2094-2108

  Herschel, Melanie ^a   Naumann, Felix ^b   Szott, Sascha ^c   Taubert, Maik ^d  

^a UNIVERSITY OF TÜBINGEN   (Germany)

^b UNIVERSITY OF POTSDAM   (Germany)

^c ZUSE INSTITUTE BERLIN   (Germany)

^d Biotronik SE and Co KG ^*  (Germany)

Author keywords

data cleaning; data integration; Duplicate detection; entity resolution; parallelization; record linkage; scalability

Indexed keywords

DATA CLEANING; DATA INTEGRATION; DUPLICATE DETECTION; PARALLELIZATIONS; RECORD LINKAGE;

DATABASE SYSTEMS; SCALABILITY;

ITERATIVE METHODS;

EID: 84866951187     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2011.99     Document Type: Article

References (30)

1. A.K. Elmagarmid, P.G. Ipeirotis, and V.S. Verykios, "Duplicate Record Detection: A Survey," IEEE Trans. Knowledge Data Eng., vol. 19, no. 1, pp. 1-16, Jan. 2007.
2. E. Rahm and H.H. Do, "Data Cleaning: Problems and Current Approaches," IEEE Data Eng. Bull., vol. 23, no. 4, pp. 3-13, Dec. 2000.
3. A. Doan, Y. Lu, Y. Lee, and J. Han, "Object Matching for Information Integration: A Profiler-Based Approach," IEEE Intelligent Systems, vol. 18, no. 5, pp. 54-59, Sept. 2003.
4. X. Dong, A. Halevy, and J. Madhavan, "Reference Reconciliation in Complex Information Spaces," Proc. ACM SIGMOD Int'l Conf. Management of Data, 2005.
5. M. Weis and F. Naumann, "Detecting Duplicates in Complex XML Data," Proc. 22nd Int'l Conf. Data Eng. (ICDE), 2006.
6. I. Bhattacharya and L. Getoor, "Collective Entity Resolution in Relational Data," ACM Trans. Knowledge Discovery from Data, vol. 1, no. 1, pp. 1-36, Mar. 2007.
7. M. Herschel and F. Naumann, "Scaling Up Duplicate Detection in Graph Data," Proc. 17th ACM Conf. Information and Knowledge Management (CIKM) Conf., 2008.
8. P. Singla and P. Domingos, "Object Identification with Attribute-Mediated Dependences," Proc. European Conf. Principles and Practice of Knowledge Discovery in Databases (PKDD), 2005.
9. W. Shen, P. DeRose, L. Vu, A. Doan, and R. Ramakrishnan, "Source-Aware Entity Matching: A Compositional Approach," Proc. IEEE 23rd Int'l Conf. Data Eng. (ICDE), 2007.
10. D.V. Kalashnikov and S. Mehrotra, "Domain-Independent Data Cleaning via Analysis of Entity-relationship Graph," ACM Trans. Database Systems, vol. 31, no. 2, pp. 716-767, 2006.
11. Z. Chen, D.V. Kalashnikov, and S. Mehrotra, "Exploiting Relationships for Object Consolidation," Proc. Second Int'l Workshop Information Quality in Information Systems (IQIS), 2005.
12. X. Yin, J. Han, and P.S. Yu, "LinkClus: Efficient Clustering via Heterogeneous Semantic Links," Proc. 32nd Int'l Conf. Very Large Data Bases (VLDB), 2006.
13. A. Arasu, C. Ré, and D. Suciu, "Large-Scale Deduplication with Constraints Using Dedupalog," Proc. IEEE 25th Int'l Conf. Data Eng. (ICDE) Conf., 2009.
14. M. Tachibana and H. Garcia-Molina, "Joint Entity Resolution," technical report, ID 900, Stanford InfoLab, 2009.
15. R. Ananthakrishna, S. Chaudhuri, and V. Ganti, "Eliminating Fuzzy Duplicates n Data Warehouses," Proc. 28th Int'l Conf. Very Large Data Bases (VLDB), 2002.
16. M.A. Hernández and S.J. Stolfo, "The Merge/purge Problem for Large Databases," Proc. ACM SIGMOD Int'l Conf. Management of Data, 1995.
17. S. Puhlmann, M. Weis, and F. Naumann, "XML Duplicate Detection Using Sorted Neigborhoods," Proc. 10th Int'l Conf. Advances in Database Technology (EDBT), 2006.
18. M.J. Quinn and N. Deo, "Parallel Graph Algorithms," ACM Computing Survey, vol. 16, no. 3, pp. 319-348, 1984.
19. H. sik Kim and D. Lee, "Parallel Linkage," Proc. ACM Int'l Conf. Information and Knowledge Management (CIKM), 2007.
20. J. Dean and S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," Proc. Sixth Conf. Symp. Operating Systems Design & Implementation (OSDI), 2004.
21. M. Weis and F. Naumann, "DogmatiX Tracks Down Duplicates in XML," Proc. ACM SIGMOD Int'l Conf. Management of Data, 2005.
22. I. Bhattacharya and L. Getoor, "Iterative Record Linkage for Cleaning and Integration," Proc. Ninth ACM SIGMOD Workshop Research Issues in Data Mining and Knowledge Discovery (DMKD), 2004.
23. A.E. Monge and C.P. Elkan, "An Efficient Domain-independent Algorithm for Detecting Approximately Duplicate Database Records," Proc. SIGMOD Workshop Data Mining and Knowledge Discovery (DMKD), 1997.
24. M. Weis and F. Naumann, "Relationship-Based Duplicate Detection," Technical Report HU-IB-206, Humboldt Univ. Berlin, 2006.
25. M. Weis, F. Naumann, U. Jehle, J. Lufter, and H. Schuster, "Industry-Scale Duplicate Detection," Proc. Int'l Conf. Very Large Data Bases (VLDB), 2008.
26. M. Weis and F. Naumann, "Space and Time Scalability of Duplicate Detection in Graph Data," technical report, Nr. 25, Hasso-Plattner-Insitut Potsdam, 2008.
27. D. Milano, M. Scannapieco, and T. Catarci, "Structure Aware XML Object Identification," Proc. First Int'l Very Large Data Bases (VLDB) Workshop Clean Databases (CleanDB), 2006.
28. W.W. Cohen, P. Ravikumar, and S.E. Fienberg, "A Comparison of String Distance Metrics for Name-Matching Tasks," Proc. IJCAI Workshop Information Integration on the Web (IIWeb), pp. 73-78, 2003.
29. B.-W. On, N. Koudas, D. Lee, and D. Srivastava, "Group Linkage," Proc. IEEE 23rd Int'l Conf. Data Eng. (ICDE), 2007.
30. N. Reddy and J.R. Haritsa, "Analyzing Plan Diagrams of Database Query Optimizers," Proc.31st Int'l Conf. Very Large Data Bases (VLDB), 2005.