Large-Scale deduplication with constraints using dedupalog

Proceedings - International Conference on Data Engineering

Volumn , Issue , 2009, Pages 952-963

  Arasu, Arvind ^a   Ré, Christopher ^a   Suciu, Dan ^b  

^a MICROSOFT RESEARCH   (United States)

^b University of Washington   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DATA CLEANING; DEDUPLICATION; DOMAIN SPECIFIC; EFFICIENT ALGORITHM; HIGH QUALITY; LARGE DATASETS; PAPER REFERENCES; PROTOTYPE IMPLEMENTATIONS; REAL WORLD DATA;

SOFTWARE PROTOTYPING;

PAPER;

EID: 67649649597     PISSN: 10844627     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDE.2009.43     Document Type: Conference Paper

References (55)

1. S. Guha, R. Rastogi, and K. Shim, "Rock: A robust clustering algorithm for categorical attributes," Inf. Syst., vol. 25, no. 5, pp. 345 -366, 2000.
2. S. Chaudhuri, A. D. Sarma, V. Ganti, and R. Kaushik, "Leveraging aggregate constraints for deduplication," in SIGMOD Conference, 2007, pp. 437 -448.
3. W. W. Cohen, "Integration of heterogeneous databases without common domains using queries based on textual similarity." in SIGMOD Conference, 1998, pp. 201 -212.
4. A. K. H. Tung, R. T. Ng, L. V. S. Lakshmanan, and J. Han, "Constraintbased clustering in large databases," in ICDT, 2001, pp. 405 -419.
5. I. Bhattacharya and L. Getoor, "A latent dirichlet model for unsupervised entity resolution," in SDM, 2006.
6. W. Shen, X. Li, and A. Doan, "Constraint-based entity matching," in AAAI, 2005, pp. 862 -867.
7. H. Pasula, B. Marthi, B. Milch, S. J. Russell, and I. Shpitser, "Identity uncertainty and citation matching," in NIPS, 2002, pp. 1401 -1408.
8. P. Andritsos, P. Tsaparas, R. J. Miller, and K. C. Sevcik, "Limbo: Scalable clustering of categorical data," in EDBT, 2004, pp. 123 -146.
9. I. Davidson, K. Wagstaff, and S. Basu, "Measuring constraint-set utility for partitional clustering algorithms," in PKDD, 2006, pp. 115 -126.
10. K. Wagstaff, S. Basu, and I. Davidson, "When is constrained clustering beneficial, and why?" in AAAI, 2006.
11. R. Ananthakrishna, S. Chaudhuri, and V. Ganti, "Eliminating fuzzy duplicates in data warehouses," in VLDB, 2002, pp. 586 -597.
12. P. Domingos and M. Richardson, "Markov logic: A unifying framework for statistical relational learning," in ICML Workshop on Statistical Relational Learning, 2004, pp. 49 -54.
13. A. Pfeffer and D. Koller, "Semantics and inference for recursive probability models," in AAAI/IAAI, 2000, pp. 538 -544.
14. P. Singla and P. Domingos, "Entity resolution with markov logic," in ICDM, 2006, pp. 572 -582.
15. E. D. Demaine, D. Emanuel, A. Fiat, and N. Immorlica, "Correlation clustering in general weighted graphs," Theor. Comput. Sci., vol. 361, no. 2-3, pp. 172 -187, 2006.
16. [Online]. Available: http://www.cs.umass.edu/ mccallum/code-data.html
17. R. Reiter, "Equality and domain closure in first-order databases," J. ACM, vol. 27, no. 2, pp. 235 -249, 1980.
18. S. Sarawagi, A. Bhamidipaty, A. Kirpal, and C. Mouli, "Alias: An active learning led interactive deduplication system," in VLDB, 2002, pp. 1103 -1106.
19. M. Bilgic, L. Licamele, L. Getoor, and B. Shneiderman, "D-dupe: An interactive tool for entity resolution in social networks," in Graph Drawing, 2005, pp. 505 -507.
20. P. Singla and P. Domingos, "Multi-relational record linkage," in Proceedings of 3rd Workshop on Multi-Relational Data Mining at ACM SIGKDD, Seattle, WA, Aug. 2004.
21. I. Bhattacharya and L. Getoor, "Collective entity resolution in relational data," TKDD, vol. 1, no. 1, 2007.
22. P. Bohannon, W. Fan, F. Geerts, X. Jia, and A. Kementsietsidis, "Conditional functional dependencies for data cleaning," in ICDE, 2007, pp. 746 -755.
23. W. Fan, "Dependencies revisited for improving data quality," in PODS, 2008, pp. 159 -170.
24. N. Bansal, A. Blum, and S. Chawla, "Correlation clustering," Machine Learning, vol. 56, no. 1-3, pp. 89 -113, 2004.
25. M. Charikar, V. Guruswami, and A. Wirth, "Clustering with qualitative information," J. Comput. Syst. Sci., vol. 71, no. 3, pp. 360 -383, 2005.
26. N. Ailon, M. Charikar, and A. Newman, "Aggregating inconsistent information: ranking and clustering," in STOC, 2005, pp. 684 -693.
27. V. V. Vazirani, Approximation Algorithms. Springer-Verlag, 2003.
28. J. D. Ullman, Principles of Database and Knowledge-Base Systems, Volume I. Computer Science Press, 1988.
29. A. Arasu, V. Ganti, and R. Kaushik, "Efficient exact set-similarity joins," in VLDB, 2006, pp. 918 -929.
30. A. Gionis, H. Mannila, and P. Tsaparas, "Clustering aggregation," TKDD, vol. 1, no. 1, 2007.
31. C. Lund and M. Yannakakis, "On the hardness of approximating minimization problems," J. ACM, vol. 41, no. 5, pp. 960 -981, 1994.
32. A. K. Elmagarmid, P. G. Ipeirotis, and V. S. Verykios, "Duplicate record detection: A survey," IEEE Trans. Knowl. Data Eng., vol. 19, no. 1, pp.1 -16, 2007.
33. M. Bilenko and R. J. Mooney, "Adaptive duplicate detection using learnable string similarity measures," in KDD, 2003, pp. 39 -48.
34. S. Chaudhuri, K. Ganjam, V. Ganti, and R. Motwani, "Robust and efficient fuzzy match for online data cleaning," in SIGMOD Conference, 2003, pp. 313 -324.
35. L. Gravano, P. G. Ipeirotis, N. Koudas, and D. Srivastava, "Text joins in an rdbms for web data integration," in WWW, 2003, pp. 90 -101.
36. M. A. Jaro, "Unimatch: A record linkage system: Users manual," US Bureau of the Census, Washington, D.C., Tech. Rep., 1976.
37. I. P. Fellegi and A. B. Sunter, "A theory for record linkage," J. Am. Statistical Assoc., vol. 64, no. 328, pp. 1183 -1210, 1969.
38. S. Sarawagi and A. Bhamidipaty, "Interactive deduplication using active learning," in KDD, 2002, pp. 269 -278.
39. M. Hadjieleftheriou, A. Chandel, N. Koudas, and D. Srivastava, "Fast indexes and algorithms for set similarity selection queries," in ICDE, 2008, pp. 267 -276.
40. A. E. Monge and C. P. Elkan, "An efficient domain-independent algorithm for detecting approximately duplicate database records," in Proc. Second ACM SIGMOD Workshop Research Issues in Data Mining and Knowledge Discovery, 1997, pp. 23 -29.
41. S. Sarawagi and A. Kirpal, "Efficient set joins on similarity predicates," in SIGMOD Conference, 2004, pp. 743 -754.
42. P. Christen, "Febrl - a freely available record linkage system with a graphical user interface," in Proc. of the Australasian Workshop on Health Data and Knowledge Management, 2008, pp. 17 -25.
43. M. G. Elfeky, A. K. Elmagarmid, and V. S. Verykios, "Tailor: A record linkage tool box," in ICDE, 2002, pp. 17 -28.
44. N. Koudas, A. Marathe, and D. Srivastava, "Spider: flexible matching in databases," in SIGMOD Conference, 2005, pp. 876 -878.
45. S. Chaudhuri, V. Ganti, and R. Motwani, "Robust identification of fuzzy duplicates," in ICDE, 2005, pp. 865 -876.
46. H. Galhardas, D. Florescu, D. Shasha, E. Simon, and C.-A. Saita, "Declarative data cleaning: Language, model, and algorithms," in VLDB, 2001, pp. 371 -380.
47. V. Raman and J. M. Hellerstein, "Potter 's wheel: An interactive data cleaning system," in VLDB, 2001, pp. 381 -390.
48. X. Dong, A. Y. Halevy, and J. Madhavan, "Reference reconciliation in complex information spaces," in SIGMOD Conference, 2005, pp. 85 -96.
49. P. Singla and P. Domingos, "Entity resolution with markov logic," in ICDM, 2006, pp. 572 -582.
50. A. Culotta and A. McCallum, "Joint deduplication of multiple record types in relational data," in CIKM, 2005, pp. 257 -258.
51. A. Chandel, N. Koudas, K. Q. Pu, and D. Srivastava, "Fast identification of relational constraint violations," in ICDE, 2007, pp. 776 -785.
52. P. Bohannon, M. Flaster, W. Fan, and R. Rastogi, "A cost-based model and effective heuristic for repairing constraints by value modification," in SIGMOD Conference, 2005, pp. 143 -154.
53. G. Cong, W. Fan, F. Geerts, X. Jia, and S. Ma, "Improving data quality: Consistency and accuracy," in VLDB, 2007, pp. 315 -326.
54. A. Fuxman, E. Fazli, and R. J. Miller, "Conquer: Efficient management of inconsistent databases," in SIGMOD Conference, 2005, pp. 155 -166.
55. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. John Wiley & Sons, 2001.