Creating probabilistic databases from duplicated data

VLDB Journal

Volumn 18, Issue 5, 2009, Pages 1141-1166

  Hassanzadeh, Oktie ^a   Miller, Renée J ^a  

^a UNIVERSITY OF TORONTO   (Canada)

Author keywords

Duplicate detection; Probabilistic databases; String databases

Indexed keywords

ALTERNATIVE APPROACH; CLEANING STRATEGIES; DEDUPLICATION; DUPLICATE DETECTION; IMPERFECT DATA; JOIN METHOD; MODULAR FRAMEWORK; PROBABILISTIC DATABASE; PROBABILISTIC DATABASES; QUERY RESULTS; REAL-WORLD ENTITIES; STRING DATABASES; THRESHOLDING TECHNIQUES;

CLUSTERING ALGORITHMS; PROBABILITY;

DATABASE SYSTEMS;

EID: 70349826301     PISSN: 10668888     EISSN: 0949877X     Source Type: Journal    
DOI: 10.1007/s00778-009-0161-2     Document Type: Article

References (51)

1. Ailon, N., Charikar, M., Newman, A.: Aggregating inconsistent information: ranking and clustering. In: ACM Symp. on Theory of Computing (STOC), pp. 684-693 (2005)
2. Andritsos, P., Fuxman, A., Miller, R.J.: Clean answers over dirty databases: a probabilistic approach. In: IEEE Proc. of the Int'l Conf. on Data Eng., p. 30 (2006)
3. Antova, L., Koch, C., Olteanu, D.: Fast and simple relational processing of uncertain data. In: IEEE Proc. of the Int'l Conf. on Data Eng., pp. 983-992 (2008)
4. Arasu, A., Ganti, V., Kaushik, R.: Efficient exact set-similarity joins. In: Proc. of the Int'l Conf. on Very Large Data Bases (VLDB), pp. 918-929 (2006)
5. Arasu, A., Ré, C., Suciu, D.: Large-scale deduplication with constraints using dedupalog. In: IEEE Proc. of the Int'l Conf. on Data Eng., pp. 952-963 (2009)
6. J.A. Aslam E. Pelekhov D. Rus 2004 The star clustering algorithm for static and dynamic information organization J. Graph Algorithm. Appl. 8 1 95 129 1068.68098 2112265
7. N. Bansal A. Blum S. Chawla 2004 Correlation clustering Mach. Learn. 56 1-3 89 113 1089.68085 10.1023/B:MACH.0000033116.57574.95
8. Bayardo, R.J., Ma, Y., Srikant, R.: Scaling up all pairs similarity search. In: Int'l World Wide Web Conference (WWW), pp. 131-140, Banff (2007)
9. O. Benjelloun H. Garcia-Molina D. Menestrina Q. Su S.E. Whang J. Widom 2009 Swoosh: a generic approach to entity resolution Int. J. Very Large Data Bases 18 1 255 276 10.1007/s00778-008-0098-x
10. Beskales, G., Soliman, M.A., Ilyas, I.F., Ben-David, S.: Modeling and querying possible repairs in duplicate detection. In: Proc. of the Int'l Conf. on Very Large Data Bases (VLDB), 2009 (To Appear). Available as University of Waterloo, Tech. Report CS-2009-15, 2009
11. Bezdek J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Kluwer Academic Publishers, Dordrecht (1981)
12. Bhattacharya, I., Getoor, L.: A latent dirichlet model for unsupervised entity resolution. In: Proc. of the SIAM International Conference on Data Mining (SDM), pp. 47-58, Bethesda (2006)
13. I. Bhattacharya L. Getoor 2006 Collective entity resolution in relational data IEEE Data Eng. Bull. 29 2 4 12
14. D.M. Blei A.Y. Ng M.I. Jordan 2003 Latent dirichlet allocation J. Mach. Learn. Res. 3 993 1022 1112.68379 10.1162/jmlr.2003.3.4-5.993
15. Boulos, J., Dalvi, N., Mandhani, B., Mathur, S., Re, C., Suciu, D.: MYSTIQ: a system for finding more answers by using probabilities. In: ACM SIGMOD Int'l Conf. on the Mgmt. of Data, pp. 891-893 (2005)
16. Chaudhuri, S., Ganjam, K., Ganti, V., Motwani, R.: Robust and efficient fuzzy match for online data cleaning. In: ACM SIGMOD Int'l Conf. on the Mgmt. of Data, pp. 313-324 (2003)
17. Chaudhuri, S., Ganti, V., Motwani, R.: Robust identification of fuzzy duplicates. In: IEEE Proc. of the Int'l Conf. on Data Eng., pp. 865-876, Washington (2005)
18. Chaudhuri, S., Das Sarma, A., Ganti, V., Kaushik, R.: Leveraging aggregate constraints for deduplication. In: ACM SIGMOD Int'l Conf. on the Mgmt. of Data, pp. 437-448 (2007)
19. R. Cheng J. Chen X. Xie 2008 Cleaning uncertain data with quality guarantees Proc. VLDB Endow. (PVLDB) 1 1 722 735
20. Cohen, W.W., Ravikumar, P., Fienberg, S.E.: A comparison of string distance metrics for name-matching tasks. In: Proc. of IJCAI-03 Workshop on Information Integration on the Web (IIWeb-03), pp. 73-78, Acapulco (2003)
21. N. Dalvi D. Suciu 2007 Efficient query evaluation on probabilistic databases Int. J. Very Large Data Bases 16 4 523 544 10.1007/s00778-006-0004-3
22. Dalvi, N., Suciu, D.: Management of probabilistic data: foundations and challenges. In: ACM SIGMOD Int'l Conf. on the Mgmt. of Data, pp. 1-12 (2007)
23. E.D. Demaine D. Emanuel A. Fiat N. Immorlica 2006 Correlation clustering in general weighted graphs Theor. Comput. Sci. 361 2 172 187 1099.68074 10.1016/j.tcs.2006.05.008 2252576 (Pubitemid 44189103)
24. Dong, X.L., Halevy, A.Y., Yu, C.: Data integration with uncertainty. In: Proc. of the Int'l Conf. on Very Large Data Bases (VLDB), pp. 687-698 (2007)
25. A.K. Elmagarmid P.G. Ipeirotis V.S. Verykios 2007 Duplicate record detection: a survey IEEE Trans. Know. Data Eng. 19 1 1 16 10.1109/TKDE.2007. 250581 (Pubitemid 44955773)
26. I.P. Fellegi A.B. Sunter 1969 A theory for record linkage J. Am. Stat. Assoc. 64 328 1183 1210 10.2307/2286061
27. G.W. Flake R.E. Tarjan K. Tsioutsiouliklis 2004 Graph clustering and minimum cut trees Internet Math. 1 4 385 408 1098.68095 2119992
28. Gravano, L., Ipeirotis, P.G., Jagadish, H.V., Koudas, N., Muthukrishnan, S., Srivastava, D.: Approximate string joins in a database (Almost) for free. In: Proc. of the Int'l Conf. on Very Large Data Bases (VLDB), pp. 491-500 (2001)
29. Gupta, R., Sarawagi, S.: Creating probabilistic databases from information extraction models. In: Proc. of the Int'l Conf. on Very Large Data Bases (VLDB), pp. 965-976 (2006)
30. Gusfield, D.: Algorithms on strings, trees, and sequences. In: Computer Science and Computational Biology. Cambridge University Press, Cambridge (1997)
31. Hassanzadeh, O.: Benchmarking Declarative Approximate Selection Predicates. Master's thesis, University of Toronto, February 2007
32. Hassanzadeh, O., Chiang, F., Lee, H.C., Miller, R.J.: Framework for evaluating clustering algorithms in duplicate detection. In: Proc. of the Int'l Conf. on Very Large Data Bases (VLDB), 2009
33. Hassanzadeh, O., Sadoghi, M., Miller, R.J.: Accuracy of approximate string joins using grams. In: Proc. of the International Workshop on Quality in Databases (QDB), pp. 11-18, Vienna (2007)
34. Haveliwala, T.H., Gionis, A., Indyk, P.: Scalable techniques for clustering the web. In: Proc. of the Int'l Workshop on the Web and Databases (WebDB), pp. 129-134, Dallas (2000)
35. M.A. Hernández S.J. Stolfo 1998 Real-world data is dirty: data cleansing and the merge/purge problem Data Min. Know. Discov. 2 1 9 37 10.1023/A:1009761603038 (Pubitemid 128063205)
36. Indyk, P., Motwani, R., Raghavan, P., Vempala, S.: Locality-preserving hashing in multidimensional spaces. In: ACM Symp. on Theory of Computing (STOC), pp. 618-625 (1997)
37. K. Kukich 1992 Techniques for automatically correcting words in text ACM Comput. Surv. 24 4 377 439 10.1145/146370.146380 (Pubitemid 23687641)
38. Li, C., Wang, B., Yang, X.: VGRAM: Improving performance of approximate queries on string collections using variable-length grams. In: Proc. of the Int'l Conf. on Very Large Data Bases (VLDB), pp. 303-314, Vienna (2007)
39. McCallum, A., Nigam, K., Ungar, L.H.: Efficient clustering of high-dimensional data sets with application to reference matching. In: Proc. of the Int'l Conf. on Knowledge Discovery & Data Mining, pp. 169-178 (2000)
40. Miller, D.R.H., Leek, T., Schwartz, R.M.: A hidden Markov model information retrieval system. In: ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 214-221 (1999)
41. Monge, A.E., Elkan, C.: An efficient domain-independent algorithm for detecting approximately duplicate database records. In: Proc. of SIGMOD Workshop on Data Mining and Knowledge Discovery (DMKD) (1997)
42. E. Rahm H. Hai Do 2000 Data cleaning: problems and current approaches IEEE Data Eng. Bull. 23 4 3 13
43. Re, C., Dalvi, N., Suciu, D.: Efficient Top-k query evaluation on probabilistic data. In: IEEE Proc. of the Int'l Conf. on Data Eng., pp. 886-895 (2007)
44. S. Robertson 2004 Understanding inverse document frequency: on theoretical arguments for IDF J. Doc. 60 5 503 520 10.1108/00220410410560582 (Pubitemid 39538229)
45. Sarawagi, S., Kirpal, A.: Efficient set joins on similarity predicates. In: ACM SIGMOD Int'l Conf. on the Mgmt. of Data, pp. 743-754, Paris (2004)
46. Sen, P., Deshpande, A., Getoor, L.: Representing tuple and attribute uncertainty in probabilistic databases. In: ICDM Workshops, pp. 507-512 (2007)
47. Slonim, N.: The Information Bottleneck: Theory and Applications. PhD thesis, The Hebrew University (2003)
48. Soliman, M.A., Ilyas, I.F., Chang, K.C.: Top-k query processing in uncertain databases. In: IEEE Proc. of the Int'l Conf. on Data Eng., pp. 896-905 (2007)
49. M.A. Soliman I.F. Ilyas K.C. Chang 2008 Probabilistic top-k and ranking-aggregate queries ACM Trans. Database Syst. (TODS) 33 3 1 54 10.1145/1386118.1386119
50. van Dongen, S.: Graph Clustering By Flow Simulation. PhD thesis, University of Utrecht (2000)
51. Widom J.: Trio: a system for integrated management of data, accuracy, and lineage. In: Proc. of the Conference on Innovative Data Systems Research (CIDR), pp. 262-276 (2005)