Improving data quality by source analysis

Journal of Data and Information Quality

Volumn 2, Issue 4, 2012, Pages

  Müller, Heiko ^a   Freytag, Johann Christoph ^b   Leser, Ulf ^b  

^a CSIRO   (Australia)

^b HUMBOLDT UNIVERSITY   (Germany)

Author keywords

Conflict resolution; Data cleaning; Quality assessment; Semantic distance measure

Indexed keywords

ALTERNATIVE APPROACH; CONFLICT RESOLUTION; DATA CLEANING; DATA PATTERNS; DATA PROPERTIES; DATA QUALITY; DATA SETS; DATA SOURCE; DATA VALUES; EXPERT USERS; HIGH QUALITY; HOTSPOTS; INTEGRATED SYSTEMS; INTEGRITY CONSTRAINTS; NP COMPLETE; OVERLAPPING DATA; OVERLAPPING PATTERNS; POTENTIAL DEPENDENCY; QUALITY ASSESSMENT; QUALITY OF DATA; RULE MINING ALGORITHMS; SEMANTIC DISTANCE MEASURE; SOURCE ANALYSIS;

DATA MINING; DATA REDUCTION; SEMANTICS;

QUALITY CONTROL;

EID: 84859023521     PISSN: 19361955     EISSN: 19361963     Source Type: Journal    
DOI: 10.1145/2107536.2107538     Document Type: Article

References (63)

1. ABITEBOUL, S., CLUET, S.,MILO, T.,MOGILEVSKY, P., SIMON, J., AND ZOHAR, S. 1999. Tools for data translation and integration. IEEE Data Engin. Bull. 22, 1, 3-8.
2. AGRAWAL, R. AND SRIKANT, R. 1994. Fast algorithms for mining association rules in large databases. In Proceedings of the 20th International Conference on Very Large Data Bases (VLDB'94).
3. ARENAS, M., BERTOSSI, L., AND CHOMICKI, J. 1999. Consistent query answers in inconsistent databases. In Proceedings of the 18th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS'99).
4. BAUMGARTNER, W. A., COHEN, K. B., FOX, L. M., ACQUAAH-MENSAH, G., AND HUNTER, L. 2007. Manual curation is not sufficient for annotation of genomic databases. Bioinformatics 23, 13, i41-i48. (Pubitemid 47244384)
5. BAY, S. D. AND PAZZANI, M. J. 2001. Detecting group differences: Mining contrast sets. Data Min. Knowl. Discov. 5, 3, 213-246.
6. BERMAN, H. M.,WESTBROOK, J., FENG, Z.,GILLILAND, G., AND BHAT, T. N. E. A. 2000. The protein data bank. Nucl. Acids Res. 28, 1, 235-242. (Pubitemid 30047768)
7. BHAT, T. N., BOURNE, P., FENG, Z.,GILLILAND, G., AND JAIN, S. E. A. 2001. The PDB data uniformity project. Nucl. Acids Res. 29, 1, 214-218. (Pubitemid 32054451)
8. BLEIHOLDER, J. AND NAUMANN, F. 2005. Declarative data fusion-Syntax, semantics, and implementation. In Proceedings of the 9th East European Conference on Advances in Databases and Information Systems.
9. BLEIHOLDER, J. AND NAUMANN, F. 2006. Conflict handling strategies in an integrated information system. In Proceedings of the IJCAI Workshop on Information on the Web (IIWeb).
10. BLEIHOLDER, J. AND NAUMANN, F. 2008. Data fusion. ACM Comput. Surv. 41, 1, 1-41.
11. BOHANNON, P., FAN, W., FLASTER, M., AND RASTOGI, R. 2005. A cost-based model and effective heuristic for repairing constraints by value modification. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD'05).
12. BOUTSELAKIS, H., DIMITROPOULOS, D., FILLON, J., GOLOVIN, A., AND ET AL., K. H. 2003. E-msd: the european bioinformatics institute macromolecular structure database. Nucl. Acids Res. 31, 1, 458-462. (Pubitemid 36150430)
13. BRENNER, S. E. 1999. Errors in genome annotation. Trends Genet. 15, 4, 132-133. (Pubitemid 29155430)
14. BUNEMAN, P., CHENEY, J., TAN, W.-C., AND VANSUMMEREN, S. 2008. Curated databases. In Proceedings of the 27th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS'08).
15. BUNEMAN, P., KHANNA, S., AND TAN, W.-C. 2002. On propagation of deletions and annotations through views. In Proceedings of the PODS Conference. 150-158. (Pubitemid 35009490)
16. BURKS, C. 1999. Molecular biology database list. Nucl. Acids Res. 27, 1, 1-9. (Pubitemid 29209390)
17. CHAWATHE, S. S. AND GARCIA-MOLINA, H. 1997. Meaningful change detection in structured data. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD'97).
18. CHIANG, F. AND MILLER, R. J. 2008. Discovering data quality rules. Proc. VLDB Endow. 1, 1, 1166-1177.
19. CHOMICKI, J. AND MARCINKOWSKI, J. 2005. Minimal-change integrity maintenance using tuple deletions. Inf. Comput. 197, 1/2, 90-121. (Pubitemid 40330051)
20. CURWEN, V., EYRAS, E., ANDREWS, T. D., CLARKE, L., AND MONGIN, E. E. A. 2004. The ensembl automatic gene annotation system. Genome Res 14, 5, 942-950. (Pubitemid 38660514)
21. DADAM, P., LUM, V. Y., AND WERNER, H. D. In Proceedings of the 10th International Conference on Very Large Data Bases (VLDB'84).
22. DENNIS, C. AND GALLAGHER, R., EDS. 2002. The Human Genome. Palgrave, McMillan.
23. DONG, X. L., BERTI-EQUILLE, L., AND SRIVASTAVA, D. 2009. Integrating conflicting data: The role of source dependence. Proc. VLDB Endow. 2, 1, 550-561.
24. ELMAGARMID, A. K., IPEIROTIS, P. G., AND VERYKIOS, V. S. 2007. Duplicate record detection: A survey. IEEE Trans. Knowl. Data Engin. 19, 1, 1-16. (Pubitemid 44955773)
25. FAGIN, R., KOLAITIS, P. G.,MILLER, R. J., AND POPA, L. 2005. Data exchange: Semantics and query answering. Theor. Comput. Sci. 336, 1, 89-124.
26. FAN, W., GEERTS, F., JIA, X., AND KEMENTSIETSIDIS, A. 2008. Conditional functional dependencies for capturing data inconsistencies. ACM Trans. Datab. Syst. 33, 2.
27. FAN,W.,GEERTS, F.,LAKSHMANAN, L. V. S., AND XIONG,M. 2009. Discovering conditional functional dependencies. In Proceedings of the 25th International Conference on Data Engineering (ICDE'09).
28. FAN, W.,LU, H.,MADNICK, S. E., AND CHEUNG, D. 2001. Discovering and reconciling value conflicts for numerical data integration. Inf. Syst. 26, 8, 635-656. (Pubitemid 33046274)
29. GALHARDAS, H.,FLORESCU, D.,SHASHA, D.,SIMON, E., AND SAITA, C.-A. 2001. Declarative data cleaning: Language, model, and algorithms. In Proceedings of 27th International Conference on Very Large Data Bases (VLDB'01).
30. GALLAND, A., ABITEBOUL, S., MARIAN, A., AND SENELLART, P. 2010. Corroborating information from disagreeing views. In Proceedings of the 3rd ACM International Conference on Web Search and Data Mining (WSDM'10).
31. GALPERIN, M. Y. AND COCHRANE, G. R. 2009. Nucleic acids research annual database issue and the NAR online molecular biology database collection in 2009. Nucl. Acids Res. 37, suppl1, D1-4.
32. GOLAB, L., KARLOFF, H., KORN, F., SRIVASTAVA, D., AND YU, B. 2008. On generating near-optimal tableaux for conditional functional dependencies. Proc. VLDB Endow. 1, 1, 376-390.
33. HAMMING, R. 1950. Error detecting and error correcting codes. Bell Syst. Techn. J. 26, 2, 147-160.
34. HERNANDEZ, T. AND KAMBHAMPATI, S. 2004. Integration of biological sources: current systems and challenges ahead. SIGMOD Rec. 33, 3, 51-60.
35. HRYCEJ, T. AND HIPP, J. 2004. Outlier detection by rareness assumption. In 34. Jahrestagung der Gesellschaft für Informatik e.V. (GI).
36. INTERNATIONAL HUMAN GENOME SEQUENCING CONSORTIUM. 2004. Finishing the euchromatic sequence of the human genome. Nature 431, 7011, 931-945.
37. LABIO, W. AND GARCIA-MOLINA, H. 1996. Efficient snapshot differential algorithms for data warehousing. In Proceedings of the 22th International Conference on Very Large Data Bases (VLDB'96).
38. LAND, A. H. AND DOIG, A. G. 1960. An automatic method of solving discrete programming problems. Econometrica 28, 3, 497-520.
39. LENZERINI, M. 2002. Data integration: A theoretical perspective. In Proceedings of the 21st ACM SIGMODSIGACT- SIGART Symposium on Principles of Database Systems (PODS'02).
40. LEVENSHTEIN, V. I. 1966. Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady 10, 8, 707-710.
41. LIU, B., HSU, W., AND MA, Y. 1998. Integrating classification and association rule mining. In Proceedings of the 4th International Conference on Knowledge Discovery and Data Mining (KDD).
42. LOUIE, B., MORK, P., MARTÍN-SÁNCHEZ, F., HALEVY, A. Y., AND TARCZY-HORNOCH, P. 2007. Data integration and genomic medicine. J. Biomed. Inf. 40, 1, 5-16. (Pubitemid 44821009)
43. MALETIC, J. I. AND MARCUS, A. 2000. Data cleansing: Beyond integrity analysis. In Proceedings of the 5th Conference on Information Quality (IQ'00).
44. MICHAEL R. GAREY, D. S. J. 1979. Computers and Intractability. A Guide to the Theory of NP-Completeness. W.H. Freeman and Co.
45. MÜLLER, H., FREYTAG, J.-C., AND LESER, U. 2006. Describing differences between databases. In Proceedings of the 15th ACM International Conference on Information and Knowledge Management (CIKM'06).
46. MÜLLER, H., LESER, U., AND FREYTAG, J.-C. 2004. Mining for patterns in contradictory data. In Proceedings of the International Workshop on Information Quality in Information Systems (IQIS'04).
47. MÜLLER, H., NAUMANN, F., AND FREYTAG, J.-C. 2003. Data quality in genome databases. In Proceedings of the 8th International Conference on Information Quality (IQ 2003).
48. NAUMANN, F. AND HÄUSSLER, M. 2002. Declarative data merging with conflict resolution. In Proceedings of the 7th International Conference on Information Quality (IQ 2002).
49. PASQUIER, N.,BASTIDE, Y., TAOUIL, R., AND LAKHAL, L. 1999. Discovering frequent closed itemsets for association rules. In Proceedings of the 7th International Conference on Database Theory (ICDT'99).
50. RAHM, E. AND BERNSTEIN, P. A. 2001. A survey of approaches to automatic schema matching. VLDB J. 10, 4, 334-350. (Pubitemid 33570972)
51. RAHM, E. AND DO, H. H. 2000. Data cleaning: Problems and current approaches. IEEE Data Engin. Bull. 23, 4, 3-13.
52. RAMAN, V. AND HELLERSTEIN, J. M. 2001. Potter's wheel: An interactive data cleaning system. In Proceedings of 27th International Conference on Very Large Data Bases (VLDB'01).
53. STEIN, L. D. 2003. Integrating biological databases. Nat. Rev. Genet. 4, 5, 337-345. (Pubitemid 36538358)
54. TRISSL, S., ROTHER, K., MÜLLER, H., STEINKE, T., AND ET AL., I. K. 2005. Columba: An integrated database of proteins, structures, and annotations. BMC Bioinformatics 6, 81.
55. VENTER, J. C., ADAMS, M. D.,MYERS, E. W., LI, P. W., AND MURAL, R. J. E. A. 2001. The sequence of the human genome. Science 291, 5507, 1304-1351. (Pubitemid 32173090)
56. VOSSEN, G. 1991. Data Models, Database Languages and Database Management Systems. Addison-Wesley Longman Publishing Co.
57. WEBB, G. I., BUTLER, S., AND NEWLANDS, D. 2003. On detecting differences between groups. In Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'03).
58. WIJSEN, J. 2002. Condensed representation of database repairs for consistent query answering. In Proceedings of the 9th International Conference on Database Theory (ICDT'03).
59. WINKLER, W. E. 1999. The state of record linkage and current research problems. Tech. rep. RR/1999/04, Statistics Research Division, U.S. Bureau of the Census.
60. XIONG, H., TAN, P.-N., AND KUMAR, V. 2003. Mining strong affinity association patterns in data sets with skewed support distribution. In Proceedings of the 3rd IEEE International Conference on Data Mining (ICDM'03).
61. YIN, X., HAN, J., AND YU, P. S. 2007. Truth discovery with multiple conflicting information providers on the web. In Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'07).
62. ZAKI, M.J. AND HSIAO, C.-J. 2002. Charm:An efficient algorithm for closed itemset mining. In Proceedings of the 2nd SIAM International Conference on Data Mining.
63. ZEHETNER, G. AND LEHRACH, H. 1994. The reference library system sharing biological material and experimental data. Nature 367, 489-491.