Data quality: The other face of Big Data

Proceedings - International Conference on Data Engineering

Volumn , Issue , 2014, Pages 1294-1297

  Saha, Barna ^a   Srivastava, Divesh ^a  

^a AT AND T LABS RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

QUALITY MANAGEMENT; REPAIR; SEMANTICS;

DATA QUALITY; DATA QUALITY MANAGEMENT; ERRONEOUS DATUM; LARGE DATABASE; QUALITY DATA; QUALITY ISSUES;

BIG DATA;

EID: 84901781647     PISSN: 10844627     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDE.2014.6816764     Document Type: Conference Paper

References (70)

1. U. Boobna and M. de Rougemont: Correctors for XML data. XSym 2004: 97-111.
2. P. Bohannon, M. Flaster, W. Fan and R. Rastogi: A cost-based model and effective heuristic for repairing constraints by value modification. SIGMOD 2005: 143-154.
3. P. Bohannon, W. Fan, F. Geerts, X. Jia and A. Kementsietsidis: Conditional functional dependencies for data cleaning. ICDE 2007: 746-755.
4. L. Bravo, W. Fan and S. Ma: Extending dependencies with conditions, VLDB 2007: 243-254.
5. G. Beskales, I. F. Ilyas and L. Golab: Sampling the repairs of functional dependency violations under hard constraints. PVLDB 3(1): 197-207 (2010).
6. G. Beskales, I. F. Ilyas, L. Golab and A. Galiullin: On the relative trust between inconsistent data and inaccurate constraints. ICDE 2013: 541-552.
7. L. Bertossi, S. Kolahi and Laks V.S. Lakshmanan: Data cleaning and query answering with matching dependencies and matching functions. ICDT 2011: 268-279.
8. G. J. Bex, F. Neven, T. Schwentick and K. Tuyls: Inference of concise DTDs from XML data. VLDB 2006: 115-126.
9. G. J. Bex, F. Neven and S. Vansummeren: Inferring XML schema definitions from XML data. VLDB 2007: 998-1009.
10. K. Chen, H. Chen, N. Conway, J. M. Hellerstein and T. S. Parikh: Usher: improving data quality with dynamic forms. IEEE Trans. Knowl. Data Eng. 23(8): 1138-1153 (2011).
11. G. Cong, W. Fan, F. Geerts, X. Jia and S. Ma: Improving data quality: consistency and accuracy. VLDB 2007: 315326.
12. Y. Cao, W. Fan and W. Yu: Determining the relative accuracy of attributes. SIGMOD 2013: 565-576.
13. G. Cormode, L. Golab, F. Korn, A. McGregor, D. Srivastava, X. Zhang: Estimating the confidence of conditional functional dependencies. SIG-MOD 2009: 469-482.
14. X. Chu, I. F. Ilyas and P. Papotti: Holistic data cleaning: putting violations into context. ICDE 2013: 458-469.
15. X. Chu, I.F. Ilyas and P. Papotti: Discovering Denial Constraints. PVLDB 6(13): 1498-1509 (2013).
16. F. Chiang and R. J. Miller: Discovering data quality rules. PVLDB 1(1): 1166-1177 (2008).
17. F. Chiang and R. J. Miller: A unified model for data and constraint repair. ICDE 2011: 446-457.
18. M. Dallachiesa, A. Ebaid, A. Eldawy, A. K. Elmagarmid, I. F. Ilyas, M. Ouzzani and N. Tang: NADEEF: a commodity data cleaning system, SIGMOD 2013: 541-552.
19. T. Dasu, T. Johnson, S. Muthukrishnan and V. Shkapenyuk: Mining database structure; or, how to build a data quality browser, SIGMOD 2002: 240-251.
20. W. W. Eckerson: Data quality and the bottom line: achieving business success through a commitment to high quality data. Data Warehousing Institute, 2002.
21. L. Berti-Equille, T. Dasu and D. Srivastava: Discovery of complex glitch patterns: a novel approach to quantitative data cleaning. ICDE 2011: 733-744.
22. T. Eiter, M. Fink, G. Greco and D. Lembo: Repair localization for query answering from inconsistent databases. TODS 33(2) (2008).
23. W. Fan: Data quality: theory and practice. WAIM 2012: 1-16.
24. A. Fuxman, E. Fazli and R. J. Miller: ConQuer: efficient management of inconsistent databases. SIGMOD 2005: 155-166.
25. W. Fan and F. Geerts: Capturing missing tuples and missing values. PODS 2010: 169-178.
26. W. Fan and F. Geerts: Foundations of data quality management. Morgan & Claypool 2012.
27. W. Fan, F. Geerts and X. Jia: Semandaq: a data quality system based on conditional functional dependencies. PVLDB 1(2) 1460-1463 (2008).
28. W. Fan, F. Geerts and X. Jia: A revival of integrity constraints for data cleaning. PVLDB 1(2): 1522-1523 (2008).
29. N. Khoussainova, M. Balazinska and D. Suciu: Towards correcting input data errors probabilistically using integrity constraints. MobiDE 2006: 43-50.
30. W. Fan, F. Geerts, X. Jia and A. Kementsietsidis: Conditional functional dependencies for capturing data inconsistencies. ACM TODS 33(2) (2008).
31. W. Fan, F. Geerts, J. Li and M. Xiong: Discovering conditional functional dependencies. IEEE Trans. Knowl. Data Eng. 23(5): 683-698 (2011).
32. W. Fan, F. Geerts, S. Ma and H. Müller: Detecting inconsistencies in distributed data: ICDE 2010: 64-75.
33. W. Fan, F. Geerts, N. Tang and W. Yu: Inferring data currency and consistency for conflict resolution. ICDE 2013: 470-481.
34. W. Fan, F. Geerts and J. Wijsen: Determining the currency of data. ACM Trans. Database Syst. 37(4): 25 (2012).
35. W. Fan, J. Li, S. Ma, N. Tang and W. Yu: Towards certain xes with editing rules and master data. PVLDB 3(1) 173184 (2010).
36. W. Fan, J. Li, S. Ma, N. Tang and W. Yu: Interaction between record matching and data repairing. SIGMOD 2011: 469-480.
37. W. Fan, J. Li, S. Ma, N. Tang and W. Yu: CerFix: a system for cleaning data with certain fixes. PVLDB 4(12) 1375-1378 (2011).
38. W. Fan, J. Li, N. Tang and W. Yu: Incremental detection of inconsistencies in distributed data. ICDE 2012: 318-329.
39. Gartner: Forecast: Data quality tools worldwide, 2006-2011. Technical report, Gartner, 2007.
40. H. Galhardas, D. Florescu, D. Shasha, E. Simon and C. Saita: Declarative data cleaning: language, model, and algorithms. VLDB 2001: 371-380.
41. M. N. Garofalakis, A. Gionis, R. Rastogi, S. Seshadri and K. Shim: DTD inference from XML documents: the XTRACT approach. IEEE Data Eng. Bull.: 19-25 (2003)
42. L. Golab, H. Karloff, F. Korn and D. Srivastava: Data Auditor: exploring data quality and semantics using pattern tableaux. PVLDB 3(2) 1641-1644 (2010).
43. L. Golab, H. J. Karloff, F. Korn, A. Saha and D. Srivastava: Sequential dependencies. PVLDB 2(1): 574-585 (2009).
44. L. Golab, H. J. Karloff, F. Korn, D. Srivastava and B. Yu: On generating near-optimal tableaux for conditional functional dependencies. PVLDB 1(1): 376-390 (2008).
45. L. Golab, H. J. Karloff, F. Korn, B. Saha and D. Srivastava: Discovering conservation rules. ICDE 2012: 738-749.
46. L. Golab, F. Korn and D. Srivastava: Efficient and effective analysis of data quality using pattern tableaux. IEEE Data Eng. Bull. 2011: 26-33.
47. S. Grijzenhout and M. Marx: The quality of the XML web. CIKM 2011: 1719-1724.
48. F. Geerts, G. Mecca, P. Papotti, and D. Santoro: The LLUNATIC data-cleaning framework, PVLDB 6(9): 625-636 (2013).
49. J. Hellerstein: Quantitative data cleaning for large databases. UNECE 2008.
50. M. Hadjieleftheriou and D. Srivastava: Approximate string processing. Foundations and Trends in Databases 2011: 267-402.
51. I. F. Ilyas, V. Markl, P. J. Haas, P. Brown and A. Aboulnaga: CORDS: automatic discovery of correlations and soft functional dependencies. SIGMOD 2004: 647-658.
52. S. Kandel, R. Parikh, A. Paepcke, J. M. Hellerstein and J. Heer: Profiler: integrated statistical analysis and visualization for data quality assessment. AVI 2012: 547-554.
53. F. Korn, B. Saha, D. Srivastava and S. Ying: On repairing structural problems in semi-structured Data. PVLDB 6(9): 601-612 (2013).
54. F. Magniez, C. Mathieu and A. Nayak: Recognizing well-parenthesized expressions in the streaming model. STOC 2010: 261-270.
55. W. Nutt and S. Razniewski: Completeness of queries over SQL databases, CIKM 2012: 902-911.
56. W. Nutt, S. Razniewski and G. Vegliach: Incomplete databases: missing records and missing values. DASFAA Workshops 2012: 298-310.
57. T. Redman: The impact of poor data quality on the typical enterprise. Commun. ACM 1998.
58. V. Raman and J. Hellerstein: Potter's Wheel: an interactive data cleaning system. VLDB 2001: 381-390.
59. S. Razniewski and W. Nutt: Completeness of queries over incomplete databases. PVLDB 4(11): 749-760 (2011).
60. N. Suzuki: Finding an optimum edit script between an XML document and a DTD. SAC 2005: 647-653.
61. S. Sarsfield: The butterfly effect of data quality, The Fifth MIT Information Quality Industry Symposium, 2011.
62. S. Song and L. Chen: Discovering matching dependencies. CIKM 2009: 1421-1424.
63. J. Tee: Handling the four 'V's of big data: volume, velocity, variety, and veracity. TheServerSide.com 2013.
64. H. Samimi, M. Schaefer, S. Artzi, T. Millstein, F. Tip and L. Hendren: Automated repair of HTML generation errors in PHP applications using string constraint solving. Int'l Conf. Software Engineering 2012
65. L. Segoufin and C. Sirangelo: Constant-memory validation of streaming XML documents against DTDs. ICDT 2007: 299-313.
66. L. Segoufin and V. Vianu: Validating streaming XML documents. PODS 2002: 53-64.
67. M. Yakout, L. Berti-Equille and A. K. Elmagarmid: Don't be SCAREd: use SCalable Automatic REpairing with maximal likelihood and bounded changes. SIGMOD 2013: 553-564.
68. M. Yakout, A. K. Elmagarmid, J. Neville, M. Ouzzani and I. F. Ilyas: Guided data repair. PVLDB 4(5): 279-289 (2011).
69. M. Zhang, M. Hadjieleftheriou, B. Ooi, C. M. Procopiuc and D. Srivastava: Automatic discovery of attributes in relational databases. SIGMOD 2011: 109-120.
70. M. Zhang, M. Hadjieleftheriou, B. Ooi, C. M. Procopiuc and D. Srivastava: On multi-column foreign key discovery. PVLDB 3(1): 805-814 (2010).