A model-based evaluation of data quality activities in KDD

Information Processing and Management

Volumn 51, Issue 2, 2015, Pages 144-166

  Mezzanzanica, Mario ^a   Boselli, Roberto ^a   Cesarini, Mirko ^a   Mercorio, Fabio ^a  

^a UNIVERSITY OF MILANO BICOCCA   (Italy)

Author keywords

Data cleansing; Data quality; Model checking; Real life application

Indexed keywords

DATA REDUCTION; MODEL CHECKING;

DATA CLEANSING; DATA QUALITY; EFFECTIVENESS OF KNOWLEDGE; FINE-GRAINED ANALYSIS; MODEL BASED APPROACH; MODEL-BASED EVALUATION; MODEL-CHECKING TECHNIQUES; REAL-LIFE APPLICATIONS;

QUALITY CONTROL;

EID: 84922363216     PISSN: 03064573     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ipm.2014.07.007     Document Type: Article

References (103)

1. J. Abello, P.M. Pardalos, and M.G. Resende Handbook of massive data sets Vol. 4 2002 Springer
2. Afanasiev, L.; Franceschet, M.; Marx, M.; & de Rijke, M. (2004). CTL model checking for processing simple xpath queries. In Proceedings of the 11th international symposium on temporal representation and reasoning (TIME) (pp. 117-124).
3. M. Arenas, L.E. Bertossi, and J. Chomicki Consistent query answers in inconsistent databases ACM symp. on principles of database systems 1999 ACM Press 68 79
4. J.M. Atlee, and J. Gannon State-based model checking of event-driven system requirements IEEE Transactions on Software Engineering 19 1993 24 40
5. C. Baier, and J.-P. Katoen Principles of model checking Vol. 26202649 2008 MIT Press Cambridge
6. D.P. Ballou, and G.K. Tayi Enhancing data quality in data warehouse environments Communications of the ACM 42 1999 73 78
7. J. Barateiro, and H. Galhardas A survey of data quality tools Datenbank-Spektrum 14 2005 15 21
8. F. Bartolucci, A. Farcomeni, and F. Pennoni Latent Markov models for longitudinal data 2012 Chapman & Hall/CRC Press Boca Raton, FL
9. C. Batini, C. Cappiello, C. Francalanci, and A. Maurino Methodologies for data quality assessment and improvement ACM Comput. Surv. 41 2009 16:1 16:52
10. C. Batini, and M. Scannapieco Data quality: Concepts, methodologies and techniques Data-centric systems and applications 2006 Springer
11. B. Bérard, M. Bidoit, A. Finkel, F. Laroussinie, A. Petit, and L. Petrucci Systems and software verification: Model-checking techniques and tools 2010 Springer Publishing Company, Incorporated
12. L. Bertossi Consistent query answering in databases ACM Sigmod Record 35 2006 68 76
13. A. Biere, A. Cimatti, E.M. Clarke, O. Strichman, and Y. Zhu Bounded model checking Advances in Computers 58 2003 117 148
14. R. Boselli, M. Cesarini, F. Mercorio, and M. Mezzanzanica Inconsistency knowledge discovery for longitudinal data management: A model-based approach SouthCHI13 special session on human-computer interaction & knowledge discovery LNCS Vol. 7947 2013 Springer
15. R. Boselli, M. Cesarini, F. Mercorio, and M. Mezzanzanica A policy-based cleansing and integration framework for labour and healthcare data SOTA survey. HCI-KDD methods: Dealing with complex data LNCS Vol. 8401 2014 Springer
16. R. Boselli, M. Cesarini, F. Mercorio, and M. Mezzanzanica Towards data cleansing via planning Intelligenza Artificiale 8 2014
17. R. Boselli, M. Mezzanzanica, M. Cesarini, and F. Mercorio Planning meets data cleansing The 24th International Conference on Automated Planning and Scheduling (ICAPS) 2014 AAAI
18. L. Bravo, W. Fan, F. Geerts, and S. Ma Increasing the expressivity of conditional functional dependencies without extra complexity Proceedings of the IEEE 24th International Conference on Data Engineering (ICDE) 2008 IEEE 516 525
19. M. Cesarini, M. Mezzanzanica, and M. Fugini Analysis-sensitive conversion of administrative data into statistical information systems Journal of Cases on Information Technology 9 2007 57 81
20. E.-H. Choi, T. Tsuchiya, and T. Kikuno Model checking active database rules under various rule processing strategies IPSJ Digital Courier 2 2006 826 839
21. J. Chomicki Efficient checking of temporal integrity constraints using bounded history encoding ACM Transactions on Database Systems (TODS) 20 1995 149 186
22. J. Chomicki, and J. Marcinkowski Minimal-change integrity maintenance using tuple deletions Information and Computation 197 2005 90 121
23. J. Chomicki, and J. Marcinkowski On the computational complexity of minimal-change integrity maintenance in relational databases Inconsistency tolerance Lecture notes in computer science Vol. 3300 2005 Springer 119 150
24. A. Cimatti, M. Roveri, and P. Traverso Automatic obdd-based generation of universal plans in non-deterministic domains J. Mostow, C. Rich, AAAI/IAAI 1998 AAAI Press/The MIT Press 875 881
25. F. Cipollini, C. Ferretti, P. Ganugi, and M. Mezzanzanica A continuous time mover-stayer model for labor market in a northern italian area Classification and data mining 2013 Springer 181 188
26. E.M. Clarke, O. Grumberg, and D.E. Long Model checking and abstraction ACM Transactions on Programming Languages and Systems (TOPLAS) 16 1994 1512 1542
27. E.M. Clarke, O. Grumberg, and D.A. Peled Model checking 1999 The MIT Press
28. P. Clemente, B. Kaba, J. Rouzaud-Cornabas, M. Alexandre, and G. Aujay Sptrack: Visual analysis of information flows within selinux policies and attack logs AMT special session on human-computer interaction and knowledge discovery LNCS Vol. 7669 2012 Springer 596 605
29. Cong, G.; Fan, W.; Geerts, F.; Jia, X.; & Ma, S. (2007). Improving data quality: Consistency and accuracy. In Proceedings of the 33rd international conference on Very large data bases (pp. 315-326). VLDB Endowment.
30. P. Cousot, and R. Cousot Abstract interpretation: A unified lattice model for static analysis of programs by construction or approximation of fixpoints Proceedings of the 4th ACM SIGACT-SIGPLAN symposium on principles of programming languages 1977 ACM 238 252
31. CRISP Research Centre Web Page (2013). < http://www.crisp-org.it >.
32. M. Dallachiesa, A. Ebaid, A. Eldawy, A. Elmagarmid, I.F. Ilyas, and M. Ouzzani Nadeef: A commodity data cleaning system Proceedings of the 2013 international conference on Management of data 2013 ACM 541 552
33. T. Dasu Data glitches: Monsters in your data Handbook of data quality 2013 Springer 163 178
34. G. Della Penna, B. Intrigila, D. Magazzeni, and F. Mercorio UPMurphi: A tool for universal planning on PDDL+ problems Proceeding of the 19th International Conference on Automated Planning and Scheduling (ICAPS) 2009 AAAI Press 106 113
35. G. Della Penna, B. Intrigila, D. Magazzeni, and F. Mercorio A PDDL+ benchmark problem: The batch chemical plant Proceedings of ICAPS 2010 2010 AAAI Press 222 224
36. G. Della Penna, B. Intrigila, D. Magazzeni, and F. Mercorio Planning for autonomous planetary vehicles Proceedings of the the sixth international conference on autonomic and autonomous systems 2010 IEEE Cancun, Mexico 131 136
37. G. Della Penna, B. Intrigila, D. Magazzeni, F. Mercorio, and E. Tronci Cost-optimal strong planning in non-deterministic domains Proceedings of the 8th international conference on informatics in control Automation and robotics (ICINCO) 2011 SciTePress 56 66
38. G. Della Penna, B. Intrigila, I. Melatti, E. Tronci, and M. Venturini Zilli Exploiting transition locality in automatic verification of finite state concurrent systems STTT 6 2004 320 341
39. G. Della Penna, D. Magazzeni, and F. Mercorio A universal planning system for hybrid domains Applied Intelligence 36 2012 932 959
40. S. Devaraj, and R. Kohli Information technology payoff in the health-care industry: A longitudinal study Journal of Management Information Systems 16 2000 41 68
41. D.L. Dill The mur verification system Computer aided verification 1996 Springer 390 393
42. A. Dovier, and E. Quintarelli Model-checking based data retrieval Database programming languages LNCS Vol. 2397 2002 Springer Berlin/Heidelberg 62 77
43. A. Dovier, and E. Quintarelli Applying model-checking to solve queries on semistructured data Computer Languages, Systems & Structures 35 2009 143 172
44. S. Edelkamp, and S. Jabbar Large-scale directed model checking ltl Model checking software 2006 Springer 1 18
45. A.K. Elmagarmid, P.G. Ipeirotis, and V.S. Verykios Duplicate record detection: A survey IEEE Transactions on Knowledge and Data Engineering 19 2007 1 16
46. European Telecommunications Standards Institute ES 201 671 (2009). Handover interface for the lawful interception of telecommunications traffic.
47. W. Fan Dependencies revisited for improving data quality Proceedings of the twenty-seventh ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems 2008 ACM 159 170
48. W. Fan, J. Li, S. Ma, N. Tang, and W. Yu Towards certain fixes with editing rules and master data Proceedings of the VLDB Endowment 3 2010 173 184
49. U. Fayyad, G. Piatetsky-Shapiro, and P. Smyth The kdd process for extracting useful knowledge from volumes of data Communications of the ACM 39 1996 27 34
50. I.P. Fellegi, and D. Holt A systematic approach to automatic edit and imputation Journal of the American Statistical association 71 1976 17 35
51. I. Fellegi, and A. Sunter A theory for record linkage Journal of the American Statistical Association 64 1969 1183 1210
52. M.C. Ferreira de Oliveira, and H. Levkowitz From visual data exploration to visual data mining: A survey IEEE Transactions on Visualization and Computer Graphics 9 2003 378 394
53. C.W. Fisher, and B.R. Kingma Criticality of data quality as exemplified in two disasters Information and Management 39 2001 109 116
54. C. Fisher, E. Lauría, S. Chengalur-Smith, and R. Wang Introduction to information quality AuthorHouse 2012
55. C. Fox, A. Levitin, and T. Redman The notion of data and its quality dimensions Information Processing & Management 30 1994 9 19
56. M. Fox, D. Long, and D. Magazzeni Automatic construction of efficient multiple battery usage policies T. Walsh, IJCAI 2011 IJCAI/AAAI 2620 2625
57. M. Fox, D. Long, and D. Magazzeni Plan-based policies for efficient multiple battery load management Journal of Artificial Intelligence Research (JAIR) 44 2012 335 382
58. F. Giunchiglia, and P. Traverso Planning as model checking Recent Advances in AI Planning 2000 1 20
59. G. Greco, S. Greco, and E. Zumpano A logic programming approach to the integration, repairing and querying of inconsistent databases Logic programming 2001 Springer 348 364
60. P. Hansen, and K. Järvelin Collaborative information retrieval in an information-intensive domain Information Processing & Management 41 2005 1101 1119
61. A. Haug, F. Zachariassen, and D. Van Liempd The costs of poor data quality Journal of Industrial Engineering and Management 4 2011 168 193
62. E. Herrera-Viedma, and E. Peis Evaluating the informative quality of documents in sgml format from judgements by means of fuzzy linguistic techniques based on computing with words Information Processing & Management 39 2003 233 249
63. Hipp, J.; Güntzer, U.; & Grimmer, U. (2001). Data quality mining-making a virute of necessity. In DMKD.
64. Holzinger, A.; Yildirim, P.; Geier, M.; & Simonic, K.-M. (2013b). Quality-based knowledge discovery from medical text on the web. In Pasi, Bordogna, and Jain (2013b).
65. Holzinger, A. (2011). Weakly structured data in health-informatics: The challenge for human-computer-interaction. In Proceedings of INTERACT 2011 workshop: Promoting and supporting healthy living by desing (pp. 5-7). IFIP.
66. A. Holzinger On knowledge discovery and interactive intelligent visualization of biomedical data - challenges in human-computer interaction & biomedical informatics M. Helfert, C. Francalanci, J. Filipe, DATA 2012 SciTePress
67. A. Holzinger, M. Bruschi, and W. Eder On interactive data visualization of physiological low-cost-sensor data with focus on mental stress A. Cuzzocrea, C. Kittl, D.E. Simos, E. Weippl, L. Xu, CD-ARES Lecture notes in computer science Vol. 8127 2013 Springer 469 480
68. A. Holzinger, and M. Zupan Knodwat: A scientific framework application for testing knowledge discovery methods for the biomedical domain BMC Bioinformatics 14 2013 191
69. G.J. Holzmann, and M.H. Smith A practical method for verifying event-driven software Proceedings of the 21st international conference on Software engineering 1999 ACM 597 607
70. A. Inselberg The plane with parallel coordinates The Visual Computer 1 1985 69 91
71. B.K. Kahn, D.M. Strong, and R.Y. Wang Information quality benchmarks: Product and service performance Communications of the ACM 45 2002 184 192
72. S. Kolahi, and L.V. Lakshmanan On approximating optimum repairs for functional dependency violations Proceedings of the 12th international conference on database theory 2009 ACM 53 62
73. D. Kroening, E. Clarke, and K. Yorav Behavioral consistency of C and Verilog programs using bounded model checking Proceedings of DAC 2003 2003 ACM Press 368 371
74. A. Levitin, and T. Redman Quality dimensions of a conceptual view Information Processing & Management 31 1995 81 88
75. P.G. Lovaglio, and M. Mezzanzanica Classification of longitudinal career paths Quality & Quantity 47 2013 989 1008
76. S.E. Madnick, R.Y. Wang, Y.W. Lee, and H. Zhu Overview and framework for data and information quality research Journal of Data and Information Quality 1 2009 2:1 2:22
77. C. Mayfield, J. Neville, and S. Prabhakar Eracer: A database approach for statistical inference and data cleaning Proceedings of the 2010 ACM SIGMOD international conference on management of data 2010 ACM 75 86
78. F. Mercorio Model checking for universal planning in deterministic and non-deterministic domains AI Communications 26 2013 257 259
79. M. Mezzanzanica, R. Boselli, M. Cesarini, and F. Mercorio Data quality through model checking techniques IDA Lecture notes in computer science Vol. 7014 2011 Springer 270 281
80. M. Mezzanzanica, R. Boselli, M. Cesarini, and F. Mercorio Automatic synthesis of data cleansing activities M. Helfert, C. Francalanci, The 2nd international conference on Data Management Technologies and Applications (DATA) 2013 Scitepress 138 149
81. F. Neven Automata theory for xml researchers SIGMOD Record 31 2002 39 46
82. H.B. Newcombe, and J.M. Kennedy Record linkage: Making maximum use of the discriminating power of identifying information Communications of the ACM 5 1962 563 566
83. C. Norris Ip, and D. Dill Better verification through symmetry Formal methods in system design 9 1996 41 75
84. G. Pasi, G. Bordogna, and L.C. Jain Quality issues in the management of web information Intelligent systems reference library Vol. 50 2013 Springer
85. Pasi, G.; Bordogna, G.; & Jain, L. C. (2013a). An introduction to quality issues in the management of web information. In Pasi et al. (2013b).
86. L.L. Pipino, Y.W. Lee, and R.Y. Wang Data quality assessment Communications of the ACM 45 2002 211 218
87. A. Prinzie, and D. Van den Poel Modeling complex longitudinal consumer behavior with dynamic bayesian networks: An acquisition pattern analysis application Journal of Intelligent Information Systems 36 2011 283 304
88. E. Rahm, and H. Do Data cleaning: Problems and current approaches IEEE Data Engineering Bulletin 23 2000 3 13
89. I. Ray, and I. Ray Detecting termination of active database rules using symbolic model checking Proceedings of the 5th East European Conference on Advances in Databases and Information Systems ADBIS '01 2001 Springer London, UK 266 279
90. T.C. Redman The impact of poor data quality on the typical enterprise Communications of the ACM 41 1998 79 82
91. T.C. Redman Data quality: The field guide 2001 Digital Pr.
92. S. Sadiq Handbook of data quality 2013 Springer
93. Schoppers, M.; 1987. Universal plans of reactive robots in unpredictable environments. In Proc. IJCAI 1987.
94. The Italian Ministry of Labour and Welfare (2012). Annual report about the CO system. < http://www.cliclavoro.gov.it/Barometro-Del-Lavoro/Documents/Rapporto-CO/Executive-summary.pdf >.
95. C. Thomsen, and T.B. Pedersen A survey of open source tools for business intelligence Data Warehousing and Knowledge Discovery 2005 Springer 74 84
96. M. Vardi Fundamentals of dependency theory Trends in Theoretical Computer Science 1987 171 224
97. R.Y. Wang, and D.M. Strong Beyond accuracy: What data quality means to data consumers Journal of Management Information Systems 12 1996 5 33
98. W.E. Winkler Editing discrete data Bureau of the Census 1997
99. W.E. Winkler Machine learning, information retrieval and record linkage Proc section on survey research methods 2000 American Statistical Association 20 29
100. W.E. Winkler Methods for evaluating and creating data quality Information Systems 29 2004 531 550
101. B.W. Wong, K. Xu, and A. Holzinger Interactive visualization for information analysis in medical diagnosis Information quality in e-Health 2011 Springer 109 120
102. M. Yakout, L. Berti-Équille, and A.K. Elmagarmid Don't be scared: Use scalable automatic repairing with maximal likelihood and bounded changes Proceedings of the 2013 international conference on Management of data 2013 ACM 553 564
103. L. Yu, S. Wang, and K.K. Lai An integrated data preparation scheme for neural network data analysis IEEE Transactions on Knowledge and Data Engineering 18 2006 217 230