Error correction for massive datasets

Optimization Methods and Software

Volumn 20, Issue 2-3, 2005, Pages 297-316

  Bruni, Renato ^a  

^a SAPIENZA UNIVERSITY OF ROME   (Italy)

Author keywords

Data correction; Inconsistency localization; Massive datasets

Indexed keywords

ALGORITHMS; COMPUTER SCIENCE; DATA MINING; DATA PROCESSING; LINEAR PROGRAMMING; MATHEMATICAL MODELS;

DATA CORRECTION; DATASETS; INCONSISTENCY LOCALIZATION; MASSIVE DATASETS;

ERROR CORRECTION;

EID: 12444250037     PISSN: 10556788     EISSN: None     Source Type: Journal    
DOI: 10.1080/10556780512331318281     Document Type: Article

References (35)

1. Ramakrishnan, R. and Gehrke, J., 2000, Database Management System (McGraw Hill).
2. Fellegi, P. and Holt, D., 1976, A systematic approach to automatic edit and imputation. Journal of the American Statistical Association, 71(353), 17-35.
3. Boros, E., Hammer, P.L., Ibaraki, T. and Kogan, A., 1997, Logical analysis of numerical data. Mathematical Programming, 79, 163-190.
4. Fayyad, U.M., Piatetsky-Shapiro, G., Smyth, P. and Uthurusamy, R., (Eds.) 1996, Advances in Knowledge Discovery and Data Mining (Menlo Park, CA: AAAI Press/The MIT Press).
5. Hand, D.J., Mannila, H. and Smyth, P., 2001, Principles of Data Mining (London: MIT Press).
6. Hastie, T., Tibshirani, R. and Friedman, J., 2002, The Elements of Statistical Learning (New York, Berlin, Heidelberg: Springer-Verlag).
7. Bankier, M., 2000, Canadian census minimum change donor imputation methodology. Proceedings of the Workshop on Data Editing, UN/ECE, Cardiff, UK.
8. De Waal, T., 2003, Processing of erroneous and unsafe data. PhD thesis, ERIM PhD series in Research Management.
9. Poirier, C., 1999, A functional evaluation of edit and imputation tools. UN/ECE Work Session on Statistical Data Editing, Working Paper n. 12, Rome, Italy.
10. Winkler, W.E., 1999, State of statistical data editing and current research problems. UN/ECE Work Session on Statistical Data Editing, Working Paper n.29, Rome, Italy.
11. Guyon, I., Matic, N. and Vapnik, V., 1996, Advances in Knowledge Discovery and Data Mining (Menlo Park, CA: AAAI Press/The MIT Press).
12. Ayel, M. and Laurent, J.P. (Eds.), 1991, Validation, Verification and Testing of Knowledge-Based Systems (Chichester, England: John Wiley & Sons Ltd.).
13. Menzies, T., 1999, Knowledge maintenance: the state of the art. Knowledge Engineering Review, 14(1), 1-46.
14. Rescher, N. and Brandom, R., 1980, The Logic of Inconsistency (Oxford: Basil Blackwell).
15. Franconi, E., Laureti Palma, A., Leone, N., Perri, S. and Scarcello, F., 2001, Census data repair: a challenging application of disjunctive logic programming. In: Proceedings of 8th International Conference on Logic for Programming, Artificial Intelligence and Reasoning (LPAR-2001), Lecture Notes in Artificial Intelligence, 2250, (Springer).
16. Garfinkel, R.S., Kunnathur, A.S. and Liepins, G.E., 1986, Optimal imputation of erroneous data: categorical data, general edits. Operations Research, 34, 744-751.
17. Garfinkel, R.S., Kunnathur, A.S. and Liepins, G.E., 1988, Error localization for erroneous data: continuous data, linear constraints. SIAM Journal on Scientific and Statistical Computing, 9, 922-931.
18. Ragsdale, C.T. and McKeown, P.G., 1996, On solving the continuous data editing problem. Computers and Operations Research, 23, 263-273.
19. Bruni, R. and Sassano, A., 2001, Error detection and correction in large scale data collecting. In: Advances in Intelligent Data Analysis, Lecture Notes in Computer Science, 2189, (Springer).
20. Amaldi, E., Pfetsch, M.E., and Trotter, L., Jr., 2003, On the maximum feasible subsystem problem, IISs and IIS-hypergraphs. Mathematical Programming, 95(3), 533-554.
21. Chinneck, J.W., 2001, Fast heuristics for the maximum feasible subsystem problem. INFORMS Journal on Computing, 13(3), 210-223.
22. Chinneck, J.W. and Dravnieks, E.W., 1991, Locating minimal infeasible constraint sets in linear programs. ORSA Journal on Computing, 3, 157-168.
23. Tamiz, M., Mardle, S.J. and Jones, D.F., 1996, Detecting US in infeasible linear programs using techniques from goal programming. Computers and Operations Research, 23, 113-191.
24. Schrijver, A., 1986, Theory of Linear and Integer Programming (New York: Wiley).
25. ILOG, 2000, ILOG Cplex 7.0. Reference Manual.
26. Manzari, A. and Reale, A., 2001, Towards a new system for edit and imputation of the 2001 Italian population census data: a comparison with the Canadian nearest-neighbour imputation methodology. Proceedings of the 53rd Session of the International Statistical Institute, Seoul, South Korea.
27. Bruni, R., Reale, A. and Torelli, R., 2001, Optimization techniques for edit validation and data imputation. Proceedings of Statistics Canada Symposium: Achieving Data Quality in a Statistical Agency, Ottawa, Canada.
28. Williams, H.P., 1993, Model Building in Mathematical Programming (Chichester: J. Wiley).
29. Guieu, O. and Chinneck, J. W., 1999, Analyzing infeasible mixed-integer and integer linear programs. INFORMS Journal on Computing, 11(1).
30. Gleeson, J. and Ryan, J., 1990, Identifying minimally infeasible subsystems of inequalities. ORSA Journal on Computing, 2(1), 61-63.
31. Nemhauser, G.L. and Wolsey, L.A., 1988, Integer and Combinatorial Optimization (New York: J. Wiley).
32. Chandru, V. and Hooker, J.N., 1999, Optimization Methods for Logical Inference (New York: Wiley).
33. Bertsimas, D. and Tsitsiklis, J.N., 1997, Introduction to Linear Optimization. Athena Scientific (Belmont, MA: Athena Scientific).
34. ILOG, 2000, ILOG Concert Technology 1.0. Reference Manual.
35. United Nations Economic Commission for Europe and the Statistical Office of the European Communities, 1998, Recommendations for the 2000 censuses of population and housing in the ECE region. Technical Report Statistical Standards and Studies No. 49, UN/ECE Statistical Division.