How to lie with bad data

Statistical Science

Volumn 20, Issue 3, 2005, Pages 231-238

  De Veaux, Richard D ^a   Hand, David J ^b  

^a WILLIAMS COLLEGE   (United States)

^b IMPERIAL COLLEGE LONDON   (United Kingdom)

Author keywords

Accuracy; Data consistency; Data mining; Data profiling; Data quality; Data rectification; Data ware housing; Distortion; Missing values; Record linkage

Indexed keywords

EID: 26444582963     PISSN: 08834237     EISSN: None     Source Type: Journal    
DOI: 10.1214/088342305000000269     Document Type: Article

References (32)

1. BAGGERLY, K. A, MORRIS, J. S. and COOMBES, K. R. (2004). Reproducibility of SELDI-TOF protein patterns in serum: Comparing datasets from different experiments. Bioinformatics 20 777-785.
2. BRUNSKILL, A. J. (1990). Some sources of error in the coding of birth weight. American J. Public Health 80 72-73.
3. CHECK, E. (2004). Proteomics and cancer: Running before we can walk? Nature 429 496-497.
4. COALE, A. J. and STEPHAN, F. F. (1962). The case of the Indians and the teen-age widows. J. Amer. Statist. Assoc. 57 338-347.
5. DE VEAUX, R. D. (2002). Data mining: A view from down in the pit. Stats (34) 3-9.
6. DE VEAUX, R. D., DONAHUE, R. and SMALL, R. D. (2002). Using data mining techniques to harvest information in clinical trials. Presentation at Joint Statistical Meetings, New York.
7. DE VEAUX, R. D., GORDON, A., COMISO, J. and BACHERER, N. E. (1993). Modeling of topographic effects on Antarctic sea-ice using multivariate adaptive regression splines. J. Geophysical Research - Oceans 98 20, 307-20, 320.
8. HAND, D. J. (2001). Reject inference in credit operations. In Handbook of Credit Scoring (E. Mays, ed.) 225-240. Glenlake Publishing, Chicago.
9. HAND, D. J. (2004a). Academic obsessions and classification realities: Ignoring practicalities in supervised classification. In Classification, Clustering and Data Mining Applications (D. Banks, L. House, F. R. McMorris, P. Arabie and W. Gaul, eds.) 209-232. Springer, Berlin.
10. HAND, D. J. (2004b). Measurement Theory and Practice: The World Through Quantification. Arnold, London.
11. HAND, D. J., BLUNT, G., KELLY, M. G. and ADAMS, N. M. (2000). Data mining for fun and profit (with discussion). Statist. Sci. 15 111-131.
12. HAND, D. J. and HENLEY, W. E. (1993). Can reject inference ever work? IMA J. of Mathematics Applied in Business and Industry 5(4) 45-55.
13. HUFF, D. (1954). How to Lie with Statistics. Norton, New York.
14. JONES, P. D. and WIGLEY, T. M. L. (1990). Global warming trends. Scientific American 263(2) 84-91.
15. KIM, W., CHOI, B.-J., HONG, E.-K., KIM, S.-K. and LEE, D. (2003). A taxonomy of dirty data. Data Mining and Knowledge Discovery 7 81-99.
16. KLEIN, B. D. (1998). Data quality in the practice of consumer product management: Evidence from the field. Data Quality 4(1).
17. KRUSKAL, W. (1981). Statistics in society: Problems unsolved and unformulated. J. Amer. Statist. Assoc. 76 505-515.
18. LAUDON, K. C. (1986). Data quality and due process in large interorganizational record systems. Communications of the ACM 29 4-11.
19. LITTLE, R. J. A. and RUBIN, D. B. (1987). Statistical Analysis with Missing Data. Wiley, New York.
20. LOSHIN, D. (2001). Enterprise Knowledge Management: The Data Quality Approach. Morgan Kaufmann, San Francisco.
21. MADNICK, S. E. and WANG, R. Y. (1992). Introduction to the TDQM research program. Working Paper 92-01, Total Data Quality Management Research Program.
22. MOREY, R. C. (1982). Estimating and improving the quality of information in a MIS. Communications of the ACM 25 337-342.
23. PERCY, T. (1986). My data, right or wrong. Datamation 32(11) 123-124.
24. PETRICOIN, E. F., III, ARDEKANI, A. M., HITT, B. A., LEVINE, P. J., FUSARO, V. A., STEINBERG, S. M., MILLS, G. B., SIMONE, C., FISHMAN, D. A., KOHN, E. C. and LIOTTA, L. A. (2002). Use of proteomic patterns in serum to identify ovarian cancer. The Lancet 359 572-577.
25. PIERCE, E. (1997). Modeling database error rates. Data Quality 3(1). Available at www.dataquality.com/dqsep97.htm.
26. PRICEWATERHOUSECOOPERS (2004). The Tech Spotlight 22. Available at www.pwc.com/extweb/manissue.nsf/docid/ 2D6E2F57E06E022F85256B8F006F389A.
27. REDMAN, T. C. (1992). Data Quality. Management and Technology. Bantam, New York.
28. STRAYHORN, J. M. (1990). Estimating the errors remaining in a data set: Techniques for quality control. Amer. Statist. 44 14-18.
29. WAINER, H. (2004). Curbstoning IQ and the 2000 presidential election. Chance 17(4) 43-46.
30. WEST, M. and WINKLER, R. L. (1991). Data base error trapping and prediction. J. Amer. Statist. Assoc. 86 987-996.
31. WILLENBORG, L. and DE WAAL, T. (2001). Elements of Statistical Disclosure Control. Springer, New York.
32. WOLINS, L. (1962). Responsibility for raw data. American Psychologist 17 657-658.