Data mining and the impact of missing data

Industrial Management and Data Systems

Volumn 103, Issue 8-9, 2003, Pages 611-621

  Brown, Marvin L ^a   Kros, John F ^b  

^a HAWAII PACIFIC UNIVERSITY   (United States)

^b EAST CAROLINA UNIVERSITY   (United States)

Author keywords

Data handling; Database management systems; Information gathering; Information retrieval

Indexed keywords

ALGORITHMS; DATA ACQUISITION; DATA HANDLING; DATABASE SYSTEMS; INFORMATION RETRIEVAL; MATHEMATICAL MODELS; MATRIX ALGEBRA; PATTERN RECOGNITION; STATISTICAL METHODS;

EXPECTATION MAXIMIZATION ALGORITHM; KNOWLEDGE DISCOVERY;

DATA MINING;

EID: 0344687410     PISSN: 02635577     EISSN: None     Source Type: Journal    
DOI: 10.1108/02635570310497657     Document Type: Article

References (59)

1. Adriaans, P. and Zantinge, D. (1997), Data Mining, Addison-Wesley, New York, NY.
2. Afifi, A. and Elashoff, R. (1966), "Missing observations in multivariate statistics I: review of the literature", Journal of the American Statistical Association, Vol. 61, pp. 595-604.
3. Barnard, J. and Meng, X. (1999), "Applications of multiple imputation in medical studies: from AIDS to NHANES", Statistical Methods in Medical Research, Vol. 8, pp. 17-36.
4. Berry, M. and Linoff, G. (1997), Data Mining Techniques, Wiley, New York, NY.
5. Berry, M. and Linoff, G. (2000), "The art and science of customer relationship", Industrial Management & Data Systems, Vol. 100 No. 5, pp. 245-6.
6. Berson, A., Smith, S. and Thearling, K. (2000), Building Data Mining Applications for CRM, McGraw-Hill, New York, NY.
7. Brick, J.M. and Kalton, G. (1996), "Handling missing data in survey research", Statistical Methods in Medical Research, Vol. 5, pp. 215-38.
8. Clogg, C., Rubin, D., Schenker, N., Schultz, B. and Weidman, L. (1991), "Multiple imputation of industry and occupation codes in census public-use samples using Bayesian logistic regression", Journal of the American Statistical Association, Vol. 86 No. 413, pp. 68-78.
9. Darling, C.B. (1997), "Datamining for the masses", Datamation, Vol. 52, pp. 5.
10. David, M., Little, R., Samuhel, M. and Triest, R. (1986), "Alternative methods for CPS income imputation", Journal of the American Statistical Association, Vol. 81, pp. 29-41.
11. Dempster, A. and Rubin, D. (1983), "Incomplete data in sample surveys", in Madow, W.G., Olkin, I. and Rubin, D. (Eds), Sample Surveys Vol. II: Theory and Annotated Bibliography, Academic Press, New York, NY, pp. 3-10.
12. Dempster, A., Laird, N. and Rubin, D. (1977), "Maximum likelihood from incomplete data via the EM algorithm (with discussion)", Journal of the Royal Statistical Society, Vol. B39, pp. 1-38.
13. Diggle, P. and Kenward, M. (1994), "Informative dropout in longitudinal data analysis (with discussion)", Applied Statistics, Vol. 43, pp. 49-94.
14. Dillman, D.A. (1999), Mail and Internet Surveys: The Tailored Design Method, John Wiley Company, New York, NY.
15. Ernst, L. (1980), "Variance of the estimated mean for several imputation procedures", American Statistical Association 1980, Proceedings of the Survey Research Methods Section, pp. 716-20.
16. Ford, B. (1981), "An overview of hot deck procedures", in Madow, W.G., Olkin, I. and Rubin, D. (Eds), Sample Surveys Vol. II: Theory and Annotated Bibliography, Academic Press, New York, NY, pp. 3-10.
17. Graham, J., Hofer, S. and Piccinin, A. (1994), "Analysis with missing data in drug prevention research", in Collins, L.M. and Seitz, L. (Eds), Advances in Data Analysis for Prevention Intervention Research NIDA Research Monograph, Series (#142), National Institute on Drub Abuse, Washington, DC.
18. Graham, J., Hofer, S., Donaldson, S., MacKinnon, D. and Schafer, J. (1997), "Analysis with missing data in prevention research", in Bryant, K., Windle, W. and West, S. (Eds), New Methodological Approaches to Alcohol Prevention Research, American Psychological Association, Washington, DC.
19. Groth, R. (2000), Data Mining: Building Competitive Advantage, Prentice-Hall, Upper Saddle River, NJ.
20. Hair, J., Anderson, R., Tatham, R. and Black, W. (1998), Multivariate Data Analysis, Prentice-Hall, Upper Saddle River, NJ.
21. Han, J. and Kamber, M. (2001), Data Mining: Concepts and Techniques, Academic Press, San Francisco, CA.
22. Hansen, M., Madow, W. and Tepping, J. (1983), "An evaluation of model-dependent and probability-sampling inferences in sample surveys", Journal of the American Statistical Association, Vol. 78, pp. 776-807.
23. Hartley, H. and Hocking, R. (1971), "The analysis of incomplete data", Biometrics, Vol. 27, pp. 783-808.
24. Herzog, T. and Rubin, D. (1983), "Using multiple imputations to handle nonresponse in sample surveys", in Madow, W. G., Olkin, I. and Rubin, D. (Eds), Incomplete Data in Sample Surveys, Volume 2: Theory and Bibliographies, Academic Press, New York, NY, pp. 209-45.
25. Howell, D.C. (1998), "Treatment of missing data", D.C. Howell personal Web site, available at: www.uvm.edu/~dhowell/StatPages/More_Stuff/Missing_Data/ Missing.html/
26. Iannacchione, V. (1982), "Weighted sequential hot deck imputation macros", Proceedings of the SAS Users Group International Conference, Vol. 7, pp. 759-63.
27. Jenkins, C.R. and Dillman, D.A. (1997), "Towards a theory of self-administered questionnaire design", in Lyberg, L. et al. (Eds), Survey Measurement and Process Quality, John Wiley Company, New York, NY.
28. Kalton, G. and Kasprzyk, D. (1982), "Imputing for missing survey responses", American Statistical Association, Proceedings of the Section on Survey Research Methods, pp. 22-31.
29. Kalton, G. and Kasprzyk, D. (1986), "The treatment of missing survey data", American Statistical Association, Proceedings of the Section on Survey Research Methods, pp. 22-31.
30. Kalton, G. and Kish, L. (1981), "Two efficient random imputation procedures", American Statistical Association 1981, Proceedings of the Survey Research Methods Section, pp. 146-51.
31. Kim, Y. (2001), "The curse of the missing data", Y. Kim personal Web site, available at: http://209.68.11:8080/2ndMoment/978476655/addPostingForm/
32. Lee, S. and Siau, K. (2001), "A review of data mining techniques", Industrial Management & Data Systems, Vol. 101 No. 1, pp. 41-6.
33. Li, K. (1985), "Hypothesis testing in multiple imputation - with emphasis on mixed-up frequencies in contingency tables", PhD thesis, The University of Chicago, Chicago, IL.
34. Little, R. (1982), "Models for nonresponse in sample surveys", Journal of the American Statistical Association, Vol. 77, pp. 237-50.
35. Little, R. (1992), "Regression with missing X's: a review", Journal of the American Statistical Association, Vol. 87, pp. 1227-37.
36. Little, R. (1995), "Modeling the drop-out mechanism in repeated-measures studies", Journal of the American Statistical Association, Vol. 90, pp. 1112-21.
37. Little, R. and Rubin, D. (1987), Statistical Analysis with Missing Data, Wiley, New York, NY.
38. Little, R. and Rubin, D. (1989) "The analysis of social science data with missing values", Sociological Methods and Research, Vol. 18, pp. 292-26.
39. Ma, C., Chou, D. and Yen, D. (2000), "Data warehousing, technology assessment and management", Industrial Management & Data Systems, Vol. 100, No. 3, pp. 125-35.
40. Nie, N., Hull, C., Jenkins, J., Steinbrenner, K. and Bent, D. (1975), SPSS, 2nd ed., McGraw-Hill, New York, NY.
41. Orchard, T. and Woodbury, M. (1972) "A missing information principle: theory and applications", Proceedings of the 6th Berkeley Symposium on Mathematical Statistics and Probability, Vol. 1, pp. 697-715.
42. Pennell, S. (1993) "Cross-sectional imputation and longitudinal editing procedures in the survey of income and program participation", Technical report, Institute of Social Research, University of Michigan, Ann Arbor, MI.
43. Royall, R. and Herson, J. (1973) "Robust estimation from finite populations", Journal of the American Statistical Association, Vol. 68, pp. 883-9.
44. Rubin, D. (1978) "Multiple imputations in sample surveys - a phenomenological Bayesian approach to nonresponse", Imputation and Editing of Faulty or Missing Survey Data, US Department of Commerce, Washington, DC, pp. 1-23.
45. Rubin, D. (1986), "Statistical matching using file concatenation with adjusted weights and multiple imputations", Journal of Business and Economic Statistics, Vol. 4, pp. 87-94.
46. Rubin, D. (1987), Multiple Imputation for Nonresponse in Surveys, Wiley, New York, NY.
47. Rubin, D. (1996) "Multiple imputation after 18+years (with discussion)", Journal of the American Statistical Association, Vol. 91, pp. 473-89.
48. Rubin, D. and Schenker, N. (1986) "Multiple imputation for interval estimation from simple random sample with ignorable nonresponse", Journal of the American Statistical Association, Vol. 81, pp. 366-74.
49. Sande, L. (1982) "Imputation in surveys: coping with reality", The American Statistician, Vol. 36, pp. 145-52.
50. Sande, L. (1983) "Hot-deck imputation procedures", in Madow, W.G. and Olkin, I. (Eds), Incomplete Data in Sample Surveys, Vol. 3, Proceedings of the Symposium, Academic Press, New York, NY, pp. 339-49.
51. Schafer, J. (1997) Analysis of Incomplete Multivariate Data, Chapman & Hall, London.
52. Schafer, J. (1999), "Multiple imputation: a primer", Statistical Methods in Medical Research, Vol. 8, pp. 3-15.
53. Schafer, J. and Olsen, M. (1998), "Multiple imputation for multivariate missing-data problems: a data analyst's perspective", Multivariate Behavioral Research, Vol. 33, pp. 545-71.
54. Statistical Services of University of Texas (2000) "General FAQ #25: handling missing or incomplete data", available at: www.utexas.edu/cc/faqs/stat/general/gen25.html
55. van Buren, S., Boshuizen, H. and Knook, D. (1999), "Multiple imputation of missing blood pressure covariates in survival analysis", Statistics in Medicine, Vol. 18, pp. 681-94.
56. Vosburg, J. and Kumar, A. (2001) "Managing dirty data in organizations using ERP: lessons from a case study", Industrial Management & Data Systems, Vol. 101 No. 1, pp. 21-31.
57. Westphal, C. and Blaxton, T. (1998) Data Mining Solutions, Wiley, New York, NY.
58. Witten, I. and Frank, E. (2000), Data Mining, Academic Press, San Francisco, CA.
59. Xu, H. Horn Nord, J., Brown, N. and Nord, G.D. (2002) "Data quality issues in implementing an ERP", Industrial Management & Data Systems, Vol. 102 No. 1, pp. 47-58.