Missing Data: Five Practical Guidelines

Organizational Research Methods

Volumn 17, Issue 4, 2014, Pages 372-411

  Newman, Daniel A ^a  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

EM algorithm; full information maximum likelihood (FIML); missing data; multiple imputation; R syntax R code

Indexed keywords

EID: 84907542751     PISSN: 10944281     EISSN: 15527425     Source Type: Journal    
DOI: 10.1177/1094428114548590     Document Type: Article

References (56)

1. Allison P. D.Missing data. Thousand Oaks, CA: Sage; 2002:
2. Anseel F.,Lievens F.,Schollaert E.,Choragwicka B.Response rates in organizational science, 1995-2008: A meta-analytic review and guidelines for survey researchers.Journal of Business and Psychology. 2010;25:335-349
3. Bennett R. J.,Robinson S. L.Development of a measure of workplace deviance.Journal of Applied Psychology. 2000;85 ((3)): 349-360
4. Bernaards C. A.,Sijtsma K.Influence of imputation and EM methods on factor analysis when item nonresponse in questionnaire data is nonignorable.Multivariate Behavioral Research. 2000;35 ((3)): 321-364
5. Brannick M. T.Implications of empirical Bayes meta-analysis for test validation.Journal of Applied Psychology. 2001;86:468-480
6. Collins L. M.,Schafer J. L.,Kam C. M.A comparison of inclusive and restrictive strategies in modern missing data procedures.Psychological Methods. 2001;6:330-351
7. Cycyota C. S.,Harrison D. A.What (not) to expect when surveying executives a meta-analysis of top manager response rates and techniques over time.Organizational Research Methods. 2006;9 ((2)): 133-160
8. Dempster A. P.,Laird N. H.,Rubin D. B.Maximum likelihood from incomplete data via the EM algorithm.Journal of the Royal Statistical Society. 1977;B39:1-38
9. Dillman D. A.Mail and telephone surveys: The total design method. New York, NY: Wiley; 1978:
10. Downey R. G.,King C. V.Missing data in Likert ratings: A comparison of replacement methods.The Journal of General Psychology. 1998;125 ((2)): 175-191
11. Enders C. K.The impact of nonnormality on full information maximum-likelihood estimation for structural equation models with missing data.Psychological Methods. 2001a;6:352-370
12. Enders C. K.A primer on maximum likelihood algorithms for use with missing data.Structural Equation Modeling. 2001b;8:128-141
13. Enders C. K.Applied missing data analysis. New York, NY: Guilford; 2010:
14. Enders C. K.,Baraldi A. N.,Cham H.Estimating interaction effects with incomplete predictor variables.Psychological Methods. 2014;19:39-55
15. Enders C. K.,Peugh J. L.Using an EM covariance matrix to estimate structural equation models with missing data: Choosing an adjusted sample size to improve the accuracy of inferences.Structural Equation Modeling. 2004;11:1-19
16. Finkbeiner C.Estimation for the multiple factor model when data are missing.Psychometrika. 1979;44 ((4)): 409-420
17. Gelman A.,Carlin J. B.,Stern H. S.,Dunson D. B.,Vehtari A.,Rubin D. B.Bayesian data analysis. (2013). Bayesian data analysis (3rd ed.). Boca Raton, FL: Taylor & Francis.Boca Raton, FL: Taylor & Francis; 2013:
18. Glynn R. J.,Laird N. M.,Rubin D. B.Drawing inferences from self-selected samples. Wainer H., ed. New York, NY: Springer-Verlag; 1986:115-142.
19. Gold M. S.,Bentler P. M.Treatments of missing data: A Monte Carlo comparison of RBHDI, iterative stochastic regression imputation, and expectation-maximization.Structural Equation Modeling. 2000;7:319-355
20. Gold M. S.,Bentler P. M.,Kim K. H.A comparison of maximum-likelihood and asymptotically distribution-free methods of treating incomplete nonnormal data.Structural Equation Modeling. 2003;10:47-79
21. Graham J. W.Adding missing-data-relevant variables to FIML-based structural equation models.Structural Equation Modeling. 2003;10:80-100
22. Graham J. W.Missing data analysis: Making it work in the real world.Annual Review of Psychology. 2009;60:549-576
23. Graham J. W.,Olchowski A. E.,Gilreath T. D.How many imputations are really needed? Some practical clarifications of multiple imputation theory.Prevention Science. 2007;8 ((3)): 206-213
24. Graham J. W.,Schafer J. L.On the performance of multiple imputation for multivariate data with small sample size. In.Statistical Strategies for Small Sample Research. 1999;:
25. Graham J. W.,Taylor B. J.,Olchowski A. E.,Cumsille P. E.Planned missing data designs in psychological research.Psychological Methods. 2006;11 ((4)): 323-343
26. Heckman J. T.Sample selection bias as a specification error.Econometrica. 1979;47:153-161
27. Little R. J. A.Pattern mixture models for multivariate incomplete data.Journal of the American Statistical Association. 1993;88:125-134
28. Little R. J. A.,Rubin D. B.Statistical analysis with missing data. New York, NY: Wiley; 1987:
29. Little R. J. A.,Rubin D. B.Statistical analysis with missing data. (2002). Statistical analysis with missing data (2nd ed.). New York, NY: Wiley.New York, NY: Wiley; 2002:
30. Marsh H. W.Pairwise deletion for missing data in structural equation models: Nonpositive definite matrices, parameter estimates, goodness of fit, and adjusted sample sizes.Structural Equation Modeling. 1998;5:22-36
31. McKnight P. E.,McKnight K. M.,Sidani S.,Figueredo A. J.Missing data: A gentle introduction. New York, NY: Guilford Press; 2007:
32. Mistler S. A.Proceedings of the SAS Global Forum. 2013:
33. The Belmont report: Ethical principles and guidelines for the protection of human subjects of research. Washington, DC: ERIC Clearinghouse; 1979:
34. Newman D. A.Longitudinal modeling with randomly and systematically missing data: A simulation of ad hoc, maximum likelihood, and multiple imputation techniques.Organizational Research Methods. 2003;6:328-362
35. Newman D. A.Statistical and methodological myths and urban legends: Doctrine, verity, and fable in the organizational and social sciences. Lance C. E.Vandenberg R. J., ed. New York, NY: Routledge; 2009:7-36.
36. Newman D. A.,Cottrell J. M.More statistical and methodological myths and urban legends. Lance C. E.Vandenberg R. J., ed. New York, NY: Routledge; in press:
37. Newman D. A.,Jacobs R. R.,Bartram D.Choosing the best method for local validity estimation: Relative accuracy of meta-analysis versus a local study versus Bayes-analysis.Journal of Applied Psychology. 2007;92:1394-1413
38. Newman D. A.,Sin H. P.How do missing data bias estimates of within-group agreement? Sensitivity of SDWG, CVWG, rWG(J), rWG(J)*, and ICC to systematic nonresponse.Organizational Research Methods. 2009;12:113-147
39. Peugh J. L.,Enders C. K.Missing data in educational research: A review of reporting practices and suggestions for improvement.Review of Educational Research. 2004;74:525-556
40. Puhani P. A.The Heckman correction for sample selection and its critique.Journal of Economic Surveys. 2000;14:53-67
41. Rogelberg S. G.,Conway J. M.,Sederburg M. E.,Spitzmuller C.,Aziz S.,Knight W. E.Profiling active and passive nonrespondents to an organizational survey.Journal of Applied Psychology. 2003;88:1104-1114
42. Rosenthal R.Science and ethics in conducting, analyzing, and reporting psychological research.Psychological Science. 1994;5 ((3)): 127-134
43. Roth P. L.,BeVier C. A.Response rates in HRM/OB survey research: Norms and correlates, 1990-1994.Journal of Management. 1998;24:97-117
44. Roth P. L.,Switzer F. S.,Switzer D. M.Missing data in multiple item scales: A Monte Carlo analysis of missing data techniques.Organizational Research Methods. 1999;2:211-232
45. Rubin D. B.Inference and missing data.Biometrika. 1976;63:581-592
46. Rubin D. B.Multiple imputation for nonresponse in surveys. Hoboken, NJ: Wiley; 1987:
47. Savalei V.,Bentler P. M.A two-stage approach to missing data: Theory and application to auxiliary variables.Structural Equation Modeling. 2009;16 ((3)): 477-497
48. Schafer J. L.Analysis of incomplete multivariate data. New York, NY: Chapman & Hall; 1997:
49. Schafer J. L.,Graham J. W.Missing data: Our view of the state of the art.Psychological Methods. 2002;7:147-177
50. Schmidt F. L.,Hunter J. E.,Urry V. W.Statistical power in criterion-related validation studies.Journal of Applied Psychology. 1976;61:473-485
51. Spitzmuller C.,Glenn D. M.,Barr C. D.,Rogelberg S. G.,Daniel P.“If you treat me right, I reciprocate”: Examining the role of exchange in survey response.Journal of Organizational Behavior. 2006;27:19-35
52. Switzer F. S.,Roth P. L.,Switzer D. M.Systematic data loss in HRM settings: A Monte Carlo analysis.Journal of Management. 1998;24:763-779
53. Thorndike R. L.Personnel selection: Test and measurement techniques. New York, NY: Wiley; 1949:
54. Winship C.,Mare R. D.Models for sample selection bias.Annual Review of Sociology. 1992;18:327-350
55. van Buuren S.Multiple imputation of multilevel data.The Handbook of Advanced Multilevel Analysis. 2011;:pp. 173-196
56. Yammarino F. J.,Skinner S. J.,Childers T. L.Understanding mail survey response behavior.Public Opinion Quarterly. 1991;55:613-629