Missing data in educational research: A review of reporting practices and suggestions for improvement

Review of Educational Research

Volumn 74, Issue 4, 2004, Pages 525-556

  Peugh, James L ^a   Enders, Craig K ^a  

^a UNIVERSITY OF NEBRASKA LINCOLN   (United States)

Author keywords

EM algorithm; Maximum likelihood estimation; Missing data; Multiple imputation; NORM

Indexed keywords

EID: 12744272198     PISSN: 00346543     EISSN: None     Source Type: Journal    
DOI: 10.3102/00346543074004525     Document Type: Review

References (49)

1. Allison, P. D. (2002). Missing data. Thousand Oaks, CA: Sage.
2. Allison, P. D. (2000). Multiple imputation for missing data: A cautionary tale. Sociological Methods and Research, 28, 301-309.
3. Arbuckle, J. L. (1996). Full information estimation in the presence of incomplete data. Mahwah, NJ: Lawrence Erlbaum.
4. Azar, B. (2002). Finding a solution for missing data. Monitor on Psychology, 33, 70.
5. Collins, L. M., Schafer, J. L., & Kam, C. M. (2001). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6(4), 330-351.
6. Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Ser. B, 39, 1-38.
7. Enders, C. K. (in press). Estimation by maximum likelihood. In B. Everitt & D. C. Howell (Eds.), Encyclopedia of behavioral statistics. West Sussex, UK: Wiley.
8. Enders, C. K. (2001a). The impact of nonnormality on full information maximum likelihood estimation for structural equation models with missing data. Psychological Methods, 6, 352-370.
9. Enders, C. K. (2001b). The performance of the full information maximum likelihood estimator in multiple regression models with missing data. Educational and Psychological Measurement, 61, 713-740.
10. Enders, C. K. (2003). Using the EM algorithm to estimate coefficient alpha for scales with item level missing data. Psychological Methods, 8, 322-337.
11. Enders, C. K., & Bandalos, D. L. (2001). The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 8, 430-457.
12. Gold, M. S., & Bentler, P. M. (2000). Treatments of missing data: A Monte Carlo comparison of RBHDI, iterative stochastic regression imputation, and expectation-maximization. Structural Equation Modeling: A Multidisciplinary Journal, 7, 319-355.
13. Graham, J. W. (2003). Adding missing-data relevant variables to FIML-based structural equation models. Structural Equation Modeling: A Multidisciplinary Journal, 10, 80-100.
14. Graham, J. W., & Hofer, S. M. (2000). Multiple imputation in multivariate research. In T. D. Little & K. U. Schnabel (Eds.), Modeling longitudinal and multilevel data: Practical issues, applied approaches, and specific examples (pp. 201-218, 269-281). Mahwah, NJ: Lawrence Erlbaum.
15. Graham, J. W., Hofer, S. M., & MacKinnon, D. P. (1996). Maximizing the usefulness of data obtained with planned missing value patterns: An application of maximum likelihood procedures. Multivariate Behavioral Research, 31(2), 197-218.
16. Graham, J. W., & Schafer, J. L. (1999). On the performance of multiple imputation for multivariate data with small sample size. In Rick H. Hoyle (Ed.), Statistical strategies for small sample research (pp. 1-29). Thousand Oaks, CA: Sage.
17. Graham, J. W., Taylor, B. J., & Cumsille, P. E. (2001). Planned missing-data designs in analysis of change. In L. M. Collins & A. G. Sayer (Eds.), New methods for the analysis of change (pp. 335-353). Washington, DC: American Psychological Association.
18. Henson, R. K. (1999). Multivariate normality: What is it and how is it assessed? In B. Thompson (Ed.), Advances in social science methodology (Vol. 5, pp. 193-212). Greenwich, CT: JAI Press.
19. Hill, J. L., Brooks-Gunn, J., & Waldfogel, J. (2003). Sustained effects of high participation in an early intervention for low-birth-weight premature infants. Developmental Psychology, 39, 730-744.
20. Horton, N. J., & Lipsitz, S. R. (2001). Multiple imputation in practice: Comparison of software packages for regression models with missing variables. The American Statistician, 55, 244-254.
21. Kaplan, D. (1995). The impact of BIB spiraling-induced missing data patterns on goodness-of-fit tests in factor analysis. Journal of Educational and Behavioral Statistics, 20(1), 69-82.
22. Kenward, M. G., & Molenberghs, G. (1998). Likelihood based frequentist inference when data are missing at random. Statistical Science, 13, 236-247.
23. Keselman, H. J., Huberty, C. J., Lix, L. M., Olejnik, S., Cribbie, R. A., Donohue, B., et al. (1998). Statistical practices of educational researchers: An analysis of their ANOVA, MANOVA, and ANCOV A techniques. Review of Educational Research, 3, 350-386.
24. Li, K. H., Raghunathan, T. E., & Rubin, D. B. (1991). Large-sample significance levels from multiple imputed data using moment-based statistics and an F distribution. Journal of the American Statistical Association, 86, 1065-1073.
25. Little, R. J. A. (1988). A test of missing completely at random for multivariate data with missing values. Journal of the American Statistical Association, 83, 1198-1202.
26. Little, R. J. A., & Rubin, D. B. (1987). Statistical analysis with missing data. New York: Wiley.
27. Little, R. J. A., & Rubin, D. B. (2002). Statistical analysis with missing data. Hoboken, NJ: Wiley.
28. Mardia, K. V. (1974). Applications of some measures of multivariate skewness and kurtosis in testing normality and robustness studies. Sankhya, Ser. B, 36, 115-128.
29. McQueen, A., Getz, J. G., & Bray, J. H. (2003). Acculturation, substance use, and deviant behavior: Examining separation and family conflict as mediators. Child Development, 74, 1737-1750.
30. Muthén, B., Kaplan, D., & Hollis, M. (1987). On structural equation modeling with data that are not missing completely at random. Psychometrika, 52(3), 431-462.
31. Muthén, L. K., & Muthén, B. O. (2004). Mplus user's guide (3rd ed.). Los Angeles: Muthén & Muthén.
32. Pedhazur, E. J. (1997). Multiple regression in behavioral research: Explanation and prediction (3rd ed.). Fort Worth, TX: Harcourt Brace.
33. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.
34. Roth, P. L. (1994). Missing data: A conceptual review for applied psychologists. Personnel Psychology, 47(3), 537-560.
35. Rubin, D. B. (1976). Inference and missing data. Biometrika (63), 581-592.
36. Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New York: Wiley.
37. Sadler, P., & Woody, E. (2003). Is who you are who you're talking to? Interpersonal style and complementarity in mixed-sex interactions. Journal of Personality and Social Psychology, 84, 80-96.
38. Schafer, J. L. (1997). Analysis of incomplete multivariate data. New York: Chapman and Hall.
39. Schafer, J. L. (1999). NORM: Multiple imputation of incomplete multivariate data under a normal model [Computer software]. University Park, PA: Department of Statistics, Pennsylvania State University.
40. Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147-177.
41. Schafer, J. L., & Olsen, M. K. (1998). Multiple imputation for multivariate missing data problems: A data analyst's perspective. Multivariate Behavioral Research, 33, 545-571.
42. Sinharay, S., Stern, H. S., & Russell, D. (2001). The use of multiple imputation for the analysis of missing data. Psychological Methods, 6(4), 317-329.
43. Snyder, J., Brooker, M., Patrick, M. R., Snyder, A., Schrepferman, L., & Stoolmiller, M. (2003). Observed peer victimization during early elementary school: Continuity, growth, and relation to risk for child antisocial and depressive behavior. Child Development, 74, 1881-1898.
44. Thompson, B. (1999, April). Common methodology mistakes in educational research, revisited, along with a primer on both effect sizes and the bootstrap. Invited address presented at the annual meeting of the American Educational Research Association, Montreal. (ERIC Document Reproduction Service No. ED 429 110)
45. Tolan, P. H., Gorman-Smith, D., & Henry, D. B. (2003). The developmental ecology of urban males' youth violence. Developmental Psychology, 39, 274-291.
46. West, S. G. (2001). New approaches to missing data in psychological research: Introduction to the special section. Psychological Methods, 6, 315-316.
47. Wilkinson, L., & Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604.
48. Wothke, W. (2000). Longitudinal and multi-group modeling with missing data. In T. D. Little, K. U. Schnabel, & J. Baumert (Eds.), Modeling longitudinal and multiple group data: Practical issues, applied approaches and specific examples (pp. 219-240). Mahwah, NJ: Lawrence Erlbaum.
49. Yuan, K.-H., & Bentler, P. M. (2000). Three likelihood-based methods for mean and covariance structure analysis with nonnormal missing data. In M. Becker & M. Sobel (Eds.), Sociological methodology, 2000 (pp. 165-200). Malden, MA: Blackwell.