Impact of missing data on the detection of differential item functioning: The case of mantel-haenszel and logistic regression analysis

Educational and Psychological Measurement

Volumn 69, Issue 1, 2009, Pages 18-34

  Robitzsch, Alexander ^a   Rupp, André A ^a,b  

^a HUMBOLDT UNIVERSITY   (Germany)

^b UNIVERSITY OF MARYLAND   (United States)

Author keywords

Differential item functioning; Logistic regression analysis; Mantel Haenszel statistic; Missing data

Indexed keywords

EID: 58849123354     PISSN: 00131644     EISSN: 15523888     Source Type: Journal    
DOI: 10.1177/0013164408318756     Document Type: Article

References (34)

1. Adams, R.J., Wilson, M., & Wu, W. (1997). The multidimensional random coefficients multinomial logit model. Applied Psychological Measurement, 21, 1-23.
2. Agresti, A. (2002). Categorical data analysis (2 nd ed.). New York: John Wiley.
3. Allison, P.D. (2002). Missing data. Thousand Oaks, CA: Sage.
4. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards of educational and psychological testing. Washington, DC: Author.
5. Banks, K., & Walker, C. (2006, April). Performance of SIBTEST when focal group examinees have missing data. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco, CA.
6. Clauser, B., & Mazor, K. (1998). Using statistical procedures to identify differentially functioning test items. Educational Measurement: Issues & Practice, 17 (1). 31-44.
7. Donoghue, J.R., Holland, P.W., & Thayer, D.T. (1993). A Monte Carlo study of factors that affect the Mantel-Haenszel and standardization measures of differential item functioning. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 137-166). Hillsdale, NJ: Lawrence Erlbaum.
8. Dorans, N.J., & Holland, P.W. (1993). DIF detection and description: Mantel-Haneszel and standardization. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 35-66). Hillsdale, NJ: Lawrence Erlbaum.
9. Durrant, G.B. (2005, June). Imputation methods in the social sciences: A methodological review. NCRM Working Paper Series. Retrieved October 14, 2007, from http://www.lshtm.ac.uk/msu/missingdata/papers/durrantOct05.pdf
10. Embretson, S.E., & Reise, S.P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum.
11. Ferne, T., & Rupp, A.A. (2007). A synthesis of 15 years of research on DIF in language testing: Methodological advances, challenges, and recommendations. Language Assessment Quarterly, 4 (2). 113-148.
12. Fidalgo, A.M., Mellenbergh, G.J., & Muniz, J. (2000). Effects of amount of DIF, test length, and purification types on robustness and power of Mantel-Haenszel procedures. Methods of Psychological Research Online, 5, 43-53.
13. Finch, H. (2005). The MIMIC model as a method for detecting DIF: Comparison with Mantel-Haenszel, SIBTEST, and the IRT likelihood ratio. Applied Psychological Measurement, 29, 278-295.
14. Glas, C.A.W., & Pimentel, J.L. (2006). Modeling nonignorable missing data processes in item calibration (Law School Admission Council Computerized Testing Report No. 04-07). Law School Admission Council. Retrieved October 14, 2007, from http://www.lsacnet.org/Research/Modeling-Nonignorable-Missing-Data- Processes-in-Item-Calibration.pdf
15. Gotzman, A.J., & Boughton, K.A. (2004, April). A comparison of Type I error and power rates for the Mantel-Haenszel and SIBTEST procedures when group difference are large and unbalanced. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.
16. Kamata, A. (2001). Item analysis by the hierarchical generalized linear model. Journal of Educational Measurement, 38, 79-93.
17. Kelderman, H., & Macready, G.B. (1990). The use of loglinear models for assessing differential item functioning across manifest and latent examinee groups. Journal of Educational Measurement, 27, 307-327.
18. Lee, Y., & Klockars, A. (2005, April). The effects of sample size, test length, and inclusion criteria on the use of SIBTEST. Paper presented at the annual meeting of the American Educational Research Association, Montraal, Quebec, Canada.
19. Monahan, P.O., McHorney, C.A., Stump, T.E., & Perkins, A.J. (2007). Odds ratio, delta, ETS classification, and standardization measures of DIF magnitude for binary logistic regression. Journal of Educational and Behavioral Statistics, 32, 92-109.
20. Penfield, R.D., & Lam, T.C.M. (2000). Assessing differential item functioning in performance assessment: Review and recommendations. Educational Measurement: Issues and Practice, 19, 5-15.
21. Peugh, J.L., & Enders, C.K. (2004). Missing data in educational research: A review of reporting practices and suggestions for improvement. Review of Educational Research, 74, 525-556.
22. Rousseau, M., Bertrand, R., & Boiteau, N. (2004, April). Impact of missing data on robustness of DIF IRT-based and non IRT-based methods. Paper presented at the annual meeting of the American Educational Research Association, San Diego, CA.
23. Rousseau, M., Bertrand, R., & Boiteau, N. (2006, April). Impact of missing data treatment on the efficiency of DIF methods. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco, CA.
24. Rubin, D.P., & Little, R.J.A. (2002). Statistical analysis with missing data (2 nd ed.). New York: John Wiley.
25. Rupp, A.A., Choi, H., & Ferne, T. (2006, April). Integration of psychometric and cognitive information into differential item functioning (DIF) analyses: Impact of imputation methods and a cognitive approach to predicting DIF in SAIP reading comprehension items. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco, CA.
26. Schafer, J.L. (1997). Analysis of incomplete multivariate data. Boca Raton, FL: Chapman & Hall/CRC.
27. Schafer, J.L. (1999). NORM: Multiple imputation of incomplete multivariate data under a normal model, version 2 [Computer software]. Retrieved October 14, 2007, from www.stat.psu.edu/ĝ̂1/4jls/ misoftwa.html; http://www.stat.psu.edu/%7Ejls/misoftwa.html
28. Schafer, J.L., & Graham, J.W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7, 147-177.
29. Sedivy, S.K., Zhang, B., & Traxel, N.M. (2006, April). Detection of differential item functioning with polytomous items in the presence of missing data. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco, CA.
30. Sinharay, S., Stern, H.S., & Russell, D. (2001). The use of multiple imputation for the analysis of missing data. Psychological Methods, 6, 317-329.
31. Swaminathan, H., & Rogers, H.J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361-370.
32. van Buuren, S., Boshuizen, H.C., & Knook, D.L. (1999). Multiple imputation of missing blood pressure covariates in survival analysis. Statistics in Medicine, 18, 681-694.
33. van Buuren, S., & Oudshoorn, K. (1999). Flexible multivariate imputation by MICE (Technical Report No. PG/VGZ/99.054). Leiden: Netherlands Organization for Applied Scientific Research.
34. Zumbo, B. (1999). A handbook on the theory and methods of differential item functioning: Logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores. Ottawa, Canada: Directorate of Human Resources Research and Evaluation, Department of National Defense.