Empirical bayes versus standard Mantel-Haenszel statistics for detecting differential item functioning under small sample conditions

Journal of Experimental Education

Volumn 75, Issue 4, 2007, Pages 293-314

  Fidalgo, Ángel M ^a,d   Hashimoto, Kanako ^b   Bartram, Dave ^c   Muñiz, Jose ^a  

^a UNIVERSITY OF OVIEDO   (Spain)

^b Welfare And Medical Service Agency   (Japan)

^c Research Department of SHL Group Plc   (United Kingdom)

^d Laboratory of Psychobiology   (Spain)

Author keywords

Differential item functioning (DIF); Empirical bayes (EB) methods; Loss functions; Mantel Haenszel; Small samples

Indexed keywords

EID: 34547492330     PISSN: 00220973     EISSN: None     Source Type: Journal    
DOI: 10.3200/JEXE.75.4.293-316     Document Type: Article

References (34)

1. Agresti, A. (1990). Categorical data analysis. New York: Wiley.
2. Azocar, E, Areán, P., Miranda, J., & Muñoz, R. F. (2001). Differential item functioning in a Spanish translation of the Beck Depression Inventory. Journal of Clinical Psychology, 57(3), 355-365.
3. Camilli, G., & Smith, J. K. (1990). Comparison of the Mantel-Haenszel test with a randomized and a jackknife test for detecting biased items. Journal of Educational Statistics, 15, 53-67.
4. Ellis, B. B. (1995). A partial test of Hulin's Psychometric theory of measurement equivalence in translated test. European Journal of Psychological Assessment, 11(3), 184-193.
5. Ellis, B. B., & Mead, A. D. (2000). Assessment of the measurement equivalence of a Spanish translation of the 16PF Questionnaire. Educational Psychological Measurement, 60, 787-807.
6. Fidalgo, A. M. (1994). MHDIF: A computer program for detecting uniform and nonuniform differential item functioning with the Mantel-Haenszel procedure. Applied Psychological Measurement, 18, 300.
7. Fidalgo, A. M. (2005). Mantel-Haenszel methods. In B. S. Everitt & D. C. Howell (Eds.), Encyclopedia of Statistics in Behavioral Science (Vol. 3, pp. 1120-1126). Chichester, England: Wiley & Sons.
8. Fidalgo, A. M., Ferreres, D., & Muñiz, J. (2004). Utility of the Mantel-Haenszel procedure for detecting differential item functioning with small samples. Educational and Psychological Measurement, 64, 925-936.
9. Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (1995). Bayesian data analysis. London: Chapman & Hall.
10. Hidalgo-Montesinos, M. D., & Lopez-Pina, J. A. (2002). Two-stage equating in differential item functioning detection under the graded response model with the Raju area measures and the Lord statistic. Educational and Psychological Measurement, 62, 32-44.
11. Holland, W. P., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129-145). Hillsdale, NJ: LEA.
12. Kim, M. (2001). Detecting DIF across the different language groups in a speaking test. Language Testing, 18(1), 89-114.
13. Little, R. J. A. (1989). Testing the equality of two independent binomial proportions. The American Statistician, 43, 283-288.
14. Longford, N. T., Holland, P. W., & Thayer, D. T. (1993). Stability of the MH D-DIF statistics across populations. In W. P. Holland & H. Wainer (Eds.), Differential item functioning (pp. 171-196). Hillsdale, NJ: LEA.
15. Mantel, N., Fleiss, J. L. (1980). Minimum expected cell size requirements for the Mantel-Haenszel one-degree of freedom chi-squared test and a related rapid procedure. American Journal of Epidemiology, 112, 129-143.
16. Meyer, J. P., Huynh, H., & Seaman, M. A. (2004). Exact small-sample differential item functioning for polytomous items with illustration based on an attitude survey. Journal of Educational Measurement, 41, 331-344.
17. Muñiz, J., Hambleton, R. K., & Xing, D. (2001). Small sample studies to detect flaws in item translations. International Journal of Testing, 1, 115-135.
18. Orlando, M., & Marshall, G. N. (2002). Differential item functioning in a Spanish translation of the PTSD Checklist: Detection and evaluation of impact. Psychological Assessment, 14(1), 50-59.
19. Parshall, C. G., & Miller, T. R. (1995). Exact versus asymptotic Mantel-Haenszel DIF statistics: A comparison of performance under small-sample conditions. Journal of Educational Measurement, 32, 302-316.
20. Penfield, R. D. (2005). DIFAS: Differential item functioning analysis system. Applied Psychological Measurement, 29, 150-151.
21. Phillips, A., & Holland, P. W. (1987). Estimation of the variance of the log-odds-ratio estimate. Biometrics, 43, 425-431.
22. Robins, J., Breslow, N., & Greenland, S. (1986). Estimators of the Mantel-Haenszel variance consistent in both sparse data and large-strata limiting models. Biometrics, 42, 311-324.
23. Rogers, H. J., & Swaminathan, H. (1993). A comparison of logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17, 105-116.
24. Roussos, L. A., Schnipke, D. L., & Pashley, P. J. (1999). A generalized formula for the Mantel-Haenszel differential item functioning parameter. Journal of Educational and Behavioral Statistics, 24, 293-322.
25. Roussos, L. A., & Stout, W. F. (1996). Simulations studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel Type I error performance. Journal of Educational Measurement, 33, 215-230.
26. Sasaki, M. (1991). A comparison of two methods for detecting differential item functioning in an ESL placement test. Language Testing, 8(2), 95-111.
27. Spray, J. A., & Miller, T. R. (1992). Performance of the Mantel-Haenszel statistics and the standardized difference in proportion correct when population ability distributions are incongruent (ACT Research Rep. No. 92-1). Iowa City, IA: American College Testing.
28. Zieky, M. (1993). Practical questions in the use of DIF statistics in test development. In W. P. Holland & H. Wainer (Eds.), Differential item functioning (pp. 337-347). Hillsdale, NJ: LEA.
29. Zwick, R., & Thayer, D. T. (2002). Application of an empirical Bayes enhancement of Mantel-Haenszel differential item functioning analysis to a computed adaptative test. Applied Psychological Measurement, 36, 1-28.
30. Zwick, R., Thayer, D. T., & Lewis, C. (1997). An investigation of the validity of an empirical Bayes approach to Mantel-Haenszel DIF analysis. ETS Research Report No. 97-21. Princenton, NJ: Educational Testing Service.
31. Zwick, R., Thayer, D. T., & Lewis, C. (1999). An empirical Bayes approach to Mantel-Haenszel DIF analysis. Journal of Educational Measurement, 36, 1-28.
32. Zwick, R., Thayer, D. T., & Lewis, C. (2000). Using loss functions for DIF detection: An empirical Bayes approach. Journal of Educational and Behavioral Statistics, 25, 225-247.
33. Zwick, R., Thayer, D. T., & Wingersky, M. ( 1994). An simulation study of methods for assessing differential item functioning in computerized adaptive tests. Applied Psychological Measurement, 18, 121-140.
34. Zwick, R., Thayer, D. T., & Wingersky, M. (1995). Effect of Rasch calibration on ability and DIF estimation in computer-adaptive tests. Journal of Educational Measurement, 32, 341-363.