Generalized Mantel-Haenszel methods for differential item functioning detection

Educational and Psychological Measurement

Volumn 68, Issue 6, 2008, Pages 940-958

  Fidalgo, Angel M ^a,b   Madeira, Jaqueline M ^a  

^a UNIVERSITY OF OVIEDO   (Spain)

^b Laboratory of Psychobiology   (Spain)

Author keywords

Cochran Mantel Haenszel methods; Differential item functioning; Generalized Mantel Haenszel statistics; Polytomous items

Indexed keywords

EID: 56749169610     PISSN: 00131644     EISSN: 15523888     Source Type: Journal    
DOI: 10.1177/0013164408315265     Document Type: Article

References (67)

1. Agresti, A. (1990). Categorical data analysis. New York: Wiley.
2. Allen, N.L., & Donoghue, J.R. (1996). Applying the Mantel-Haenszel procedure to complex samples of items. Journal of Educational Measurement, 33, 231-251.
3. Ankenmann, R.D., Witt, E.A., & Dunbar, S.B. (1999). An investigation of the power of the likelihood ratio goodness-of-fit statistic in detecting differential item functioning. Journal of Educational Measurement, 36 (4). 277-300.
4. Birch, M.W. (1964). The detection of partial association, I: The 2 × 2 case. Journal of the Royal Statistical Society, Series B, 26, 313-324.
5. Birch, M.W. (1965). The detection of partial association, II: The general case. Journal of the Royal Statistical Society, Series B, 27, 111-124.
6. Camilli, G., & Congdon, P. (1999). Application of a method of estimating DIF for polytomous test items. Journal of Educational and Behavioural Statistics, 4, 323-341.
7. Camilli, G., & Penfield, D.A. (1997). Variance estimation for differential test functioning based on Mantel-Haenszel statistics. Journal of Educational Measurement, 34, 123-139.
8. Camilli, G., & Shepard, L.A. (1994). Methods for identifying biased test items. Newbury Park, CA: Sage.
9. Camilli, G., & Smith, J.K. (1990). Comparison of the Mantel-Haenszel test with a randomized and a jackknife test for detecting biased items. Journal of Educational Statistics, 15, 53-67.
10. Chang, H.H., Mazzeo, J., & Roussos, J. (1996). Detecting DIF for polytomously scored items: An adaptation of the SIBTEST procedure. Journal of Educational Measurement, 33, 333-353.
11. Clauser, B.E., & Mazor, K.M. (1998). Using statistical procedures to identify differentially functioning test items. Educational Measurement: Issues and Practice, 17, 31-44.
12. Clauser, B., Mazor, K.M., & Hambleton, R.K. (1993). The effects of purification of the matching criterion on the identification of DIF using the Mantel-Haenszel procedure. Applied Measurement in Education, 6, 269-279.
13. Cochran, W.G. (1954). Some methods for strengthening the common χ2 test. Biometrics, 10, 417-451.
14. Cox, D.R. (1958). The regression analysis of binary sequences. Journal of the Royal Statistical Society, Series B, 20, 215-232.
15. Donoghue, J.R., Holland, P.W., & Thayer, D.T. (1993). A Monte Carlo study of factors that affect the Mantel-Haenszel and standardization measures of differential item functioning. In W. P. Holland & H. Wainer (Eds.), Differential item functioning (pp. 137-166). Hillsdale, NJ: Lawrence Erlbaum.
16. Fidalgo, A.M. (2005). Mantel-Haenszel methods. In B. S. Everitt & D. C. Howell (Eds.), Encyclopedia of statistics in behavioral science (Vol. 3, pp. 1120-1126). Chichester, UK: Wiley.
17. Fidalgo, A.M., & Bartram, D. (2008). DIF detection in polytomous items using the generalized ordinal Mantel-Haenszel statistic. What type of scores should be selected ? Manuscript submitted for publication.
18. Fidalgo, A.M., Ferreres, D., & Muiz, J. (2004). Utility of the Mantel-Haenszel procedure for detecting differential item functioning with small samples. Educational and Psychological Measurement, 64, 925-936.
19. Fidalgo, A.M., Hashimoto, K., Bartram, D., & Muiz, J. (2007). Empirical Bayes versus standard Mantel-Haenszel statistics for detecting differential item functioning under small sample conditions. Journal of Experimental Education, 75, 293-314.
20. Fidalgo, A.M., Mellenbergh, G.J., & Muiz, J. (2000). Effects of amount of DIF, test length, and purification type on robustness and power of Mantel-Haenszel procedures. Methods of Psychological Research, 5, 43-53.
21. Fischer, G. (1995). Some neglected problems in IRT. Psychometrika, 60, 459-487.
22. Graubard, B.I., & Korn, E.L. (1987). Choice of column scores for testing independence in ordered 2 × K contingency tables. Biometrics, 43, 471-476.
23. Greenland, S. (1989). Generalized Mantel-Haenszel estimators for K 2 × J tables. Biometrics, 45, 183-191.
24. Hidalgo, M.D., & Lpez-Pina J. A. (2004). Differential item functioning detection and effect size: A comparison between logistic regression and Mantel-Haenszel procedures. Educational and Psychological Measurement, 64, 903-915.
25. Holland, W.P., & Thayer, D.T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum.
26. Koch, G.G., Imrey, P.B., Singer, J.M., Atkinson, S.S., & Stokes, M.E. (1985). Analysis of categorial data. Montreal, Canada: Presses de l'Universit de Montreal.
27. Koch, G.G., Sen, P.K., & Amara, I.A. (1985). Log-rank scores, statistics and test. In N. L. Johnson & S. Kotz (Eds.), Encyclopedia of statistical sciences (Vol. 5, pp. 136-142). New York: John Wiley.
28. Kristjansson, E., Aylesworth, R., McDowell, I., & Zumbo, B.D. (2005). A comparison of four methods for detecting differential item functioning in ordered response items. Educational and Psychological Measurement, 65, 935-953.
29. Kuritz, S.J., Landis, J.R., & Koch, G.G. (1988). A general overview of Mantel-Haenszel methods: Applications and recent developments. Annual Review of Public Health, 9, 123-160.
30. Landis, J.R., Heyman, E.R., & Koch, G.G. (1978). Average partial association in three-way contingency tables: A review and discussion of alternative tests. International Statistical Review, 46, 237-254.
31. Liu, I.-M., & Agresti, A. (1996). Mantel-Haenszel-type inference for cumulative odds ratios with a stratified ordinal response. Biometrics, 52, 1223-1234.
32. Mantel, N. (1963). Chi-square tests with one degree of freedom: Extensions of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58, 690-700.
33. Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719-748.
34. Mazor, K.M., Clauser, B.E., & Hambleton, R.K. (1992). The effect of sample size on the functioning of the Mantel-Haenszel statistic. Educational and Psychological Measurement, 52, 443-452.
35. Mazor, K.M., Clauser, B.E., & Hambleton, R.K. (1994). Identification of nonuniform differential item functioning using a variation of the Mantel-Haenszel procedure. Educational and Psychological Measurement, 54, 284-291.
36. Meyer, J.P., Huynh, H., & Seaman, M.A. (2004). Exact small-sample differential item functioning for polytomous items with illustration based on an attitude survey. Journal of Educational Measurement, 41, 331-344.
37. Mickey, R.M., & Elashoff, R.M. (1985). A generalization of the Mantel-Haenszel estimator of partial association for 2 × J × K tables. Biometrics, 41, 623-635.
38. Millsap, R.E., & Everson, H.T. (1993). Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement, 17, 297-334.
39. Muiz, J., Hambleton, R.K., & Xing, D. (2001). Small sample studies to detect flaws in item translations. International Journal of Testing, 1, 115-135.
40. Narayanan, P., & Swaminathan, H. (1994). Performance of the Mantel-Haenszel and simultaneous item bias procedures for detecting differential item functioning. Applied Psychological Measurement, 18, 315-328.
41. Narayanan, P., & Swaminathan, H. (1996). Identification of items that show nonuniform DIF. Applied Psychological Measurement, 20, 257-274.
42. Parshall, C.G., & Miller, T.R. (1995). Exact versus asymptotic Mantel-Haenszel DIF statistics: A comparison of performance under small-sample conditions. Journal of Educational Measurement, 32, 302-316.
43. Penfield, R.D. (2001). Assessing differential item functioning among multiple groups: A comparison of three Mantel-Haenszel procedures. Applied Measurement in Education, 14, 235-259.
44. Penfield, R.D., & Algina, J. (2003). Applying the Liu-Agresti estimator of the cumulative common odds ratio to DIF detection in polytomous items. Journal of Educational Measurement, 40 (4). 353-370.
45. Penfield, R.D., & Lam, T.C.M. (2000). Assessing differential item functioning in performance assessment: Review and recommendations. Educational Measurement: Issues and Practice, 19, 5-15.
46. Penny, J., & Johnson, R.L. (1999). How group differences in matching criterion distribution and IRT item difficulty can influence the magnitude of the Mantel-Haenszel chi-square DIF index. Journal of Experimental Education, 67, 343-366.
47. Potenza, M.T., & Dorans, N.J. (1995). DIF assessment for polytomously scored items: A framework for classification and evaluation. Applied Psychological Measurement, 19, 23-37.
48. Robins, J., Breslow, N., & Greenland, S. (1986). Estimators of the Mantel-Haenszel variance consistent in both sparse data and large-strata limiting models. Biometrics, 42, 311-324.
49. Rogers, H.J., & Swaminathan, H. (1993). A comparison of logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17, 105-116.
50. Roussos, L.A., Schnipke, D.L., & Pashley, P.J. (1999). A generalized formula for the Mantel-Haenszel differential item functioning parameter. Journal of Educational and Behavioral Statistics, 24 (3). 293-322.
51. Roussos, L.A., & Stout, W.F. (1996). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel Type I error performance. Journal of Educational Measurement, 33, 215-230.
52. Ryan, K.E., & Chiu, S. (2001). An examination of item context effects, DIF, and gender DIF. Applied Measurement in Education, 14, 73-90.
53. Shealy, R., & Stout, W. (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58, 159-194.
54. Stokes, M.E., Davis, C.S., & Koch, G.G. (2000). Categorical data analysis using the SAS system (2 nd ed.). Cary, NC: SAS Institute.
55. Su, Y.-H., & Wang, W.-C. (2005). Efficiency of the Mantel, generalized Mantel-Haenszel, and logistic discriminant function analysis methods in detecting differential item functioning in polytomous items. Applied Measurement in Education, 18, 313-350.
56. Uttaro, T., & Millsap, R.E. (1994). Factors influencing the Mantel-Haenszel procedure in the detection of differential item functioning. Applied Psychological Measurement, 18, 15-25.
57. Walker, C.M., & Beretvas, S.N. (2001). An empirical investigation demonstrating the multidimensional DIF paradigm: A cognitive explanation for DIF. Journal of Educational Measurement, 38, 147-163.
58. Wang, W.-C., & Su, Y.-H. (2004 a). Effect of average signed area between two item characteristic curves and test purification procedures on the DIF detection via the Mantel-Haenszel method. Applied Measurement in Education, 17, 113-144.
59. Wang, W.-C., & Su, Y.-H. (2004 b). Factors influencing the Mantel and generalized Mantel-Haenszel methods for the assessment of differential item functioning in polytomous items. Applied Psychological Measurement, 28, 450-480.
60. Welch, C., & Hoover, H.D. (1993). Procedures for extending item bias detection techniques to polytomously scored items. Applied Measurement in Education, 6, 1-19.
61. Yanagawa, T., & Fujii, Y. (1995). Projection-method Mantel-Haenszel estimator for K 2 × J tables. Journal of the American Statistical Association, 90, 649-656.
62. Zieky, M. (1993). Practical questions in the use of DIF statistics in test development. In W.P. Holland & H. Wainer (Eds.), Differential item functioning (pp. 337-347). Hillsdale, NJ: Lawrence Erlbaum.
63. Zwick, R. (1990). When do item response function and Mantel-Haenszel definitions of differential item functioning coincide ? Journal of Educational Statistics, 15, 185-197.
64. Zwick, R., Donoghue, J.R., & Grima, A. (1993 a). Assessment of differential item functioning for performance tasks. Journal of Educational Measurement, 30, 233-251.
65. Zwick, R., Donoghue, J.R., & Grima, A. (1993 b). Assessing differential item functioning in performance tests (ETS Research Rep. No. 93-14). Princeton, NJ: Educational testing Service.
66. Zwick, R., & Thayer, D.T. (1996). Evaluating the magnitude of differential item functioning in polytomous items. Journal of Educational and Behavioural Statistics, 21, 187-201.
67. Zwick, R., Thayer, D.T., & Mazzeo, J. (1997). Descriptive and inferential procedures for assessing differential item functioning in polytomous items. Applied Measurement in Education, 10, 321-344.