Effect of unequal variances in proficiency distributions on type-I error of the Mantel-Haenszel chi-square test for differential item functioning

Journal of Educational Measurement

Volumn 42, Issue 2, 2005, Pages 101-131

  Monahan, Patrick O ^a,c   Ankenmann, Robert D ^b  

^a INDIANA UNIVERSITY   (United States)

^b UNIVERSITY OF IOWA   (United States)

^c INDIANA UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 17144419380     PISSN: 00220655     EISSN: None     Source Type: Journal    
DOI: 10.1111/j.1745-3984.2005.00006     Document Type: Review

References (50)

1. ACT. (1992). High school profile report. Iowa City: ACT.
2. Alien, N. L., & Donoghue, J. R. (1996). Applying the Mantel-Haenszel procedure to complex samples of items. Journal of Educational Measurement, 33(2), 231-251.
3. Benbow, C. P. (1988). Sex differences in mathematical reasoning ability in intellectually talented preadolescents: Their nature, effects, and possible causes. Behavioral and Brain Sciences, 11, 169-232.
4. Benbow, C. P., & Stanley, J. C. (1980). Sex differences in mathematical ability: Fact or artifact? Science, 210, 1262-1264.
5. Benbow, C. P., & Stanley, J. C. (1983). Sex differences in mathematical reasoning ability: More facts. Science, 222, 1029-1031.
6. Birch, M. W. (1964). The detection of partial association I: The 2 × 2 case. Journal of the Royal Statistical Society, B26, 313-324.
7. Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 395-479). Menlo Park, CA: Addison-Wesley.
8. Chang, S.-W. (1998). A comparative study of item exposure control methods in computerized adaptive testing. Unpublished doctoral dissertation, The University of Iowa, Iowa City.
9. Clauser, B., Mazor, K., & Hambleton, R. K. (1993). The effects of purification of the matching criterion on the identification of DIF using the Mantel-Haenszel procedure. Applied Measurement in Education, 6(4), 269-279.
10. Donoghue, J. R., Holland, P. W., & Thayer, D. T. (1993). A Monte Carlo study of factors that affect the Mantel-Haenszel and standardization measures of differential item functioning. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 137-166). Hillsdale, NJ: Lawrence Erlbaum.
11. Feingold, A. (1992). Sex differences in variability in intellectual abilities: A new look at an old controversy. Review of Educational Research, 62, 61-84.
12. Feingold, A. (1994). Gender differences in variability in intellectual abilities: A cross-cultural perspective. Sex Roles, 30, 81-90.
13. Feingold, A. (1995). The additive effects of differences in central tendency and variability are important in comparisons between groups. American Psychologist, 50, 5-13.
14. Feldt, L. S., Forsyth, R. A., Ansley, T. N., & Alnot, S. D. (1993). Iowa Tests of Educational Development. Iowa City: The University of Iowa.
15. Hedges, L. V., & Friedman, L. (1993). Gender differences in variability in intellectual abilities: A reanalysis of Feingold's results. Review of Educational Research, 63, 94-105.
16. Hedges, L. V., & Nowell, A. (1995). Sex differences in mental test scores, variability, and numbers of high-scoring individuals. Science, 269,41-45.
17. Holland, P. W. (1985, October). On the study of differential item performance without IRT. Paper presented at the Proceedings of the Military Testing Association.
18. Holland, P. W., & Thayer, D. T. (1985). An alternate definition of the ETS scale of item difficulty (Research Report RR-85-43). Princeton, NJ: Educational Testing Service.
19. Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum.
20. Holland, P. W., & Wainer, H. (Eds.). (1993). Differential item functioning. Hillside, NJ: Lawrence Erlbaum.
21. Hoover, H. D., Hieronymus, A. N., Frisbie, D. A., & Dunbar, S. B. (1993). Iowa Tests of Basic Skills. Chicago: The Riverside Publishing Company.
22. Humphreys, L. G. (1988). Sex differences in variability may be more important than sex differences in means. Behavioral and Brain Sciences, 11, 195-196.
23. Lewis, C. (1993). A note on the value of including the studied item in the test score when analyzing test items for DIF. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 317-319). Hillsdale, NJ: Lawrence Erlbaum.
24. Lord, F. M. (1977). A study of item bias using item characteristic curve theory. In Y. H. Poortinga (Ed.), Basic problems in cross-cultural psychology (pp. 19-29). Amsterdam: Swets & Zeitlinger.
25. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
26. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
27. Lu, S.-M. (1996). The relationship between item statistics and the Mantel-Haenszel and standardization DIF statistics when comparison groups differ in ability (Doctoral dissertation, The University of Iowa, 1996). Dissertation Abstracts International, 457(07), 2987.
28. Mantel, N., & Haenszel, W. M. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719-748.
29. Martin, D. J., & Hoover, H. D. (1987). Sex differences in educational achievement: A longitudinal study. Journal of Early Adolescence, 7(1), 65-83.
30. Meredith, W., & Millsap, R. E. (1992). On the misuse of manifest variables in the detection of measurement bias. Psychometrika, 57, 289-311.
31. Millsap, R. E., & Meredith, W. (1992). Inferential conditions in the statistical detection of measurement bias. Applied Psychological Measurement, 16(4), 389-402.
32. Mislevy, R. J., & Bock, R. D. (1990). BILOG: Item analysis and test scoring with binary logistic models (2nd ed.). Mooresville, IN: Scientific Software.
33. Narayanan, P., & Swaminathan, H. (1994). Performance of the Mantel-Haenszel and simultaneous item bias procedures for detecting differential item functioning. Applied Psychological Measurement, 18(4), 315-328.
34. Pommerich, M., Spray, J. A., & Parshall, C. G. (1995). An analytical evaluation of two common-odds ratios as population indicators of DIF (ACT Research Report Series 95-1). Iowa City, IA: ACT.
35. Rasch, G. (1966a). An individualistic approach to item analysis. In P. F. Lazarsfeld & N. W. Henry (Eds.), Readings in mathematical social science. Chicago: Science Research Associates.
36. Rasch, G. (1966b). An item analysis which takes individual differences into account. The British Journal of Mathematical and Statistical Psychology, 19, 49-57.
37. Rogers, H. J., & Swaminathan, H. (1993). A comparison of logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17(2), 105-116.
38. Roussos, L., & Stout, W. (1996). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel type I error performance. Journal of Educational Measurement, 33(2), 215-230.
39. Roussos, L. A., Schnipke, D. L., & Pashley, P. J. (1999). A generalized formula for the Mantel-Haenszel differential item functioning parameter. Journal of Educational and Behavioral Statistics, 24(3), 293-322.
40. SAS Institute Inc. (1989). SAS/STAT user's guide, version 6 (4th ed., Vol. 1). Gary, NC: SAS Institute Inc.
41. Shealy, R., & Stout, W. (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58(2), 159-194.
42. Shepard, L. A., Camilli, G., & Averill, M. (1981). Comparison of procedures for detecting test-item bias with both internal and external ability criteria. Journal of Educational Statistics, 6, 317-375.
43. Shepard, L. A., Camilli, G., & Williams, D. M. (1984). Accounting for statistical artifacts in item bias research. Journal of Educational Statistics, 9, 93-128.
44. Thissen, D., & Wainer, H. (Eds.). (2001). Test scoring. Mahwah, NJ: Lawrence Erlbaum.
45. Uttaro, T., & Millsap, R. E. (1994). Factors influencing the Mantel-Haenszel procedure in the detection of differential item functioning. Applied Psychological Measurement, 18(1), 15-25.
46. Wright, B. D. (1977). Solving measurement problems with the Rasch model. Journal of Educational Measurement, 14, 97-116.
47. Zieky, M. (1993). Practical questions in the use of DIF statistics in test development. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 337-347). Hillsdale, NJ: Lawrence Erlbaum.
48. Zwick, R. (1990). When do item response function and Mantel-Haenszel definitions of differential item functioning coincide? Journal of Educational Statistics, 75(3), 185-197.
49. Zwick, R., Thayer, D. T., & Mazzeo, J. (1997). Descriptive and inferential procedures for assessing differential item functioning in polytomous items. Applied Measurement in Education, 10(4), 321-344.
50. Zwick, R., Thayer, D. T., & Wingersky, M. (1994). A simulation study of methods for assessing differential item functioning in computerized adaptive tests. Applied Psychological Measurement, 18(2), 121-140.