DIF detection and effect size measures for polytomously scored items

Journal of Educational Measurement

Volumn 44, Issue 2, 2007, Pages 93-116

  Kim, Seock Ho ^a   Cohen, Allan S ^a   Alagoz, Cigdem ^a   Kim, Sukwoo ^a,b  

^a UNIVERSITY OF GEORGIA   (United States)

^b Pusan National University   (South Korea)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 34247850754     PISSN: 00220655     EISSN: 17453984     Source Type: Journal    
DOI: 10.1111/j.1745-3984.2007.00029.x     Document Type: Article

References (55)

1. Agresti, A. (1990). Categorical data analysis. New York : Wiley.
2. Agresti, A. (1996). An introduction to categorical data analysis. New York : Wiley.
3. American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME) (1999). Standards for educational and psychological testing. Washington, DC : Author.
4. Ankenmann, R. D., Witt, E. A., & Dunbar, S. B. (1999). An investigation of the power of the likelihood ratio goodness-of-fit statistic in detecting differential item functioning. Journal of Educational Measurement, 36, 279 300.
5. Baker, F. B., & Kim, S.-H. (2004). Item response theory: Parameter estimation techniques (2nd ed.). New York : Dekker.
6. Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443 459.
7. Chang, H.-H., Mazzeo, J., & Roussos, L. (1996). Detecting DIF for polytomously scored items: An adaptation of the SIBTEST procedure. Journal of Educational Measurement, 33, 333 353.
8. Cohen, A. S., Kim, S.-H., & Baker, F. B. (1993). Detection of differential item functioning in the graded response model. Applied Psychological Measurement, 17, 335 350.
9. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ : Erlbaum.
10. Cox, D. R., & Snell, E. J. (1989). The analysis of binary data (2nd ed.). New York : Chapman and Hall.
11. Dorans, N. J., & Holland, P. W. (1993). DIF detection and description: Mantel-Haenszel and standardization. In P. W. Holland & H. Wainer (Eds Differential item functioning (pp. 35 66). Hillsdale, NJ : Erlbaum.
12. Dorans, N. J., & Kulick, E. (1986). Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the Scholastic Aptitude Test. Journal of Educational Measurement, 23, 355 368.
13. Dorans, N. J., & Schmitt, A. P. (1991). Constructed response and differential item functioning: A pragmatic approach (Research Rep. No. RR-91-47). Princeton, NJ : Educational Testing Service.
14. Dunbar, S. B., Koretz, D. M., & Hoover, H. D. (1991). Quality control in the development and use of performance assessment. Applied Psychological Measurement, 4, 289 303.
15. du Toit, M. (Ed 2002) IRT from SSI. Lincolnwood, IL : Scientific Software International.
16. Fischer, G. H. (1983). Logistic latent trait models with linear constraints. Psychometrika, 48, 3 26.
17. French, A. W., & Miller, T. R. (1996). Logistic regression and its use in detecting differential item functioning in polytomous items. Journal of Educational Measurement, 33, 315 332.
18. Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds Test validity (pp. 129 145). Hillsdale, NJ : Erlbaum.
19. Jodoin, M. G., & Gierl, M. J. (2001). Evaluating Type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education, 14, 329 349.
20. Joint Committee on Testing Practices (2004). Code of fair testing practices in education. Washington, DC : Author.
21. Kim, S.-H. (April 2000). An investigation of the likelihood ratio test, the Mantel test, and the generalized Mantel-Haenszel test of DIF. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
22. Kim, S.-H., & Cohen, A. S. (1995). A comparison of Lord's chi-square, Raju's area measures, and the likelihood ratio test on detection of differential item functioning. Applied Measurement in Education, 8, 291 312.
23. Kim, S.-H., & Cohen, A. S. (1998). Detection of differential item functioning under the graded response model with the likelihood ratio test. Applied Psychological Measurement, 22, 345 355.
24. Kristjansson, E., Aylesworth, R., McDowell, I., & Zumbo, B. D. (2005). A comparison of four methods for detecting differential item functioning in ordered response items. Educational and Psychological Measurement, 65, 935 953.
25. Lee, Y.-W., Breland, H., & Muraki, E. (2004). Comparability of TOEFL CBT writing prompts for different native language groups (TOEFL Research Rep. No. RR-77). Princeton, NJ : Educational Testing Service.
26. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ : Erlbaum.
27. Mantel, N. (1963). Chi-square tests with one degree of freedom: Extensions of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58, 690 700.
28. Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719 748.
29. Miller, T. A., & Spray, J. A. (1993). Logistic discriminant function analysis for DIF identification of polytomously scored items. Journal of Educational Measurement, 30, 107 122.
30. Muthén, L. K., & Muthén, B. O. (2004). Mplus user's guide (3rd ed.). Los Angeles : Muthén & Muthén.
31. Nagelkerke, N. J. D. (1991). A note on a general definition of the coefficient of determination. Biometrika, 78, 691 692.
32. Raju, N. S. (1988). The area between two item characteristic curves. Psychometrika, 53, 495 502.
33. Rao, C. R. (1973). Linear statistical inference and its applications (2nd ed.). New York : Wiley.
34. Reise, S. P., & Yu, J. (1990). Parameter recovery in the graded response model using MULTILOG. Journal of Educational Measurement, 27, 133 144.
35. Riger, R. (Ed 2004). The new SAT. New York : Spark Publishing.
36. Rost, J. (1990). Rasch models in latent classes: An integration of two approaches to item analysis. Applied Psychological Measurement, 3, 271 282.
37. Samejima, F. (1969). Estimation of a latent ability using a response pattern of graded scores. Psychometric Monographs, No. 17.
38. Serlin, R. C., & Lapsley, D. K. (1993). Rational appraisal of psychological research and the good-enough principle. In G. Keren & C. Lewis (Eds A handbook for data analysis in the behavioral sciences: Methodological issues (pp. 199 228). Hillsdale, NJ : Erlbaum.
39. Shaffer, D. R. (2000). Social and personality development (4th ed.). Belmont, CA : Wadsworth.
40. Su, Y.-H., & Wang, W.-C. (2005). Efficiency of the Mantel, generalized Mantel-Haenszel, and logistic discriminant function analysis methods in detecting differential item functioning for polytomous items. Applied Measurement in Education, 18, 313 350.
41. Subkoviak, M. J., Mack, J. S., Ironson, G. H., & Craig, R. D. (1984). Empirical comparison of selected item detection procedures with bias manipulation. Journal of Educational Measurement, 21, 49 58.
42. Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361 370.
43. Thissen, D. (1991). MULTILOG: Multiple, categorical item analysis and test scoring using item response theory (Version 6.0) [Computer program]. Chicago: Scientific Software.
44. Thissen, D. (2001). IRTLRDIF v.2.0b [Computer program]. University of North Carolina at Chapel Hall: L. L. Thurstone Psychometric Laboratory.
45. Thissen, D., Chen, W.-H., & Bock, R. D. (2002). MULTILOG [Computer program]. Lincoln-wood, IL: Scientific Software International.
46. Thissen, D., Steinberg, L., & Gerrard, M. (1986). Beyond group-mean differences: The concept of item bias. Psychological Bulletin, 99, 118 128.
47. Thissen, D., Steinberg, L., & Wainer, H. (1988). Use of item response theory in the study of group differences in trace lines. In H. Wainer & H. I. Braun (Eds Test validity (pp. 147 169). Hillsdale, NJ : Erlbaum.
48. Thissen, D., Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds Differential item functioning (pp. 67 113). Hillsdale, NJ : Erlbaum.
49. Wainer, H. (1993). Model-based standardized measurement of an item's differential impact. In P. W. Holland & H. Wainer (Eds Differential item functioning (pp. 123 135). Hillsdale, NJ : Erlbaum.
50. Wainer, H., Sireci, S. G., & Thissen, D. (1991). Differential testlet functioning: Definitions and detection. Journal of Educational Measurement, 28, 197 219.
51. Welch, C., & Hoover, H. D. (1993). Procedures for extending item bias detection techniques to polytomously scored items. Applied Measurement in Education, 6, 1 19.
52. Wells, C. S., Cohen, A. S., & Serlin, R. C. (April 2005). An equivalence test for detection of differential item functioning. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada.
53. Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF). Ottawa, Ontario, Canada : Directorate of Human Resources Research and Evaluation, Department of National Defense.
54. Zwick, R., Donoghue, J. R., & Grima, A. (1993). Assessment of differential item functioning for performance tasks. Journal of Educational Measurement, 30, 233 251.
55. Zwick, R., & Thayer, D. T. (1996). Evaluating the magnitude of differential item functioning in polytomous items. Journal of Educational and Behavioral Statistics, 21, 187 201.