A REVIEW OF RECENT DEVELOPMENTS IN DIFFERENTIAL ITEM FUNCTIONING

ETS Research Report Series

Volumn 2008, Issue 2, 2008, Pages i-32

  Mapuranga, Raymond ^a   Dorans, Neil J ^a   Middleton, Kyndra ^a  

^a EDUCATIONAL TESTING SERVICE   (United States)

Author keywords

differential item functioning; practical criteria for differential item functioning; statistical criteria for differential item functioning

Indexed keywords

EID: 85164495774     PISSN: None     EISSN: 23308516     Source Type: Journal    
DOI: 10.1002/j.2333-8504.2008.tb02129.x     Document Type: Article

References (82)

1. Bolt, D. M. (2002). A Monte Carlo comparison of parametric and nonparametric polytomous DIF detection methods. Applied Measurement in Education, 15(2), 113–141.
2. Bolt, D. M., & Gierl, M. J. (2006). Testing features of graphical DIF: Application of a regression correction to three nonparametric statistical tests. Journal of Educational Measurement, 43, 313–333.
3. Camilli, G. (1992). A conceptual analysis of differential item functioning in terms of a multidimensional item response model. Applied Psychological Measurement, 16(2), 129–147.
4. Camilli, G. (2006). Test fairness. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 221–256). Westport, CT: Praeger.
5. Camilli, G., & Congdon, P. (1999). Application of a method of estimating DIF for polytomous test items. Journal of Educational and Behavioral Statistics, 24(4), 323–341.
6. Camilli, G., & Penfield, D. A. (1997). Variance estimation for differential test functioning based on Mantel-Haenszel statistics. Journal of Educational Measurement, 34(2), 123–139.
7. Chang, H.-H, Mazzeo, J., & Roussos, L. A. (1993, April). Extension of Shealy-Stout's DIF procedures to polytomously scored items. Paper presented at the annual meeting of the National Council on Measurement in Education, Atlanta, GA.
8. Chang, H., Mazzeo, J., & Roussos, L. (1996). Detecting DIF for polytomously scored items: An adaptation of the SIBTEST procedure. Journal of Educational Measurement, 33(3), 333–353.
9. Cheong, Y. F. (2006). Analysis of school context effects on differential item functioning using hierarchical generalized linear models. International Journal of Testing, 6(1), 57–79.
10. Clauser, B. E., & Mazor, K. M. (1998). Using statistical procedures to identify differentially functioning test items. Educational Measurement: Issues and Practice, 17(1), 31–44.
11. Cohen, A. S., & Bolt, D. M. (2005). A mixture model analysis of differential item functioning. Journal of Educational Measurement, 42(2), 133–148.
12. de Ayala, R. J., Kim, S.-H., Stapleton, L. M., & Dayton, C. M. (2002). Differential item functioning: A mixture distribution conceptualization. International Journal of Testing, 2(3 & 4), 243–276.
13. Dorans, N. J. (2004). Using subpopulation invariance to assess test score equity. Journal of Educational Measurement, 41(1), 43–68.
14. Dorans, N. J., & Kulick, E. (1986). Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the Scholastic Aptitude Test. Journal of Educational Measurement, 23(4), 355–368.
15. Dorans, N. J., & Schmitt, A. P. (1993). Constructed response and differential item functioning: A pragmatic perspective. In R. E. Bennett & W. C. Ward (Eds.), Construction versus choice in cognitive measurement (pp. 135–165). Hillsdale, NJ: Lawrence Erlbaum.
16. Dorans, N. J., Schmitt, A. P., & Bleistein, C. A. (1992). The standardization approach to assessing comprehensive differential item functioning. Journal of Educational Measurement, 29(4), 309–319.
17. Douglas, J., Roussos, L., & Stout, W. (1996). Item-bundle DIF hypothesis testing: Identifying suspect bundles and assessing their differential functioning. Journal of Educational Measurement, 33(4), 465–484.
18. Douglas, J. A., Stout, W., & DiBello, L. V. (1996). A kernel-smoothed version of SIBTEST with application to local DIF inference and function estimation. Journal of Educational Measurement, 21(4), 333–363.
19. Flowers, C. P., Oshima, T. C., & Raju, N. S. (1999). A description and demonstration of the polytomous-DFIT framework. Applied Psychological Measurement, 23(4), 309–326.
20. French, A. W., & Miller, T. R. (1996). Logistic regression and its use in detecting differential item functioning in polytomous items. Journal of Educational Measurement, 33(3), 315–332.
21. Gierl, M., Bisanz, J., Bisanz, G., & Boughton, K. (2003). Identifying content and cognitive skills that produce gender differences in mathematics: A demonstration of the multidimensionality-based DIF analysis paradigm. Journal of Educational Measurement, 40(4), 281–306.
22. Gierl, M., Bisanz, J., Bisanz, G., Boughton, K., & Khaliq, S. (2001). Illustrating the utility of differential bundle functioning analyses to identify and interpret group differences on achievement tests. Educational Measurement: Issues and Practice, 20(2), 26–36.
23. Gierl, M. J., & Bolt, D. M. (2001). Illustrating the use of nonparametric regression to assess differential item and bundle functioning among multiple groups. International Journal of Testing, 1(3 & 4), 249–270.
24. Glas, C.A.W. (1999). Detection of differential item functioning using Lagrange multiplier tests. Statistica Sinica, 8(1), 647–667.
25. Green, B. F., Crone, C. R., & Folk, V. G. (1989). A method for studying differential distractor functioning. Journal of Educational Measurement, 26(2), 147–160.
26. Haladyna, T. M., & Downing, S. M. (2004). Construct-irrelevant variance in high-stakes testing. Educational Measurement: Issues and Practice, 23(1), 17–26.
27. Holland, P. W. (1994). Measurements or contests? Comments on Zwick, Bond, and Allen/Donoghue. 1994 proceedings of section on social statistics, American Statistical Association, 27–29. Alexandria, VA: American Statistical Association.
28. Holland, P. W., & Thayer, D. T. (1988). Differential item functioning and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129–145). Hillsdale, NJ: Lawrence Erlbaum.
29. Individuals with Disabilities Education Act of 1991, 20 U.S.C. § 1400 et seq. (1991).
30. Individuals with Disabilities Education Act of 1997, 20 U.S.C. § 1412(a) (17)(A). (1997).
31. Jodoin, M. G., & Gierl, M.J. (2001). Evaluating Type I error and power rates using an effect size measure with logistic regression procedure for DIF detection. Applied Measurement in Education, 14(4), 329–349.
32. Kristjansson, E., Aylesworth, R., McDowell, I, & Zumbo, B. D. (2005). A comparison of four methods for detecting DIF in ordered response items. Educational and Psychological Measurement, 65(6), 935–953.
33. Li, H., & Stout, W. (1996). A new procedure for detection of crossing DIF. Psychometrika, 61(4), 647–677.
34. Lyu, F., Dorans, N., & Ramsay, J. O. (1995, April). Smoothed standardization assessment of testlet level DIF on a math free-response item (ETS Research Rep. No. RR-95-38). Princeton, NJ: ETS.
35. Mantel, N. (1963). Chi-square tests with one degree of freedom; Extensions of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58(303), 690–700.
36. Mantel, N., & Haenszel, W. M. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22(4), 719–748.
37. Mazor, K. M., Kanjee, A., & Clauser, B. E. (1995). Using logistic regression and the Mantel-Haenszel with multiple ability estimates to detect differential item functioning. Journal of Educational Measurement, 32(2), 131–144.
38. McCullagh, P. & Nelder, J.A. (1989). Generalized linear models. Chapman and Hall: London.
39. McDonald, R. P. (1999). Test theory: A unified treatment. Hillsdale, NJ: Erlbaum.
40. Meyer, J. P., Huynh, H., & Seaman, M. A. (2004). Exact small-sample differential item functioning methods for polytomous items with illustration based on an attitude survey. Journal of Educational Measurement, 41(4), 331–344.
41. Millsap, R. E., & Everson, H. T. (1993). Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement, 17(4), 297–334.
42. Monahan, P. O., McHorney, C. A., Stump, T. E., & Perkins, A. J. (2007). Odds ratio, delta, ETS classification, and standardization measures of DIF magnitude for binary logistic regression. Journal of Educational and Behavioral Statistics, 32(1), 92–109.
43. Moore, S. (1996). Estimating differential item functioning in the polytomous case with the random coefficient multinomial logits (RCML) model. In G. Englehard & M. Wilson (Eds.), Objective measurement III: Theory into practice (pp. 219–238). Norwood, NJ: Ablex.
44. Muraki, E. (1993, April). Implementing item parameter drift and bias in polytomous item response models. Paper presented at the annual meeting of the National Council on Measurement in Education, Atlanta, GA.
45. Muthén, B. O. (2002). Beyond SEM: General latent variable modeling. Behaviormetrika, 29(1), 81–117.
46. Oshima, T. C., Raju, N. S., & Flowers, C. P. (1997). Development and demonstration of multidimensional IRT-based internal measures of differential functioning of items and tests. Journal of Educational Measurement, 34(3), 253–272.
47. Oshima, T. C., Raju, N. S., Flowers, C. P., & Slinde, J. A. (1998). Differential bundle functioning using the DFIT framework: Procedures for identifying possible sources of differential functioning. Applied Measurement in Education, 11(4), 353–369.
48. Parshall, C. G., & Miller, T. R. (1995). Exact versus asymptotic Mantel-Haenszel DIF statistics: A comparison of performance under small-sample conditions. Journal of Educational Measurement, 32(3), 302–316.
49. Penfield, R. D. (2001). Assessing differential item functioning among multiple groups: A comparison of three Mantel-Haenszel procedures. Applied Measurement in Education, 14(3), 235–259.
50. Penfield, R. D., & Algina, J. (2003). Applying the Liu-Agresti estimator of the cumulative common odds ratio to DIF detection in polytomous items. Journal of Educational Measurement, 40(4), 353–370.
51. Penfield, R. D., & Algina, J. (2006). A generalized DIF effect variance estimator for measuring unsigned differential test functioning in mixed format tests. Journal of Educational Measurement 43(4), 295–312.
52. Potenza, M. T., & Dorans, N. J. (1995). DIF assessment for polytomously scored items: A framework for classification and evaluation. Applied Psychological Measurement, 19(1), 23–37.
53. Puhan, G., Moses, T. P., Yu, L., & Dorans, N. J. (2007). Small-sample DIF estimation using loglinear smoothing: A SIBTEST application (ETS Research Rep. No. RR-07-10). Princeton, NJ: ETS.
54. Raju, N. S., van der Linden, W. J., & Fleer, P. F. (1995). IRT-based measures of differential item functioning of items and tests. Applied Psychological Measurement, 19(4), 353–368.
55. Ramsay, J. O. (2000). TestGraf: A program for the graphical analysis of multiple-choice test and questionnaire data [Computer program and manual]. Retrieved June 2, 2008, from http://www.psych.mcgill.ca/faculty/ramsay/ramsay.html.
56. Rogers, H. J., & Swaminathan, H. (1993, April). Differential item functioning procedures for non-dichotomous responses. Paper presented at the annual meeting of the National Council on Measurement in Education, Atlanta, GA.
57. Roussos, L., & Stout, W. (1996). A multidimensionality-based DIF analysis paradigm. Applied Psychological Measurement, 20(4), 355–371.
58. Schmitt, A. P., Curley, W. E., Bleistein, C. A., & Dorans, N. J. (1988, April). Experimental evaluation of language and interest factors related to differential item functioning for Hispanic examinees on the SAT-Verbal. Paper presented at the annual meeting of the National Council on Measurement in Education, New Orleans.
59. Schmitt, A. P, Holland, P. W., & Dorans, N. J. (1993). Evaluating hypotheses about differential item functioning. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 281–315). Hillsdale, NJ: Lawrence Erlbaum.
60. Scrams, D. J., & McLeod, L. D. (2000). An expected response function approach to graphical differential item functioning. Journal of Educational Measurement, 37(3), 263–280.
61. Shealy, R., & Stout, W. F. (1993). A model-based standardization approach that separates true bias/DIF from group differences and detects test bias/DTF as well as item bias/DIF. Psychometrika, 58, 159–194.
62. Sinharay, S., Dorans, N. J., Grant, M., & Blew, E. (in press). Using past data to enhance small-sample DIF estimation: A Bayesian approach. Journal of Educational and Behavioral Statistics.
63. Sinharay, S., Liang, L., & Dorans, N. J. (2008, March). English proficiency and its effects on fairness assessment. Paper presented at the annual meeting of the National Council on Measurement in Education, New York.
64. Stone, E. A., Cook, L. L., Cline, F., & Cahalan-Laitusis, C. (2007, April). Using differential item functioning to investigate the impact of testing accommodations on an English language arts assessment for students who are blind and visually impaired. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, IL.
65. Stout, W., Li, H.-H., Nandakumar, R., & Bolt, D. (1997). MULTISIB: A procedure to investigate DIF when a test is intentionally two-dimensional. Applied Psychological Measurement, 21(3), 195–215.
66. Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27(4), 361–370.
67. Swanson, D. B., Clauser, B. E., Case, S. M., Nungster, R. J., & Featherman, C. (2002). Analysis of differential item functioning (DIF) using hierarchical logistic regression models. Journal of Educational and Behavioral Statistics, 27(1), 53–75.
68. Van den Noortgate, W., & De Boeck, P. (2005) Assessing and explaining differential item functioning using logistic mixed models. Journal of Educational and Behavioral Statistics, 30(4), 443–464.
69. Wainer, H. (1983). Pyramid power: Searching for an error in test. scoring with 830,000 helpers. The American Statistician, 37, 87–81.
70. Wainer, H. (1995). Precision and differential item functioning on a testlet-based test: The 1991 Law School Admissions Test as an example. Applied Measurement in Education. 8(2), 157–186.
71. Walker, C. M., & Beretvas, S. N. (2001). An empirical investigation demonstrating the multidimensional DIF paradigm: A cognitive explanation for DIF. Journal of Educational Measurement, 38(2), 147–163.
72. Welch, C., & Hoover, H. D. (1993). Procedures for extending item bias techniques to polytomously scored items. Applied Measurement in Education, 6, 1–19.
73. Williams, N. J., & Beretvas, S. N. (2006). DIF identification using HGLM for polytomous items. Applied Psychological Measurement, 30(1), 22–42.
74. Zhang, Y., Dorans, N., & Matthews-Lopez, J. (2005). Using DIF dissection method to assess effects of item deletion (ETS Research Rep. No. RR-05-23). Princeton, NJ: ETS.
75. Zieky, M. J. (1993). Practical questions in the use of DIF statistics in test development. In P. W. Holland & H. Wainer (Eds.), Differential item functioning. (pp. 337–47). Hillsdale, NJ: Lawrence Erlbaum.
76. Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF). Ottawa, Ontario, Canada: Directorate of Human Resources Research and Evaluation, Department of National Defense.
77. Zwick, R., Donoghue, J. R., & Grima, A. (1993). Assessment of differential item functioning for performance tasks. Journal of Educational Measurement, 30(3), 233–251.
78. Zwick R., & Thayer, D. T. (1996). Evaluating the magnitude of differential item functioning in polytomous items. Journal of Educational Statistics, 21(3), 187–201.
79. Zwick, R., Thayer, D. T., & Lewis, C. (1999). An empirical Bayes approach to Mantel-Haenszel DIF analysis. Journal of Educational Measurement, 36(1), 1–28.
80. Zwick, R., & Thayer, D. T., & Lewis, C. (1997). An investigation of the validity of an empirical Bayes approach to Mantel-Haenszel DIF analysis (ETS Research Rep. No. RR-97-21). Princeton, NJ: ETS.
81. Zwick, R., Thayer, D. T., & Lewis, C. (2000). Using loss functions for DIF detection: An empirical Bayes approach. Journal of Educational and Behavioral Statistics, 25(1), 225–247.
82. Zwick, R., Thayer, D. T., & Mazzeo, J. (1997). Descriptive and inferential procedures for assessing differential item functioning in polytomous items. Applied Measurement in Education, 10(4), 321–344.