Decisions that make a difference in detecting differential item functioning

Educational Research and Evaluation

Volumn 19, Issue 2-3, 2013, Pages 170-187

  Sireci, Stephen G ^a   Rios, Joseph A ^a  

^a UNIVERSITY OF MASSACHUSETTS   (United States)

Author keywords

differential item functioning; item bias; validity

Indexed keywords

EID: 84875871533     PISSN: 13803611     EISSN: 17444187     Source Type: Journal    
DOI: 10.1080/13803611.2013.767621     Document Type: Article

References (53)

1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. 1999. Standards for educational and psychological testing, Washington, DC: American Educational Research Association.
2. Buzick, H. and Stone, E. 2011. Recommendations for conducting differential item functioning (DIF) analyses for students with disabilities based on previous DIF studies, Princeton, NJ: Educational Testing Service.
3. Camilli, G. 2006. "Test fairness". In Educational Measurement, 4, Edited by: Brennan, R. 221-256. Westport, CT: Praeger.
4. Camilli, G. and Penfield, D. A. 1997. Variance estimation for differential test functioning based on the Mantel-Haenszel log-odds ratio. Journal of Educational Measurement, 34: 123-139. (doi:10.1111/j.1745-3984.1997.tb00510.x)
5. Camilli, G. and Shepard, L. A. 1994. Methods for identifying biased test items, Thousand Oaks, CA: Sage.
6. Clauser, B. E. and Mazor, K. M. 1998. Using statistical procedures to identify differential item functioning test items. Educational Measurement: Issues and Practice, 17: 31-44. (doi:10.1111/j.1745-3992.1998.tb00619.x)
7. Clauser, B. E., Nungester, R. J. and Swaminathan, H. 1996. Improving the matching for DIF analysis by conditioning on both test score and an educational background variable. Journal of Educational Measurement, 33: 453-463. (doi:10.1111/j.1745-3984.1996.tb00501.x)
8. Colvin, K. 2011. A review of recent findings on DIF analysis techniques, Amherst, MA: Center for Educational Assessment, University of Massachusetts Amherst. (Center for Educational Assessment Research Report No. 797).
9. DeMars, C. E. 2011. An analytic comparison of effect sizes for differential item functioning. Applied Measurement in Education, 24: 189-209. (doi:10.1080/08957347.2011.580255)
10. Dorans, N. J. and Kulick, E. 1986. Demonstrating the utility of the standardization approach to assessing the unexpected differential item functioning on the Scholastic Aptitude Test. Journal of Educational Measurement, 23: 355-368. (doi:10.1111/j.1745-3984.1986.tb00255.x)
11. Emenogu, B. C., Falenchuk, O. and Childs, R. A. 2010. The effect of missing data treatment on Mantel-Haenszel DIF detection. The Alberta Journal of Educational Research, 56: 459-469.
12. Fidalgo, A. M., Mellenbergh, G. J. and Muñiz, J. 1998. Comparación del procedimiento Mantel Haenszel frente a los modelos loglineales en la detección del funcionamiento diferencial de los ítems [Comparison of the Mantel-Haenszel procedure versus the log linear models for detecting differential item functioning]. Psicothema, 10: 219-228.
13. French, B. and Maller, S. 2007. Iterative purification and effect size use with logistic regression for differential item functioning detection. Educational and Psychological Measurement, 67: 373-393. (doi:10.1177/0013164406294781)
14. Gómez-Benito, J., Hidalgo, M. D. and Padilla, J. L. 2009. Efficacy of effect size measures in logistic regression: An application for detecting DIF. Methodology, 5: 18-25.
15. Hambleton, R. K. 1989. "Principles and selected applications of item response theory". In Educational measurement, Edited by: Linn, R. 147-200. New York, NY: Macmillan.
16. Hambleton, R. K. 2006. Good practices for identifying differential item functioning. Medical Care, 44: 182-188. (doi:10.1097/01.mlr.0000245443.86671.c4)
17. Hambleton, R. K., Swaminathan, H. and Rogers, H. J. 1991. Fundamentals of item response theory, Newbury Park, CA: Sage.
18. Hidalgo, M. D. and Gomez, J. 2003. Test purification and the evaluation of differential item functioning with multinomial logistic regression. European Journal of Psychological Assessment, 19: 1-11. (doi:10.1027//1015-5759.19.1.1)
19. Holland, P. W. and Thayer, D. T. 1988. "Differential item performance and the Mantel-Haenszel procedure". In Test validity, Edited by: Wainer, H. and Braun, H. 129-145. Hillsdale, NJ: Lawrence Erlbaum Associates.
20. Jodoin, M. G. and Gierl, M. J. 2001. Evaluating power and Type I error rates using an effect size with the logistic regression procedure for DIF. Applied Measurement in Education, 14: 329-349. (doi:10.1207/S15324818AME1404_2)
21. Kim, S.-H., Cohen, A. S. and Kim, H.-O. 1994. An investigation of Lord's procedure for the detection of differential item functioning. Applied Psychological Measurement, 18: 217-228. (doi:10.1177/014662169401800303)
22. Lord, F. M. 1980. Applications of item response theory to practical testing problems., Hillsdale, NJ: Lawrence Erlbaum Associates.
23. Mazor, K. M., Clauser, B. E. and Hambleton, R. K. 1992. The effect of sample size on the functioning of the Mantel-Haenszel statistic. Educational and Psychological Measurement, 52: 443-452. (doi:10.1177/0013164492052002020)
24. McLaughlin, M. E. and Drasgow, F. 1987. Lord's chi-square test of item bias with estimated and with known person parameters. Applied Psychological Measurement, 11: 161-173. (doi:10.1177/014662168701100205)
25. Miller, T. R. and Spray, J. A. 1993. Logistic discriminant function analysis for DIF identification of polytomously scored items. Journal of Educational Measurement, 30: 107-122. (doi:10.1111/j.1745-3984.1993.tb01069.x)
26. Muñiz, J., Hambleton, R. K. and Xing, D. 2001. Small sample studies to detect flaws in item translations. International Journal of Testing, 1: 115-135. (doi:10.1207/S15327574IJT0102_2)
27. Nandakumar, R. 1993. Simultaneous DIF amplification and cancellation: Shealy-Stout's Test for DIF. Journal of Educational Measurement, 30: 293-311. (doi:10.1111/j.1745-3984.1993.tb00428.x)
28. Narayanan, P. and Swaminathan, H. 1996. Identification of items that show nonuniform DIF. Applied Psychological Measurement, 20: 257-274. (doi:10.1177/014662169602000306)
29. Navas-Ara, M. J. and Gomez-Benito, J. 2002. Effects of ability scale purification on the identification of dif. European Journal of Psychological Assessment, 18: 9-15. (doi:10.1027//1015-5759.18.1.9)
30. Parshall, C. G. and Miller, T. R. 1995. Exact versus asymptotic Mantel-Haenszel DIF statistics: A comparison of performance under small-sample conditions. Journal of Educational Measurement, 32: 302-316. (doi:10.1111/j.1745-3984.1995.tb00469.x)
31. Raju, N. S. 1988. The area between two item characteristic curves. Psychometrika, 53: 495-502. (doi:10.1007/BF02294403)
32. Ramsey, P. A. 1993. "Sensitivity review: The ETS experience as a case study". In Differential item functioning, Edited by: Holland, P. W. and Wainer, H. 367-388. Hillsdale, NJ: Erlbaum.
33. Robin, F. 2001. STDIF: Standardization-DIF analysis program., Amherst, MA: University of Massachusetts, School of Education.
34. Rogers, H. J. and Swaminathan, H. 1993. A comparison of the logistic regression and Mantel-Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17: 105-116. (doi:10.1177/014662169301700201)
35. Sireci, S. G. 1997. Problems and issues in linking tests across languages. Educational Measurement: Issues and Practice, 16: 12-19. (doi:10.1111/j.1745-3992.1997.tb00581.x)
36. Sireci, S. G. 2011. "Evaluating test and survey items for bias across languages and cultures". In Cross-cultural research methods in psychology, Edited by: Matsumoto, D. and van de Vijver, F. 216-243. New York, NY: Cambridge University Press.
37. Sireci, S. G. and Mullane, L. A. 1994. Evaluating test fairness in licensure testing: The sensitivity review process. CLEAR Exam Review, 5(2): 22-28.
38. Sireci, S. G., Patsula, L. and Hambleton, R. K. 2005. "Statistical methods for identifying flaws in the test adaptation process". In Adapting educational and psychological tests for cross-cultural assessment, Edited by: Hambleton, R., Merenda, P. and Spielberger, C. 93-115. Mahwah, NJ: Lawrence Erlbaum Associates.
39. Sireci, S. G. and Wells, C. S. 2010. "Evaluating the comparability of English and Spanish video accommodations for English language learners". In Evaluating the comparability of scores from achievement test variations, Edited by: Winter, P. 33-68. Washington, DC: Council of Chief State School Officers.
40. Spray, J. A. 1989. Performance of three conditional DIF statistics in detecting differential item functioning on simulated tests, Iowa City, IA: American College Testing. (Research Report No. 89-7).
41. Stokes, M. E., Davis, C. S. and Koch, G. G. 2000. Categorical data analysis using the SAS system, Cary, NC: SAS Institute and Wiley.
42. Swaminathan, H. and Rogers, H. J. 1990. Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27: 361-370. (doi:10.1111/j.1745-3984.1990.tb00754.x)
43. Thissen, D. 2003. "MULTILOG 7: Multiple categorical item analysis and test scoring using item response theory [computer program]". Chicago, IL: Scientific Software.
44. Thissen, D. (2006). IRTLRDIF [computer program]. Retrieved fromhttp://www.unc.edu/~dthissen/dl/irtlrdif201.zip
45. Thissen, D, Steinberg, L. and Wainer, H. 1988. "Use of item response theory in the study of group differences in trace lines". In Test validity, Edited by: Wainer, H. and Braun, H. I. 147-169. Hillsdale, NJ: Lawrence Erlbaum.
46. Thissen, D., Steinberg, L. and Wainer, H. 1993. "Detection of differential item functioning using the parameters of item response models". In Differential item functioning, Edited by: Holland, P. and Wainer, H. 67-113. Hillsdale, NJ: Lawrence Erlbaum Associates.
47. Wainer, H., Sireci, S. G. and Thissen, D. 1991. Differential testlet functioning: Definitions and detection. Journal of Educational Measurement, 28: 197-219. (doi:10.1111/j.1745-3984.1991.tb00354.x)
48. Walker, C. M. 2011. What's the DIF? Why differential item functioning analyses are an important part of instrument development and validation. Journal of Psychoeducational Assessment, 29: 364-376. (doi:10.1177/0734282911406666)
49. Wood, R. 1988. "Item analysis". In Educational research, methodology, and measurement: An international handbook, Edited by: Keeves, J. P. 376-384. New York, NY: Pergamon Press.
50. Zieky, M. 1993. "Practical questions in the use of DIF statistics in test development". In Differential item functioning, Edited by: Holland, P. W. and Wainer, H. 337-347. Hillsdale, NJ: Erlbaum.
51. Zumbo, B. D. 1999. A handbook on the theory and methods of differential item functioning (DIF): Logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores, Ottawa, Canada: Directorate of Human Resources Research and Evaluation, Department of National Defense.
52. Zwick, R., Donoghue, J. R. and Grima, A. 1993. Assessment of differential item functioning for performance tasks. Journal of Educational Measurement, 30: 233-251. (doi:10.1111/j.1745-3984.1993.tb00425.x)
53. Zwick, R., Thayer, D. T. and Mazzeo, J. 1997. Descriptive and inferential procedures for assessing differential item functioning in polytomous items. Applied Measurement in Education, 10: 321-334. (doi:10.1207/s15324818ame1004_2)