Item Purification Does Not Always Improve DIF Detection: A Counterexample With Angoff's Delta Plot

Educational and Psychological Measurement

Volumn 73, Issue 2, 2013, Pages 293-311

  Magis, David ^a,b   Facon, Bruno ^c  

^a UNIVERSITY OF LIÈGE   (Belgium)

^b UNIVERSITY OF LEUVEN   (Belgium)

^c UNIV LILLE   (France)

Author keywords

delta plot; differential item functioning; item purification; power; Type I error

Indexed keywords

EID: 84874543803     PISSN: 00131644     EISSN: 15523888     Source Type: Journal    
DOI: 10.1177/0013164412451903     Document Type: Article

References (36)

1. Angoff W. H. (1972, September). A technique for the investigation of cultural differences. Paper presented at the annual meeting of the American Psychological Association,Honolulu.
2. Angoff W. H.,Ford S. F.Item-race interaction on a test of scholastic aptitude.Journal of Educational Measurement. 1973;10:95-106.
3. Candell G. L.,Drasgow F.An iterative procedure for linking metrics and assessing item bias in item response theory.Applied Psychological Measurement. 1988;12:253-260.
4. Clauser B. E.,Mazor K. M.,Hambleton R. K.The effects of purification of the matching criterion on the identification of DIF using the Mantel-Haenszel procedure.Applied Measurement in Education. 1993;6:269-279.
5. Cohen A. S.,Bolt D. M.A mixture model analysis of differential item functioning.Journal of Educational Measurement. 2005;42:133-148.
6. De Boeck P.Random item IRT models.Psychometrika. 2008;73:533-559.
7. Donoghue J. R.,Allen N. L.Thin vs. thick matching in the Mantel-Haenszel procedure for detecting DIF.Journal of Educational Statistics. 1993;18:131-154.
8. Donoghue J. R.,Holland P. W.,Thayer D. T.Differential item functioning. Holland P.Wainer H., ed. Hillsdale, NJ: Lawrence Erlbaum; 1993:137-166.
9. Dorans N. J.,Kulick E.Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the Scholastic Aptitude Test.Journal of Educational Measurement. 1986;23:355-368.
10. Facon B.,Magis D.,Belmont J. M.Beyond matching on the mean in developmental disabilities research.Research in Developmental Disabilities. 2011;32:2134-2147.
11. Facon B.,Magis D.,Courbois Y.On the difficulty of relational concepts among participants with Down syndrome.Research in Developmental Disabilities. 2012;33:60-68.
12. Facon B.,Nuchadee M.-L.An item analysis of Raven's colored Progressive Matrices among participants with Down syndrome.Research in Developmental Disabilities. 2010;31:243-249.
13. Fidalgo A. M.,Ferreres D.,Muniz J.Utility of the Mantel-Haenszel procedure for detecting differential item functioning in small samples.Educational and Psychological Measurement. 2004;64:925-936.
14. Fidalgo A. M.,Mellenbergh G. J.,Muniz J.Effects of amount of DIF, test length, and purification type on robustness and power of Mantel-Haenszel procedures.Methods of Psychological Research. 2000;5:43-53.
15. Frederickx S.,Tuerlinckx F.,De Boeck P.,Magis D.RIM: A random item mixture model to detect differential item functioning.Journal of Educational Measurement. 2010;47:432-457.
16. Holland P. W.,Thayer D. T.Test validity. Wainer H.Braun H. I., ed. Hillsdale, NJ: Lawrence Erlbaum; 1988:129-145.
17. Kamata A.Item analysis by the hierarchical generalized linear model.Journal of Educational Measurement. 2001;38:79-93.
18. Lightbody A.,Reiss A.Gene, brain, and behavior relationships in fragile X syndrome: Evidence from neuroimaging studies.Developmental Disabilities Research Reviews. 2009;15:343-352.
19. Lord F. M.Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum Associates; 1980:.
20. Magis D.,De Boeck P.Identification of differential item functioning in multiple-group settings: A multivariate outlier detection approach.Multivariate Behavioral Research. 2011;46:733-755.
21. Magis D.,De Boeck P.A robust outlier approach to prevent Type I error inflation in DIF.Educational and Psychological Measurement. 2012;72:291-311.
22. Magis D.,Facon B.Angoff's delta method revisited: improving the DIF detection under small samples.British Journal of Mathematical and Statistical Psychology. 2012;65:302-321.
23. Mervis C. B.,John A. E.Cognitive and behavioral characteristics of children with Williams syndrome: Implications for intervention approaches.American Journal of Medical Genetics Part C, 154C. 2010;:229-248.
24. Michaelides M. P.Sensitivity of equated aggregate scores to the treatment of misbehaving common items.Applied Psychological Measurement. 2010;34:365-369.
25. Muniz J.,Hambleton R. K.,Xing D.Small sample studies to detect flaws in item translations.International Journal of Testing. 2001;1:115-135.
26. Osterlind S. J.Test item bias. Newbury Park, CA: Sage; 1983:.
27. Penfield R. D.Application of the Breslow-Day test of trend in odds ratio heterogeneity to the detection of nonuniform DIF.Alberta Journal of Educational Research. 2003;49:231-243.
28. Raju N. S.Determining the significance of estimated signed and unsigned areas between two item response functions.Applied Psychological Measurement. 1990;14:197-207.
29. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2011:.
30. Robin F.,Sireci S. G.,Hambleton R. K.Evaluating the equivalence of different language versions of a credentialing exam.International Journal of Testing. 2003;3:1-20.
31. Rudner L. M. (1977, June). Efforts toward the development of unbiased selection and assessment instruments. Paper presented at the Third International Symposium on Educational Testing, Leiden, Netherlands.
32. Rudner L. M.Using standard tests with the hearing impaired: the problems of item bias.Volta Review. 1978;80:31-40.
33. Swanson D. B.,Clauser B. E.,Case S. M.,Nungester R. J.,Featherman C.Analysis of differential item functioning (DIF) using hierarchical logistic regression models.Journal of Educational and Behavioral Statistics. 2002;27:53-75.
34. Thissen D.,Steinberg L.,Wainer H.Test validity. Wainer H.Braun H., ed. Hillsdale, NJ: Lawrence Erlbaum; 1988:147-170.
35. Wang W.-C.,Su Y.-H.Effects of average signed area between two item characteristic curves and test purification procedures on the DIF detection via the Mantel-Haenszel method.Applied Measurement in Education. 2004;17:113-144.
36. Zwick R.,Donoghue J. R.,Grima A.Assessment of differential item functioning for performance tasks.Journal of Educational Measurement. 1993;30:233-251.