Angoff's delta method revisited: Improving DIF detection under small samples

British Journal of Mathematical and Statistical Psychology

Volumn 65, Issue 2, 2012, Pages 302-321

  Magis, David ^a   Facon, Bruno ^b  

^a UNIVERSITY OF LIÈGE   (Belgium)

^b UNIV LILLE   (France)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; COMPUTER SIMULATION; SAMPLE SIZE; STATISTICAL MODEL; STATISTICS;

COMPUTER SIMULATION; MODELS, STATISTICAL; SAMPLE SIZE;

EID: 80053646587     PISSN: 00071102     EISSN: 20448317     Source Type: Journal    
DOI: 10.1111/j.2044-8317.2011.02025.x     Document Type: Article

References (59)

1. Abedalaziz, N. (2010). A gender-related differential item functioning of mathematics test items. International Journal of Educational and Psychological Assessment, 5, 101-116.
2. Ackerman, T. A. (1992). A didactic explanation of item bias, item impact, and item validity from a multidimensional perspective. Journal of Educational Measurement, 29, 67-91. doi:
3. Agresti, A. (1996). Introduction to categorical data analysis. New York : Wiley.
4. Agresti, A., & Coull, B. A. (1998). Approximate is better than 'exact' for interval estimation of binomial proportions. American Statistician, 52, 119-126.
5. Angoff, W. H. (1982). Use of difficulty and discrimination indices for detecting item bias. In R. A. Beck (Ed.), Handbook of methods for detecting item bias (pp. 96-116). Baltimore, MD : Johns Hopkins University Press.
6. Angoff, W. H. (1993). Perspectives on differential item functioning methodology. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 3-23). Hillsdale, NJ : Lawrence Erlbaum Associates.
7. Angoff, W. H., & Ford, S. F. (1973). Item-race interaction on a test of scholastic aptitude. Journal of Educational Measurement, 10, 95-106. doi:
8. Bellugi, U., Lichtenberger, L., Jones, W., & Lai, Z. (2000). I. The neurocognitive profile of Williams syndrome: A complex pattern of strengths and weaknesses. Journal of Cognitive Neuroscience, 12, 7-29.
9. Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. Thousand Oaks, CA : Sage.
10. Candell, G. L., & Drasgow, F. (1988). An iterative procedure for linking metrics and assessing item bias in item response theory. Applied Psychological Measurement, 12, 253-260. doi:
11. Cole, N. S. (1978, March). Approaches to examining bias in achievement test items. Paper presented at the annual meeting of the American Personnel and Guidance Association, Washington, DC
12. Clauser, B. E., Mazor, K. M., & Hambleton, R. K. (1993). The effects of purification of the matching criterion on the identification of DIF using the Mantel-Haenszel procedure. Applied Measurement in Education, 6, 269-279. doi:
13. Donoghue, J. R., & Allen, N. L. (1993). Thin vs. thick matching in the Mantel-Haenszel procedure for detecting DIF. Journal of Educational Statistics, 18, 131-154. doi:
14. Donoghue, J. R., Holland, P. W., & Thayer, D. T. (1993). A Monte-Carlo study of factors that affect the Mantel-Haenszel and standardization measures of differential item functioning. In P. Holland & H. Wainer (Eds.), Differential item functioning. Hillsdale, NJ : Lawrence Erlbaum Associates.
15. Dorans, N. J., & Kulick, E. (1986). Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the Scholastic Aptitude Test. Journal of Educational Measurement, 23, 355-368. doi:
16. Dorans, N. J., Schmitt, A. P., & Bleistein, C. A. (1992). The standardization approach to assessing comprehensive differential item functioning. Journal of Educational Measurement, 29, 309-319. doi:
17. Facon, B., Magis, D., Nuchadee, M.-L., & De Boeck, P. (in press). Do Raven's Colored Progressive Matrices function in the same way in typical and clinical populations? Insight from the intellectual disability field. Intelligence.
18. Facon, B., & Nuchadee, M.-L. (2010). An item analysis of Raven's Colored Progressive Matrices among participants with Down syndrome. Research in Developmental Disabilities, 31, 243-249. doi:
19. Holland, P. W., & Thayer, D. T. (1985). An alternative definition of the ETS delta scale of item difficulty. Research Report RR-85-43 Princeton, NJ : Educational Testing Service.
20. Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 129-145). Hillsdale, NJ : Lawrence Erlbaum Associates.
21. Hunter, J. E. (1975, December). A critical analysis of the use of item means and item-test correlations to determine the presence or absence of content bias in achievement test items. Paper presented at the National Institute of Education Conference on Test Bias, Annapolis, MD
22. Järvinen-Pasley, A., Bellugi, U., Reilly, J., Mills, D. L., Galaburda, A., Reiss, A. L., & Korenberg, J. R. (2008). Defining the social phenotype in Williams syndrome: A model for linking gene, the brain, and behavior. Development and Psychopathology, 20, 1-35. doi:
23. Jodoin, M. G., & Gierl, M. J. (2001). Evaluating Type I error and power rates using an effect size measure with logistic regression procedure for DIF detection. Applied Measurement in Education, 14, 329-349. doi:
24. Johnson, R. A., & Wichern, D. W. (1998). Applied multivariate statistical analysis(4th ed.). Upper Saddle River, NJ : Prentice Hall.
25. Karkee, T., & Choi, S. (2005, April). Impact of eliminating anchor items flagged from statistical criteria on test score classifications in common item equating. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Canada
26. Lord, F. M. (1977). A study of item bias using item characteristic curve theory. In N. H. Poortinga (Ed.), Basic problems in cross-cultural psychology (pp. 19-29). Amsterdam : Swets and Zeitlinger.
27. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ : Lawrence Erlbaum Associates.
28. Magis, D., & De Boeck, P. (in press). A robust outlier approach to prevent Type I error inflation in DIF. Educational and Psychological Measurement.
29. Magis, D., & De Boeck, P. (in press). Identification of differential item functioning in multiple-groups settings: A multivariate outlier detection approach. Multivariate Behavioral Research.
30. Mervis, C. B., & John, A. E. (2010). Cognitive and behavioral characteristics of children with Williams syndrome: Implications for intervention approaches. American Journal of Medical Genetics Part C, 154, 229-248. doi:
31. Merz, W. R., & Grossen, N. E. (1979). An empirical investigation of six methods for examining test item bias. Unpublished research report. Sacramento : California State University.
32. Michaelides, M. P. (2010). Sensitivity of equated aggregate scores to the treatment of misbehaving common items. Applied Psychological Measurement, 34, 365-369. doi:
33. Moon, S., McLean, J. E., & Kaufman, A. S. (2003). A cross-cultural validation of the sequential simultaneous theory of intelligence in children. School Psychology International, 24, 449-461. doi:
34. Muniz, J., Hambleton, R. K., & Xing, D. (2001). Small sample studies to detect flaws in item translations. International Journal of Testing, 1, 115-135. doi:
35. Oosterhof, A. C., Atash, M. N., & Lassiter, K. L. (1984). Facilitating identification of item bias through use of delta plots. Educational and Psychological Measurement, 44, 619-627. doi:
36. Osterlind, S. J. (1983). Test item bias. Newbury Park, CA : Sage.
37. Osterlind, S. J., & Everson, H. T. (2009). Differential item functioning(2nd ed.). Thousand Oaks, CA : Sage.
38. Parshall, C. G., & Miller, T. R. (1995). Exact versus asymptotic Mantel-Haenszel DIF statistics: A comparison of performance under small-sample conditions. Journal of Educational Measurement, 32, 302-316. doi:
39. Penfield, R. D. (2001). Assessing differential item functioning among multiple groups: A comparison of three Mantel-Haenszel procedures. Applied Measurement in Education, 14, 235-259. doi:
40. Penfield, R. D. (2003). Application of the Breslow-Day test of trend in odds ratio heterogeneity to the detection of nonuniform DIF. Alberta Journal of Educational Research, 49, 231-243.
41. Penfield, R. D., & Camilli, G. (2007). Differential item functioning and item bias. In C. R. Rao & S. Sinharray (Eds.), Handbook of Statistics 26: Psychometrics (pp. 125-167). Amsterdam : Elsevier.
42. Raju, N. S. (1988). The area between two item characteristic curves. Psychometrika, 53, 495-502.
43. Raju, N. S. (1990). Determining the significance of estimated signed and unsigned areas between two item response functions. Applied Psychological Measurement, 14, 197-207. doi:
44. Robin, F., Sireci, S. G., & Hambleton, R. K. (2003). Evaluating the equivalence of different language versions of a credentialing exam. International Journal of Testing, 3, 1-20. doi:
45. Rudner, L. M. (1977, June). Efforts toward the development of unbiased selection and assessment instruments. Paper presented at the third International Symposium on Educational Testing, Leyden, The Netherlands
46. Rudner, L. M. (1978). Using standard tests with the hearing impaired: The problems of item bias. Volta Review, 80, 31-40.
47. Rudner, L. M., Getson, P. R., & Knight, D. L. (1980). A Monte Carlo comparison of seven biased item detection techniques. Journal of Educational Measurement, 17, 1-10. doi:
48. Schneider, A., Hagerman, R., & Hessl, D. (2009). Fragile X syndrome - From genes to cognition. Developmental Disabilities Research Reviews, 15, 333-342. doi:
49. Shealy, R. T., & Stout, W. (1993). A model based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DIF as well as item bias/DIF. Psychometrika, 58, 159-194.
50. Shepard, L. A., Camilli, G., & Williams, D. M. (1985). Validity of approximation techniques for detecting item bias. Journal of Educational Measurement, 22, 77-105. doi:
51. Silverman, W. (2007). Down syndrome: Cognitive phenotype. Mental Retardation and Developmental Disabilities Research Reviews, 13, 228-236. doi:
52. Sireci, S. G., & Allalouf, A. (2003). Appraising item equivalence across multiple languages and cultures. Language Testing, 20, 148-166. doi:
53. Sireci, S. G., Patsula, L., & Hambleton, R. K. (2005). Statistical methods for identifying flawed items in the test adaptation process. In R. K. Hambleton, P. F. Merenda, & C. D. Spielberger (Eds.), Adapting educational and psychological tests for cross-cultural assessment (pp. 93-115). Hillsdale, NJ : Lawrence Erlbaum Associates.
54. Strassberg-Rosenberg, B., & Donlon, T. F. (1975, June). Context influences on sex differences in performance and aptitude tests. Paper presented at the annual meeting of the National Council on Measurement in Education, Washington, DC
55. Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361-370. doi:
56. Thissen, D., Steinberg, L., & Wainer, H. (1988). Use of item response theory in the study of group difference in trace lines. In H. Wainer & H. Braun (Eds.), Test validity (pp. 147-170). Hillsdale, NJ : Lawrence Erlbaum Associates.
57. Van Herwegen, J., Farran, E., & Annaz, D. (2011). Item and error analysis on Raven's Colored Progressive Matrices in Williams syndrome. Research in Developmental Disabilities, 32, 93-99. doi:
58. Wainer, H. (2010). Fourteen conversations about three things. Journal of Educational and Behavioral Statistics, 35, 5-25. doi:
59. Zwick, R., Donoghue, J.R., & Grima, A. (1993). Assessment of differential item functioning for performance tasks. Journal of Educational Measurement, 30, 233-251. doi: