A Comparison of Procedures to Detect Item Parameter Drift

Applied Psychological Measurement

Volumn 22, Issue 1, 1998, Pages 33-51

  Donoghue, John R ^a   Isham, Steven P ^b  

^a EDUCATIONAL TESTING SERVICE   (United States)

^b NONE

Author keywords

Differential item function ing; Item bias; Item parameter drift; Item response theory, parameter drift; Lord's 2; Mantel Haenszel statistic; Signed area; Unsigned area

Indexed keywords

EID: 0032345993     PISSN: 01466216     EISSN: None     Source Type: Journal    
DOI: 10.1177/01466216980221002     Document Type: Article

References (44)

1. Agresti, A. (1990). Categorical data analysis. New York: Wiley.
2. Baker, F. B., Al-Karni, A., & Al-Dosary, I. M. (1991). EQUATE: A computer program for the test characteristic curve method of IRT equating. Applied Psychological Measurement, 15, 78.
3. Bock, R. D., Muraki, E., & Pfeiffenberger, W. (1988). Item pool maintenance in the presence of item parameter drift. Journal of Educational Measurement, 25, 275-285.
4. Campbell, J. R., Voelkl, K. E., & Donahue, P. L. (1997). NAEP 1996 trends in academic progress. Washington DC: National Center for Educational Statistics.
5. Candell, G. L., & Drasgow, F. (1988). An iterative procedure for linking metrics and assessing item bias in item response theory. Applied Psychological Measurement, 12, 253-260.
6. Chang, H., Mazzeo, J., & Roussos, L. (1996). Detecting DIF for polytomously scored items: An adaptation of the SIBTEST procedure. Journal of Educational Measurement, 33, 333-353.
7. 2 and Raju's area measures in detection of DIF. Applied Psychological Measurement, 17, 39-52.
8. Cohen, A. S., Kim, S.-H., & Baker, F. B. (1993). Detection of differential item functioning in the graded response model. Applied Psychological Measurement, 17, 335-350.
9. Cook, L. L., Eignor, D. R., & Taft, H. L. (1988). A comparative study of the effects of recency of instruction on the stability of IRT and conventional item parameter estimates. Journal of Educational Measurement, 25, 31-45.
10. Donoghue, J. R. (1996). Some formulae for computing IRT-DIF procedures using item parameters from NAEP BILOG/PARSCALE. Unpublished manuscript.
11. Donoghue, J. R. (1997). Implementing Shaffer's multiple comparison procedure for large numbers of groups. Under review.
12. Fleiss, J. L. (1973). Statistical methods for rates and proportions. New York: Wiley.
13. French, A. W., & Miller, T. R. (1996). Logistic regression and its use in detecting differential item functioning in polytomous items. Journal of Educational Measurement, 33, 315-332.
14. Goldstein, H. (1983). Measuring changes in educational attainment over time: Problems and possibilities. Journal of Educational Measurement, 20, 369-377.
15. Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer and H. Braun (Eds.), Test validity, pp. 129-145. Hillsdale NJ: Erlbaum.
16. Hulin, C. L., Lissak, R. I., & Drasgow, F. (1982). Recovery of two- and three-parameter logistic item characteristic curves: A monte carlo study. Applied Psychological Measurement, 6, 249-260.
17. Kim, S-H., & Cohen, A. S. (1991). A comparison of two area measures for detecting differential item functioning. Applied Psychological Measurement, 15, 269-278.
18. Kim, S-H., & Cohen, A. S. (1992a). Effects of linking methods on detection of DIF. Journal of Educational Measurement, 29, 51-66.
19. Kim, S-H., & Cohen, A. S. (1992b). FRTDIF: A computer program for IRT differential item functioning analysis. Applied Psychological Measurement, 16, 158.
20. Lord, F. M. (1968). An analysis of the verbal Scholastic Achievement Test using Birnbaum's three-parameter logistic model. Educational and Psychological Measurement, 28, 989-1020.
21. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale NJ: Erlbaum.
22. Miller, M. D., & Oshima, T. C. (1992). Effect of sample size, number of biased items, and magnitude of bias on a two-stage item bias estimation method. Applied Psychological Measurement, 16, 381-388.
23. Miller, T. R., & Spray, J. A. (1993). Logistic discriminant function analysis for DIF identification of polytomously scored items. Journal of Educational Measurement, 30, 107-122.
24. Millsap, R. E., & Everson, H. T. (1993). Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement, 17, 297-334.
25. Mislevy, R. J. (1982, March). Five steps toward controlling item parameter drift. Paper presented at the annual meeting of the American Educational Research Association, New York.
26. Mislevy, R. J., & Bock, R. D. (1992). BILOG: Item analysis and test scoring with binary logistic models [computer program]. Mooresville IN: Scientific Software.
27. Muraki, E. & Bock, R. D. (1991). PARSCALE: Parameter scaling of rating data [computer program]. Chicago: Scientific Software, Inc.
28. Narayanan, P., & Swaminathan, H. (1996). Identification of items that show nonuniform DIF. Applied Psychological Measurement, 20, 257-274.
29. Park, D.-G., & Lautenschlager, G. J. (1990). Improving IRT item bias detection with iterative linking and ability scale purification. Applied Psychological Measurement, 14, 163-173.
30. Potenza, M. T., & Dorans, N. J. (1995). DIF assessment for polytomously scored items: A framework for classification and evaluation. Applied Psychological Measurement, 19, 23-37.
31. Raju, N. S. (1988). The area between two item characteristic curves. Psychometrika, 53, 495-502.
32. Raju, N. S. (1990). Determining the significance of estimated signed and unsigned area between two item response functions. Applied Psychological Measurement, 14, 197-207.
33. Rogers, A. M. (1996). PARSCALE/NAEP User's guide [computer program]. Princeton NJ: Educational Testing Service.
34. Rogers, H. J., & Swaminathan, H. (1993). A comparison of the logistic regression and Mantel Haenszel procedures for detecting differential item functioning. Applied Psychological Measurement, 17, 105-116.
35. Roussos, L. A., & Stout, W. F. (1996). Simulation studies of the effects of small sample size and studied item parameters on SIBTEST and Mantel-Haenszel Type I error performance. Journal of Educational Measurement, 33, 215-230.
36. Shaffer, J. P. (1986). Modified sequentially rejective multiple test procedures. Journal of the American Statistical Association, 81, 826-831.
37. Shealy, R., & Stout, W. F. (1993). A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DIF as well as item bias/DIF. Psychometrika, 58, 159-194.
38. Stocking, M. L., & Lord, F. M. (1983). Developing a common metric in item response theory. Applied Psychological Measurement, 7, 201-210.
39. Stone, C. A., & Lane, S. (1991). Use of restricted item response theory models for examining the stability of item parameter estimates over time. Applied Measurement in Education, 4, 125-141.
40. Stone, C. A., Mislevy, R. J., & Mazzeo, J. (1994, April). Classification error and goodness-of-fit in IRT models. Paper presented at the annual meeting of the American Educational Research Association, New Orleans LA.
41. Stuart, A., & Ord, J. K. (1987). Kendall's advanced theory of statistics. Volume 1: Distributional theory (5th ed.). New York: Oxford University Press.
42. Thissen, D., Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response theory. In P. W. Holland and H. Wainer (Eds.). Differential item functioning: Theory and practice, pp. 67-113. Hillsdale NJ: Erlbaum
43. Uttaro, T., & Millsap, R. E. (1994). Factors influencing the Mantel-Haenszel procedure in the detection of differential item functioning. Applied Psychological Measurement, 18, 15-25.
44. Zwick, R., Donoghue, J. R., & Grima, A. (1993). Assessment of differential item functioning for performance tasks. Journal of Educational Measurement, 30, 233-251.