An NCME instructional module on using differential step functioning to refine the analysis of dif in polytomous items

Educational Measurement: Issues and Practice

Volumn 28, Issue 1, 2009, Pages 38-49

  Penfield, Randall D ^a   Gattamorta, Karina ^a   Childs, Ruth A ^b  

^a UNIVERSITY OF MIAMI   (United States)

^b UNIVERSITY OF TORONTO   (Canada)

Author keywords

Differential item functioning; Graded response model; Polytomous items

Indexed keywords

EID: 60249091295     PISSN: 07311745     EISSN: 17453992     Source Type: Journal    
DOI: 10.1111/j.1745-3992.2009.01135.x     Document Type: Article

References (57)

1. Agresti, A. (1990). Categorical data analysis. New York : Wiley.
2. Alvarez, K., & Penfield, R. D. (2007). Using differential step functioning (DSF) to refine the analysis of DIF in polytomous items: An illustration. Poster presented at the annual conference of the Institute of Education Sciences, Washington, DC.
3. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (1999). Standards for educational and psychological testing. Washington, DC : American Psychological Association.
4. Ankenmann, R. D., Witt, E. A., & Dunbar, S. B. (1999). An investigation of the power of the likelihood ratio goodness-of-fit statistic in detecting differential item functioning. Journal of Educational Measurement, 36, 277 300.
5. Baker, F. B. (1992). Item response theory: Parameter estimation techniques. New York : Marcel Dekker.
6. Bolt, D. M. (2000). A SIBTEST approach to testing DIF hypotheses using experimentally designed test items. Journal of Educational Measurement, 37, 307 327.
7. Camilli, G. (2006). Test fairness. In R. L. Brennan (Ed. Educational measurement (4th ed., pp. 65 110). Westport, CT : Praeger.
8. Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. Newbury Park, CA : Sage.
9. Chang, H., Mazzeo, J., & Roussos, L. (1996). Detecting DIF for polytomously scored items: An adaptation of the SIBTEST procedure. Journal of Educational Measurement, 33, 333 353.
10. Cohen, A. S., Kim, S.-H., & Baker, F. B. (1993). Detection of differential item functioning in the graded response model. Applied Psychological Measurement, 17, 335 350.
11. DiBello, L. V., Roussos, L. A., & Stout, W. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In S. Sinharay & C. R. Rao (Eds. Handbook of statistics, volume 26: Psychometrics (pp. 979 1030). New York : Elsevier.
12. Dorans, N. J., & Schmitt, A. P. (1993). Constructed response and differential item functioning: A pragmatic approach. In R. E. Bennett & W. C. Ward (Eds. Construction versus choice in cognitive assessment: Issues in constructed response, performance testing, and portfolio assessment (pp. 135 166). Hillsdale, NJ : Erlbaum.
13. Douglas, J., Roussos, L., & Stout, W. (1996). Item bundle DIF hypothesis testing: Identifying suspect bundles and assessing their DIF. Journal of Educational Measurement, 33, 465 484.
14. French, A. W., & Miller, T. R. (1996). Logistic regression and its use in detecting differential item functioning in polytomous items. Journal of Educational Measurement, 33, 315 332.
15. Gierl, M. J., & Khaliq, S. N. (2001). Identifying sources of differential item and bundle functioning on translated achievement tests: A confirmatory analysis. Journal of Educational Measurement, 38, 164 187.
16. Hauck, W. W. (1979). The large sample variance of the Mantel-Haenszel estimator of a common odds ratio. Biometrics, 35, 817 819.
17. Holland, P. W., & Thayer, D. T. (1988). Differential item performance and the Mantel-Haenszel procedure. In H. Wainer & H. I. Braun (Eds. Test validity (pp. 129 145). Hillsdale, NJ : Erlbaum.
18. Holland, P. W., & Wainer, H. (1993). Differential item functioning. Hillsdale, NJ : Erlbaum.
19. Jodoin, M. G., & Gierl, M. J. (2001). Evaluating Type I error and power rates using an effect size measure with the logistic regression procedure for DIF detection. Applied Measurement in Education, 14, 329 349.
20. Kim, S.-H., & Cohen, A. S. (1998). Detection of differential item functioning under the graded response model with the likelihood ratio test. Applied Psychological Measurement, 22, 345 355.
21. Leighton, J. P., & Gierl, M. J. (2007). Defining and evaluating models of cognition used in educational measurement to make inferences about examinees' thinking processes. Educational Measurement: Issues and Practice, 26 (2 3 16.
22. Liu, I.-M., & Agresti, A. (1996). Mantel-Haenszel-type inference for cumulative odds ratios with a stratified ordinal response. Biometrics, 52, 1223 1234.
23. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ : Erlbaum.
24. Mantel, N. (1963). Chi-square tests with one degree of freedom: Extension of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58, 690 700.
25. Mantel, N., & Haenszel, W. (1959). Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22, 719 748.
26. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, 149, 174.
27. Mazor, K. M., Clauser, B. E., & Hambleton, R. K. (1994). Identification of nonuniform differential item functioning using a variation of the Mantel-Haenszel procedure. Educational and Psychological Measurement, 54, 284 291.
28. Mislevy, R. J. (2006). Cognitive psychology and educational assessment. In R. L. Brennan (Ed. Educational measurement (4th ed., pp. 257 305). Westport, CT : Praeger.
29. Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159 176.
30. Muraki, E. (1997). A generalized partial credit model. In W. J. van der Linden & R. K. Hambleton (Eds. Handbook of modern item response theory (pp. 153 164). New York : Springer.
31. Oshima, T. C., Raju, N. S., Flowers, C. P., & Slinde, J. A. (1998). Differential bundle functioning using the DFIT framework: Procedures for identifying possible sources of differential functioning. Applied Psychological Measurement, 11, 353 369.
32. Penfield, R. D. (2003). Application of the Breslow-Day test of trend in odds ratio heterogeneity to the detection of nonuniform DIF. Alberta Journal of Educational Research, 49, 231 243.
33. Penfield, R. D. (2005). DIFAS: Differential item functioning analysis system. Applied Psychological Measurement, 29, 150 151.
34. Penfield, R. D. (2006). A nonparametric method for assessing differential step functioning in polytomous items. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco, CA.
35. Penfield, R. D. (2007). Assessing differential step functioning in polytomous items using a common odds ratio estimator. Journal of Educational Measurement, 44, 187 210.
36. Penfield, R. D., & Algina, J. (2003). Applying the Liu-Agresti estimator of the cumulative common odds ratio to DIF detection in polytomous items. Journal of Educational Measurement, 40, 353 370.
37. Penfield, R. D., & Camilli, G. (2007). Differential item functioning and item bias. In S. Sinharay & C. R. Rao (Eds. Handbook of statistics, volume 26: Psychometrics (pp. 125 167). New York : Elsevier.
38. Penfield, R. D., & Lam, T. C. M. (2000). Assessing differential item functioning in performance assessment: Review and recommendations. Educational Measurement: Issues and Practice, 19 (3 5 15.
39. Penfield, R. D., Alvarez, K., & Lee, O. (2009). Using a taxonomy of differential step functioning to improve the interpretation of DIF in polytomous items: An illustration. Applied Measurement in Education, 22, 61 78.
40. Potenza, M. T., & Dorans, N. J. (1995). DIF assessment for polytomously scored items: A framework for classification and evaluation. Applied Psychological Measurement, 19, 23 37.
41. Raju, N. S. (1988). The area between two item characteristic curves. Psychometrika, 53, 495 502.
42. Roussos, L. A., & Stout, W. (2004). Differential item functioning analysis: Detecting DIF item and testing DIF hypotheses. In D. Kaplan (Ed. The Sage handbook of quantitative methodology for the social sciences (pp. 107 115). Thousand Oaks, CA : Sage.
43. Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph Supplement, 34 (4, Part 2, No. 17).
44. Samejima, F. (1972). A general model for free response data. Psychometrika Monograph Supplement, 37 (1, Part 2, No. 18).
45. Samejima, F. (1997). Graded response model. In W. V. van der Linden & R. K. Hambleton (Eds. Handbook of modern item response theory (pp. 85 100). New York : Springer.
46. Scheuneman, J. D. (1987). An experimental, exploratory study of causes of bias in test items. Journal of Educational Measurement, 24, 97 118.
47. Schmitt, A. P., Holland, P. W., & Dorans, N. J. (1993). Evaluating hypotheses about differential item functioning. In P. W. Holland & H. Wainer (Eds. Differential item functioning (pp. 281 315). Hillsdale, NJ : Erlbaum.
48. Somes, G. W. (1986). The generalized Mantel-Haenszel statistic. The American Statistician, 40, 106 108.
49. Swaminathan, H., & Rogers, H. J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361 370.
50. Swanson, D. B., Clauser, B. E., Case, S. M., Nungester, R. J., & Featherman, C. (2002). Analysis of differential item functioning (DIF) using hierarchical logistic regression models. Journal of Educational and Behavioral Statistics, 27, 53 75.
51. Thissen, D. (2001). IRTLRDIF (version 2) [Computer software]. Retrieved January 17, 2006, from http://www.unc.edu/∼dthissen/dl.html.
52. Thissen, D., Chen, W.-H., & Bock, D. (2003). MULTILOG 7 [Computer software]. Lincolnwood, IL : Scientific Software International.
53. Thissen, D., Steinberg, L., & Wainer, H. (1993). Detection of differential item functioning using the parameters of item response models. In P. W. Holland & H. Wainer (Eds. Differential item functioning (pp. 67 113). Hillsdale, NJ : Erlbaum.
54. Wang, W.-C., & Su, Y.-H. (2004). Factors influencing the Mantel and generalized Mantel-Haenszel methods for the assessment of differential item functioning in polytomous items. Applied Psychological Measurement, 28, 450 480.
55. Zieky, M. (1993). Practical questions in the use of DIF statistics in item development. In P. W. Holland & H. Wainer (Eds. Differential item functioning (pp. 337 347). Hillsdale, NJ : Erlbaum.
56. Zimowski, M., Muraki, E., Mislevy, R., & Bock, D. (2003). BILOG-MG 3 [Computer software]. Lincolnwood, IL : Scientific Software International.
57. Zumbo, B. D., & Thomas, D. R. (1996, October). A measure of DIF effect size using logistic regression procedures. Paper presented at the National Board of Medical Examiners, Philadelphia, PA.