Assessing Differential Item Functioning in Performance Assessment: Review and Recommendations

Educational Measurement: Issues and Practice

Volumn 19, Issue 3, 2000, Pages 5-15

  Penfield, Randall D ^a   Lam, Tony C M ^b  

^a UNIVERSITY OF FLORIDA   (United States)

^b UNIVERSITY OF TORONTO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0010040755     PISSN: 07311745     EISSN: 17453992     Source Type: Journal    
DOI: 10.1111/j.1745-3992.2000.tb00033.x     Document Type: Article

References (46)

1. Agresti, A. (1990) Categorical data analysis. New York: Wiley.
2. Baker, F. B. (1992) Item response theory: Parameter estimation techniques. New York: Marcel Dekker.
3. Camilli, G. (1993) The case against item bias detection techniques based on internal criteria: Do item bias procedures obscure test fairness issues? In P. W. Holland & H. Wainer, (Eds.), Differential item functioning (pp. 397–413). Hillsdale, NJ: Erlbaum.
4. Camilli, G, & Penfield, D. A. (1997) Variance estimation for differential test functioning based on Mantel-Haenszel statistics. Journal of Educational Measurement, 34, 123–139.
5. Camilli, G., & Shepard, L. A. (1994) Methods for identifying biased test items. Newbury Park, CA: Sage.
6. Chang, H., Mazzeo, J., & Roussos, L. (1996) Detecting DIF for polytomously scored items: An adaptation of the SIBTEST procedure. Journal of Educational Measurement, 33, 333–353.
7. Clauser, B. E., & Mazor, K. M. (1998) Using statistical procedures to identify differentially functioning test items. Educational Measurement: Issues and Practice, 17(1), 31–44.
8. Dorans, N. J., & Holland, P. W. (1993) DIF detection and description: Mantel-Haenszel and Standardization. In P. W. Holland & H. Wainer, (Eds.), Differential item functioning (pp. 35–66). Hillsdale, NJ: Erlbaum.
9. Dorans, N. J., & Schmitt, A. J. (1991) Constructed response and differential item functioning: A pragmatic approach (Research Rep. No. 91-47). Princeton, NJ: Educational Testing Service.
10. French, A. W., & Miller, T. R. (1996) Logistic regression and its use in detecting differentially item functioning in polytomous items. Journal of Educational Measurement, 33, 315–332.
11. Hambleton, R. K., & Murphy, E. (1992) A psychometric perspective on authentic measurement. Applied Measurement in Education, 5, 1–16.
12. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991) Fundamentals of item response theory. Newbury Park, CA Sage.
13. Hamilton, L. S. (1999) Detecting gender-based differential item functioning on a constructed-response science test. Applied Measurement in Education, 12, 211–235.
14. Herman, J. L., Aschbacher, P. R., & Winters, L. (1992) A practical guide to alternative assessment. U.S.A.: Regents of the University of California.
15. Hills, J. R. (1989) Screening for potentially biased items in testing programs. Educational Measurement: Issues and Practice, 8(4), 5–11.
16. Holland, P. W., & Wainer, H. (Eds.). (1993) Differential item functioning. Hillsdale, NJ: Erlbaum.
17. Johnson, R. A., & Wichern, D. W. (1992) Applied multivariate statistical analysis (3rd ed.). Upper Saddle River, NJ: Prentice Hall.
18. Lam, T. C. M. (1995) Fairness in performance assessment. ERIC Digest (EDOCG-95-25).
19. Lord, F. M., & Novick, M. R. (1968) Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
20. Magnusson, D. (1967) Test theory. Palo Alto: Addison-Wesley.
21. Mantel, N. (1963) Chi-square tests with one degree of freedom: Extensions of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58, 690–700.
22. Meredith, V., & Williams, P. (1984) Issues in direct writing assessment: Problem identification and control. Educational Measurement: issues and Practice, 3, 11, 11–15.
23. Miller, T. R., & Spray, J. A. (1993) Logistic discriminant function analysis for DIF identification of polytomously scored items. Journal of Educational Measurement, 30, 107–122.
24. Miller, T. R., & Welch, C. J. (1993) Issues and problems in assessing differential item functioning in performance assessments. Paper presented at the annual meeting of the National Council on Measurement in Education, Atlanta.
25. Millsap, R. E., & Everson, H. T. (1993) Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement, 17, 297–334.
26. Muraki, E. (1992) A generalized partial credit model: Applications of an EM algorithm. Applied Psychological Measurement, 16, 159–176.
27. Neter, J., Kutner, M. H., Nachtsheim, C. J., & Wasserman, W. (1990) Applied linear regression models (3rd ed.). Toronto: Irwin.
28. Potenza, M. T., & Dorans, N. J. (1995) DIF assessment for polytomously scored items: a framework for classification and evaluation. Applied Psychological Measurement, 19, 23–37.
29. Reardon, S. F., Scott, K., & Verre, J. (1994) Introduction. Symposium: Equity and educational assessment. Harvard Educational Review, 64, 1–4.
30. Rousos, L., & Stout, W. (1996) A multidimensionality-based DIF analysis paradigm. Applied Psychological Measurement, 20, 355–371.
31. Sax, G. (1997) Principles of educational and psychological measurement and evaluation (4th ed.). Toronto: Wadsworth.
32. Shavelson, R. J., Baxter, G.P., & Gao, X. (1993) Sampling variability of performance assessments. Journal of Educational Assessment, 30, 215–232.
33. Shealy, R. T., & Stout, W. F. (1993) A model-based standardization approach that separates true bias/DIF from group ability differences and detects test bias/DIF as well as item bias/DIF. Psychometrika, 54, 159–194.
34. Somes, G. W. (1986) The generalized Mantel-Haenszel statistic. The American Statistician, 40, 106–108.
35. Stiggins, R., & Bridgeford, N. (1983) An analysis of published tests of writing proficiency. Educational Measurement: Issues and Practice, 2, 6–10.
36. Supovitz, J. A., & Brennan, R. T. (1997) Mirror, mirror on the wall, which is the fairest test of all? An examination of the equitability of portfolio assessment relative to standardized tests. Harvard Educational Review, 67, 472–506.
37. Swaminathan, H., & Rogers, H. J. (1990) Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361–370.
38. Tatsuoka, M. M. (1988) Multivariate analysis: Techniques for educational and psychological research (2nd ed.). New York: Macmillan.
39. Thissen, D., Steinberg, L., & Wainer, H. (1988) Use of item response theory in the study of group differences in trace lines. In H. Wainer & H. Braun, (Eds.), Test validity (pp. 147–169). Hillsdale, NJ: Erlbaum.
40. Traub, R. E. (1994a.) Facing the challenge of multidimensionality in performance assessment. Presented at the Sixth Ottowa Conference on Medical Examination, Toronto, Ontario.
41. Traub, R. E. (1994b) Reliability for the social sciences. Newbury Park, CA: Sage.
42. Welch, C., & Hoover, H. D. (1993) Procedures for extending item bias detection techniques to polytomously scored items. Applied Measurement in Education, 6, 1–19.
43. Welch, C., & Miller, T. R. (1995) Assessing differential item functioning in direct writing assessments: Problems and an example. Journal of Educational Measurement, 32, 163–178.
44. Zwick, R. (1990) When do item response function and Mantel-Haenszel definitions of differential item functioning coincide Journal of Educational Statistics, 15, 185–197.
45. Zwick, R., Donoghue, J. R., & Grima, A. (1993) Assessment of differential item functioning for performance tasks. Journal of Educational Measurement, 30, 233–251.
46. Zwick, R., Thayer, D. T., & Mazzeo, J. (1997) Descriptive and inferential procedures for assessing differential item functioning in polytomous items. Applied Measurement in Education, 10, 321–334.