An illustration of a mantel-haenszel procedure to flag misbehaving common items in test equating

Practical Assessment, Research and Evaluation

Volumn 13, Issue 7, 2008, Pages

  Michaelides, Michalis P ^a  

^a EUROPEAN UNIVERSITY CYPRUS   (Cyprus)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77954195778     PISSN: None     EISSN: 15317714     Source Type: Journal    
DOI: None     Document Type: Article

References (44)

1. American Educational Research Association, American Psychological Association, National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, D.C.: AERA.
2. Angoff, W. H. (1972, Sept.). A technique for the investigation of cultural differences. Paper presented at the annual meeting of the American Psychological Association, Honolulu. (ERIC Document Reproduction Service No. ED 069686)
3. Angoff, W. H. (1993). Perspectives on differential item functioning methodology. In P. W. Holland, and H. Wainer (Eds.), Differential item functioning (pp. 3-23). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.
4. Bock, R. D., Muraki, E., & Pfeiffenberger, W. (1988). Item pool maintenance in the presence of item parameter drift. Journal of Educational Measurement, 25(4), 275-285.
5. Camilli, G. (2006). Test Fairness. In R. L. Brennan (Ed.), Educational Measurement (4th edition, pp. 221-256). Westport, CT: ACE/Praeger.
6. Cook, L. L., Eignor, D. R., & Taft, H. L. (1988). A comparative study of the effects of recency of instruction on the stability of irt and conventional item parameter estimates. Journal of Educational Measurement, 25(1), 31-45.
7. Donoghue, J. R., Holland, P. W., & Thayer, D. T. (1993). A Monte Carlo study of factors that affect the mantel-haenszel and standardization measures of differential item functioning. In P. W. Holland, and H. Wainer (Eds.), Differential item functioning (pp. 137-166). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.
8. Dorans, N. J., & Holland, P. W. (1993). DIF Detection and Description: Mantel-Haenszel and Standardization. In P. W. Holland, and H. Wainer (Eds.), Differential Item Functioning (pp. 35-66). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.
9. Dorans, N. J., & Kulick, E. (1986). Demonstrating the utility of the standardization approach to assessing unexpected differential item performance on the Scholastic Aptitude Test. Journal of Educational Measurement, 23, 355-368.
10. Dorans, N. J., & Schmitt, A. P. (1991). Constructed-response and differential item functioning: A pragmatic approach (ETS Research Report No. 91-47). Princeton, NJ: Educational Testing Service. [Also appears in Construction vs. Choice in Cognitive Measurement (pp. 135-166), R. E. Bennett & W. C. Ward (Eds.), 1993, Hillsdale, NJ: Erlbaum.]
11. Hanson, B. A., & Feinstein, Z. S. (1997). Application of a Polynomial Loglinear Model to Assessing Differential Item Functioning for Common Items in the Common-Item Equating Design (ACT Research Report Series No. 97-1). Iowa City, IA: ACT Inc.
12. Hambleton, R. K., Swaminathan, H., & Rogers H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage Publications, Inc.
13. Holland, P. W., & Thayer, D. T. (1988). Differential Item Performance and the Mantel-Haenszel Procedure. In H. Wainer, and H. I. Brown (Eds.), Test Validity (pp. 129-145). Hillsdale, NJ: Lawrence Erlbaum Associates.
14. Kim, S.-H., Cohen, A.S., Alagoz, C., & Kim, S. (2007). DIF Detection and Effect Size Measures for Polytomously Scored Items. Journal of Educational Measurement, 44(2), 93-116.
15. Kingston, N. M., & Dorans, N. J. (1984). Item location effects and their implications for IRT equating and adaptive testing. Applied Psychological Measurement, 8, 147-154.
16. Kolen, M. J., & Brennan, R. L. (2004). Test Equating Scaling, and Linking: Methods and Practices (2nd edition). New York: Springer.
17. Kristjansson, E., Aylesworth, R, McDowell, I., & Zumbo, B. D. (2005). A comparison of four methods for detecting differential item functioning in ordered response items. Educational and Psychological Measurement, 65(6), 935-953.
18. Linn, R. L. (1993). The use of differential item functioning statistics: A discussion of current practice and future implications. In P. W. Holland, and H. Wainer (Eds.), Differential item functioning (pp. 349-364). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.
19. Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.
20. Mantel, N. (1963). Chi-square tests with one degree of freedom; Extensions of the Mantel-Haenszel procedure. Journal of the American Statistical Association, 58, 690-700.
21. Mantel, N., & Haenszel, W. (1959). Statistical Aspects of the Analysis of Data from Retrospective Studies of Disease. Journal of the National Cancer Institute, 22, 719-748.
22. Masters, G. N. (1988). Item discrimination: When more is worse. Journal of Educational Measurement, 25(1), 15-29.
23. Michaelides, M. P. (2003). Effects of common-item selection on item response theory test equating with nonequivalent groups. Unpublished doctoral dissertation, Stanford University, Stanford, CA.
24. Michaelides, M. P. (2005). An application of a Mantel-Haenszel procedure to flag misbehaving common items in test equating. Poster presented at the 2005 Annual Meeting of the American Educational Research Association, Montreal, Canada.Practical Assessment, Research & Evaluation, Vol 13, No 7
25. Miller, A. D., & Linn, R. L. (1988). Invariance of item characteristic functions with variations in instructional coverage. Journal of Educational Measurement, 25(3), 205-219.
26. Mitzel, H. C., Weber, M. M., & Sykes, R. C. (1999). Test item disclosure: How much difference does it really make? Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal, Canada.
27. Phillips, A., & Holland, P. W. (1987). Estimators of the variance of the Mantel-Haenszel log-odds-ratio estimate. Biometrics, 43, 425-431.
28. Shepard, L. A. (1982). Definitions of bias. In R. A. Berk (Ed.), Handbook of methods for detecting test bias (pp. 9-30). Baltimore: John Hopkins University Press.
29. Simpson, E. H. (1951). Interpretation of interaction contingency tables. Journal of the Royal Statistical Society, (Series B), 13, 238-241.
30. Su, Y.-H., & Wang, W.-C. (2005). Efficiency of the Mantel, generalized Mantel-Haenszel, and logistic discriminant function analysis methods in detecting differential item functioning for polytomous items. Applied Measurement in Education, 18, 313-350.
31. Swaminathan, H., & Rogers, J. (1990). Detecting differential item functioning using logistic regression procedures. Journal of Educational Measurement, 27, 361-370.
32. Sykes, R. C., & Fitzpatrick, A. R. (1992). The stability of IRT b values. Journal of Educational Measurement, 29(3), 201-211.
33. U.S. Department of Education, Office of Educational Research and Improvement, National Center for Education Statistics. The NAEP 1998 Technical Report, NCES 2001-509, by Allen, N. L., Donoghue, J. R., & Schoeps, T. L. (2001). Washington, DC: National Center for Education Statistics.
34. Vukmirovic, Z., Hu, H., & Turner, J. C. (2003, April). The effects of outliers on IRT equating with fixed common item parameters. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, Il.
35. Wainer, H. (1999). Comparing the Incomparable: An Essay on the Importance of Big Assumptions and Scant Evidence. Educational Measurement: Issues and Practice, 18(4), 10-16.
36. Welch, C. J., & Miller, T. R. (1995). assessing differential item functioning in direct writing assessments: Problems and an example. Journal of Educational Measurement, 32(2), 163-178.
37. Yen, W. M. (1980). The extent, causes and importance of context effects on item parameters for two latent trait models. Journal of Educational Measurement, 17(4), 297-311.
38. Yen, W. M., Green, D. R., & Burket, G. R. (1987). Valid information from customized achievement tests. Educational Measurement: Issues and Practice, 6(1), 7-13.
39. Zieky, M. (1993). Practical Questions in the Use of DIF Statistics in Test Development. In P. W. Holland, and H. Wainer (Eds.), Differential Item Functioning (pp. 337-347). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.
40. Zumbo, B. D. (1999). A handbook on the theory and methods of differential item functioning (DIF). Ottawa, Ontario, Canada: Directorate of Human Resources Research and Evaluation, Department of National Defense.
41. Zwick, R. (1990). When do item response function and Mantel-Haenszel definitions of differential item functioning coincide? Journal of Educational Statistics, 15(3), 185-197.
42. Zwick, R., Donoghue, J. R., & Grima, A. (1993). Assessment of Differential Item Functioning for Performance Tasks. Journal of Educational Measurement, 30(3), 233-251.
43. Zwick, R., & Thayer, D. T. (1996). Evaluating the magnitude of differential item functioning in polytomous items. Journal of Educational and Behavioral Statistics, 21(3), 187-201.
44. Zwick, R., Thayer, D. T., & Mazzeo, J. (1997). Descriptive and inferential procedures for assessing differential item functioning in polytomous items. Applied Measurement in Education, 10(4), 321-344.