Evaluating the Accuracy of Judgments Obtained from Item Review Committees

Applied Measurement in Education

Volumn 12, Issue 2, 1999, Pages 199-210

  Engelhard Jr , George ^b   Davis, Melodee ^a   Hansche, Linda ^c  

^a The Atlanta School   (United States)

^b EMORY UNIVERSITY   (United States)

^c NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0033472619     PISSN: 08957347     EISSN: None     Source Type: Journal    
DOI: 10.1207/s15324818ame1202_6     Document Type: Article

References (22)

1. Andrich, D. (1982). An index of person separation in latent trait theory, the traditional KR.20 index and the Guttman scale response pattern. Education Research and Perspectives, 9, 95-104.
2. Andrich, D. (1988). Rasch models for measurement. Newbury Park, CA: Sage.
3. Camilli, G., & Shepard, L. A. (1994). Methods for identifying biased test items. Thousand Oaks, CA: Sage.
4. Educational Testing Service. (1992). ETS sensitivity review process. Princeton, NJ: Author.
5. Engelhard, G. (1996). Evaluating rater accuracy in performance assessments. Journal of Educational Measurement, 33, 56-70.
6. Engelhard, G., Anderson, D., & Gabrielson, S. (1990). An empirical comparison of Mantel - Haenszel and Rasch procedures for studying differential item functioning on teacher certification tests. Journal of Research and Development in Education, 23, 172-179.
7. Engelhard, G., Hansche, L., & Rutledge, K. E. (1990). Accuracy of bias review judges in identifying differential item functioning on teacher certification tests. Applied Measurement in Education, 3, 347-360.
8. Georgia Department of Education. (1993). Guide to writing and reviewing multiple-choice items. Atlanta, GA: Author.
9. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. San Diego, CA: Academic.
10. Hoover, H. D., & Kolen, M. J. (1984). The reliability of six item bias indices. Applied Psychological Measurement, 8, 173-181.
11. Linacre, J. M. (1989). Many-faceted Rasch measurement. Chicago: MESA.
12. Linacre, J. M., Engelhard, G., Tatum, D. S., & Myford, C. M. (1994). Measurement with judges: Many-faceted conjoint measurement. International Journal of Educational Research, 21, 569-577.
13. Linacre, J. M., & Wright, B. D. (1994). A user's guide to FACETS: Rasch measurement computer program. Chicago: MESA.
14. Lunz, M. E., Wright, B. D., & Linacre, J. M. (1990). Measuring the impact of judge severity on examination scores. Applied Measurement in Education, 3, 331-345.
15. Plake, B. S. (1980). A comparison of a statistical and subjective procedure to ascertain item validity: One step in the test validation process. Educational and Psychological Measurement, 40, 397-404.
16. Ramsey, P. A. (1993). Sensitivity review: The ETS experience as a case study. In P. W. Holland & H. Wainer (Eds.), Differential item functioning (pp. 367-388). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.
17. Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests. Chicago: University of Chicago Press.
18. Rengel, E. (1986, August). Agreement between statistical and judgmental item bias methods. Paper presented at the annual meeting of the American Educational Research Association, Washington, DC. (ERIC Document Reproduction No. ED 289 890)
19. Sandoval, J., & Mülle, M. P. W. (1980). Accuracy of judgments of WISC-R item difficulty for minority groups. Journal of Consulting and Clinical Psychology, 48, 249-253.
20. Sulsky, L. M., & Balzer, W. K. (1988). Meaning and measurement of performance rating accuracy: Some methodological and theoretical concerns. Journal of Applied Psychology, 73, 497-506.
21. Wright, B. D., & Masters, G. N. (1982). Rating scale analysis: Rasch measurement. Chicago: MESA.
22. Wright, B. D., & Stone, M. H. (1979). Best test design: Rasch measurement. Chicago: MESA.