Effects of assigning raters to items

Educational Measurement: Issues and Practice

Volumn 27, Issue 1, 2008, Pages 47-55

  Sykes, Robert C ^a   Ito, Kyoko ^b,d   Wang, Zhen ^c,e  

^a CTB MCGRAW HILL   (United States)

^b HUMAN RESOURCES RESEARCH ORGANIZATION   (United States)

^c Harcourt Assessment ^*

^d Human Resources Research Organization   (United States)

^e Harcourt Assessment Inc   (United States)

Author keywords

Correlated error; Measurement model; Rater effects; RIB; Tau equivalency

Indexed keywords

EID: 40749095269     PISSN: 07311745     EISSN: 17453992     Source Type: Journal    
DOI: 10.1111/j.1745-3992.2008.00114.x     Document Type: Article

References (17)

1. Bock, R. D., Brennan, R. L., & Muraki, E. (2002). The information in multiple ratings. Applied Psychological Measurement, 26, 364 375.
2. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds. Testing structural equation models. CA : Sage Publications.
3. Englehard, G. Jr., & Myford, C. M. (2003). Monitoring faculty consultant performance in the advanced placement English literature and composition program with a many-faceted Rasch model. College Board Research Report No. 2003-1. New York, NY : The College Board.
4. Haladyna, T. (2007). The consequences of rater severity on pass/fail decisions in a performance assessment. Unpublished manuscript, Arizona State University.
5. Hombo, C. M., Donoghue, J. R., & Thayer, D. T. (2001). A simulation study of the effect of rater designs on ability estimation. ETS Research Report, RR-01-05. Princeton, NJ : Educational Testing Service.
6. Joreskog, K., & Sorbom, D. (1993). LISREL 8: Structural modeling with the SIMPLIS command language. Chicago, IL : Scientific Software International.
7. Joreskog, K., & Sorbom, D. (2004). LISREL (version 8.71.) [Computer software]. Lincolnwood, IL Scientific Software International.
8. Landy, R. J., & Farr, J. L. (1980). Performance rating. Psychological Bulletin, 87, 72 107.
9. Linacre, J. M. (1994). Facets Computer Program Chicago, IL : Mesa Press.
10. McDonald, R. P. (1985). Factor Analysis and Related Methods. Hillsdale, NJ : Lawrence Erlbaum Associates.
11. Myford, C., & Wolfe, E. W. (2003). Detecting and measuring rater effects using the many-faceted Rasch model. Journal of Applied Measurement, 4, 386 422.
12. Patz, R. J., Junker, B. W., Johnson, M. S., & Mariano, L. T. (2002). The hierarchical rater model for rated test items and its application to large-scale educational assessment data. Journal of Educational and Behavioral Statistics, 27, 341 384.
13. Rudner, L. M. (1992). Reducing errors due to the use of judges. ERIC/TM Digest. Washington, DC : ERIC Clearinghouse on Tests, Measurements, and Evaluation. EDO-TM-92-10.
14. Steiger, J. H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25, 173 180.
15. Sykes, R. C., Heidorn, M., & Lee, G. (1999). The assignment of raters to items: Controlling for rater effects. Paper presented at the annual meeting of the National Council on Measurement in Education, Montreal.
16. Wilson, M., & Hoskins, M. (2001). The rater bundle model. Journal of Educational and Behavioral Statistics, 26, 283 306.
17. Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30, 187 213.