The Relation between Score Resolution Methods and Interrater Reliability: An Empirical Study of an Analytic Scoring Rubric

Applied Measurement in Education

Volumn 13, Issue 2, 2000, Pages 121-138

  Johnson, Robert L ^b   Penny, James ^a   Gordon, Belita ^c  

^a New Initiatives   (United States)

^b UNIVERSITY OF SOUTH CAROLINA   (United States)

^c NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0034386497     PISSN: 08957347     EISSN: None     Source Type: Journal    
DOI: 10.1207/S15324818AME1302_1     Document Type: Article

References (14)

1. Breland, H. (1983). The direct assessment of writing skill: A measurement review (Tech. Rep. No. 83-6). Princeton, NJ: College Entrance Examination Board.
2. Brennan, R. (1992). NCME instructional module: Generalizability theory. Educational Measurement: Issues and Practices, 11(4), 27-34.
3. Crick, J., & Brennan, R. (1984). GENOVA [Computer software]. Philadelphia: National Board of Medical Examiners.
4. Cronbach, L., Linn, R., Brennan, R., & Haertel, E. (1995). Generalizability analysis for educational assessments. Los Angeles: Center for the Study of Evaluation, Standards, and Student Testing, University of California at Los Angeles.
5. Georgia Department of Education. (1993). Georgia High School Writing Test: Assessment and instructional guide. Athens: Author.
6. Johnson, R. L., Penny, J., & Johnson, C. (1998, June). Score resolution in the rating ofperformance assessments: Practices and issues. Paper presented at the Council of Chief State School Officers National Conference on Large Scale Assessment, Colorado Springs, CO.
7. Johnson, R. L., Willeke, M., Bergman, T., & Steiner, D. (1997). Family literacy portfolios: Development and implementation. Window on the World of Family Literacy, 2(2), 10-17.
8. Margolis, M., & Ross, L. (1995, April). Halo and related effects in ratings by standardized patients in clinical evaluation. Paper presented at the annual meeting of the National Council of Measurement in Education, San Francisco.
9. Mehrens, W., & Lehmann, I. (1991). Measurement and evaluation in education and psychology (4th ed.). Fort Worth, TX: Harcourt Brace.
10. National Board of Professional Teaching Standards. (1993). Candidate guide [Brochure]. San Antonio, TX: Author.
11. National Education Goals Panel. (1996). Profile of 1994-1995 state assessment systems and reported results. Washington, DC: Author.
12. Shavelson, R., & Webb, N. (1991). Generalizability theory: A primer. Newbury Park, CA: Sage.
13. Smith, W. (1993). Assessing the reliability and adequacy of using holistic scoring of essays as a college composition placement technique. In M. Williamson & B. Huot (Eds.), Validating holistic scoring for writing assessment: Theoretical and empirical foundations (pp. 142-205). Cresskill, NJ: Hampton.
14. Welch, C., & Martinovich-Barhite, D. (1997, March). Reliability issues and possible solutions. Paper presented at the annual meeting of the American Educational Research Association, Chicago.