The exact variance of weighted kappa with multiple raters

Psychological Reports

Volumn 101, Issue 2, 2007, Pages 655-660

  Mielke Jr , Paul W ^a   Berry, Kenneth J ^a   Johnston, Janis E ^a  

^a Department of Ecosystem Science and Sustainability   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; HUMAN; OBSERVER VARIATION; PSYCHOLOGICAL MODEL; PSYCHOLOGY; STATISTICAL ANALYSIS; STATISTICS;

DATA INTERPRETATION, STATISTICAL; HUMANS; MODELS, PSYCHOLOGICAL; OBSERVER VARIATION; PSYCHOLOGY;

EID: 36849015912     PISSN: 00332941     EISSN: 1558691X     Source Type: Journal    
DOI: 10.2466/PR0.101.2.655-660     Document Type: Article

References (19)

1. ANDRÉS, A. M., & MARZO, P. F. (2004) Delta: a new measure of agreement between two raters. British Journal of Mathematical and Statistical Psychology, 57, 1-19.
2. ANDRÉS, A. M., & MARZO, P. F. (2005) Chance-corrected measures of reliability and validity in K x K tables. Statistical Methods in Medical Research, 14, 473-492.
3. BANERJEE, M., CAPOZZOLI, M., MCS WEENEY, L., & SINHA, D. (1999) Beyond kappa: a review of interrater agreement measures. The Canadian Journal of Statistics, 27, 3-23.
4. BRENNAN, R. L., & PREDIGER, D. J. (1981) Coefficient kappa: some uses, misuses, and alternatives. Educational and Psychological Measurement, 41, 687-699.
5. COHEN, J. (1960) A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.
6. COHEN, J. (1968) Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70, 213-220.
7. DE MAST, J. (2007) Agreement and kappa-type indices. The American Statistician, 61, 148-153.
8. EVERITT, B. S. (1968) Moments of the statistics kappa and weighted kappa. British Journal of Mathematical and Statistical Psychology, 21, 97-103.
9. FLEISS, J. L., COHEN, J., & EVERITT, B. S. (1969) Large sample standard errors of kappa and weighted kappa. Psychological Bulletin, 12, 323-327.
10. GRAHAM, P., & JACKSON, R. (1993) The analysis of ordinal agreement data: beyond weighted kappa. Journal of Clinical Epidemiology, 46, 1055-1062.
11. n, Cohen's kappa, Scott's π, and Aickin's α. Understanding Statistics, 2, 205-219.
12. KUNDEL, H. L., & POLANSKY, M. (2003) Measurement of observer agreement. Radiology, 228, 303-308.
13. MACLURE, M., & WILLETT, W. C. (1987) Misinterpretation and misuse of the kappa statistic. American Journal of Epidemiology, 126, 161-169.
14. MIELKE, P. W., & BERRY, K. J. (1988) Cumulant methods for analyzing independence of r-way contingency tables and goodness-of-fit frequency data. Biometrika, 75, 790-793.
15. MIELKE, P. W., BERRY, K. J., & JOHNSTON, J. E. (2005) A FORTRAN program for computing the exact variance of weighted kappa. Perceptual and Motor Skills, 101, 468-472.
16. NELSON, J. C., & PEPE, M. S. (2000) Statistical description of interrater variability in ordinal ratings. Statistical Methods in Medical Research, 9, 475-496.
17. SCHUSTER, C. (2004) A note on the interpretation of weighted kappa and its relations to other rater agreement statistics for metric scales. Educational and Psychological Measurement, 64, 243-253.
18. SCHUSTER, C., & SMITH, D. A. (2005) Dispersion-weighted kappa: an integrative framework for metric and nominal scale agreement coefficients. Psychometrika, 70, 135-146.
19. ZWICK, R. (1988) Another look at interrater agreement. Psychological Bulletin, 103, 374-378.