Cohen's kappa can always be increased and decreased by combining categories

Statistical Methodology

Volumn 7, Issue 6, 2010, Pages 673-677

  Warrens, Matthijs J ^a  

^a LEIDEN UNIVERSITY   (Netherlands)

Author keywords

Cohen's kappa; Collapsing categories; Merging categories; Nominal agreement

Indexed keywords

EID: 77956601127     PISSN: 15723127     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.stamet.2010.05.003     Document Type: Article

References (19)

1. Agresti A. Categorical Data Analysis 1990, Wiley, New York.
2. Bishop Y.M.M., Fienberg S.E., Holland P.W. Discrete Multivariate Analysis. Theory and Practice 1976, MIT Press, Cambridge.
3. Brennan R.L., Prediger D.J. Coefficient kappa: some uses, misuses, and alternatives. Educational and Psychological Measurement 1981, 41:687-699.
4. Cohen J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 1960, 20:213-220.
5. Fleiss J.L. Measuring nominal scale agreement among many raters. Psychological Bulletin 1971, 76:378-382.
6. Fleiss J.L. Measuring agreement between two judges on the presence or absence of a trait. Biometrics 1975, 31:651-659.
7. James I.R. Analysis of nonagreement among multiple raters. Biometrics 1983, 39:651-657.
8. Kraemer H.C. Ramifications of a population model for K{cyrillic} as a coefficient of reliability. Psychometrika 1979, 44:461-472.
9. Kraemer H.C. Extension of the kappa coefficient. Biometrics 1980, 36:207-216.
10. Kraemer H.C. Measurement of reliability for categorical data in medical research. Statistical Methods in Medical Research 1992, 1:183-199.
11. Krippendorff K. Reliability in content analysis: some common misconceptions and recommendations. Human Communication Research 2004, 30:411-433.
12. Roberts C., McNamee R. A matrix of kappa-type coefficients to assess the reliability of nominal scales. Statistics in Medicine 1998, 17:471-488.
13. Schouten H.J.A. Nominal scale agreement among observers. Psychometrika 1986, 51:453-466.
14. Visser H., De Nijs T. The map comparison kit. Environmental Modelling & Software 2006, 21:346-358.
15. Warrens M.J. On the equivalence of Cohen's kappa and the Hubert-Arabie adjusted Rand index. Journal of Classification 2008, 25:177-183.
16. Warrens M.J. On similarity coefficients for 2×2 tables and correction for chance. Psychometrika 2008, 73:487-502.
17. Warrens M.J. On association coefficients for 2×2 tables and properties that do not depend on the marginal distributions. Psychometrika 2008, 73:777-789.
18. Warrens M.J. Inequalities between kappa and kappa-like statistics for k×k tables. Psychometrika 2010, 75:176-185.
19. Zwick R. Another look at interrater agreement. Psychological Bulletin 1988, 103:374-378.