Kappa coefficients in medical research

Statistics in Medicine

Volumn 21, Issue 14, 2002, Pages 2109-2129

  Kraemer, Helena Chmura ^a,c   Periyakoil, Vyjeyanthi S ^b   Noda, Art ^a  

^a STANFORD UNIVERSITY SCHOOL OF MEDICINE   (United States)

^b VA PALO ALTO HEALTH CARE SYSTEM   (United States)

^c STANFORD UNIVERSITY   (United States)

Author keywords

Consensus; Kappa; Reliability; Validity

Indexed keywords

ARTICLE; BIOSTATISTICS; CONSTANTS AND COEFFICIENTS; CONTROLLED STUDY; MEDICAL RESEARCH; METHODOLOGY; PROCESS DESIGN; RELIABILITY; VALIDATION PROCESS;

DATA INTERPRETATION, STATISTICAL; HUMANS; RESEARCH; STATISTICS;

EID: 0037199386     PISSN: 02776715     EISSN: None     Source Type: Journal    
DOI: 10.1002/sim.1180     Document Type: Article

References (66)

1. Large sample variance of kappa in the case of different sets of raters
2. Reliability of content analysis: The case of nominal scale coding
3. A coefficient of agreement for nominal scales
4. A bibliography of publications on observer variability
5. Beyond kappa: A review of interrater agreement measures
6. On the methods and theory of reliability
7. Coefficient kappa; some uses, misuses, and alternatives
8. Measuring agreement between two judges on the presence or absence of a trait
9. A comparison of three indexes of agreement between observers: Proportion of agreement, G-index, and kappa
10. Kappa coefficients in epidemiology: An appraisal of a reappaisal
11. A review of statistical methods in the analysis of data arising from observer reliability studies
12. Measures of response agreement for qualitative data: Some generalizations and alternatives
13. Misinterpretation and misuse of the Kappa statistic
14. Measurement reliability and agreement in psychiatry
15. A reappraisal of the kappa coefficient
16. Bias, prevalence and kappa
17. High agreement but low kappa: I. The problems of two paradoxes
18. Bias and prevalence effects on kappa viewed in terms of sensitivity and specificity
19. Behavior and interpretation of the k statistic: Resolution of the two paradoxes
20. Modelling observer agreement-an alternative to kappa
21. Measuring nominal scale agreement among many raters
22. Extensions of the kappa coefficient
23. On assessing interrater agreement for multiple attribute responses
24. The measurement of observer agreement for categorical data
25. Agreement measurement and the judgment process
26. A proposed solution to the base rate problem in the kappa statistic
27. A computer program for determining the significance of the difference between pairs of independently derived values of kappa or weighted kappa
28. Sample size requirements for reliability studies
29. Testing the homogeneity of kappa statistics
30. Sample size requirements for the comparison of two or more coefficients of inter-observer agreement
31. Ramifications of a population model for k as a coefficient of reliability
32. Maximum likelihood estimation of agreement in the constant predictive probability model, and its relation to Cohen's kappa
33. Coefficients of agreement between observers and their interpretations
34. Estimating false alarms and missed events from interobserver agreement: Comment on Kaye
35. The Dependability of Behavioral Measurements
36. Measurement of reliability for categorical data in medical research
37. Reliability and validity in binary ratings
38. Reliability of multiple classifications
39. Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit
40. Category distinguishability and observer agreement
41. A hierarchical approach to inferences concerning interobserver agreement for multinomial data
42. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability
43. Jackknifing functions of multinomial frequencies, with an application to a measure of concordance
44. A goodness-of-fit approach to inference procedures for the kappa statistics: Confidence interval construction, significance-testing and sample size estimation
45. 2 x 2 kappa coefficients: Measures of agreement or association
46. A one-way components of variance model for categorical data
47. Standard error of the kappa statistic
48. Bootstrap methods: Another look at the jackknife
49. Confidence intervals for the interrater agreement measure kappa
50. Inference about weighted kappa in the non-null case
51. Interval estimation for Cohen's kappa as a measure of agreement
52. Hypothesis testing and effect size estimation in clinical trials
53. Kappa, measures of marginal symmetry and intraclass correlations
54. Measuring the potency of a risk factor for clinical or policy significance
55. Testing patterned hypotheses in multi-way contingency tables using weighted kappa and weighted chi square
56. The cost of dichotomization
57. How many raters are needed for a reliable diagnosis?
58. How many raters? Toward the most reliable diagnostic consensus
59. What is the 'right' statistical measure of twin concordance (or diagnostic reliability and validity)?
60. Statistical issues in assessing comorbidity
61. Modeling agreement among raters