Statistical description of interrater variability in ordinal ratings

Statistical Methods in Medical Research

Volumn 9, Issue 5, 2000, Pages 475-496

  Nelson, J C ^a   Pepe, M S ^a  

^a UNIVERSITY OF WASHINGTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DATA ANALYSIS; HUMAN; IMAGE ANALYSIS; MAMMOGRAPHY; MEDICAL RESEARCH; METHODOLOGY; REVIEW; SAMPLE; SCORING SYSTEM; STATISTICAL ANALYSIS; STATISTICAL MODEL; STATISTICAL PARAMETERS; VARIANCE;

DATA INTERPRETATION, STATISTICAL; FEMALE; HUMANS; MAMMOGRAPHY; MODELS, STATISTICAL; OBSERVER VARIATION; REPRODUCIBILITY OF RESULTS; RESEARCH; RESEARCH DESIGN; UNITED STATES;

EID: 0034521639     PISSN: 09622802     EISSN: None     Source Type: Journal    
DOI: 10.1191/096228000701555262     Document Type: Review

References (41)

1. Improved detection of abnormal respiratory function using forced expiration from raised lung volume in infants with cystic fibrosis
2. Clinical evaluation of the blood glucose monitors Accutrend, Companion 2, Glucometer 3 and One Touch II in comparison with the glucose oxidase reference method
3. New imaging techniques for assessing cardiac function
4. Variability in the interpretation of screening mammograms by U.S. radiologists: Findings from a national sample
5. Poor reproducibility of classification of proximal humeral fractures
6. Interobserver reliability and intraobserver reproducibility of the modified Ficat classification system of osteonecrosis of the femoral head
7. Analysis of interobserver disagreement in the assessment of subarachnoid blood and acute hydrocephalus on CT scans
8. Inter- and intraobserver variability in the qualitative categorization of coronary angiograms
9. Inter and intraobserver variability of total skin thickness score (modified Rodnan TSS) in systemic sclerosis
10. Variability in radiologists' interpretations of mammograms
11. Reaching rural mammographers for quality improvement
12. Measurement of reliability for categorical data in medical research
13. Modelling patterns of agreement and disagreement
14. Beyond kappa: A review of interrater agreement measures
15. A coefficient of agreement for nominal scales
16. Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit
17. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability
18. Measuring nominal scale agreement among many raters
19. Measures of response agreement for qualitative data: Some generalizations and alternatives
20. An application of hierarchical kappa-type statistics in the assessment of majority of agreement among multiple observers
21. The measurement of observer agreement for categorical data
22. Measuring agreement for multinomial data
23. Extension of the kappa coefficient
24. General observer-agreement measures on individual subjects and groups of subjects
25. Measuring pairwise interobserver agreement when all subjects are judged by the same observers
26. Measuring interrater reliability among multiple raters: An example of methods for nominal data
27. Ramifications of a population model for kappa as a coefficient of reliability
28. Misinterpretation and misuse of the kappa statistic
29. Issues in the use of kappa
30. A reappraisal of the kappa coefficient
31. How reliable are chance-corrected measures of agreement?
32. High agreement but low kappa
33. Coefficient kappa: Some uses, misuses, and alternatives
34. A model for agreement between ratings on an ordinal scale
35. Modeling agreement among raters
36. Simple models for the analysis of association in cross-classifications having ordered categories
37. Modeling approaches for the analysis of observer agreement
38. A latent trait finite mixture model for the analysis of rating agreement
39. Assessing interrater agreement from dependent data