Measuring agreement for ordered ratings in 3 × 3 tables

Methods of Information in Medicine

Volumn 45, Issue 5, 2006, Pages 541-547

  Neveu, Dorine ^a,d   Aubas, P ^b   Seguret, F ^b   Kramar, A ^c   Dujols, P ^a,b  

^a UNIV MONTPELLIER   (France)

^b MONTPELLIER UNIVERSITY HOSPITAL   (France)

^c CENTRE VAL D AURELLE   (France)

^d HÔPITAL ARNAUD DE VILLENEUVE   (France)

Author keywords

Agreement; Concordance; Maximal and partial disagreement; Order relationship; Weighted kappa

Indexed keywords

ARTICLE; CLINICAL RESEARCH; CONTINGENCY TABLE; KAPPA STATISTICS; PRIORITY JOURNAL; STATISTICAL MODEL; STATISTICAL PARAMETERS;

EID: 33750219370     PISSN: 00261270     EISSN: None     Source Type: Journal    
DOI: 10.1055/s-0038-1634116     Document Type: Article

References (25)

1. Brenner H, Kliebsch U. Dependence of weighted kappa coefficients on the number of categories. Epidemiology 1996; 7 (2): 199-202.
2. Hasman A, de Bruijn LM, Arends JW. Evaluation of a method that supports pathology report coding. Methods Inf Med 2001; 40 (4): 293-7.
3. Bloom HJ, Richardson WW. Histological grading and prognosis in breast cancer; a study of 1409 cases of which 359 have been followed for 15 years. British Journal of Cancerology 1957; 11 (3): 359-77.
4. Elston CW, Ellis I0. Pathological prognostic factors in breast cancer. I. The value of histological grade in breast cancer: experience from a large study with long-term follow-up. Histopathology 1991; 19 (5): 403-10.
5. Cohen J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 1960; 20 (1): 37-46.
6. Cohen J. Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin 1968; 70 (4): 213-20.
7. Fleiss JL. Statistical methods for rates and proportions. Wiley: New York 1981; 236: pp 212-36.
8. Kraemer HC, Periyakoil VS, Noda A. Kappa coefficients in medical research. Statistics in Medicine 2002; 21: 2109-29.
9. Cicchetti DV, Allison T. A new procedure for assessing reliability of scoring EEG sleep recordings. American Journal of EEG Technology 1971; 11 (3): 101-9.
10. Fleiss JL, Cohen J. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement 1973; 33: 613-9.
11. Feinstein AR, Cicchetti DV High agreement but low kappa: 1. the problems of two paradoxes. Journal of Clinical Epidemiology 1990; 43 (6): 543-9.
12. Graham P, Jackson R. The analysis of ordinal agreement data: beyond weighted kappa. Journal of Clinical Epidemiology 1993; 46 (9): 1055-62.
13. Byrt T, Bishop J, Carlin JB. Bias, prevalence and kappa. Journal of clinical epidemiology 1993; 46 (5): 423-9.
14. King TS, Chinchilli VM. A generalized concordance correlation coefficient for continuous and categorical data. Statistics in Medicine 2001; 20: 2131-47.
15. Tanner MA, Young MA. Modeling agreement among raters. Journal of the American Statistical Association 1985; 80: 175-80.
16. Agresti A. A model for agreement between ratings on a categorical scale. Biometrics 1988; 44: 539-48.
17. Becker MP. Using association models to analyse agreement data: two examples. Statistics in Medicine 1989; 8: 1199-1207.
18. May SM. Modelling observer agreement - an alternative to kappa. Journal of Clinical Epidemiology 1994; 47 (11): 1315-24.
19. Feinstein AR, Cicchetti DV. High agreement but low kappa: II. Resolving the paradoxes. Journal of Clinical Epidemiology 1990; 43 (6): 551-8.
20. Bishop J, Carlin J, Nolan T. Evaluation of the properties and reliability of a clinical severity scale for acute asthma in children. Journal of Clinical Epidemiology 1992; 45 (1): 71-6.
21. Rao C. Linear statistical inference and its applications. New York: Wiley; 1965.
22. Harris GC, Denley HE, Pinder SE, Lee AH, Ellis IO, Elston CW, et al. Correlation of histologic prognostic factors in core biopsies and therapeutic excisions of invasive breast carcinoma. American Journal of Surgical Pathology 2003; 27 (1): 11-5.
23. Fleiss JL, Cohen J, Everitt BS. Large sample standard errors of kappa and weighted kappa. Psychological Bulletin 1969; 72 (5): 323-7.
24. Pinder SE, Murray S, Ellis IO, Trihia H, Elston CW, Gelber RD, et al. The importance of the histologic grade of invasive breast carcinoma and response to chemotherapy. Cancer 1998; 83 (8): 1529-39.
25. Bowker AH. Bowker's test for symmetry. Journal of the American Statistical Association 1948; 43: 572-574.