Statistical inferences for interobserver agreement studies with nominal outcome data

Journal of the Royal Statistical Society Series D: The Statistician

Volumn 50, Issue 2, 2001, Pages 135-146

  Bartfay, Emma ^a,c   Donner, Allan ^b  

^a QUEEN S UNIVERSITY   (Canada)

^b UNIVERSITY OF WESTERN ONTARIO   (Canada)

^c KINGSTON GENERAL HOSPITAL   (Canada)

Author keywords

Homogeneity; Hypothesis testing; Reliability studies;

Indexed keywords

EID: 0008555768     PISSN: 00390526     EISSN: None     Source Type: Journal    
DOI: 10.1111/1467-9884.00266     Document Type: Article

References (25)

1. Agresti, A. (1989) An agreement model with kappa as parameter. Statist Probab. Lett., 7, 271-273.
2. _(1992) Modelling patterns of agreement and disagreement. Statist. Meth. Med. Res., 1, 201-218.
3. Banerjee, M., Capozzoli, L., McSweeney, L. and Sinha, D. (1999) Beyond kappa: a review of interrater agreement measures. Can. J. Statist., 27, 3-23.
4. Bartfay, E. and Donner, A. (2000) The effect of collapsing multinomial data when assessing agreement. Int. J. Epidem., 29, 1070-1075.
5. _(2001) Statistical inferences for a twin correlation with multinomial outcomes. Statist. Med., 20, 249-262.
6. Basu, S., Banerjee, M. and Sen, A. (2000) Bayesian inference for kappa from single and multiple samples. Biometrics, 56, 583-585.
7. Basu, S. and Basu, A. (1995) Comparison of several goodness-of-fit tests for the kappa statistic based on exact power and coverage probability. Statist. Med., 14, 347-356.
8. Basu, S., Basu, A. and Raychaudhuri, A. (1999) Measuring agreement between two raters for ordinal response: a model-based approach. Statistician, 48, 339-348.
9. Becker, M. P. (1989) Using association models to analyse agreement data: two examples. Statist. Med., 8, 1199-1207.
10. Bianchi, S., Palli, D., Ciatto, S., Galli, M., Girogi, D., Vezzosi, V., Del Turco, M. R., Cataliotti, L., Cardona, G. and Zampi, G. (1995) Accuracy and reliability of frozen section diagnosis in a series of 672 nonpalpable breast lesions. Am. J. Clin. Pathol., 103, 199-205.
11. Bloch, D. A. and Kraemer, H. C. (1989) 2 x 2 kappa coefficients: measures of agreement or association. Biometrics, 45, 269-287.
12. Cohen, J. (1960) A coefficient of agreement for nominal scales. Educ. Psychol. Measmnt, 20, 37-46.
13. Donner, A. and Eliasziw, M. (1992) A goodness-of-fit approach to inference procedures for the kappa statistic: confidence interval construction, significance-testing and sample size estimation. Statist. Med., 11, 1511-1519.
14. _(1997) A hierarchical approach to inferences concerning interobserver agreement for multinomial data. Statist. Med., 16, 1097-1106.
15. Donner, A., Eliasziw, M. and Klar, N. (1996) Testing the homogeneity of kappa statistics. Biometrics, 52, 176-183.
16. Donner, A., Shoukri. M. M., Klar, N. and Bartfay, E. (2000) Testing the equality of two dependent kappa statistics. Statist. Med., 19, 373-387.
17. Fleiss, J. L. (1975) Measuring agreement between two judges on the presence or absence of a binary trait. Biometrics, 31, 651-659.
18. Guggenmoos-Holzmann, I. and Vonk, R. (1998) Kappa-like indices of observer agreement viewed from a latent class perspective. Statist. Med., 17, 797-812.
19. Kraemer, H. C. (1980) Extension of the kappa coefficient. Biometrics, 86, 207-216.
20. _(1992) Measurement of reliability for categorical data in medical research. Statist. Meth. Med. Res., 1, 183-199.
21. Landis, J. R. and Koch, G. G. (1977) The measurements of observer agreement for categorical data. Biometrics, 33, 159-174.
22. Lau, T. S. (1993) Higher-order kappa-type statistics for a dichotomous attribute in multiple ratings. Biometrics, 49, 535-542.
23. Nam, J. M. (2000) Interval estimation of the kappa statistic with binary classification and an equal marginal probability model. Biometrics, 56, 577-582.
24. Roberts, C. and McNamee, R. (1998) A matrix of kappa-type coefficients to assess the reliability of nominal scales. Statist. Med., 17, 471-488.
25. Robey, R. R. and Barcikowski, R. S. (1992) Type I error and the number of iterations in Monte Carlo studies of robustness. Br. J. Math. Statist. Psychol., 45, 283-288.