An exact bootstrap confidence interval for κ in small samples

Journal of the Royal Statistical Society Series D: The Statistician

Volumn 51, Issue 4, 2002, Pages 467-478

  Klar, Neu ^a   Lipsitz, Stuart R ^b   Parzen, Michael ^d   Leong, Traci ^c  

^a CANCER CARE ONTARIO   (Canada)

^b MEDICAL UNIVERSITY OF SOUTH CAROLINA   (United States)

^c EMORY UNIVERSITY   (United States)

^d UNIVERSITY OF CHICAGO   (United States)

Author keywords

Interrater agreement; Monte Carlo bootstrap

Indexed keywords

EID: 0039484150     PISSN: 00390526     EISSN: None     Source Type: Journal    
DOI: 10.1111/1467-9884.00331     Document Type: Article

References (17)

1. Bishop, Y, Fienberg, S. and Holland, P. (1975) Discrete Multivariate Analysis: Theory and Practice. Cambridge: Massachusetts Institute of Technology Press.
2. Blackman, N. and Koval, J. (2000) Interval estimation for Cohen's kappa as a measure of agreement. Statist. Med., 19, 723-741.
3. Bloch, D. and Kraemer, H. (1989) 2 × 2 kappa coefficients: measures of agreement or association. Biometrics, 45, 269-287.
4. Cohen, J. (1960) A coefficient of agreement for nominal scales. Educ. Psychol. Measmnt, 20, 37-46.
5. Efron, B. (1982) The Jackknife, the Bootstrap, and Other Resampling Plans. Philadelphia: Society for Industrial and Applied Mathematics.
6. Flack, V. (1987) Confidence intervals for the interrater agreement measure kappa. Communs Statist. Theory Meth 16, 953-968.
7. Fleiss, J. (1981) Statistical Methods for Rates and Proportions, 2nd edn. New York: Wiley.
8. Fleiss, J., Cohen, J. and Everitt, B. (1969) Large-sample standard errors of kappa and weighted kappa. Psychol. Bull, 72, 323-327.
9. Fleiss, J. and Davies, M. (1982) Jackknifing functions of multinomial frequencies, with an application to a measure of concordance. Am. J. Epidem., 115, 841-845.
10. Hale, C. and Fleiss, J. (1993) Interval estimation under two study designs for kappa with binary classifications. Biometrics, 49, 523-533.
11. Hansen, R., Ryan, L., Anderson, T., Krzywda, B., Quebbeman, E., Benson, A., Haller, D. and Tormey, D. (1996) Phase III study of bolus versus infusion fluorouracil with or without cisplatin in advanced colorectal cancer. J. Natn. Cancer Inst., 88, 668-674.
12. Kinsella, A. (1987) The 'exact' bootstrap approach to confidence intervals for the relative difference statistic. Statistician, 36, 345-347; correction, 37 (1988), 97.
13. Kinsella, A. (1987) The 'exact' bootstrap approach to confidence intervals for the relative difference statistic. Statistician, 36, 345-347; correction, 37 (1988), 97.
14. Landis, J. and Koch, G. (1977) The measurement of observer agreement for categorical data. Biometrics, 33, 159-174.
15. Lee, J. and Tu, Z. (1994) A better confidence interval for kappa (κ) on measuring agreement between two raters with binary outcomes. J. Comput. Graph. Statist., 3, 301-321.
16. McKenzie, D., Mackinnon, A. J., Peladeau, N., Onghena, P., Bruce, P. C., Clarke, D., Harrigan, S. and McGorry, P. D. (1996) Comparing correlated kappas by resampling: is one level of agreement significantly different from another? J. Psychiatr. Res., 30, 483-492.
17. Ridout, M., Demtrio, C. and Firth, D. (1999) Estimating intraclass correlation for binary data. Biometrics, 55, 137-148.