A confidence interval approach to sample size estimation for interobserver agreement studies with multiple raters and outcomes

Journal of Clinical Epidemiology

Volumn 65, Issue 7, 2012, Pages 778-784

  Rotondi, Michael A ^a   Donner, Allan ^b  

^a YORK UNIVERSITY   (Canada)

^b UNIVERSITY OF WESTERN ONTARIO   (Canada)

Author keywords

Agreement; Experimental design; Kappa; Reliability; Sample size estimation

Indexed keywords

ARTICLE; CLINICAL PRACTICE; CONFIDENCE INTERVAL; DENTIST; DENTISTRY; EMERGENCY HEALTH SERVICE; HUMAN; KAPPA STATISTICS; NURSE; OUTCOME ASSESSMENT; PATHOLOGY; PHYSIOTHERAPY; PRIORITY JOURNAL; PSYCHIATRY; RADIOLOGY; RELIABILITY; SAMPLE SIZE;

CANADA; CONFIDENCE INTERVALS; HUMANS; MANDIBLE; MATHEMATICAL COMPUTING; OBSERVER VARIATION; PILOT PROJECTS; RADIOGRAPHY, DENTAL; RADIOGRAPHY, PANORAMIC; REPRODUCIBILITY OF RESULTS; SAMPLE SIZE;

EID: 84861606926     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2011.10.019     Document Type: Article

References (28)

1. J. Jeronimo, L.S. Stewart, P. Castle, S. Wacholder, and M. Schiffman Interobserver agreement in the evaluation of digitized cervical images Obstet Gynecol 110 2007 833 840
2. K. Berger, C.S. Kase, and J.E. Buring Interobserver agreement in the classification of stroke in the physician's health study Stroke 27 1996 238 242
3. J. Sim, and C.C. Wright The kappa statistic in reliability studies: use, interpretation, and sample size requirements Phys Ther 85 2005 257 268
4. J.M. Jones Validity of nutritional screening and assessment tools Nutrition 20 2004 312 317
5. A.J. Viera, and J.M. Garrett Understanding interobserver agreement: the kappa statistic Fam Med 37 2005 360 363
6. J.R. Landis, and G.G. Koch The measurement of observer agreement for categorical data Biometrics 33 1977 159 179
7. A. Donner, and M. Eliasziw A goodness-of-fit approach to inference procedures for the kappa statistic: confidence interval construction, significance-testing and sample size estimation Stat Med 11 1992 1511 1519
8. M. Altaye, A. Donner, and M. Eliasziw A general goodness-of-fit approach for inference procedures concerning the kappa statistic Stat Med 20 2001 2479 2488
9. M. Altaye, A. Donner, and N. Klar Inference procedures for assessing interobserver agreement among multiple raters Biometrics 57 2001 584 588
10. M. Gardner, and D. Altman Estimating with confidence D.G. Altman, D. Machin, T.N. Bryant, M.J. Gardner, Statistics with Confidence 2007 British Medical Journal Books 3 6
11. A. Donner Sample size requirements for interval estimation of the intraclass kappa statistic Commun Stat Simul Comput 28 1999 415 429
12. A. Donner, and M.A. Rotondi Sample size requirements for interval estimation of the intraclass kappa statistic for interobserver agreement studies with a binary outcome and multiple raters Int J Biostat 6 2010 31
13. R. Zwick Another look at interrater agreement Psych Bull 103 1988 374 378
14. Q. McNemar Note on the sampling error of the difference between correlated proportions or percentages Psychometrika 12 1947 153 157
15. A. Stuart A test for homogeneity of the marginal distributions in a two-way classification Biometrika 42 1955 412 416
16. A.E. Maxwell Comparing the classification of subjects by two independent judges Br J Psychiatry 116 1970 651 655
17. A. Agresti Categorical data analysis 2002 John Wiley & Sons Inc Hoboken, NJ
18. D. Manos, D.A. Petrie, R.C. Beveridge, S. Walter, and J. Ducharme Inter-observer agreement using the Canadian Emergency Department Triage and Acuity Scale CJEM 4 2002 16 22
19. A. Taguchi, Y. Suei, K. Horner, H. Devlin, T. Nakamoto, and A. Yamashina Is there any difference between the British and Japanese definitions of the mandibular cortical index (MCI) on panoramic radiographs? A pilot study Oral Radiol 20 2004 44 48
20. E. Klemetti, S. Kolmakov, and H. Kroger Pantomography in assessment of the osteoporosis risk group Scand J Dent Res 102 1994 68 72
21. D.K. Zlataric, A. Celebic, and P. Kobler Relationship between body mass index and local quality of mandibular bone structure in elderly individuals J Gerontol A Biol Sci Med Sci 57 2002 588 593
22. B. Cakur, A. Sahin, S. Dagistan, O. Altun, F. Caglayan, and O. Miloglu Dental panoramic radiography in the diagnosis of osteoporosis J Int Med Res 36 2008 792 799
23. M.J. Bullard, B. Unger, J. Spence, and E. Grafstein Revisions to the Canadian emergency department triage and acuity scale (CTAS) adult guidelines CJEM 10 2008 136 142
24. E. Grafstein, G. Innes, J. Westman, J. Christenson, and A. Thorne Inter-rater reliability of a computerized presenting-complaint linked triage system in an urban emergency department CJEM 5 2003 323 329
25. J. Cohen The cost of dichotomization Appl Psychol Meas 7 1983 249 253
26. P.E. Shrout Measurement reliability and agreement in psychiatry Stat Methods Med Res 7 1998 301 317
27. SAS software version 9.2 of the SAS System for Unix 2008 SAS Institute Cary, NC
28. R Development Core Team. R: a language and environment for statistical computing. Vienna, Austria: 2011.