Sample size requirements for the comparison of two or more coefficients of inter-observer agreement

Statistics in Medicine

Volumn 17, Issue 10, 1998, Pages 1157-1168

  Donner, Allan ^a  

^a UNIVERSITY OF WESTERN ONTARIO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; COMPARATIVE STUDY; MATHEMATICAL ANALYSIS; SAMPLE; STATISTICAL ANALYSIS;

HUMANS; OBSERVER VARIATION; OUTCOME ASSESSMENT (HEALTH CARE); SAMPLING STUDIES; SENSITIVITY AND SPECIFICITY;

EID: 0031897109     PISSN: 02776715     EISSN: None     Source Type: Journal    
DOI: 10.1002/(SICI)1097-0258(19980530)17:10<1157::AID-SIM792>3.0.CO;2-W     Document Type: Article

References (44)

1. Bloch, D. A. and Kraemer, H. C. '2 × 2 kappa coefficients: measures of agreement or association', Biometrics, 45, 269-287 (1989).
2. Donner, A. and Eliasziw, M. 'A goodness-of-fit approach to inference procedures for the kappa statistic: confidence interval construction, significance-testing and sample size estimation', Statistics in Medicine, 11, 1511-1519 (1992).
3. Hale, C. A. and Fleiss, J. L. 'Interval estimation under two study designs for kappa with binary classifications', Biometrics, 49, 523-534 (1993).
4. Dunn, G. Design and Analyses of Reliability Studies, Oxford University Press, New York, 1989.
5. Walter, S. D., Eliasziw, M. and Donner, A. 'Sample size and optimal designs for reliability studies', Statistics in Medicine, (in press).
6. Kraemer, H. C. and Korner, A. F. 'Statistical alternatives in assessing reliability, consistency, and individual differences for quantitative measures: application to behavioral measures of neonates', Psychological Bulletin, 83, (5), 914-921 (1976).
7. Kraemer, H. C. 'The small sample non-null properties of Kendall's coefficient of concordance for normal populations', Journal of the American Statistical Association, 71, 608-613 (1976).
8. Kraemer, H. C. and Thiemann, S. How Many Subjects? Statistical Power Analysis in Research, Sage Publications, Inc., Beverley Hills, California, 1987.
9. Donner, A. and Eliasziw, M. 'Sample size requirements for reliability studies', Statistics in Medicine, 6, 441-448 (1987).
10. Barlow, W., Lai, M-Y. and Azen, S. P. 'A comparison of methods for calculating a stratified kappa', Statistics in Medicine, 10, 1465-1472 (1991).
11. McLellan, A. T., Luborsky, L., Cacciola, J., Griffith, J., Evans, F., Barr, H. L. and O'Brien, C. P. 'New data from the addiction severity index', Journal of Nervous and Mental Disease, 173, 412-423 (1985).
12. McLaughlin, J. K., Dietz, M. S., Mehl, E. S. and Blot, W. J. 'Reliability of surrogate information on cigarette smoking by type of informant', American Journal of Epidemiology, 126, 1, 144-146 (1987).
13. Alsawalmeh, Y. M. and Feldt, L. S. 'Test of the hypothesis that the intraclass reliability coefficient is the same for two measurement procedures', Applied Psychological Measurement, 16, 192-205 (1992).
14. Donner, A. and Koval, J. J. 'The estimation of intraclass correlation in the analysis of family data,' Biometrics, 36, 19-25 (1980).
15. Fisher, R. A. Statistical Methods for Research Workers, 13th edn, Hafner Publishing Company, Inc., Riverside, New Jersey, 1958.
16. Schwartz, D., Flamant, R. and Lellouch, J. Clinical Trials, Academic Press, London, 1980.
17. Zerbe, G. O. and Goldgar, D. E. 'Comparison of intraclass correlation coefficients with the ratio of two independent F-statistics', Communications in Statistics - Theory and Methods A9(15), 1641-1655 (1980).
18. Kraemer, H. C. 'Extension of Feldt's approach to testing homogeneity of coefficients of reliability,' Psychometrika, 45, 41-45 (1981).
19. Schumann, D. E. W. and Bradley, R. A. 'The comparison of the sensitivities of similar experiments: Model II of the analysis of variance', Biometrika, 15, 405-416 (1959).
20. Donner, A. and Bull, S. 'Inferences concerning a common intraclass correlation coefficient', Biometrics, 39, 771-775 (1983).
21. Kraemer, H. C. 'Ramifications of a population model for κ as a coefficient of reliability', Psychometrika, 44, 461-412 (1979).
22. Mak, T. K. 'Analyzing intraclass correlation for dichotomous variables', Applied Statistics, 37, 344-352 (1988).
23. Fleiss, J. L. Statistical Methods for Rates and Proportions, 2nd edn, Wiley New York, 1981.
24. Donner, A., Eliasziw, M. and Klar, N. 'Testing the homogeneity of kappa statistics', Biometrics, 52, 176-183 (1996).
25. Thompson, W. D. and Walter, S. D. 'A reappraisal of the kappa statistic', Journal of Clinical Epidemiology, 41, 949-958 (1988).
26. Kendall, M. G. and Stuart, A. The Advanced Theory of Statistics, Vol. 2, 5th edn, Arnold, London, 1991.
27. Cohen, J. 'The cost of dichotomization', Applied Psychological Measurement, 7, 249-253 (1983).
28. Kraemer, H. C. 'A measure of 2 × 2 association with stable variance and approximately normal small-sample distribution: planning cost-effective studies', Biometrics, 42, 359-370 (1986).
29. Donner, A. and Eliasziw, M. 'Statistical implications of the choice between a dichotomous or continuous trait in studies of interobserver agreement', Biometrics, 50, 550-555 (1994).
30. Haynam, G. E., Govindarajulu, Z. and Leone, G. C. 'Tables of the cumulative non-central chi-square distribution', Case Statistical Laboratory, Publication No. 104, 1962, Part of the tables have been published in Selected Tables in Mathematical Statistics Vol. 1, edited by H. L. Harter and D. B. Owen, Markham, Chicago.
31. Browman, G. P., Markman, S., Thompson, G., Minuk, T., Chirawatkul, A. and Roberts, R. S. 'Assessment of observer variation in measuring the radiographic vertebral index in patients with multiple myeloma', Journal of Clinical Epidemiology, 43, 833-840 (1990).
32. Alsawalmeh, Y. M. and Feldt, L. S. 'Testing the equality of two related intraclass reliability coefficients', Applied Psychological Measurement, 18, 183-190 (1994).
33. Feldt, D. S. 'A test of the hypothesis that Cronbach's alpha reliability coefficient is the same for two tests administered to the same sample', Psychometrika, 45, 99-105 (1980).
34. Donner, A., Koval, J. J. and Bull, S. 'Testing the effect of sex differences on sib-sib correlations', Biometrics, 40, 349-356 (1984).
35. Elston, R. C. 'The correlation between correlations', Biometrika, 62, 133-148 (1975).
36. Rao, D. C. 'Joint distribution of Z-transformations estimated from the same sample', Human Heredity, 29, 334-336 (1979).
37. Berg, K. 'Twin research in coronary heart disease, Twin Research 3: Epidemiological and Clinical Studies', in the Proceedings of the Second International Congress on Twin Studies, Alan R. Lins Inc., New York, 1981, pp. 117-130.
38. Berson, E. L., Rosner, B. and Simonoff, E. 'An outpatient population of retinitis pigmentosa and their normal relatives: risk factors for genetic typing and detection derived from their ocular examinations', American Journal of Ophthalmology, 89, 763-775 (1980).
39. Larsen, F. and Vaglum, S. 'Interrater reliability of the DSM-III diagnoses in two Norwegian studies on psychiatric and super obese patients', Acta Psychiatrica Scandinavica, 17, 18-21 (1986).
40. World Health Organization. ICD-10. Draft of Chapter V, categories F00-F99. Mental, behaviourial and development disorders, clinical descriptions and diagnostic guidelines, Geneva, 1987.
41. MacLure, M. and Willett, W. C. 'Misinterpretation and misuse of the kappa statistic', American Journal of Epidemiology, 126, (2), 161-169 (1987).
42. Landis, J. R. and Koch, G. G. 'A one-way components of variance model for categorical data', Biometrics, 33, 671-679 (1977).
43. Ishmail, A. A., Wing, S., Ferguson, J., Hutchinson, T. A., Magder, S. and Flegel, K. M. 'Interobserver agreement by auscultation in the presence of a third heart sound in patients with congestive heart failure', Chest, 91, 870-873 (1987).
44. Hannah, M. C., Hopper, J. L. and Mathews, J. D. 'Twin concordance for a binary trait. II. Nested analysis of ever-smoking and ex-smoking traits and unnested analysis of a 'Committed-Smoking' trait', American Journal of Human Genetics, 37, 153-165 (1985).