Selected issues in the design of studies of interrater agreement

Applied Optimal Designs

Volumn , Issue , 2005, Pages 181-195

  Donner, Allan ^a,c   Altaye, Mekibib ^b  

^a WESTERN UNIVERSITY   (Canada)

^b UNIVERSITY OF CINCINNATI COLLEGE OF MEDICINE   (United States)

^c ROBARTS RESEARCH INSTITUTE   (Canada)

Author keywords

Binary outcome variable; Budgetary constraints; Design features; Dichotomous outcome variable; Interrater agreement; Polychotomous outcome data; Power considerations; Relative efficiency (RE)

Indexed keywords

EID: 33847026116     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/0470857005.ch7     Document Type: Chapter

References (22)

1. Altaye, M., Donner, A. and Eliasziw, M. (2001a). A general goodness-of-fit approach for inference procedures concerning the kappa statistic. Statistics in Medicine, 20: 2479-2488.
2. Altaye, M., Donner, A. and Klar, N. (2001b). Inference procedures for assessing interobserver agreement among multiple raters. Biometrics, 57: 584-588.
3. Bahadur, R. R. (1961). A representation of the joint distribution of responses to n dichotomous items. In: Studies in Item Analysis and Prediction. Solomon, H. (ed.). Stanford Mathematical Studies in the Social Sciences VI. Stanford, Calif.: Stanford University Press.
4. Banerjee, M., Capozzoli, M., McSweeney, L. and Sinha, D. (1999). Beyond kappa: a review of interrater agreement measures. The Canadian Journal of Statistics, 27: 3-23.
5. Barlow, W. (1996). Measurement of interrater agreement with adjustments for covariates. Biometrics, 52: 695-702.
6. Bartfay, E. and Donner, A. (2001). Statistical inferences for interobserver agreement studies with nominal outcome data. The Statistician, 50: 135-146.
7. Cantor, A. B. (1996). Sample-size calculations for Cohen’s kappa. Psychological Methods, 1: 150-153.
8. Cohen, J. (1983). Thecost of dichotomization. AppliedPsychologicalMeasurement, 7: 249-253.
9. Donner, A. and Eliasziw, M. (1992). A goodness-of-fit approach to inference procedures for the kappa statistic: confidence interval construction, significance-testing and sample size estimation. Statistics in Medicine, 11: 1511-1519.
10. Donner, A. and Eliasziw, M. (1994). Statistical implications of the choice between a dichotomous or continuous trait in studies of interobserver agreement. Biometrics, 50: 550-555.
11. Eliasziw, M. and Donner, A. (1987). A cost-function approach to the design of reliability studies. Statistics in Medicine, 6: 647-655.
12. Flack, V. F., Afifi, A. A. and Lachenbruch, P. A. (1988). Sample size determinations for the two rater kappa statistic. Psychometrika, 53: 321-325.
13. Fleiss, J. L. Statistical Methods for Rates and Proportions, 2nd edn 1981. New York: John Wiley & Sons, Ltd.
14. Kraemer, H. C. (1979). Ramifications of a population model for κ as a coefficient of reliability. Psychometrika, 44: 461-472.
15. Kraemer, H. C. (1986). A measure of 2 × 2 association with stable variance and approximately normal small-sample distribution: planning cost-effective studies. Biometrics, 42: 359-370.
16. Kraemer, H. C. (1992). Measurement of reliability for categorical data in medical research. Statistical Methods in Medical Research, 1: 183-199.
17. Lipsitz, S. R., Laird, N. M. and Brennan, T. A. (1994). Simple moment estimates of the κ coefficient and its variance. Applied Statistics, 43: 309-323.
18. Prentice, R. L. (1988). Correlated binary regression with covariates specific to each binary observation. Biometrics, 44: 1033-1048.
19. Scheffé, H. The Analysis of Variance, 1959. New York: John Wiley & Sons, Ltd.
20. Shrout, P. E. (1998). Measurement reliability and agreement in psychiatry. Statistical Methods in Medical Research, 7: 301-317.
21. Walter, S. D., Eliasziw, M. and Donner, A. (1998). Sample size and optimal designs for reliability studies. Statistics in Medicine, 17: 101-110.
22. Westlund, K. B. and Kurland, L. T. (1953). Studies on multiple sclerosis in Winnipeg, Manitoba, and New Orleans, Louisiana. I. Prevalence, comparison between the patient groups in Winnipeg and New Orleans. American Journal of Hygiene, 57: 380-396.