Assessing observer agreement in studies involving replicated binary observations

Journal of Biopharmaceutical Statistics

Volumn 17, Issue 4, 2007, Pages 757-766

  Haber, Michael ^a   Gao, Jingjing ^a   Barnhart, Huiman X ^b  

^a EMORY UNIVERSITY   (United States)

^b DUKE CLINICAL RESEARCH INSTITUTE   (United States)

Author keywords

Binary data; Coefficient of individual agreement; Observer agreement

Indexed keywords

ARTICLE; BREAST CANCER; CLINICAL ARTICLE; CONTROLLED STUDY; CORRELATION COEFFICIENT; ERROR; FEMALE; GOLD STANDARD; HUMAN; MAMMOGRAPHY; NONPARAMETRIC TEST; OBSERVATIONAL STUDY; PARTICIPANT OBSERVATION; PRIORITY JOURNAL; PROBABILITY; READING; RELIABILITY; RESEARCH SUBJECT; SIMULATION; STATISTICAL ANALYSIS; VALIDITY;

ALGORITHMS; BREAST NEOPLASMS; COMPUTER SIMULATION; DIAGNOSTIC TECHNIQUES AND PROCEDURES; FEMALE; HUMANS; MAMMOGRAPHY; MODELS, STATISTICAL; OBSERVER VARIATION; PROBABILITY; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; STATISTICS, NONPARAMETRIC;

EID: 34347374460     PISSN: 10543406     EISSN: 15205711     Source Type: Journal    
DOI: 10.1080/10543400701329547     Document Type: Article

References (10)

1. Baker, S. G., Freedman, L. S., Parmar, M. K. B. (1991). Using replicated observations in observer agreement studies with binary assessments. Biometrics 47:1327-1338.
2. Barnhart, H. X., Kosinski, A. S., Haber, M. (2007) Assessing individual agreement. Journal of Biopharmaceutical Statistics 17(4):697-719.
3. Baughman, A. L. (2000). Latent Structure Models for Evaluating Diagnostic Agreement Using Replicate Binary Measurements. Ph.D. dissertation, Emory University, Atlanta, GA.
4. Dawid, A. P., Skene, A. M. (1979). Maximum likelihood estimation of observer error-rates using the EM algorithm. Appl. Statist. 28:20-28.
5. Elmore, J. G., Wells, C. K., Lee, C. H., Howard, D. H., Feinstein, A. R. (1994). Variability in radiologists' interpretation of mammograms. New Eng. J. Med. 331:1493-1499.
6. Feinstein, A. R., Cicchetti, D. V. (1990). High agreement but low kappa. J. Clin. Epi. 43:543-549, 551-558.
7. Haber, M., Barnhart, H. X. (2007). A general approach to evaluating agreement between two observers or methods of measurement. Statistical Methods in Medical Research (accepted).
8. Kirchner, H. L., Lemke, J. H. (2002). Simultaneous estimation of intrarater and interrater agreement for multiple raters under order restrictions for a binary trait. Statistics in Medicine 21:1761-1772.
9. Kraemer, H. C. (1979). Ramifications of a population model for kappa as a coefficient of reliability. Psychometrika 44:461-472.
10. Lin, L., Hedayat, A. S., Sinha, B., Yang, M. (2002). Statistical methods in assessing agreement: models, issues and tools. J. Amer. Stat. Assoc. 97:257-270.