Using pooled kappa to summarize interrater agreement across many items

Field Methods

Volumn 20, Issue 3, 2008, Pages 272-282

  De Vries, Han ^a   Elliott, Marc N ^a   Kanouse, David E ^a   Teleki, Stephanie S ^a  

^a RAND CORPORATION   (United States)

Author keywords

Interviews; Qualitative analysis; Reliability; Simulations

Indexed keywords

EID: 46649097009     PISSN: 1525822X     EISSN: 15523969     Source Type: Journal    
DOI: 10.1177/1525822X08317166     Document Type: Article

References (15)

1. Adamson, L.B., R. Bakeman, and D.F. Deckner. 2004. The development of symbol-infused joint engagement. Child Development 75 (4): 1171-87.
2. Bakeman, R., and J.M. Gottman. 1986. Observing interaction. An introduction to sequential analysis. London: Cambridge University Press.
3. Basilevsky, A. 1980. The ratio estimator and maximum-likelihood weighted least squares regression. Quality and Quantity 14 (3): 377-95.
4. Bernard, H.R. 2001. Research methods in anthropology: Qualitative and quantitative approaches. Walnut Creek, CA: AltaMira.
5. Brennan, R.L., and D.J. Prediger. 1981. Coefficient kappa: Some uses, misuses, and alternatives. Educational and Psychological Measurement 41 (3): 687-99.
6. Cohen, J. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20 (1): 37-46.
7. Kravitz, R.L., R.A. Bell, C.E. Franz, M.N. Elliott, C. Willis, and L. Silverio. 2002. Characterizing patient requests and physician responses in office practice. Health Services Research 37 (1): 218-38.
8. Landis, J.R., and G.G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics 33 (1): 159-74.
9. Matsumoto, M., and T. Nishimura. 1998. Mersenne Twister: A 623-dimensionally equidistributed uniform pseudo-random number generator. ACM Transactions on Modeling and Computer Simulation 8 (1): 3-30.
10. Oden, N.L. 1991. Estimating kappa from binocular data. Statistics in Medicine 10 (8): 1303-11.
11. R Development Core Team. 2006. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. http://www.R-project.org (accessed February 29, 2008).
12. Schouten, H.J.A. 1993. Estimating kappa from binocular data and comparing marginal probabilities. Statistics in Medicine 12 (23): 2207-17.
13. Simon, P. 2006. Including omission mistakes in Cohen's kappa and an analysis of the coefficient's paradox features. Educational and Psychological Measurement 66 (5): 765-77.
14. Teleki, S.S., D.E. Kanouse, M.N. Elliott, L. Hiatt, H. de Vries, and D.D. Quigley. 2007. Understanding the reporting practices of CAHPS sponsors. Health Care Financing Review 28 (3): 1-14.
15. von Eye, A. 2006. Alternatives to Cohen's k. European Psychologist 11 (1): 12-24.