Reproducibility of clinical data II: Categorical outcomes

Pharmaceutical Statistics

Volumn 3, Issue 2, 2004, Pages 109-122

  Blackman, Nicole Jill Marie ^a  

^a GLAXOSMITHKLINE   (United States)

Author keywords

Agreement; Intraclass correlation; Intraclass kappa; Jackknife; Reproducibility

Indexed keywords

ANALYSIS OF VARIANCE; ANALYTICAL ERROR; ARTICLE; CLINICAL RESEARCH; CLINICAL STUDY; CORRELATION ANALYSIS; DRINKING BEHAVIOR; HUMAN; ISCHEMIC HEART DISEASE; REPRODUCIBILITY; RISK ASSESSMENT; RISK FACTOR; SCORING SYSTEM; STATISTICAL ANALYSIS; STATISTICAL SIGNIFICANCE;

EID: 10644266128     PISSN: 15391604     EISSN: None     Source Type: Journal    
DOI: 10.1002/pst.105     Document Type: Article

References (50)

1. Blackman NJM. Reproducibility of clinical data I: Continuous outcomes. Pharmaceutical Statistics 2004; 3: XXX-XXX.
2. Cronbach LJ, Gleser GC, Nanda H, Rajaratnam N. The dependability of behavioral measurements: Theory of generalisability for scores and profiles. Wiley: New York, 1972.
3. Lord FM, Novick MR. Statistical theories in mental test scores. Addison-Wesley: Reading, MA, 1968.
4. Rothery P. A nonparametric measure of intraclass correlation. Biometrika 1979; 66:629-639.
5. Shirahata S. Intraclass rank tests for independence. Biometrika 1981; 68:451-456.
6. Shirahata S. Nonparametric measures of intraclass correlation. Communications in Statistics 1982; A11 1707-1721.
7. Jackson R, Scragg R, Beaglehole R. Alcohol consumption and risk of coronary heart disease. British Medical Journal 1991; 303:211-216.
8. Fleiss JL. Statistical methods for rates and proportions Wiley: New York, 1981.
9. Fleiss JL. Measuring agreement between two judges on the presence or absence of a trait. Biometrics 1975; 31:651-659.
10. Koran LM. The reliability of clinical methods, data and judgements (first of two parts). New England Journal of Medicine 1975; 293:642-646.
11. Koran, LM. The reliability of clinical methods, data and judgments (second of two parts). New England Journal of Medicine 1975; 293:695-701.
12. Cohen J. A coefficient of agreement for nominal scales. Educational and Psychological Measurement 1960; 20:37-46.
13. Cohen J. Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin 1968; 70:213-220.
14. Light RJ. Measures of response agreement for qualitative data: Some generalisations and alternatives. Psychological Bulletin 1971; 71:365-377.
15. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977; 33:159-174.
16. Conger AJ. Integration and generalisation of kappas for multiple raters. Psychological Bulletin 1980; 88:322-328.
17. Davies M, Fleiss JL. Measuring agreement for multinomial data. Biometrics 1982; 38:1047-1051.
18. Schouten HJA. Measuring pairwise interobserver agreement when all subjects are judged by the same set of observers. Statistica Neerlandica 1982; 36:45-61.
19. Fleiss JL. Measuring nominal scale agreement among many raters. Psychological Bulletin 1971; 76:378-382.
20. Landis JR, Koch GG. A one-way components variance model for categorical data. Biometrics 1977; 33:671-679.
21. Fleiss JL, Cuzick J. The reliability of dichotomous judgements: Unequal number of judges per subject. Applied Psychological Measurement 1979; 3:537-542.
22. Kraemer HC. Extension of the kappa coefficient. Biometrics 1980; 36:207-216.
23. Kupper JL, Hafner KB. On assessing interrater agreement for multiple attribute responses. Biometrics 1989; 54:957-967.
24. Banerjee M, Capozzoli LM, Sinha D. Beyond kappa: a review of interrater agreement measures. Canadian Journal of Statistics 1999; 27:3-23.
25. Kraemer HC, Periyakoil VS, Noda A. Kappa coefficients in medical research. Statistics in Medicine 2002; 21:2109-2129.
26. Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The problems of the two paradoxes. Journal of Clinical Epidemiology 1990; 43:543-549.
27. Guggenmoss-Holzmann, I. The meaning of kappa: probabilistic concepts of reliability and validity revisited. Journal of Clinical Epidemiology 1996; 49:775-782.
28. Maclure M, Willett WC. Misinterpretation and misuse of the kappa statistic. American Journal of Epidemiology 1987; 126 161-169.
29. Fleiss JL, Cohen J. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement 1973; 33:613-619.
30. Graham P, Jackson R. The analysis of ordinal agreement data: beyond weighted kappa. Journal of Clinical Epidemiology 1993; 9 1055-1062.
31. Bloch DA, Kraemer HC. 2 × 2 kappa coefficients: Measures of agreement or association. Biometrics 1989; 45:269-287.
32. Kraemer HC. Ramifications of a population model for κ as a coefficient of reliability. Psychometrika 1979; 44:461-473.
33. Kraemer HC. The robustness of common measures of 2 × 2 association to bias due to misclassification. American Statistician 1985; 39 b:286-290.
34. Blackman NJM, Koval JJ. Estimating rater agreement in 2 × 2 tables: Correction for chance and intraclass correlation. Applied Psychological Measurement 1993; 17:211-223.
35. Scott WA. Reliability of content analysis: The case of nominal scale coding. Public Opinion Quarterly 1955; 76 365-377.
36. Mak TK. Analysing intraclass correlation for dichotomous variables. Applied Statistics 1988; 37:344-352.
37. Koval JJ, and Blackman NJM. Estimators of kappa-exact small sample properties. Journal of Statistical Computation and Simulation 1996; 15:315-336.
38. Fleiss JL, Davies M. Jack-knifing functions of multinomial frequencies with an application to a measure of concordance. American Journal of Epidemiology 1982; 115:841-845.
39. Quenouille MH. Notes on bias in estimation. Biometrika 1956; 43:353-360.
40. Tukey JW. Bias and confidence intervals in not-quite large samples. Annals of Mathematical Statistics 1958; 29:614-616.
41. Miller RG. A trustworthy jackknife. Annals of Mathematical Statistics 1964; 35:1594-1605.
42. Blackman NJM, Koval JJ. Interval estimation for Cohen's kappa as a measurement of agreement. Statistics in Medicine 2000; 19 723-741.
43. Hale CA, Fleiss JL. Interval estimation under two study designs for kappa with binary classifications. Biometrics 1993; 49:523-534.
44. Longford NT. Essay rating and score adjustment. Journal of Educational and Behavioural Statistics 1994; 19:171-200.
45. Kendall MG, Stuart A. The Advanced Theory of Statistics 3rd edition. Griffin: London, 1969.
46. Walter SD, Eliasziw M, Donner A. Sample size and optimal designs for reliability studies. Statistics in Medicine 1998; 17 101-110.
47. Donner A. Sample size requirements for the comparison of two or more coefficients of interobserver agreement. Statistics in Medicine 1997; 15:1157-1168.
48. Kraemer HC. Measurement of reliability for categorical data in medical research. Statistical Methods in Medical Research 1992; 1:183-199.
49. Shrout PE. Measurement reliability and agreement in medical research. Statistical Methods in Medical Research 1998; 7 301-317.
50. Kraemer HC. Evaluating medical tests. Objective and quantitative guidelines. Sage: Newbury Park, CA, 1992.