Information-based measure of disagreement for more than two observers: A useful tool to compare the degree of observer disagreement

BMC Medical Research Methodology

Volumn 13, Issue 1, 2013, Pages

  Henriques, Teresa ^a,b,c   Antunes, Luis ^b,c   Bernardes, João ^a,c,d   Matias, Mara ^b   Sato, Diogo ^a   Costa Santos, Cristina ^a,c  

^a University of Porto   (Portugal)

^b INSTITUTO DE TELECOMUNICAÇÕES   (Portugal)

^c UNIVERSITY OF PORTO   (Portugal)

^d INEB INSTITUTO DE ENGENHARIA BIOMÉDICA   (Portugal)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; BODY SIZE; CARDIOTOCOGRAPHY; FEMALE; FETUS HEART RATE; HEART RATE; HUMAN; INTERNET; LABOR STAGE 1; LABOR STAGE 3; OBSERVER VARIATION; PREGNANCY; STATISTICAL MODEL;

BODY SIZE; CARDIOTOCOGRAPHY; FEMALE; HEART RATE; HEART RATE, FETAL; HUMANS; INTERNET; LABOR STAGE, FIRST; LABOR STAGE, THIRD; MODELS, STATISTICAL; OBSERVER VARIATION; PREGNANCY;

EID: 84875159444     PISSN: None     EISSN: 14712288     Source Type: Journal    
DOI: 10.1186/1471-2288-13-47     Document Type: Article

References (33)

1. Bland M, An introduction to medical statistics. 3rd edition Oxford: Oxford University Press 2000
2. Agreement Between Methods of Measurement with Multiple Observations Per Individual. Bland JM, Altman DG, J Biopharm Stat 2007 17 4 571 582 10.1080/10543400701329422 17613642 (Pubitemid 47027010)
3. A multivariate hierarchical Bayesian approach to measuring agreement in repeated measurement method comparison studies. Schluter PJ, BMC Med Res Methodol 2009 9 6 10.1186/1471-2288-9-6 19161599
4. Observer Reliability and Agreement. De Vet H, Encyclopedia of Biostatistics John Wiley & Sons, Ltd 2005
5. Intraclass Correlations-Uses in Assessing Rater Reliability. Shrout PE, Fleiss JL, Psychol Bull 1979 86 2 420 428 18839484
6. A critical discussion of intraclass correlation coefficients. Muller R, Buttner P, Stat Med 1994 13 23-24 2465 2476 7701147
7. The limits of agreement and the intraclass correlation coefficient may be inconsistent in the interpretation of agreement. Costa-Santos C, Bernardes J, Ayres-de-Campos D, Costa A, Amorim-Costa C, J Clin Epidemiol 2011 64 3 264 269 10.1016/j.jclinepi.2009.11.010 20189765
8. An overview on assessing agreement with continuous measurements. Barnhart HX, Haber MJ, Lin LI, J Biopharm Stat 2007 17 4 529 569 10.1080/ 10543400701376480 17613641 (Pubitemid 47027007)
9. The concordance correlation coefficient for repeated measures estimated by variance components. Carrasco JL, King TS, Chinchilli VM, J Biopharm Stat 2009 19 1 90 105 10.1080/10543400802527890 19127469
10. A repeated measures concordance correlation coefficient. King TS, Chinchilli VM, Carrasco JL, Stat Med 2007 26 16 3095 3113 10.1002/sim.2778 17216594 (Pubitemid 47019009)
11. A unified approach for assessing agreement for continuous and categorical data. Lin L, Hedayat AS, Wu W, J Biopharm Stat 2007 17 4 629 652 10.1080/10543400701376498 17613645 (Pubitemid 47027011)
12. A concordance correlation coefficient to evaluate reproducibility. Lin LI, Biometrics 1989 45 1 255 268 10.2307/2532051 2720055 (Pubitemid 19109172)
13. Statistical methods in assessing agreement: models, issues and tools. Lin LI, Hedayat AS, Sinha B, Yang M, J Am Stat Assoc 2002 97 257 270 10.1198/016214502753479392
14. A Note on A Concordance Correlation Coefficient to Evaluate Reproducibility Nickerson C, Biometrics 1997 53 1503 1507 10.2307/2533516 (Pubitemid 28013591)
15. Estimating the generalized concordance correlation coefficient through variance components. Carrasco JL, Jover L, Biometrics 2003 59 4 849 858 10.1111/j.0006-341X.2003.00099.x 14969463 (Pubitemid 38021213)
16. Comment on the Use of Concordance Correlation to Assess the Agreement between Two Variables. Atkinson G, Nevill A, Biometrics 1997 53 775 777
17. Rejoinder to the Letter to the Editor from Atkinson and Nevill. Lin L, Chinchilli V, Biometrics 1997 53 2 777 778
18. Assessment of disagreement: a new information-based approach. Costa-Santos C, Antunes L, Souto A, Bernardes J, Ann Epidemiol 2010 20 7 555 561 10.1016/j.annepidem.2010.02.011 20538199
19. The mathematical theory of communication. 1963. Shannon CE, MD Comput 1997 14 4 306 317 9230594
20. Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians. Carpenter J, Bithell J, Stat Med 2000 19 9 1141 1164 10.1002/(SICI)1097-0258(20000515)19:9<1141::AID-SIM479>3.0.CO;2-F 10797513 (Pubitemid 30265928)
21. Efron B, Tibshirani RJ, An Introduction to the Bootstrap (Chapman & Hall/CRC Monographs on Statistics & Applied Probability) Chapman and Hall/CRC 1994 15754206
22. Davison AC, Hinkley DV, Methods and their Application (Cambridge Series in Statistical and Probabilistic Mathematics) Cambridge University Press 1997
23. On-line calculator of IBMD, [ http://disagreement.med.up.pt ]
24. WINPEPI (PEPI-for-Windows): computer programs for epidemiologists. Abramson J, Epidemiologic Perspectives & Innovations 2004 1 1 6 10.1186/1742-5573-1-6 23520591 (Pubitemid 40404933)
25. WINPEPI updated: computer programs for epidemiologists, and their teaching potential. Abramson JH, Epidemiol Perspect Innov 2011 8 1 1 10.1186/1742-5573-8-1 21288353
26. A new permutation-based method for assessing agreement between two observers making replicated binary readings. Pan Y, Haber M, Barnhart HX, Stat Med 2011 30 8 839 853 10.1002/sim.4136 21432878
27. A new permutation-based method for assessing agreement between two observers making replicated quantitative readings. Pan Y, Haber M, Gao J, Barnhart HX, Stat Med 2012 31 20 2249 2261 10.1002/sim.5323 22415910
28. Assessment of agreement of a quantitative variable: a new graphical approach. Luiz RR, Costa AJ, Kale PL, Werneck GL, J Clin Epidemiol 2003 56 10 963 967 10.1016/S0895-4356(03)00164-1 14568627 (Pubitemid 37268083)
29. Folded empirical distribution function curves-mountain plots. Monti K, Am Stat 1995 33 525 527
30. Assessing reproducibility by the within-subject coefficient of variation with random effects models. Quan H, Shih WJ, Biometrics 1996 52 4 1195 1203 10.2307/2532835 8962450 (Pubitemid 26425271)
31. Total deviation index for measuring individual agreement with applications in laboratory performance and bioequivalence. Lin LI, Stat Med 2000 19 2 255 270 10.1002/(SICI)1097-0258(20000130)19:2<255::AID-SIM293>3.0. CO;2-8 10641028 (Pubitemid 30051246)
32. The total deviation index estimated by tolerance intervals to evaluate the concordance of measurement devices. Escaramis G, Ascaso C, Carrasco JL, BMC Med Res Methodol 2010 10 31 10.1186/1471-2288-10-31 20377875
33. Assessing individual agreement. Barnhart HX, Kosinski AS, Haber MJ, J Biopharm Stat 2007 17 4 697 719 10.1080/10543400701329489 17613649 (Pubitemid 47027016)