A comparison of Cohen's Kappa and Gwet's AC1 when calculating inter-rater reliability coefficients: A study conducted with personality disorder samples

BMC Medical Research Methodology

Volumn 13, Issue 1, 2013, Pages

  Wongpakaran, Nahathai ^a   Wongpakaran, Tinakon ^a   Wedding, Danny ^b   Gwet, Kilem L ^c  

^a Faculty of Medicine Chiang Mai University   (Thailand)

^b ALLIANT INTERNATIONAL UNIVERSITY   (United States)

^c LLC   (United States)

Author keywords

Coefficients; Cohen's Kappa; Gwet's AC1; Inter rater reliability; Personality disorders

Indexed keywords

ARTICLE; COMPUTER GRAPHICS; COMPUTER NETWORK; HUMAN; META ANALYSIS (TOPIC);

COMPUTER COMMUNICATION NETWORKS; COMPUTER GRAPHICS; HUMANS; META-ANALYSIS AS TOPIC;

MLCS; MLOWN;

EID: 84876804850     PISSN: None     EISSN: 14712288     Source Type: Journal    
DOI: 10.1186/1471-2288-13-61     Document Type: Article

References (29)

1. First MB, Gibbon M, Spitzer RL, Williams JBW, Benjamin LS, Structured Clinical Interview for DSM-IV Axis II Personality Disorder (SCID-II) Washington, DC: merican Psychiatric Press 1997
2. A coefficient of agreement for nominal scales. Cohen J, Educ Psychol Meas 1960 20 37 46 10.1177/001316446002000104
3. Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Cohen J, Psychol Bull 1968 70 213 220 19673146
4. Interrater reliability of Thai version of the Structured Clinical Interview for DSM-IV Axis II Personality Disorders (T-SCID II). Wongpakaran T, Wongpakaran N, Bookkamana P, Boonyanaruthee V, Pinyopornpanish M, Likhitsathian S, Suttajit S, Srisutadsanavong U, J Med Assoc Thai 2012 95 264 269 22435258
5. Short-interval test-retest interrater reliability of the Structured Clinical Interview for DSM-III-R personality disorders (SCID-II) in outpatients. Dreessen L, Arntz A, J Pers Disord 1998 12 138 148 10.1521/pedi.1998.12.2.138 9661100 (Pubitemid 28339852)
6. Short-interval test-retest interrater reliability of the Dutch version of the Structured Clinical Interview for DSM-IV personality disorders (SCID-II). Weertman A, Arntz A, Dreessen L, van Velzen C, Vertommen S, J Pers Disord 2003 17 562 567 10.1521/pedi.17.6.562.25359 14744081 (Pubitemid 38084410)
7. High agreement but low kappa: II. Resolving the paradoxes. Cicchetti DV, Feinstein AR, J Clin Epidemiol 1990 43 551 558 10.1016/0895-4356(90)90159-M 2189948 (Pubitemid 20178544)
8. The Kappa Statistic: A Second Look. Di Eugenio B, Glass M, Comput Linguist 2004 30 95 101 10.1162/089120104773633402 (Pubitemid 38549035)
9. Gwet KL, Handbook of Inter-Rater Reliability. The Definitive Guide to Measuring the Extent of Agreement Among Raters Gaithersburg, MD 20886-2696, USA: Advanced Analytics, LLC 2 2010
10. Computing inter-rater reliability and its variance in the presence of high agreement. Gwet KL, Br J Math Stat Psychol 2008 61 29 48 10.1348/000711006X126600 18482474
11. The Validity of the Mini International Neuropsychiatric Interview (M.I.N.I.)-ThaiVersion. Kittirattanapaiboon P, Khamwongpin M, Journal of Mental Health of Thailand 2005 13 126 136
12. Inter-Rater Reliability: Dependency on Trait Prevalence and Marginal Homogeneity. Gwet K, http://www.agreestat.com/research-papers/inter-rater- reliability-dependency.pdf
13. Kappa is not satisfactory for assessing the extent of agreement between raters. Gwet K, http://www.google.ca/url?sa=t&rct=j&q= kappa%20statistic%20is%20not%
14. A consideration of the measurement and reporting of interrater reliability: answers to the July 2009 Journal Club questions. Day FC, Schriger DL, Annals Of Emergency Medicine Journal Club, Ann Emerg Med 2009 54 843 853 10.1016/j.annemergmed.2009.07.013 19942071
15. The interrater reliability of a Dutch version of the Structured Clinical Interview for DSM-III-R Personality Disorders. Arntz A, van Beijsterveldt B, Hoekstra R, Hofman A, Eussen M, Sallaerts S, Acta Psychiatr Scand 1992 85 394 400 10.1111/j.1600-0447.1992.tb10326.x 1605061
16. Inter-rater reliability of the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID I) and Axis II Disorders (SCID II). Lobbestael J, Leurgans M, Arntz A, Clin Psychol Psychother 2011 18 75 79 10.1002/cpp.693 20309842
17. Screening for personality disorder in incarcerated adolescent boys: preliminary validation of an adolescent version of the standardised assessment of personality-abbreviated scale (SAPAS-AV). Kongerslev M, Moran P, Bo S, Simonsen E, BMC Psychiatry 2012 12 94 10.1186/1471-244X-12-94 22846474
18. Biostatistics 104: correlational analysis. Chan YH, Singapore Med J 2003 44 614 619 14770254
19. Testing a tool for the classification of study designs in systematic reviews of interventions and exposures showed moderate reliability and low accuracy. Hartling L, Bond K, Santaguida PL, Viswanathan M, Dryden DM, J Clin Epidemiol 2011 64 861 871 10.1016/j.jclinepi.2011.01.010 21531537
20. Diagnostic accuracy and reliability of ultrasonography for the detection of fatty liver: a meta-analysis. Hernaez R, Lazo M, Bonekamp S, Kamel I, Brancati FL, Guallar E, Clark JM, Hepatology 2011 54 1082 1090 21618575
21. Reliability and validity of the Mini International Neuropsychiatric Interview for Children and Adolescents (MINI-KID). Sheehan DV, Sheehan KH, Shytle RD, Janavs J, Bannon Y, Rogers JE, Milo KM, Stock SL, Wilkinson B, J Clin Psychiatry 2010 71 313 326 10.4088/JCP.09m05305whi 20331933
22. Interrater reliability for Kernberg's structural interview for assessing personality organization. Ingenhoven TJ, Duivenvoorden HJ, Brogtrop J, Lindenborn A, van den Brink W, Passchier J, J Pers Disord 2009 23 528 534 10.1521/pedi.2009.23.5.528 19817632
23. Classification of bipolar disorder in psychiatric hospital. Øiesvold T, Nivison M, Hansen V, Sørgaard KW, Østensen L, Skre I, A prospective cohort study. BMC Psychiatry 2012 12 13
24. Development and psychometric properties the Barriers to Access to Care Evaluation scale (BACE) related to people with mental ill health. Clement S, Brohan E, Jeffery D, Henderson C, Hatch SL, Thornicroft G, BMC Psychiatry 2012 12 36 10.1186/1471-244X-12-36 22546012
25. Interrater agreement of nasal endoscopy in patients with a prior history of endoscopic sinus surgery. McCoul ED, Smith TL, Mace JC, Anand VK, Senior BA, Hwang PH, Stankiewicz JA, Tabaee A, Int Forum Allergy Rhinol 2012 2 453 459 10.1002/alr.21058 22696506
26. Development of the Persian version of the Modified Modified Ashworth Scale: translation, adaptation, and examination of interrater and intrarater reliability in patients with poststroke elbow flexor spasticity. Ansari NN, Naghdi S, Forogh B, Hasson S, Atashband M, Lashgari E, Disabil Rehabil 2012 34 1843 1847 10.3109/09638288.2012.665133 22432437
27. Interrater agreement and interrater reliability: Key concepts, approaches, and applications. Gisev N, Bell JS, Chen TF, Res Social Adm Pharm In press 23632161
28. Neurofilament ELISA validation. Petzold A, Altintas A, Andreoni L, Bartos A, Berthele A, Blankenstein MA, Buee L, Castellazzi M, Cepok S, Comabella M, J Immunol Methods 2010 352 23 31 10.1016/j.jim.2009.09.014 19857497
29. Frequency, types and severity of medication use-related problems among medical outpatients in Nigeria. Yusuff KB, Tayo F, Int J Clin Pharm 2011 33 558 564 10.1007/s11096-011-9508-z 21526413