메뉴 건너뛰기




Volumn 4, Issue 4, 2010, Pages 271-286

Inequalities between multi-rater kappas

Author keywords

Arithmetic harmonic means inequality; Cauchy Schwarz inequality; Cohen's kappa; Fleiss' kappa; Hubert's kappa; Light's kappa; Nominal agreement; Randolph's kappa; Scott's pi

Indexed keywords

CAUCHY-SCHWARZ INEQUALITY; COHEN'S KAPPA; FLEISS' KAPPA; HARMONIC MEAN; HUBERT'S KAPPA; NOMINAL AGREEMENT; RANDOLPH'S KAPPA; SCOTT'S PI;

EID: 78650064898     PISSN: 18625347     EISSN: 18625355     Source Type: Journal    
DOI: 10.1007/s11634-010-0073-4     Document Type: Article
Times cited : (187)

References (46)
  • 2
  • 3
    • 0040822661 scopus 로고
    • Communications through limited response questioning
    • Bennett EM, Alpert R, Goldstein AC (1954) Communications through limited response questioning. Public Opin Q 18: 303-308.
    • (1954) Public Opin Q , vol.18 , pp. 303-308
    • Bennett, E.M.1    Alpert, R.2    Goldstein, A.C.3
  • 4
    • 0000996283 scopus 로고
    • A generalization of Cohen's kappa agreement measure to interval measurement and multiple raters
    • Berry KJ, Mielke PW (1988) A generalization of Cohen's kappa agreement measure to interval measurement and multiple raters. Educ Psychol Meas 48: 921-933.
    • (1988) Educ Psychol Meas , vol.48 , pp. 921-933
    • Berry, K.J.1    Mielke, P.W.2
  • 5
    • 0001057572 scopus 로고
    • Coefficient kappa: some uses, misuses, and alternatives
    • Brennan RL, Prediger DJ (1981) Coefficient kappa: some uses, misuses, and alternatives. Edu Psychol Meas 41: 687-699.
    • (1981) Edu Psychol Meas , vol.41 , pp. 687-699
    • Brennan, R.L.1    Prediger, D.J.2
  • 6
    • 84973587732 scopus 로고
    • A coefficient of agreement for nominal scales
    • Cohen J (1960) A coefficient of agreement for nominal scales. Edu Psychol Meas 20: 37-46.
    • (1960) Edu Psychol Meas , vol.20 , pp. 37-46
    • Cohen, J.1
  • 7
    • 58149412516 scopus 로고
    • Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit
    • Cohen J (1968) Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol Bull 70: 213-220.
    • (1968) Psychol Bull , vol.70 , pp. 213-220
    • Cohen, J.1
  • 8
    • 0001059534 scopus 로고
    • Integration and generalization of kappas for multiple raters
    • Conger AJ (1980) Integration and generalization of kappas for multiple raters. Psychol Bull 88: 322-328.
    • (1980) Psychol Bull , vol.88 , pp. 322-328
    • Conger, A.J.1
  • 9
    • 0039263823 scopus 로고
    • Generalization of Scott's index of intercoder agreement
    • Craig RT (1981) Generalization of Scott's index of intercoder agreement. Public Opin Q 45: 260-264.
    • (1981) Public Opin Q , vol.45 , pp. 260-264
    • Craig, R.T.1
  • 10
    • 0001463960 scopus 로고
    • Measuring agreement for multinomial data
    • Davies M, Fleiss JL (1982) Measuring agreement for multinomial data. Biometrics 38: 1047-1051.
    • (1982) Biometrics , vol.38 , pp. 1047-1051
    • Davies, M.1    Fleiss, J.L.2
  • 11
    • 34248390977 scopus 로고    scopus 로고
    • Agreement and kappa-type indices
    • De Mast J (2007) Agreement and kappa-type indices. Am Stat 61: 148-153.
    • (2007) Am Stat , vol.61 , pp. 148-153
    • de Mast, J.1
  • 12
    • 2142668188 scopus 로고    scopus 로고
    • The kappa statistic: a second look
    • Di Eugenio B, Glass M (2004) The kappa statistic: a second look. Comput Linguist 30: 95-101.
    • (2004) Comput Linguist , vol.30 , pp. 95-101
    • Di Eugenio, B.1    Glass, M.2
  • 14
    • 3343019470 scopus 로고
    • Measuring nominal scale agreement among many raters
    • Fleiss JL (1971) Measuring nominal scale agreement among many raters. Psychol Bull 76: 378-382.
    • (1971) Psychol Bull , vol.76 , pp. 378-382
    • Fleiss, J.L.1
  • 15
    • 52949085846 scopus 로고    scopus 로고
    • Variance estimation of nominal-scale inter-rater reliability with random selection of raters
    • Gwet KL (2008) Variance estimation of nominal-scale inter-rater reliability with random selection of raters. Psychometrika 73: 407-430.
    • (2008) Psychometrika , vol.73 , pp. 407-430
    • Gwet, K.L.1
  • 16
    • 33845312187 scopus 로고
    • Beoordelaarsovereenstemming
    • In: Eggen TJHM, Sanders PF (eds), Cito Instituut voor Toestontwikkeling, Arnhem
    • Heuvelmans APJM, Sanders PF (1993) Beoordelaarsovereenstemming. In: Eggen TJHM, Sanders PF (eds) Psychometrie in de Praktijk. Cito Instituut voor Toestontwikkeling, Arnhem, pp 443-470.
    • (1993) Psychometrie in de Praktijk , pp. 443-470
    • Heuvelmans, A.P.J.M.1    Sanders, P.F.2
  • 17
    • 2542510882 scopus 로고    scopus 로고
    • n, Cohen's kappa, Scott's π and Aickin's α
    • n, Cohen's kappa, Scott's π and Aickin's α. Underst Stat 2: 205-219.
    • (2003) Underst Stat , vol.2 , pp. 205-219
    • Hsu, L.M.1    Field, R.2
  • 18
    • 0000075085 scopus 로고
    • Kappa revisited
    • Hubert L (1977) Kappa revisited. Psychol Bull 84: 289-297.
    • (1977) Psychol Bull , vol.84 , pp. 289-297
    • Hubert, L.1
  • 19
    • 0018421651 scopus 로고
    • An extension of the random error coefficient of agreement to N × N tables
    • Janes CL (1979) An extension of the random error coefficient of agreement to N × N tables. Br J Psychiatry 134: 617-619.
    • (1979) Br J Psychiatry , vol.134 , pp. 617-619
    • Janes, C.L.1
  • 20
    • 0035531769 scopus 로고    scopus 로고
    • A measure of agreement for interval or nominal multivariate observations
    • Janson H, Olsson U (2001) A measure of agreement for interval or nominal multivariate observations. Educ Psychol Meas 61: 277-289.
    • (2001) Educ Psychol Meas , vol.61 , pp. 277-289
    • Janson, H.1    Olsson, U.2
  • 21
    • 0040822657 scopus 로고
    • On generalizations of the G index and the Phi coefficient to nominal scales
    • Janson S, Vegelius J (1979) On generalizations of the G index and the Phi coefficient to nominal scales. Multivar Behav Res 14: 255-269.
    • (1979) Multivar Behav Res , vol.14 , pp. 255-269
    • Janson, S.1    Vegelius, J.2
  • 22
    • 0001586045 scopus 로고
    • Ramifications of a population model for κ as a coefficient of reliability
    • Kraemer HC (1979) Ramifications of a population model for κ as a coefficient of reliability. Psychometrika 44: 461-472.
    • (1979) Psychometrika , vol.44 , pp. 461-472
    • Kraemer, H.C.1
  • 23
    • 0019025893 scopus 로고
    • Extensions of the kappa coefficient
    • Kraemer HC (1980) Extensions of the kappa coefficient. Biometrics 36: 207-216.
    • (1980) Biometrics , vol.36 , pp. 207-216
    • Kraemer, H.C.1
  • 24
    • 0037199386 scopus 로고    scopus 로고
    • Tutorial in biostatistics: kappa coefficients in medical research
    • Kraemer HC, Periyakoil VS, Noda A (2002) Tutorial in biostatistics: kappa coefficients in medical research. Stat Med 21: 2109-2129.
    • (2002) Stat Med , vol.21 , pp. 2109-2129
    • Kraemer, H.C.1    Periyakoil, V.S.2    Noda, A.3
  • 25
    • 4444293059 scopus 로고
    • Association, agreement, and equity
    • Krippendorff K (1987) Association, agreement, and equity. Qual Quant 21: 109-123.
    • (1987) Qual Quant , vol.21 , pp. 109-123
    • Krippendorff, K.1
  • 26
    • 0017360990 scopus 로고
    • The measurement of observer agreement for categorical data
    • Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 33: 159-174.
    • (1977) Biometrics , vol.33 , pp. 159-174
    • Landis, J.R.1    Koch, G.G.2
  • 27
    • 0001587595 scopus 로고
    • Measures of response agreement for qualitative data: some generalizations and alternatives
    • Light RJ (1971) Measures of response agreement for qualitative data: some generalizations and alternatives. Psychol Bull 76: 365-377.
    • (1971) Psychol Bull , vol.76 , pp. 365-377
    • Light, R.J.1
  • 32
    • 84980163290 scopus 로고
    • Measuring agreement among many observers
    • Schouten HJA (1980) Measuring agreement among many observers. Biom J 22: 497-504.
    • (1980) Biom J , vol.22 , pp. 497-504
    • Schouten, H.J.A.1
  • 33
    • 84980170795 scopus 로고
    • Measuring pairwise agreement among many observers
    • Schouten HJA (1982) Measuring pairwise agreement among many observers. Biom J 24: 431-435.
    • (1982) Biom J , vol.24 , pp. 431-435
    • Schouten, H.J.A.1
  • 34
    • 0001028067 scopus 로고
    • Nominal scale agreement among observers
    • Schouten HJA (1986) Nominal scale agreement among observers. Psychometrika 51: 453-466.
    • (1986) Psychometrika , vol.51 , pp. 453-466
    • Schouten, H.J.A.1
  • 35
    • 33748063869 scopus 로고
    • Reliability of content analysis: the case of nominal scale coding
    • Scott WA (1955) Reliability of content analysis: the case of nominal scale coding. Public Opin Q 19: 321-325.
    • (1955) Public Opin Q , vol.19 , pp. 321-325
    • Scott, W.A.1
  • 36
    • 61749098649 scopus 로고    scopus 로고
    • A note on the linearly weighted kappa coefficient for ordinal scales
    • Vanbelle S, Albert A (2009) A note on the linearly weighted kappa coefficient for ordinal scales. Stat Methodol 6: 157-163.
    • (2009) Stat Methodol , vol.6 , pp. 157-163
    • Vanbelle, S.1    Albert, A.2
  • 37
    • 52949107579 scopus 로고    scopus 로고
    • On similarity coefficients for 2 × 2 tables and correction for chance
    • Warrens MJ (2008a) On similarity coefficients for 2 × 2 tables and correction for chance. Psychometrika 73: 487-502.
    • (2008) Psychometrika , vol.73 , pp. 487-502
    • Warrens, M.J.1
  • 38
    • 58149527803 scopus 로고    scopus 로고
    • Bounds of resemblance measures for binary (presence/absence) variables
    • Warrens MJ (2008b) Bounds of resemblance measures for binary (presence/absence) variables. J Classif 25: 195-208.
    • (2008) J Classif , vol.25 , pp. 195-208
    • Warrens, M.J.1
  • 39
    • 62949147818 scopus 로고    scopus 로고
    • On association coefficients for 2 × 2 tables and properties that do not depend on the marginal distributions
    • Warrens MJ (2008c) On association coefficients for 2 × 2 tables and properties that do not depend on the marginal distributions. Psychometrika 73: 777-789.
    • (2008) Psychometrika , vol.73 , pp. 777-789
    • Warrens, M.J.1
  • 40
    • 58149498320 scopus 로고    scopus 로고
    • On the equivalence of Cohen's kappa and the Hubert-Arabie adjusted Rand index
    • Warrens MJ (2008d) On the equivalence of Cohen's kappa and the Hubert-Arabie adjusted Rand index. J Classif 25: 177-183.
    • (2008) J Classif , vol.25 , pp. 177-183
    • Warrens, M.J.1
  • 41
    • 49349103080 scopus 로고    scopus 로고
    • On the indeterminacy of resemblance measures for (presence/absence) data
    • Warrens MJ (2008e) On the indeterminacy of resemblance measures for (presence/absence) data. J Classif 25: 125-136.
    • (2008) J Classif , vol.25 , pp. 125-136
    • Warrens, M.J.1
  • 42
    • 77950076141 scopus 로고    scopus 로고
    • Inequalities between kappa and kappa-like statistics for k × k tables
    • Warrens MJ (2010a) Inequalities between kappa and kappa-like statistics for k × k tables. Psychometrika 75: 176-185.
    • (2010) Psychometrika , vol.75 , pp. 176-185
    • Warrens, M.J.1
  • 43
    • 84895901446 scopus 로고    scopus 로고
    • A formal proof of a paradox associated with Cohen's kappa
    • (in press)
    • Warrens MJ (2010b) A formal proof of a paradox associated with Cohen's kappa. J Classif (in press).
    • (2010) J Classif
    • Warrens, M.J.1
  • 44
    • 77956601127 scopus 로고    scopus 로고
    • Cohen's kappa can always be increased and decreased by combining categories
    • Warrens MJ (2010c) Cohen's kappa can always be increased and decreased by combining categories. Stat Methodol 7: 673-677.
    • (2010) Stat Methodol , vol.7 , pp. 673-677
    • Warrens, M.J.1
  • 45
    • 77953726321 scopus 로고    scopus 로고
    • A Kraemer-type rescaling that transforms the odds ratio into the weighted kappa coefficient
    • Warrens MJ (2010d) A Kraemer-type rescaling that transforms the odds ratio into the weighted kappa coefficient. Psychometrika 75: 328-330.
    • (2010) Psychometrika , vol.75 , pp. 328-330
    • Warrens, M.J.1
  • 46
    • 0024005773 scopus 로고
    • Another look at interrater agreement
    • Zwick R (1988) Another look at interrater agreement. Psychol Bull 103: 374-378.
    • (1988) Psychol Bull , vol.103 , pp. 374-378
    • Zwick, R.1


* 이 정보는 Elsevier사의 SCOPUS DB에서 KISTI가 분석하여 추출한 것입니다.