Unfolding the phenomenon of interrater agreement: A multicomponent approach for in-depth examination was proposed

Journal of Clinical Epidemiology

Volumn 65, Issue 9, 2012, Pages 1016-1025

  Slaug, Björn ^a   Schilling, Oliver ^b   Helle, Tina ^a,c   Iwarsson, Susanne ^a   Carlsson, Gunilla ^a   Brandt, Åse ^d  

^a LUND UNIVERSITY   (Sweden)

^b UNIVERSITY OF HEIDELBERG   (Germany)

^c UNIVERSITY COLLEGE OF NORTHERN DENMARK   (Denmark)

^d National Board of Social Services   (Denmark)

Author keywords

Agreement; Interrater; Kappa; Methodology; Recommendations; Reliability

Indexed keywords

ARTICLE; BIOMETRY; BIOSTATISTICS; CLINICAL ASSESSMENT TOOL; CLINICAL PROTOCOL; CONCEPTUAL FRAMEWORK; CONTROLLED STUDY; ENVIRONMENTAL FACTOR; HUMAN; INTERRATER RELIABILITY; MULTILEVEL ANALYSIS; PREDICTOR VARIABLE; PREVALENCE; PRIORITY JOURNAL; PROFESSIONAL COMPETENCE; RATING SCALE; STANDARDIZATION; STATISTICAL ANALYSIS;

CONSENSUS; HUMANS; OBSERVER VARIATION; REGRESSION ANALYSIS; REPRODUCIBILITY OF RESULTS; RESEARCH;

EID: 84864282727     PISSN: 08954356     EISSN: 18785921     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2012.02.016     Document Type: Article

References (30)

1. P.M. Fayers, and D. Machin Quality of life - assessment, analysis and interpretation 2000 John Wiley & Sons, Ltd New York, NY
2. S.W.J. Kozlowski, and K. Hattrup A disagreement about within-group agreement: disentangling issues of consistency versus consensus J Appl Psychol 77 1992 161 167
3. H.C.W. De Vet, C.B. Terwee, D.L. Knol, and L.M. Bouter When to use agreement versus reliability measures J Clin Epidemiol 59 2006 1033 1039
4. J. Kottner, and D.L. Streiner The difference between reliability and agreement J Clin Epidemiol 64 2011 701 702
5. D.L. Streiner, and G.R. Norman Health measurement scales. A practical guide to their development and use 4th ed 2008 Oxford University Press Oxford, UK
6. T. Helle, C. Nygren, B. Slaug, Å Brandt, A. Pikkarainen, and A.-G. Hansen The Nordic Housing Enabler: inter-rater reliability in cross-Nordic occupational therapy practice Scand J Occup Ther 17 2010 258 266
7. S. Iwarsson, C. Nygren, and B. Slaug Cross-national and multi professional inter-rater reliability of the Housing Enabler Scand J Occup Ther 12 2005 29 39
8. J.-K. Park, J. Boyer, J. Tessler, J. Casey, L. Schemm, R. Gore Promoting Healthy and Safe Employment (PHASE) in Healthcare Project Team Inter-rater reliability of PATH observations for assessment of ergonomic risk factors in hospital work Ergonomics 52 2009 820 829
9. J. Kottner, L. Audige, S. Brorson, A. Donner, B.J. Gajewski, and A. Hróbjartsson Guideline for reporting reliability and agreement studies (GRRAS) were proposed J Clin Epidemiol 64 2011 96 106
10. C.E. Schwartz, and B.D. Rapkin Reconsidering the psychometrics of quality of life assessment in light of response shift and appraisal Health Qual Life Outcomes 2 2004 16 Bio Med Central, Available at http://www.hqlo.com/content/2/ 1/16 Accessed May 18, 2009
11. R.L. Brennan Generalizability theory 2001 Springer-Verlag New York, NY
12. D.A. Bloch, and H.C. Kraemer 2 × 2 kappa coefficients: measures of agreement or association Biometrics 45 1989 269 287
13. H.C. Kraemer Ramifications for a population model for κ as a coefficient of reliability Psychometrika 44 1979 461 472
14. C.D. Phillips, A. Patnaik, J.A. Dyer, E. Naiser, C. Hawes, and C.J. Fournier Reliability and the measurement of activity limitations (ADLs) for children with special health care needs (CSHCN) living in the community Disabil Rehabil 33 2011 2013 2022
15. L.J. Cronbach, G.C. Gleser, H. Nanda, and N. Rajaratnam The dependability of behavioral measurements: theory of generalizability of scores and profiles 1972 Wiley New York, NY
16. S.K. Mitchell Interobserver agreement, reliability, and generalizability of data collected in observational studies Psychol Bull 86 1979 376 390
17. S. Iwarsson, and B. Slaug The Housing Enabler: an instrument for assessing and analysing accessibility problems in housing 2001 Veten & Skapen HB & Slaug Data Management Nävlinge & Staffanstorp, Sweden
18. S. Iwarsson, H.-W. Wahl, C. Nygren, F. Oswald, A. Sixsmith, and J. Sixsmith Importance of the home environment for healthy aging: conceptual and methodological background of the European ENABLE-AGE Project Gerontologist 47 2007 78 84
19. F. Oswald, H.-W. Wahl, O. Schilling, C. Nygren, A. Fänge, and A. Sixsmith Relationships between housing and healthy ageing aspects in very old age: results from the European ENABLE-AGE Project Gerontologist 47 2007 96 107
20. G. Hripcsak, and D.F. Heitjan Measuring agreement in medical informatics reliability studies J Biomed Inform 35 2002 99 110
21. J.L. Fleiss, B. Levin, and M.C. Paik Statistical methods for rates and proportions 3rd ed 2003 John Wiley & Sons New York, NY
22. M.M. Shoukri Measures of inter-observer agreement 2004 Chapman & Hall/CRC Washington, DC
23. F.K. Hohler Bias and prevalence effects on kappa viewed in terms of sensitivity and specificity J Clin Epidemiol 53 2000 499 503
24. J.J. Hox Multilevel analysis 2002 Lawrence Erlbaum Mahwah, NJ
25. J.D. Singer Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models J Educ Behav Stat 23 1998 323 355
26. N. Ghotbi, N.N. Ansari, S. Naghdi, S. Hasson, B. Jamshidpour, and S. Amiri Inter-rater reliability of the modified Ashworth scale in assessing lower limb muscle spasticity Brain 23 2009 815 819
27. M. Holmefur, L. Krumlinde-Sundholm, and A.-C. Eliasson Interrater and intrarater reliability of the assisting hand assessment Am J Occup Ther 61 2007 79 84
28. B.A. Bell, G.B. Morgan, J.A. Schoeneberger, B.L. Loudermilk, J.D. Kromrey, and J.M. Ferron Dancing the sample size limbo with mixed models: how low can you go? SAS Global Forum 2010 Paper 197-2010. Presented at the Global Forum, 11-14 April 2010, Seattle, WA.
29. L.D. Broemeling Bayesian methods for measures of agreement 2009 Taylor & Francis Boca Raton, FL
30. L.D. Broemeling A Bayesian analysis of inter-rater agreement Commun Stat Simul Comput 30 2001 437 446