Calculating kappas from adjusted data improved the comparability of the reliability of triage systems: A comparative study

Journal of Clinical Epidemiology

Volumn 63, Issue 11, 2010, Pages 1256-1263

  Van Der Wulp, Ineke ^a   Van Stel, Henk F ^a  

^a UNIVERSITY MEDICAL CENTER UTRECHT   (Netherlands)

Author keywords

Emergency medicine; Emergency services hospital; Reproducibility of results; Statistics as topic; Triage; Validation studies as topic

Indexed keywords

ARTICLE; CALCULATION; COMPARATIVE STUDY; HUMAN; KAPPA STATISTICS; METHODOLOGY; PRIORITY JOURNAL; RELIABILITY; SIMULATION; STATISTICAL DISTRIBUTION; SYSTEMS THEORY; TRIAGE SYSTEM;

ALGORITHMS; DATA INTERPRETATION, STATISTICAL; EMERGENCY SERVICE, HOSPITAL; HUMANS; REPRODUCIBILITY OF RESULTS; SEVERITY OF ILLNESS INDEX; TRIAGE;

EID: 77956976310     PISSN: 08954356     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2010.01.012     Document Type: Article

References (60)

1. C.M. Fernandes, P. Tanabe, N. Gilboy, L.A. Johnson, R.S. McNair, and A.M. Rosenau Five-level triage: a report from the ACEP/ENA Five-level Triage Task Force J Emerg Nurs 31 2005 39 50
2. Australasian College for Emergency Medicine. Policy Document: The Australasian Triage Scale; 2000.
3. Beveridge R, Clarke B, Janes L, Savage N, Thompson J, Dodd G, et al. Guidelines for the Canadian Emergency Department Triage & Acuity Scale (CTAS); 1998.
4. N. Gilboy, P. Tanabe, D. Travers, A. Rosenau, and D.R. Eitel Emergency Severity Index, version 4: implementation handbook 2005 Agency for Healthcare Research and Quality Rockville, IN
5. K. Mackway-Jones Emergency triage 2nd ed 2005 Manchester Triage Group London, UK
6. J.C. Brillman, D. Doezema, D. Tandberg, D.P. Sklar, K.D. Davis, and S. Simms Triage: limitations in predicting need for emergent care and hospital admission Ann Emerg Med 27 1996 493 500
7. J.C. Brillman, D. Doezema, D. Tandberg, D.P. Sklar, and B.J. Skipper Does a physician visual assessment change triage? Am J Emerg Med 15 1997 29 33
8. D.A. Travers, A.E. Waller, J.M. Bowling, D. Flowers, and J. Tintinalli Five-level triage system more effective than three-level in tertiary emergency department J Emerg Nurs 28 2002 395 400
9. R.C. Wuerz, C.M. Fernandes, and J. Alarcon Inconsistency of emergency department triage. Emergency Department Operations Research Working Group Ann Emerg Med 32 1998 431 435
10. M.R. Baumann, and T.D. Strout Evaluation of the Emergency Severity Index (version 3) triage algorithm in pediatric patients Acad Emerg Med 12 2005 219 224
11. D.R. Eitel, D.A. Travers, A.M. Rosenau, N. Gilboy, and R.C. Wuerz The emergency severity index triage algorithm version 2 is reliable and valid Acad Emerg Med 10 2003 1070 1080
12. M.F. Gerdtz, and T.K. Bucknall Influence of task properties and subjectivity on consistency of triage: a simulation study J Adv Nurs 58 2007 180 190
13. P.A. Maningas, D.A. Hime, and D.E. Parker The use of the Soterion Rapid Triage System in children presenting to the Emergency Department J Emerg Med 31 2006 353 359
14. P.A. Maningas, D.A. Hime, D.E. Parker, and T.A. McMurry The Soterion Rapid Triage System: evaluation of inter-rater reliability and validity J Emerg Med 30 2006 461 469
15. O.T. Rutschmann, M. Kossovsky, A. Geissbuhler, T.V. Perneger, B. Vermeulen, and J. Simon Interactive triage simulator revealed important variability in both process and outcome of emergency triage J Clin Epidemiol 59 2006 615 621
16. P. Tanabe, R. Gimbel, P.R. Yarnold, D.N. Kyriacou, and J.G. Adams Reliability and validity of scores on The Emergency Severity Index version 3 Acad Emerg Med 11 2004 59 65
17. R.C. Wuerz, D. Travers, N. Gilboy, D.R. Eitel, A. Rosenau, and R. Yazhari Implementation and refinement of the emergency severity index Acad Emerg Med 8 2001 170 176
18. H. Brenner, and U. Kliebsch Dependence of weighted kappa coefficients on the number of categories Epidemiology 7 1996 199 202
19. J. Cohen A coefficient of agreement for nominal scales Educ Psychol Meas 20 1960 37 46
20. M. Maclure, and W.C. Willett Misinterpretation and misuse of the kappa statistic Am J Epidemiol 126 1987 161 169
21. C.A. Lantz, and E. Nebenzahl Behavior and interpretation of the kappa statistic: resolution of the two paradoxes J Clin Epidemiol 49 1996 431 434
22. S.L. Dong, M.J. Bullard, D.P. Meurer, I. Colman, S. Blitz, and B.R. Holroyd Emergency triage: comparing a novel computer triage program with standard triage Acad Emerg Med 12 2005 502 507
23. S.L. Dong, M.J. Bullard, D.P. Meurer, S. Blitz, A. Ohinmaa, and B.R. Holroyd Reliability of computerized emergency triage Acad Emerg Med 13 2006 269 275
24. S. George, S. Read, L. Westlake, A. Fraser-Moodie, P. Pritty, and B. Williams Differences in priorities assigned to patients by triage nurses and by consultant physicians in accident and emergency departments J Epidemiol Community Health 47 1993 312 315
25. J. Gravel, S. Gouin, S. Manzano, M. Arsenault, and D. Amre Interrater agreement between nurses for the Pediatric Canadian Triage and Acuity Scale in a tertiary care center Acad Emerg Med 15 2008 1262 1267
26. E. Hay, L. Bekerman, G. Rosenberg, and R. Peled Quality assurance of nurse triage: consistency of results over three years Am J Emerg Med 19 2001 113 117
27. A. Worster, A. Sardo, K. Eva, C.M. Fernandes, and S. Upadhye Triage tool inter-rater reliability: a comparison of live versus paper case scenarios J Emerg Nurs 33 2007 319 323
28. I. van der Wulp, M.E. van Baar, and A.J.P. Schrijvers Reliability and validity of the Manchester triage system in a general emergency department patient population in The Netherlands: results of a simulation study Emerg Med J 25 2008 431 434
29. D.V. Cicchetti, and A.R. Feinstein High agreement but low kappa: II. Resolving the paradoxes J Clin Epidemiol 43 1990 551 558
30. A.R. Feinstein, and D.V. Cicchetti High agreement but low kappa: I. The problems of two paradoxes J Clin Epidemiol 43 1990 543 549
31. I. van der Wulp, and H.F. van Stel Adjusting weighted kappa for severity of mistriage decreases reported reliability of emergency department triage systems: a comparative study J Clin Epidemiol 62 2009 1196 1201
32. Google Scholar Available at http://scholar.google.com 2008
33. T. Byrt, J. Bishop, and J.B. Carlin Bias, prevalence and kappa J Clin Epidemiol 46 1993 423 429
34. AGREE for Windows [computer program] Available at http://www.scienceplus. nl 1999 Version 7.002
35. J. Considine, S.A. LeVasseur, and E. Villanueva The Australasian Triage Scale: examining emergency department nurses' performance using computer and paper scenarios Ann Emerg Med 44 2004 516 523
36. D. Manos, D.A. Petrie, R.C. Beveridge, S. Walter, and J. Ducharme Inter-observer agreement using the Canadian Emergency Department Triage and Acuity Scale CJEM 4 2002 16 22
37. S. Bergeron, S. Gouin, B. Bailey, and H. Patel Comparison of triage assessments among pediatric registered nurses and pediatric emergency physicians Acad Emerg Med 9 2002 1397 1401
38. S. Bergeron, S. Gouin, B. Bailey, D.K. Amre, and H. Patel Agreement among pediatric health care professionals with the pediatric Canadian triage and acuity scale guidelines Pediatr Emerg Care 20 2004 514 518
39. R. Beveridge, J. Ducharme, L. Janes, S. Beaulieu, and S. Walter Reliability of the Canadian emergency department triage and acuity scale: interrater agreement Ann Emerg Med 34 1999 155 159
40. J. Considine, L. Ung, and S. Thomas Triage nurses' decisions using the National Triage Scale for Australian emergency departments Accid Emerg Nurs 8 2000 201 209
41. J. Considine, L. Ung, and S. Thomas Clinical decisions using the National Triage Scale: how important is postgraduate education? Accid Emerg Nurs 9 2001 101 108
42. T. Maldonado, and J.R. Avner Triage of the pediatric patient in the emergency department: are we all in agreement? Pediatrics 114 2004 356 360
43. A. Wollaston, P. Fahey, M. McKay, D. Hegney, P. Miller, and J. Wollaston Reliability and validity of the Toowoomba adult trauma triage tool: a Queensland, Australia study Accid Emerg Nurs 12 2004 230 237
44. S. Doherty Application of the National Triage Scale is not uniform Emerg Nurs J 1 1996 26
45. Y. Durani, D. Brecher, D. Walmsley, M.W. Attia, and J.M. Loiselle The Emergency Severity Index (version 4): reliability in pediatric patients Acad Emerg Med 14 Suppl 1 2007 S95 S96
46. K. Goransson, A. Ehrenberg, B. Marklund, and M. Ehnfors Accuracy and concordance of nurses in emergency department triage Scand J Caring Sci 19 2005 432 438
47. K.E. Goransson, A. Ehrenberg, B. Marklund, and M. Ehnfors Emergency department triage: is there a link between nurses' personal characteristics and accuracy in triage decisions? Accid Emerg Nurs 14 2006 83 88
48. J. Gravel, S. Gouin, B. Bailey, M. Roy, S. Bergeron, and D. Amre Reliability of a computerized version of the Pediatric Canadian Triage and Acuity Scale Acad Emerg Med 14 2007 864 869
49. A. Worster, N. Gilboy, C.M. Fernandes, D. Eitel, K. Eva, and R. Geisler Assessment of inter-observer reliability of two five-level triage and acuity scales: a randomized controlled trial CJEM 6 2004 240 245
50. R.C. Wuerz, L.W. Milne, D.R. Eitel, D. Travers, and N. Gilboy Reliability and validity of a new five-level triage instrument Acad Emerg Med 7 2000 236 242
51. P. Graham, and R. Jackson The analysis of ordinal agreement data: beyond weighted kappa J Clin Epidemiol 46 1993 1055 1062
52. J.R. Landis, and G.G. Koch The measurement of observer agreement for categorical data Biometrics 33 1977 159 174
53. M.W. Cooke, and S. Jinks Does the Manchester triage system detect the critically ill? J Accid Emerg Med 16 1999 179 181
54. E. Grafstein, G. Innes, J. Westman, J. Christenson, and A. Thorne Inter-rater reliability of a computerized presenting-complaint-linked triage system in an urban emergency department CJEM 5 2003 323 329
55. T.G. Kim, J.K. Cho, S.H. Kim, H.S. Lee, H.D. Gu, and S.W. Chung Reliability and validity of the modified Emergency Severity Index-2 ad a triage tool J Korean Soc Emerg Med 17 2006 154 164
56. K. Toulson, L. Laskowski-Jones, and L.A. McConnell Implementation of the five-level emergency severity index in a level I trauma center emergency department with a three-tiered triage scheme J Emerg Nurs 31 2005 259 264
57. S. George, S. Read, L. Westlake, B. Williams, A. Fraser-Moodie, and P. Pritty Evaluation of nurse triage in a British accident and emergency department BMJ 304 1992 876 878
58. J. Nakagawa, S. Ouk, B. Schwartz, and D.L. Schriger Interobserver agreement in emergency department triage Ann Emerg Med 41 2003 191 195
59. T.W. Wong, G. Tseng, and L.W. Lee Report of an audit of nurse triage in an accident and emergency department J Accid Emerg Med 11 1994 91 95
60. S.W. Goodacre, M. Gillett, R.D. Harris, and K.P. Houlihan Consistency of retrospective triage decisions as a standardised instrument for audit J Accid Emerg Med 16 1999 322 324