Objective structured assessment of nontechnical skills: Reliability of a global rating scale for the in-training assessment in the operating room

Surgery (United States)

Volumn 157, Issue 6, 2015, Pages 1002-1013

  Dedy, Nicolas J ^a   Szasz, Peter ^a   Louridas, Marisa ^a   Bonrath, Esther M ^a   Husslein, Heinrich ^b   Grantcharov, Teodor P ^a  

^a UNIVERSITY OF TORONTO   (Canada)

^b ST MICHAEL'S HOSPITAL   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CORRELATION COEFFICIENT; EVIDENCE BASED PRACTICE; FEASIBILITY STUDY; HUMAN; INTERNAL CONSISTENCY; OBJECTIVE STRUCTURED ASSESSMENT OF NONTECHNICAL SKILL; OPERATING ROOM; PILOT STUDY; PRIORITY JOURNAL; RATING SCALE; RELIABILITY; RESIDENCY EDUCATION; SIMULATION; SURGICAL TRAINING; VALIDITY; CANADA; CLINICAL COMPETENCE; COMPARATIVE STUDY; EDUCATION; EVIDENCE BASED MEDICINE; FEMALE; IN SERVICE TRAINING; INTERDISCIPLINARY COMMUNICATION; MALE; MEDICAL EDUCATION; OBSERVER VARIATION; ORGANIZATION AND MANAGEMENT; PATIENT CARE; PROCEDURES; REPRODUCIBILITY; SURGERY; TEACHING;

CLINICAL COMPETENCE; EDUCATIONAL MEASUREMENT; EVIDENCE-BASED MEDICINE; FEMALE; HUMANS; INSERVICE TRAINING; INTERDISCIPLINARY COMMUNICATION; INTERNSHIP AND RESIDENCY; MALE; OBSERVER VARIATION; ONTARIO; OPERATING ROOMS; PATIENT CARE TEAM; PATIENT SIMULATION; PILOT PROJECTS; REPRODUCIBILITY OF RESULTS; SURGICAL PROCEDURES, OPERATIVE;

EID: 84929285498     PISSN: 00396060     EISSN: 15327361     Source Type: Journal    
DOI: 10.1016/j.surg.2014.12.023     Document Type: Article

References (43)

1. L. Lingard, S. Espin, S. Whyte, G. Regehr, G.R. Baker, and R. Reznick et al. Communication failures in the operating room: an observational classification of recurrent types and effects Qual Saf Health Care 13 2004 330 334
2. C.J. Kohn LT, J.M. Corrigan, and M.S. Donaldson To err is human: building a safer health system 6th ed 2000 National Academies Press Washington, DC
3. K. Mazzocco, D.B. Petitti, K.T. Fong, D. Bonacum, J. Brookey, and S. Graham et al. Surgical team behaviors and patient outcomes Am J Surg 197 2009 678 685
4. C.C. Greenberg, S.E. Regenbogen, D.M. Studdert, S.R. Lipsitz, S.O. Rogers, and M.J. Zinner et al. Patterns of communication breakdowns resulting in injury to surgical patients J Am Coll Surg 204 2007 533 540
5. Royal College of Physicians and Surgeons of Canada. The CanMEDS Framework. Available from: http://www.royalcollege.ca/portal/page/portal/rc/canmeds/framework. Accessed February 7, 2015.
6. Accreditation Council for Graduate Medical Education. ACGME Common Program Requirements, 2013. Available from: http://www.acgme.org/acgmeweb/Portals/0/PFAssets/ProgramRequirements/CPRs2013.pdf. Accessed February 7, 2015.
7. H. Sanfey, R. Williams, and G. Dunnington Recognizing residents with a deficiency in operative performance as a step closer to effective remediation J Am Coll Surg 216 2013 114 122
8. N.J. Dedy, B. Zevin, E.M. Bonrath, and T.P. Grantcharov Current concepts of team training in surgical residency: a survey of North American program directors J Surg Educ 70 2013 578 584
9. L. Hull, S. Arora, E. Kassab, R. Kneebone, and N. Sevdalis Observational teamwork assessment for surgery: content validation and tool refinement J Am Coll Surg 212 2011 234 243.e1-5
10. N. Sevdalis, R. Davis, M. Koutantji, S. Undre, A. Darzi, and C. Vincent Reliability of a revised NOTECHS scale for use in surgical teams Am J Surg 196 2008 184 190
11. A. Mishra, K. Catchpole, and P. McCulloch The Oxford NOTECHS System: reliability and validity of a tool for measuring teamwork behaviour in the operating theatre Qual Saf Health Care 18 2009 104 108
12. E.R. Robertson, M. Hadi, L.J. Morgan, S.P. Pickering, G. Collins, and S. New et al. Oxford NOTECHS II: a modified theatre team non-technical skills scoring system PLoS One 9 2014 e90320
13. S. Yule, R. Flin, N. Maran, D. Rowley, G. Youngson, and S. Paterson-Brown Surgeons' non-technical skills in the operating room: reliability testing of the NOTSS behavior rating system World J Surg 32 2008 548 556
14. S. Yule, D. Rowley, R. Flin, N. Maran, G. Youngson, and J. Duncan et al. Experience matters: comparing novice and expert ratings of non-technical skills using the NOTSS system ANZ J Surg 79 2009 154 160
15. L. Hull, S. Arora, N.R. Symons, R. Jalil, A. Darzi, and C. Vincent et al. Training faculty in nontechnical skill assessment: national guidelines on program requirements Ann Surg 258 2013 370 375
16. R. Phitayakorn, R. Minehart, M.C. Pian-Smith, M.W. Hemingway, T. Milosh-Zinkus, and D. Oriol-Morway et al. Practicality of intraoperative teamwork assessments J Surg Res 190 2014 22 28
17. J. Norcini, B. Anderson, V. Bollela, V. Burch, M.J. Costa, and R. Duvivier et al. Criteria for good assessment: consensus statement and recommendations from the Ottawa 2010 Conference Med Teach 33 2011 206 214
18. L. Lingard, G. Regehr, S. Espin, and S. Whyte A theory-based instrument to evaluate team communication in the operating room: balancing measurement authenticity and reliability Qual Saf Health Care 15 2006 422 426
19. S. Henrickson Parker, R. Flin, A. McKinley, and S. Yule The Surgeons' Leadership Inventory (SLI): a taxonomy and rating system for surgeons' intraoperative leadership skills Am J Surg 205 2013 745 751
20. Accreditation Council for Graduate Medical Education. Program and Institutional Accreditation - Surgical Specialties. Available from: http://www.acgme.org/acgmeweb/tabid/369/ProgramandInstitutionalAccreditation/SurgicalSpecialties.aspx. Accessed on February 7, 2015.
21. Royal College of Physicians and Surgeons of Canada. Credentials, Examinations & Accreditation. Available from: http://www.royalcollege.ca/portal/page/portal/rc/credentials. Accessed February 7, 2015.
22. Intercollegiate Surgical Curriculum Programme. Overview of the Assessment System. Available from: https://www.iscp.ac.uk/surgical/assessment-overview.aspx. Accessed February 7, 2015.
23. Royal Australasian College of Surgeons. Nine RACS Competencies; 2014. Available from: http://www.surgeons.org/becoming-a-surgeon/surgical-education-training/competencies. Accessed February 7, 2015.
24. I. Ghaderi, F. Manji, Y.S. Park, D. Juul, M. Ott, and I. Harris et al. Technical Skills Assessment Toolbox: A Review Using the Unitary Framework of Validity Ann Surg 261 2015 251 262
25. S. Messick Validity of Psychological Assessment. Validation of Inferences from Persons' Responses and Performances as Scientific Inquiry into Score Meaning Am Psychol 50 1995 741 749
26. M.A. Rosen, E. Salas, K.A. Wilson, H.B. King, M. Salisbury, and J.S. Augenstein et al. Measuring team performance in simulation-based training: adopting best practices for healthcare Simul Healthc 3 2008 33 41
27. S.R. Swing Assessing the ACGME general competencies: general considerations and assessment methods Acad Emerg Med 9 2002 1278 1288
28. J. Crossley, G. Humphris, and B. Jolly Assessing health professionals Med Educ 36 2002 800 804
29. M.R. Endsley Toward a theory of situation awareness in dynamic systems Human Factors 37 1995 32 64
30. A. Mishra, K. Catchpole, T. Dale, and P. McCulloch The influence of non-technical performance on technical outcome in laparoscopic cholecystectomy Surg Endosc 22 2008 68 73
31. K. Pauley, R. Flin, S. Yule, and G. Youngson Surgeons' intraoperative decision making and risk management Am J Surg 202 2011 375 381
32. N.J. Dedy, E.M. Bonrath, B. Zevin, and T.P. Grantcharov Teaching nontechnical skills in surgical residency: a systematic review of current approaches and outcomes Surgery 154 2013 1000 1008
33. P. McCulloch, J. Rathbone, and K. Catchpole Interventions to improve teamwork and communications among healthcare staff Br J Surg 98 2011 469 479
34. E. Salas, D.E. Sims, and C.S. Burke Is there a "big five" in teamwork? Small Group Res 36 2005 555 599
35. L. Lingard, G. Regehr, C. Cartmill, B. Orser, S. Espin, and J. Bohnen et al. Evaluation of a preoperative team briefing: a new communication routine results in improved clinical practice BMJ Qual Saf 20 2011 475 482
36. J. Neily, P.D. Mills, Y. Young-Xu, B.T. Carney, P. West, and D.H. Berger et al. Association between implementation of a medical team training program and surgical mortality JAMA 304 2010 1693 1700
37. The Joint Commission. Sentinel Event Data - Root Causes by Event Type: 2004 - June 2013. 2013; Available from: http://www.jointcommission.org/assets/1/18/Root-Causes-by-Event-Type-2004-2Q2013.pdf. Accessed February 7, 2015.
38. S. Yule, R. Flin, N. Maran, G. Youngson, A. Mitchell, and D. Rowley et al. Debriefing surgeons on non-technical skills (NOTSS) Cogn Technol Work 10 2008 265 274
39. J. Crossley, J. Marriott, H. Purdie, and J.D. Beard Prospective observational study to evaluate NOTSS (Non-Technical Skills for Surgeons) for assessing trainees' non-technical performance in the operating theatre Br J Surg 98 2011 1010 1020
40. L.M. Napolitano, M. Savarise, J.C. Paramo, L.C. Soot, S.R. Todd, and J. Gregory et al. Are general surgery residents ready to practice? A survey of the american college of surgeons board of governors and young fellows association J Am Coll Surg 218 2014 1063 1072.e31
41. A. Yaghoubian, J. Galante, A. Kaji, M. Reeves, M. Melcher, and A. Salim et al. General surgery resident remediation and attrition: a multi-institutional study Arch Surg 147 2012 829 833
42. M.A. Papadakis, G.K. Arnold, L.L. Blank, E.S. Holmboe, and R.S. Lipner Performance during internal medicine residency training and subsequent disciplinary action by state licensing boards Ann Intern Med 148 2008 869 876
43. S.M. Downing Reliability: on the reproducibility of assessment data Med Educ 38 2004 1006 1012