Simulation-based assessment to identify critical gaps in safe anesthesia resident performance

Anesthesiology

Volumn 120, Issue 1, 2014, Pages 129-141

  Blum, Richard H ^a   Boulet, John R ^b   Cooper, Jeffrey B ^c   Muret Wagstaff, Sharon L ^d  

^a BOSTON CHILDREN S HOSPITAL   (United States)

^b FOUNDATION FOR ADVANCEMENT OF INTERNATIONAL MEDICAL EDUCATION AND RESEARCH   (United States)

^c MASSACHUSETTS GENERAL HOSPITAL   (United States)

^d HARVARD MEDICAL SCHOOL   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACCREDITATION; ANESTHESIA; ARTICLE; COMMUNICATION SKILL; DELPHI STUDY; FEMALE; HUMAN; MALE; MEDICAL EDUCATION; MEDICAL STAFF; POSTOPERATIVE PERIOD; PREOPERATIVE PERIOD; PRIORITY JOURNAL; RATING SCALE; RELIABILITY; RESIDENT; SIMULATION; VALIDITY; ADULT; ANESTHESIOLOGY; AUDIOVISUAL EQUIPMENT; CLINICAL COMPETENCE; EDUCATION; INFORMATION PROCESSING; INTERPERSONAL COMMUNICATION; LEARNING; METHODOLOGY; OPERATING ROOM; OPERATING ROOM PERSONNEL; ORGANIZATION AND MANAGEMENT; PATIENT SAFETY; PILOT STUDY; PSYCHOMETRY; REPRODUCIBILITY; STANDARD; SURGERY; TEACHING;

ADULT; ANESTHESIA; ANESTHESIOLOGY; CLINICAL COMPETENCE; COMMUNICATION; DATA COLLECTION; FEMALE; HUMANS; INTERNSHIP AND RESIDENCY; LEARNING; MALE; MANIKINS; OPERATING ROOM TECHNICIANS; OPERATING ROOMS; PATIENT SAFETY; PATIENT SIMULATION; PILOT PROJECTS; PSYCHOMETRICS; REPRODUCIBILITY OF RESULTS; SURGICAL PROCEDURES, OPERATIVE;

EID: 84894650167     PISSN: 00033022     EISSN: 15281175     Source Type: Journal    
DOI: 10.1097/ALN.0000000000000055     Document Type: Article

References (50)

1. Mudumbai SC, Gaba DM, Boulet JR, Howard SK, Davies MF: External validation of simulation-based assessments with other performance measures of third-year anesthesiology residents. Simul Healthc 2012; 7: 73-80
2. Berkenstadt H, Ben-Menachem E, Dach R, Ezri T, Ziv A, Rubin O, Keidan I: Deficits in the provision of cardiopulmonary resuscitation during simulated obstetric crises: Results from the Israeli Board of Anesthesiologists. Anesth Analg 2012; 115: 1122-6
3. Ben-Menachem E, Ezri T, Ziv A, Sidi A, Brill S, Berkenstadt H: Objective Structured Clinical Examination-based assessment of regional anesthesia skills: The Israeli National Board Examination in Anesthesiology experience. Anesth Analg 2011; 112: 242-5
4. Flin R, Patey R, Glavin R, Maran N: Anaesthetists' non-technical skills. Br J Anaesth 2010; 105: 38-44
5. Fehr JJ, Boulet JR, Waldrop WB, Snider R, Brockel M, Murray DJ: Simulation-based assessment of pediatric anesthesia skills. Anesthesiology 2011; 115: 1308-15
6. Blum RH, Raemer DB, Carroll JS, Dufresne RL, Cooper JB: A method for measuring the effectiveness of simulation-based team training for improving communication skills. Anesth Analg 2005; 100: 1375-80
7. Malec JF, Torsher LC, Dunn WF, Wiegmann DA, Arnold JJ, Brown DA, Phatak V: The mayo high performance teamwork scale: Reliability and validity for evaluating key crew resource management skills. Simul Healthc 2007; 2: 4-10
8. Kogan JR, Holmboe ES, Hauer KE: Tools for direct observation and assessment of clinical skills of medical trainees: A systematic review. JAMA 2009; 302: 1316-26
9. Cook DA, Hatala R, Brydges R, Zendejas B, Szostek JH, Wang AT, Erwin PJ, Hamstra SJ: Technology-enhanced simulation for health professions education: A systematic review and meta-analysis. JAMA 2011; 306: 978-88
10. Nasca TJ, Philibert I, Brigham T, Flynn TC: The next GME accreditation system-Rationale and benefits. N Engl J Med 2012; 366: 1051-6
11. Hauer KE, Ciccone A, Henzel TR, Katsufrakis P, Miller SH, Norcross WA, Papadakis MA, Irby DM: Remediation of the deficiencies of physicians across the continuum from medical school to practice: A thematic review of the literature. Acad Med 2009; 84: 1822-32
12. Zbieranowski I, Takahashi SG, Verma S, Spadafora SM: Remediation of residents in difficulty: A retrospective 10-year review of the experience of a postgraduate board of examiners. Acad Med 2013; 88: 111-6
13. Kane MT: Validation, Educational Measurement, 4th edition. Edited by Brennan RL. Westport, Praeger, 2006, pp 17-64
14. Boulet JR, Jeffries PR, Hatala RA, Korndorffer JR Jr, Feinstein DM, Roche JP: Research regarding methods of assessing learning outcomes. Simul Healthc 2011; 6(suppl): S48-51
15. S haw S, Crisp V, Johnson N: A framework for evidencing assessment validity in large-scale, high-stakes international examinations. Assess Educ Princ Pol Pract 2012; 19: 159-76
16. Joint Council on Anesthesiology Examinations; American Board of Anesthesiology, American Society of Anesthesiologists: Content Outline Basic/Advanced. Raleigh, American Board of Anesthesiology, 2012
17. Accreditation Council for Graduate Medical Education: ACGME Program Requirements for Graduate Medical Education in Anesthesiology, effective July 1, 2008, rev. July 1, 2011. Chicago, ACGME, 2011
18. Preston CC, Colman AM: Optimal number of response categories in rating scales: Reliability, validity, discriminating power, and respondent preferences. Acta Psychol (Amst) 2000; 104: 1-15
19. American Educational Research Association, American Psychological Association, National Council on Measurement in Education: Standards for Educational and Psychological Testing. Washington, American Educational Research Association, 1999, pp 7-68, 137-50
20. DeVellis RF: Scale Development. Theory and Applications, 3rd edition. Los Angeles, Sage, 2012, pp 31-114, 185-92
21. Brennan RL: Generalizability Theory. New York, Springer- Verlag, 2001, pp 4-20, 53-140
22. Boulet JR: Generalizability theory: Basics, Encyclopedia of Statistics in Behavioral Science. Edited by Evritt BS, Howell DC. Chichester, Wiley, 2005, pp 704-11
23. Mercer SJ, Moneypenny MJ, Fredy O, Guha A: What should be included in a simulation course for anaesthetists? The Merseyside trainee perspective. Eur J Anaesthesiol 2012; 29: 137-42
24. Glavin RJ: Excellence in anesthesiology: The role of nontechnical skills. Anesthesiology 2009; 110: 201-3
25. Weller JM, Jolly B, Misur MP, Merry AF, Jones A, Crossley JG, Pedersen K, Smith K: Mini-clinical evaluation exercise in anaesthesia training. Br J Anaesth 2009; 102: 633-41
26. Fletcher G, Flin R, McGeorge P, Glavin R, Maran N, Patey R: Anaesthetists' Non-Technical Skills (ANTS): Evaluation of a behavioural marker system. Br J Anaesth 2003; 90: 580-8
27. Goldszmidt M, Minda JP, Bordage G: Developing a unified list of physicians' reasoning tasks during clinical encounters. Acad Med 2013; 88: 390-7
28. Regehr G, Ginsburg S, Herold J, Hatala R, Eva K, Oulanova O: Using "standardized narratives" to explore new ways to represent faculty opinions of resident performance. Acad Med 2012; 87: 419-27
29. Carraccio CL, Englander R: From Flexner to competencies: Reflections on a decade and the journey ahead. Acad Med 2013; 88: 1067-73
30. Murray DJ, Boulet JR, Avidan M, Kras JF, Henrichs B, Woodhouse J, Evers AS: Performance of residents and anesthesiologists in a simulation-based skill assessment. Anesthesiology 2007; 107: 705-13
31. Adler MD, Vozenilek JA, Trainor JL, Eppich WJ, Wang EE , Beaumont JL, Aitchison PR, Pribaz PJ, Erickson T, Edison M, McGaghie WC: Comparison of checklist and anchored global rating instruments for performance rating of simulated pediatric emergencies. Simul Healthc 2011; 6: 18-24
32. Margolis MJ, Clauser BE, Swanson DB, Boulet JR: Analysis of the relationship between score components on a standardized patient clinical skills examination. Acad Med 2003; 78(10 suppl): S68-71
33. Henrichs BM, Avidan MS, Murray DJ, Boulet JR, Kras J, Krause B, Snider R, Evers AS: Performance of certified registered nurse anesthetists and anesthesiologists in a simulation-based skills assessment. Anesth Analg 2009; 108: 255-62
34. van Zanten M, Boulet JR, McKinley D: Using standardized patients to assess the interpersonal skills of physicians: Six years' experience with a high-stakes certification examination. Health Commun 2007; 22: 195-205
35. McBride ME, Waldrop WB, Fehr JJ, Boulet JR, Murray DJ: Simulation in pediatrics: The reliability and validity of a multiscenario assessment. Pediatrics 2011; 128: 335-43
36. DeAnda A, Gaba DM: Role of experience in the response to simulated critical incidents. Anesth Analg 1991; 72: 308-15
37. Schwid HA, Rooke GA, Carline J, Steadman RH, Murray WB, Olympio M, Tarver S, Steckner K, Wetstone S; Anesthesia Simulator Research Consortium: Evaluation of anesthesia residents using mannequin-based simulation: A multiinstitutional study. Anesthesiology 2002; 97: 1434-44
38. Friedman Z, Siddiqui N, Katznelson R, Devito I, Davies S: Experience is not enough: Repeated breaches in epidural anesthesia aseptic technique by novice operators despite improved skill. Anesthesiology 2008; 108: 914-20
39. E ricsson KA, Nandagopal K, Roring RW: Toward a science of exceptional achievement: Attaining superior performance through deliberate practice. Ann N Y Acad Sci 2009; 1172: 199-217
40. Fraser SA, Feldman LS , Stanbridge D, Fried GM: Characterizing the learning curve for a basic laparoscopic drill. Surg Endosc 2005; 19: 1572-8
41. Choudhry NK, Fletcher RH, Soumerai SB: Systematic review: The relationship between clinical experience and quality of health care. Ann Intern Med 2005; 142: 260-73
42. Weinger MB: Experience not equal expertise: Can simulation be used to tell the difference? Anesthesiology 2007; 107: 691-4
43. Steinmann AF: Threats to graduate medical education funding and the need for a rational approach: A statement from the alliance for academic internal medicine. Ann Intern Med 2011; 155: 461-4
44. Holmboe E, Rizzolo MA, Sachdeva AK, Rosenberg M, Ziv A: Simulation-based assessment and the regulation of healthcare professionals. Simul Healthc 2011; 6(suppl): S58-62
45. Boulet JR, Murray D, Kras J, Woodhouse J: Setting performance standards for mannequin-based acute-care scenarios: An examinee-centered approach. Simul Healthc 2008; 3: 72-81
46. McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ: A critical review of simulation-based medical education research: 2003-2009. Med Educ 2010; 44: 50-63
47. S teadman RH, Huang YM: Simulation for quality assurance in training, credentialing and maintenance of certification. Best Pract Res Clin Anaesthesiol 2012; 26: 3-15
48. Dieckmann P, Phero JC, Issenberg SB, Kardong-Edgren S, Ostergaard D, Ringsted C: The first Research Consensus Summit of the Society for Simulation in Healthcare: Conduction and a synthesis of the results. Simul Healthc 2011; 6(suppl): S1-9
49. Boulet JR, Murray DJ: Simulation-based assessment in anesthesiology: Requirements for practical implementation. Anesthesiology 2010; 112: 1041-52
50. Norcini J, Anderson B, Bollela V, Burch V, Costa MJ, Duvivier R, Galbraith R, Hays R, Kent A, Perrott V, Roberts T: Criteria for good assessment: Consensus statement and recommendations from the Ottawa 2010 Conference. Med Teach 2011; 33: 206-14