Debriefing assessment for simulation in healthcare: Development and psychometric properties

Simulation in Healthcare

Volumn 7, Issue 5, 2012, Pages 288-294

  Brett Fleegler, Marisa ^a,b   Rudolph, Jenny ^b,c,d   Eppich, Walter ^e   Monuteaux, Michael ^a,b   Fleegler, Eric ^a,b   Cheng, Adam ^f   Simon, Robert ^b,c,d  

^a BOSTON CHILDREN S HOSPITAL   (United States)

^b HARVARD MEDICAL SCHOOL   (United States)

^c MASSACHUSETTS GENERAL HOSPITAL   (United States)

^d Center for Medical Simulation Cambridge   (United States)

^e NORTHWESTERN UNIVERSITY   (United States)

^f UNIVERSITY OF CALGARY   (Canada)

Author keywords

Assessment; Behaviorally anchored rating scale; Debriefing; Health care education; Medical education; Psychometrics; Simulation

Indexed keywords

ARTICLE; CANADA; CHECKLIST; COMPUTER INTERFACE; COMPUTER SIMULATION; COOPERATION; EDUCATION; HUMAN; MEDICAL STAFF; METHODOLOGY; OBSERVER VARIATION; PEDIATRICS; PILOT STUDY; PROFESSIONAL COMPETENCE; PSYCHOMETRY; REPRODUCIBILITY; RESUSCITATION; STANDARD; UNITED STATES; VALIDATION STUDY;

CANADA; CHECKLIST; COMPUTER SIMULATION; EDUCATIONAL MEASUREMENT; HELPING BEHAVIOR; HUMANS; MEDICAL STAFF, HOSPITAL; OBSERVER VARIATION; PEDIATRICS; PILOT PROJECTS; PROFESSIONAL COMPETENCE; PSYCHOMETRICS; REPRODUCIBILITY OF RESULTS; RESUSCITATION; UNITED STATES; USER-COMPUTER INTERFACE;

EID: 84867204484     PISSN: 15592332     EISSN: 1559713X     Source Type: Journal    
DOI: 10.1097/SIH.0b013e3182620228     Document Type: Article

References (67)

1. Harden RM. Trends and the future of postgraduate medical education. Emerg Med J 2006;23:798-802.
2. Issenberg SB, McGaghie WC, Hart IR, et al. Simulation technology for health care professional skills training and assessment. JAMA 1999;282:861-866.
3. POISE. 2010. Available at: http://www.members.poisenetwork.com/. Accessed December 9, 2010.
4. Cheng A, Nadkarni V, Hunt E, Qayumi K, EXPRESS Investigators. A multifunctional online research portal for facilitation of simulation-based research: A report from the EXPRESS pediatric simulation research collaborative. Simul Healthc 2011;6:239-243.
5. Cheng A, Hunt EA, Donoghue A, et al. EXPRESSVExamining Pediatric Resuscitation Education Using Simulation and Scripting. The birth of an international pediatric simulation research collaborativeVfrom concept to reality. Simul Healthc 2011;6: 34-41.
6. International Pediatric Simulation Society. 2011. Available at: http://www.ipssw2011.com/. Accessed April 25, 2011.
7. Cook DA, Hatala R, Brydges R, et al. Technology-enhanced simulation for health professions education: A systematic review and meta-analysis. JAMA 2011;306:978-988.
8. Nishisaki A, Nguyen J, Colborn S, et al. Evaluation of multidisciplinary simulation training on clinical performance and team behavior during tracheal intubation procedures in a pediatric intensive care unit. Pediatr Crit Care Med 2011;12:406-414.
9. Eppich WJ, Adler MD, McGaghie WC. Emergency and critical care pediatrics: Use of medical simulation for training in acute pediatric emergencies. Curr Opin Pediatr 2006;18:266-271.
10. Volk MS, Ward J, Irias N, Navedo A, Pollart J, Weinstock PH. Using medical simulation to teach crisis resource management and decision-making skills to otolaryngology housestaff. Otolaryngol Head Neck Surg 2011;145:35-42.
11. Halamek LP. Teaching versus learning and the role of simulation-based training in pediatrics. J Pediatr 2007;151:329-330.
12. Cheng A, Duff J, Grant E, Kissoon N, Grant VJ. Simulation in paediatrics: An educational revolution. Paediatr Child Health 2007;12:465-468.
13. Adler MD, Trainor JL, Siddall VJ, McGaghie WC. Development and evaluation of high-fidelity simulation case scenarios for pediatric resident education. Ambul Pediatr 2007;7:182-186.
14. Nishisaki A, Hales R, Biagas K, et al. A multi-institutional high-fidelity simulation ''boot camp'' orientation and training program for first year pediatric critical care fellows. Pediatr Crit Care Med 2009;10:157-162.
15. Shilkofski NA, Nelson KL, Hunt EA. Recognition and treatment of unstable supraventricular tachycardia by pediatric residents in a simulation scenario. Simul Healthc 2008;3:4-9.
16. Brett-Fleegler MB, Vinci RJ, Weiner DL, Harris SK, Shih MC, Kleinman ME. A simulator-based tool that assesses pediatric resident resuscitation competency. Pediatrics 2008;121:e597Ye603.
17. Weinstock PH, Kappus LJ, Garden A, Burns JP. Simulation at the point of care: Reduced-cost, in situ training via a mobile cart. Pediatr Crit Care Med 2009;10:176-181.
18. Nishisaki A, Donoghue AJ, Colborn S, et al. Effect of just-in-time simulation training on tracheal intubation procedure safety in the pediatric intensive care unit. Anesthesiology 2010;113:214-223.
19. Cooper JB, Singer SJ, Hayes J, et al. Design and evaluation of simulation scenarios for a program introducing patient safety, teamwork, safety leadership, and simulation to healthcare leaders and managers. Simul Healthc 2011;6:231-238.
20. Morgan PJ, Kurrek MM, Bertram S, Leblanc V, Przybyszewski T. Nontechnical skills assessment after simulation-based continuing medical education. Simul Healthc 2011;6:255-259.
21. Hunt EA, Hohenhaus SM, Luo X, Frush KS. Simulation of pediatric trauma stabilization in 35 North Carolina emergency departments: Identification of targets for performance improvement. Pediatrics 2006;117:641-648.
22. Dismukes RK, Gaba DM, Howard SK. So many roads: facilitated debriefing in healthcare. Simul Healthc 2006;1:23-25.
23. Fanning RM, Gaba DM. The role of debriefing in simulation-based learning. Simul Healthc 2007;2:115-125.
24. Van Heukelom JN, Begaz T, Treat R. Comparison of postsimulation debriefing versus in-simulation debriefing in medical simulation. Simul Healthc 2010;5:91-97.
25. Rudolph JW, Simon R, Raemer DB, Eppich W. Debriefing as formative assessment: Closing performance gaps in medical attention. Acad Emerg Med 2008;15:1110-1116.
26. Kolb DA. Experiential Learning: Experience as the Source of Learning and Development. Englewood Cliffs, NJ: Prentice Hall; 1984.
27. Issenberg SB, McGaghie WC, Petrusa ER, Gordon DL, Scalese RJ. What are the features and uses of high-fidelity medical simulations that lead to most effective learning: A BEME systematic review. Med Teach 2005;27:10-12.
28. Schön D. Educating the Reflective Practitioner: Toward a New Design for Teaching and Learning in the Professions. San Francisco, CA: Jossey-Bass; 1987.
29. Rudolph JW, Simon R, Dufresne RL, Raemer DB. There's no such thing as a ''nonjudgmental'' debriefing: A theory and method for debriefing with good judgment. Simul Healthc 2006;1:49-55.
30. Rudolph JW, Simon R, Rivard P, Dufresne RL, Raemer DB. Debriefing with good judgment: Combining rigorous feedback with genuine inquiry. Anesthesiol Clin 2007;25:361-376.
31. Dismukes RK, McDonnell LK, Jobe KK. Facilitating LOFT debriefings: Instructor techniques and crew participation. Int J Aviat Psychol 2000;10:35-57.
32. Bylund C, Brown R, Lubrano di Ciccone B, Diamond C, Eddington J, Kissane DW. Assessing facilitator competence in a comprehensive communication skills training programme. Med Educ 2009;43:342-349.
33. Gururaja RP, Yang T, Paige JT, Chauvin SW. Examining the effectiveness of debriefing at the point of care in simulation-based operating room team training. In: Henriksen K, Battles JB
34. Keyes MA, Gary ML, eds. Advances in Patient Safety: New Directions and Alternative Approaches (Vol 3: Performance and Tools). Rockville, MD: Agency for Healthcare Research and Quality; 2008.
35. Reed SJ. Debriefing experience scale: Development of a tool to evaluate the student learning experience in debriefing. Clin Simul Nurs 2012;8(6):e211Ye217.
36. Arora S, Ahmed M, Paige J, et al. Objective Structured Assessment of Debriefing (OSAD): Bringing science to the art of debriefing in surgery. Ann Surg August 14, 2012 [epub ahead of print].
37. American Educational Research Association, APA, and National Council on Measurement in Education. Standards for Educational and Psychological Testing. Washington, DC: AERA; 1999.
38. Simon R, Rudolph JW, Raemer DB. Debriefing Assessment for Simulation in Healthcare. Cambridge, MA; 2009. Available at: http:// www.harvardmedsim.org/ debriefing-assesment-simulation-healthcare.php
39. Smith PC, Kendall LM. Retranslation of expectations: An approach to the construction of unambiguous anchors for rating scales. J Appl Psychol 1963;47:149-155.
40. Shapira Z, Shirom A. New issues in the use of behaviorally anchored rating scales: Level of analysis, the effect of incidence frequency, and external validation. J Appl Psychol 1980;65: 517-523.
41. Jacobs RAU, Kafry D, Zedeck S. Expectations of behaviorally anchored rating scales. Pers Psychol 1980;33:595-640.
42. Argyris C, Putnam R, Smith DM. Action Science: Concepts, Methods and Skills for Research and Intervention. San Francisco, CA: Jossey-Bass; 1985.
43. Edmondson A. Psychological safety and learning behavior in work teams. Adm Sci Q 1999;44:350-383.
44. Knowles MS, Holton EF, Swanson RA. The Adult Learner: The Definitive Classic in Adult Education and Human Resource Development. 6th ed. Burlington, MA: Elsevier; 2005.
45. Watzlawick P, Weakland JH, Fisch R. Change: Principles of Problem Formation and Problem Resolution. New York, NY: Horton; 1974.
46. Darling M, Parry C, Moore J. Learning in the thick of it. Harv Bus Rev 2005;83:84-92, 192.
47. Ende J. Feedback in clinical medical education. JAMA 1983;250:777-781.
48. Harlen W, James M. Assessment and learning: Differences and relationship between formative and summative assessment. Assess Educ Princ Pol Pract 1997;4:365-377.
49. Vaughan D. Theory elaboration: The heuristics of case analysis. In: Becker H, Ragin C, eds. What Is a Case? New York, NY: Cambridge University Press; 1992:173-202.
50. Vaughan D. The Challenger Launch Decision: Risky Technology, Culture and Deviance at NASA. Chicago, IL: University of Chicago Press; 1996.
51. Davis JP, Eisenhardt KM, Bingham CB. Developing theory through simulation methods. Acad Manage Rev 2007;32:480-499.
52. Martin JA, Regehr G, Reznick R, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997;84:273-278.
53. Regehr G, MacRae H, Reznick RK, Szalay D. Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format examination. Acad Med 1998;73:993-997.
54. Hodges B, Regehr G, McNaughton N, Tiberius R, Hanson M. OSCE checklists do not capture increasing levels of expertise. Acad Med 1999;74:1129-1134.
55. Berk RA. A Guide to Criterion-Referenced Test Construction. Baltimore, MD: The Johns Hopkins University Press; 1984.
56. Dumas JC, Redish JC. A Practical Guide to Usability Testing. 2nd ed. Bristol, UK: Intellect; 1999.
57. Weisinger H. The Critical Edge: How to Criticize Up and Down Your Organization and Make It Pay Off. New York, NY: Little Brown & Co; 1989.
58. Weisinger H. The Power of Positive Criticism. New York: AMACOM; 2000.
59. Kegan R, Lahey LL. How the Way We Talk Can Change the Way We Work. San Francisco, CA: Jossey-Bass; 2001.
60. Bernardin HJ, Buckley MR. Strategies in rater training. Acad Manage Rev 1982;6:206-212.
61. Lederman LC. Toward a systematic assessment of theory and practice. Simul Gaming 1992;23:145-160.
62. Steinwachs B. How to facilitate a debriefing. Simul Gaming 1992;23:186-195.
63. Savoldelli GL, Naik VN, Park J, Joo HS, Chow R, Hamstra SJ. Value of debriefing during simulated crisis management: Oral versus video-assisted oral feedback. Anesthesiology 2006;105: 279-285.
64. Draycott TJ, Crofts JF, Ash JP, et al. Improving neonatal outcome through practical shoulder dystocia training. Obstet Gynecol 2008;112:14-20.
65. Crofts JF, Fox R, Ellis D, Winter C, Hinshaw K, Draycott TJ. Observations from 450 shoulder dystocia simulations: Lessons for skills training. Obstet Gynecol 2008;112:906-912.
66. Siassakos D, Draycott T, O'Brien K, Kenyon C, Bartlett C, Fox R. Exploratory randomized controlled trial of hybrid obstetric simulation training for undergraduate students. Simul Healthc 2010;5:193-198.
67. Donoghue A, Ventre K, Boulet J, Brett-Fleegler M, Nishisaki A, Overly F, Cheng A; Donoghue A. EXPRESS Pediatric Simulation Research Investigators. Simul Healthc 2011 Apr;6(2):71-77.