Going DEEP: Guidelines for building simulation-based team assessments

BMJ Quality and Safety

Volumn 22, Issue 5, 2013, Pages 436-448

  Grand, James A ^a   Pearce, Marina ^b   Rench, Tara A ^b   Chao, Georgia T ^b   Fernandez, Rosemarie ^c   Kozlowski, Steve W J ^b  

^a UNIVERSITY OF AKRON   (United States)

^b MICHIGAN STATE UNIVERSITY   (United States)

^c UNIVERSITY OF WASHINGTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLINICAL ASSESSMENT TOOL; CLINICAL PRACTICE; CONCEPTUAL FRAMEWORK; CONTENT VALIDITY; HUMAN; MEASUREMENT; PRACTICE GUIDELINE; PROFESSIONAL KNOWLEDGE; REVIEW;

BENCHMARKING; CLINICAL COMPETENCE; EMPLOYEE PERFORMANCE APPRAISAL; GUIDELINES AS TOPIC; HEALTH KNOWLEDGE, ATTITUDES, PRACTICE; HEALTH PERSONNEL; HUMANS; INSERVICE TRAINING; INTERPROFESSIONAL RELATIONS; PATIENT CARE TEAM; PATIENT SAFETY; PATIENT SIMULATION; REPRODUCIBILITY OF RESULTS;

EID: 84876556006     PISSN: 20445415     EISSN: None     Source Type: Journal    
DOI: 10.1136/bmjqs-2012-000957     Document Type: Review

References (61)

1. Kohn LT, Corrigan JM, Donaldson MS. To err is human. Washington, DC: National Academies Press, 1999.
2. Risser DT, Rice MM, Salisbury ML, et al. The potential for improved teamwork to reduce medical errors in the emergency department. Ann Emer Med 1999;34:373-83. (Pubitemid 29416031)
3. Clancy CM, Tornberg DN. TeamSTEPPS: assuring optimal teamwork in clinical settings. Am J Med Qual 2007;22:214-17.
4. Cheung DS, Kelly JJ, Beach C, et al. Improving handoffs in the emergency department. Ann Emer Med 2010;55:171-80.
5. Walker S, Brett S. Oiling the wheels of intensive care to reduce "machine friction:" the best way to improve outcomes? Crit Care Med 2010;38:S642-48.
6. Alonso A, Baker D, Holtzman A, et al. Reducing medical error in the military health system: how can team training help? Hum Resour Manag Rev 2006;16:396-415. (Pubitemid 44375478)
7. Baker DP, Salas E, King H, et al. The role of teamwork in the professional education of physicians: current status and assessment recommendations. Jt Comm J Qual Patient Saf 2005;31:185-202.
8. Salas E, DiazGranados D, Weaver SJ, et al. Does team training work? Principles for health care. Acad Emerg Med 2008;15:1002-9.
9. Kendall D, Salas E. Measuring team performance: review of current methods and consideratioon of future needs. In: Ness JW, Tepe V, Ritzer D, eds. The science and simulation of human performance: advances in human performance and cognitive engineering research. Philadelphia, PA: Elsevier, 2004: 307-26.
10. Kozlowski SWJ, Klein KJ. A multilevel approach to theory and research in organizations: contextual, temporal, and emergent processes. In: Klein KJ, Kozlowski SWJ, eds. Multilevel theory, research, and methods in organizations: foundations, extensions, and new directions. San Francisco, CA: Jossey-Bass, 2000: 3-90.
11. Rosen MA, Salas E, Silvestri S, et al. A measurement tool for simulation-based training in emergency medicine: the Simulation Module for Assessment of Resident Targeted Event Responses (SMARTER) approach. Simul Healthc 2008;3:170-9.
12. Shapiro MJ, Gardner R, Godwin SA, et al. Defining team performance for simulation-based training: methodology, metrics, and opportunities for emergency medicine. Acad Emerg Med 2008;15:1088-97.
13. Rosen MA, Salas E, Wu TS, et al. Promoting teamwork: an event-based approach to simulation-based teamwork training for emergency medicine residents. Acad Emerg Med 2008;15:1190-8.
14. Rosen MA, Salas E, Wilson KA, et al. Measuring team performance in simulation-based training: adopting best practices for healthcare. Simul Healthc 2008;3:33-41. (Pubitemid 351280864)
15. Bond WF, Lammers RL, Spillane LL, et al. The use of simulation in emergency medicine: a research agenda. Acad Emerg Med 2007;14:353-63. (Pubitemid 46464029)
16. Ziv A, Wolpe PR, Small SD, et al. Simulation-based medical education: an ethical imperative. Acad Med 2003;78:783-8. (Pubitemid 36993078)
17. Gaba DM. The future vision of simulation in health care. Qual Saf Health Care 2004;13(Suppl 1):i2-10.
18. Shapiro MJ, Morey JC, Small SD, et al. Simulation based teamwork training for emergency department staff: does it improve clinical team performance when added to an existing didactic teamwork curriculum? Qual Saf Health Care 2004;13:417-21. (Pubitemid 39647351)
19. Salas E, Wilson KA, Burke CS, et al. Using simulation-based training to improve patient safety: what does it take? Jt Comm J Qual Patient Saf 2005;31:363 -71.
20. Gaba DM, Howard SK, Flanagan B, et al. Assessment of clinical performance during simulated crises using both technical and behavioral ratings. Anesthesiology 1998;89:8-18. (Pubitemid 28319561)
21. Baker DP, Gustafson S, Beaubien J, et al. Medical teamwork and patient safety: the evidence-based relation. Publication No. 05-053, Agency for Healthcare Research and Quality, 2005. http://www.ahrq.gov/qual/medteam/ (accessed Feb 2012).
22. Henriksen K, Battles JB, Keyes MA, et al., eds. Advances in patient safety: new directions and alternative approaches (Vol. 3. Performance and Tools). Publication No. 08-0034-3, Agency for Healthcare Research and Quality, 2008. http://www.ahrq.gov/qual/advances2/#v3 (accessed Feb 2012).
23. Salas E, Rosen MA, King H. Managing teams managing crises: principles of teamwork to improve patient safety in the emergency room and beyond. Theor Issues Ergon Sci 2007;8:381-94.
24. Salas E, Wilson-Donnelly K, Sims D, et al. Teamwork training for patient safety: best practices and guiding principles. In: Carayon P, ed. Handbook of human factors and ergonomics in health care and patient safety. Mahwah, NJ: Erlbaum, 2007:803-22.
25. Cannon-Bowers JA, Tannenbaum SI, Salas E, et al. Defining competencies and establishing team training requirements. In: Guzzo R, Salas E, eds. Team effectiveness and decision making in organizations. San Francisco, CA: Jossey-Bass, 1995: 333-80.
26. Cannon-Bowers JA, Salas E. A framework for developing team performance measures in training. In: Brannick MT, Salas E, Prince C, eds. Team performance and measurement: theory, methods, and application. Mahwah, NJ: Erlbaum, 1997: 45-62.
27. Brannick MT, Prince C. An overview of team performance measurement. In: Brannick MT, Salas E, Prince C, eds. Team performance and measurement: theory, methods, and application. Mahwah, NJ: Erlbaum, 1997: 3-16.
28. Gladstein DL. Groups in context: A model of task group effectiveness. Adm Sci Q 1984;29:499-517.
29. Kozlowski SWJ, Ilgen DR. Enhancing the effectiveness of work groups and teams. Psychol Sci Public Interest 2006;7:77-124.
30. Fernandez R, Kozlowski SWJ, Shapiro MJ, et al. Toward a definition of teamwork in emergency medicine. Acad Emerg Med 2008;15:1104-2.
31. Marks MA, Mathieu JE, Zaccaro SJ. A temporally based framework and taxonomy of team processes. Acad Manage Rev 2001;26:356-76.
32. Salas E, Sims DE, Burke CS. Is there a "big five" in teamwork? Small Group Res 2005;36:555-99.
33. Yule S, Flin R, Paterson-Brown S, et al. Non-technical skills for surgeons in the operating room. A review of the literature. Surgery 2006;139:140-9. (Pubitemid 43170313)
34. Hull L, Arora S, Kassab E, et al. Observational teamwork assessment for surgery: content validation and tool refinement. J Am Coll Surg 2011;212:234-43.
35. Flin R, Maran N. Identifying and training non-technical skills for teams in acute medicine. Qual Saf Health Care 2004;13 (Suppl 1):i80-4. (Pubitemid 39389383)
36. Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med 2006;119:166.e7-.e16.
37. Murphy KR, Davidshofer CO. Psychological testing: practices and applications. Upper Saddle River, NJ: Pearson, 2005.
38. Hoyle R, Harris M, Judd C. Research Methods in Social Relations. Belmont, CA: Wadsworth, 2002.
39. Rosen MA, Bedwell WL, Wildman JL, et al. Managing adaptive performance in teams: guiding principles and behavioral markers for measurement. Hum Resour Manag Rev 2011;21:107-22.
40. Fletcher G, Flin R, McGeorge R, et al. Rating non-techincal skills: developing a behavioural marker system for use in anaesthesia. Cogn Technol Work 2004;6:165-71. (Pubitemid 39203628)
41. Flin R, Martin L. Behavioural markers for crew resource management: a survey of current practice. Int J Aviat Psychol 2001;11:95-118.
42. Krueger J. Enhancement bias in descriptions of self and others. Pers Soc Psychol Bull 1998;24:505-16.
43. Paulhus DL. Two-component models of socially desirable responding. J Pers Soc Psychol 1984;46:598-609.
44. Hinkin TR. A brief tutorial on the development of measures for use in survey questionnaires. Organ Res Methods 1998;1:104-21.
45. Spector PE. Summated rating scales. Thousand Oaks, CA: Sage, 1992.
46. Smith-Jentsch KA, Johnston JH, Payne SC. Measuring team-related expertise in complex environments. In: Cannon-Bowers JA, Salas E, eds. Making decisions under stress: implications for individual and team training. Washington, DC: American Psychological Association, 1998: 61-87.
47. Prince A, Brannick MT, Prince C, et al. The measurement of team process behaviors in the cockpit: lessons learned. In: Brannick MT, Salas E, Prince C, eds. Team performance assessment and measurement: theory, methods, and applications. Mahwah, NJ: Lawrence Erlbaum Associates Publishers, 1997: 289-310.
48. Subject matter experts and VV&A. United States Department of Defense, 2000. http://vva.msco.mil/Special-Topics/SME/sme-pr.pdf (accessed Feb 2012).
49. Alinier G. Developing high-fidelity health care simulation scenarios: a guide for educators and professionals. Simul Gaming 2011;42:9-26.
50. Kozlowski SWJ, DeShon RP. A psychological fidelity approach to simulation-based training: theory, research, and principles. In: Schiflett SG, Elliot LR, Salas E, Coovert MD, eds. Scaled worlds: development, validation and applications. Burlington, VT: Ashgate Publishing, 2004: 75-99.
51. Neily J, Mills PD, Young-Xu YN, et al. Association between implementation of a medical team training program and surgical mortality. JAMA 2010;304:1693-700.
52. Salas E, Weaver SJ, DiazGranados D, et al. Sounding the call for team training in health care: some insights and warnings. Acad Med 2009;84:S128-31.
53. Hoyt WT. Rater bias in psychological research: when is it a problem and what can we do about it? Psychol Methods 2000;5:64-86.
54. Lievens F, Sanchez JI. Can training improve the quality of inferences made by raters in competency modeling? A quasi-experiment. J Appl Psychol 2007;92:812-19.
55. Uggerslev KL, Sulsky LM. Using frame-of-reference training to understand the implications of rater idiosyncrasy for rating accuracy. J Appl Psychol 2008;93:711-19.
56. Day DV, Sulsky LM. Effects of frame-of-reference training and information configuration on memory organization and rating accuracy. J Appl Psychol 1995;80:158-67.
57. Bernardin HJ, Buckley MR. Strategies in rater training. Acad Manage Rev 1981;6:205-12.
58. Thornton GC, Zorich S. Training to improve observer accuracy. J Appl Psychol 1980;65:351-4.
59. Castorr AH, Thompson KO, Ryan JW, et al. The process of rater training for observational instruments: implications for intterater reliability. Res Nurs Health 1990;13:311-18.
60. Bliese P. Within-group agreement, non-independence, and reliability: implications for data aggregation and analysis. In: Klein KJ, Kozlowski SWJ, eds. Multilevel theory, research, and methods in organizations: foundations, extensions, and new directions. San Francisco, CA: Jossey-Bass, 2000:349-81.
61. Klein KJ, Bliese P, Kozlowski SWJ, et al.Multilevel analytical techniques: commonalities, differences, and continuing questions. In: Klein KJ, Kozlowski SWJ, eds. Multilevel theory, research, and methods in organizations: foundations, extensions, and new directions. San Francisco, CA: Jossey-Bass, 2000: 512-56.