Tools for the direct observation and assessment of psychomotor skills in medical trainees: A systematic review

Medical Education

Volumn 47, Issue 7, 2013, Pages 650-673

  Jelovsek, J Eric ^a   Kow, Nathan ^a   Diwadkar, Gouri B ^b  

^a LERNER RESEARCH INSTITUTE   (United States)

^b Kaiser Permanente   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLINICAL COMPETENCE; COGNITION; EDUCATION; HUMAN; MEDICAL EDUCATION; MOTOR PERFORMANCE; PROCEDURES; REPRODUCIBILITY; STANDARDS; METHODOLOGY; REVIEW; STANDARD;

CLINICAL COMPETENCE; COGNITION; EDUCATION, MEDICAL, GRADUATE; EDUCATIONAL MEASUREMENT; HUMANS; MOTOR SKILLS; REPRODUCIBILITY OF RESULTS;

EID: 84878965573     PISSN: 03080110     EISSN: 13652923     Source Type: Journal    
DOI: 10.1111/medu.12220     Document Type: Article

References (71)

1. Kirk RM. Basic Surgical Techniques, 5th edn. Edinburgh: Churchill Livingstone 2002.
2. Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990;65 (Suppl 9):63-7.
3. van der Vleuten CPM. The assessment of professional competence: developments, research and practical implications. Adv Health Sci Educ 1996;1 (1):41-67.
4. van der Vleuten CPM, Schuwirth LW. Assessing professional competence: from methods to programmes. Med Educ 2005;39 (3):309-17.
5. Swing SR, Clyman SG, Holmboe ES, Williams RG. Advancing resident assessment in graduate medical education. J Grad Med Educ 2009;1:278-86.
6. Kogan JR, Holmboe ES, Hauer KE. Tools for direct observation and assessment of clinical skills of medical trainees: a systematic review. JAMA 2009;302 (12):1316-26.
7. Hammick M, Dornan T, Steinert Y. Conducting a best evidence systematic review. Part 1: from idea to data coding. BEME guide no. 13. Med Teach 2010;32 (1):3-15.
8. Moher D, Liberati A, Tetzlaff J, Altman DG, for the PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol 2009;62 (10):1006-12.
9. Nestel D, Walker K, Simon R, Aggarwal R, Andreatta P. Non-technical skills: an inaccurate and unhelpful descriptor? Simul Healthc 2011;6 (1):2-3.
10. American Psychological Association, American Educational Research Association, and National Council for Measurement in Education. The Standards for Educational and Psychological Testing. Washington, DC: AERA Publishing 1999.
11. Aaronson N, Alonso J, Burnam A, Lohr KN, Patrick DL, Perrin E, Stein REK. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res 2002;11 (3):193-205.
12. Banks E, Pardanani S, King M, Chudnoff S, Damus K, Freda MC. A surgical skills laboratory improves residents' knowledge and performance of episiotomy repair. Am J Obstet Gynecol 2006;195 (5):1463-7.
13. Banks EH, Chudnoff S, Karmin I, Wang C, Pardanani S. Does a surgical simulator improve resident operative performance of laparoscopic tubal ligation? Am J Obstet Gynecol 2007;197 (5):541 e1-e5.
14. Calatayud D, Arora S, Aggarwal R, Kruglikova I, Schulze S, Funch-Jensen P, Grantcharov T. Warm-up in a virtual reality environment improves performance in the operating room. Ann Surg 2010;251 (6):1181-5.
15. Naik VN, Matsumoto ED, Houston PL, Hamstra SJ, Yeung RY, Mallon JS, Martire TM. Fibreoptic orotracheal intubation on anaesthetised patients: do manipulation skills learned on a simple model transfer into the operating room? Anesthesiology 2001;95 (2):343-8.
16. Martin JA, Regehr G, Reznick R, MacRae H, Murnaghan J, Hutchison C, Brown M. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997;84 (2):273-8.
17. Aggarwal R, Grantcharov T, Moorthy K, Milland T, Papasavas P, Dosis A, Bello F, Darzi A. An evaluation of the feasibility, validity, and reliability of laparoscopic skills assessment in the operating room. Ann Surg 2007;245 (6):992-9.
18. Beard JD, Choksy S, Khan S, for the Vascular Society of Great Britain and Ireland. Assessment of operative competence during carotid endarterectomy. Br J Surg 2007;94 (6):726-30.
19. Chen CC, Korn A, Klingele C, Barber MD, Paraiso MF, Walters MD, Jelovsek JE. Objective assessment of vaginal surgical skills. Am J Obstet Gynecol 2010;203 (1):79.e1-8.
20. Jelovsek JE, Walters MD, Korn A, Klingele C, Zite N, Ridgeway B, Barber MD. Establishing cut-off scores on assessments of surgical skills to determine surgical competence. Am J Obstet Gynecol 2010;203 (1):81.e1-6.
21. Naik VN, Perlas A, Chandra DB, Chung DY, Chan VW. An assessment tool for brachial plexus regional anaesthesia performance: establishing construct validity and reliability. Reg Anesth Pain Med 2007;32 (1):41-5.
22. Bodle JF, Kaufmann SJ, Bisson D, Nathanson B, Binney DM. Value and face validity of objective structured assessment of technical skills (OSATS) for work-based assessment of surgical skills in obstetrics and gynaecology. Med Teach 2008;30 (2):212-6.
23. Datta V, Bann S, Beard J, Mandalia M, Darzi A. Comparison of bench test evaluations of surgical skill with live operating performance assessments. J Am Coll Surg 2004;199 (4):603-6.
24. Friedman Z, Katznelson R, Devito I, Siddiqui M, Chan V. Objective assessment of manual skills and proficiency in performing epidural anaesthesia - video-assisted validation. Reg Anesth Pain Med 2006;31 (4):304-10.
25. Rowley D, Pitts D, Galasko C. The Competence Assessment Working Party Report for the Joint Committee on Higher Surgical Training. Edinburgh: JCHST Publications 2002.
26. Beard JD, for the Education and Training Committee of the Vascular Society of Great Britain and Ireland. Setting standards for the assessment of operative competence. Eur J Vasc Endovasc Surg 2005;30 (2):215-8.
27. Beard JD, Jolly BC, Newble DI, Thomas WE, Donnelly J, Southgate LJ. Assessing the technical skills of surgical trainees. Br J Surg 2005;92 (6):778-82.
28. James K, Cross K, Lucarotti ME, Fowler AL, Cook TA. Undertaking procedure-based assessment is feasible in clinical practice. Ann R Coll Surg Engl 2009;91 (2):110-2.
29. Marriott J, Purdie H, Crossley J, Beard JD. Evaluation of procedure-based assessment for assessing trainees' skills in the operating theatre. Br J Surg 2011;98 (3):450-7.
30. Chan WY, Niranjan N, Ramakrishnan V. Structured assessment of microsurgery skills in the clinical setting. J Plast Reconstr Aesthet Surg 2010;63 (8):1329-34.
31. Chang L, Hogle NJ, Moore BB, Graham MJ, Sinanan MN, Bailey R, Fowler DL. Reliable assessment of laparoscopic performance in the operating room using videotape analysis. Surg Innov 2007;14 (2):122-6.
32. Cremers SL, Lora AN, Ferrufino-Ponce ZK. Global rating assessment of skills in intraocular surgery (GRASIS). Ophthalmology 2005;112 (10):1655-60.
33. Doyle JD, Webber EM, Sidhu RS. A universal global rating scale for the evaluation of technical skills in the operating room. Am J Surg 2007;193 (5):551-5; discussion 555.
34. Francis HW, Masood H, Chaudhry KN, Laeeq K, Carey JP, Della Santina CC, Limb CJ, Niparko JK, Bhatti NI. Objective assessment of mastoidectomy skills in the operating room. Otol Neurotol 2010;31 (5):759-65.
35. Gumbs AA, Hogle NJ, Fowler DL. Evaluation of resident laparoscopic performance using global operative assessment of laparoscopic skills. J Am Coll Surg 2007;204 (2):308-13.
36. Hiemstra E, Kolkman W, Wolterbeek R, Trimbos B, Jansen FW. Value of an objective assessment tool in the operating room. Can J Surg 2011;54 (2):116-22.
37. Ishman SL, Brown DJ, Boss EF, Skinner ML, Tunkel DE, Stavinoha R, Lin SY. Development and pilot testing of an operative competency assessment tool for paediatric direct laryngoscopy and rigid bronchoscopy. Laryngoscope 2010;120 (11):2294-300.
38. Kim MJ, Williams RG, Boehler ML, Ketchum JK, Dunnington GL. Refining the evaluation of operating room performance. J Surg Educ 2009;66 (6):352-6.
39. Kurashima Y, Feldman LS, Al-Sabah S, Kaneva PA, Fried GM, Vassiliou MC. A tool for training and evaluation of laparoscopic inguinal hernia repair: the global operative assessment of laparoscopic skills-groin hernia (GOALS-GH). Am J Surg 2011;201 (1):54-61.
40. Laeeq K, Waseem R, Weatherly RA, Reh DD, Lin SY, Lane AP, Ishii M, Cummings CW, Bhatti NI. In-training assessment and predictors of competency in endoscopic sinus surgery. Laryngoscope 2010;120 (12):2540-5.
41. Larsen CR, Grantcharov T, Schouenborg L, Ottosen C, Soerensen JL, Ottesen B. Objective assessment of surgical competence in gynaecological laparoscopy: development and validation of a procedure-specific rating scale. BJOG 2008;115 (7):908-16.
42. Paisley AM, Baldwin PJ, Paterson-Brown S. Validity of surgical simulation for the assessment of operative skill. Br J Surg 2001;88 (11):1525-32.
43. Roberson DW, Kentala E, Forbes P. Development and validation of an objective instrument to measure surgical performance at tonsillectomy. Laryngoscope 2005;115 (12):2127-37.
44. Sarker SK, Chang A, Vincent C. Technical and technological skills assessment in laparoscopic surgery. JSLS 2006;10 (3):284-92.
45. Sarker SK, Chang A, Vincent C, Darzi SA. Development of assessing generic and specific technical skills in laparoscopic surgery. Am J Surg 2006;191 (2):238-44.
46. Sarker SK, Albrani T, Zaman A, Patel B. Procedural performance in gastrointestinal endoscopy: an assessment and self-appraisal tool. Am J Surg 2008;196 (3):450-5.
47. Sarker SK, Maciocco M, Zaman A, Kumar I. Operative performance in laparoscopic cholecystectomy using the procedural-based assessment tool. Am J Surg 2010;200 (3):334-40.
48. Stack BC Jr, Siegel E, Bodenner D, Carr MM. A study of resident proficiency with thyroid surgery: creation of a thyroid-specific tool. Otolaryngol Head Neck Surg 2010;142 (6):856-62.
49. Syme-Grant J, White PS, McAleer JPG. Measuring competence in endoscopic sinus surgery. Surgeon 2008;6 (1):37-44.
50. Van Sickle KR, Baghai M, Huang IP, Goldenberg A, Smith CD, Ritter EM. Construct validity of an objective assessment method for laparoscopic intracorporeal suturing and knot tying. Am J Surg 2008;196 (1):74-80.
51. Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondre K, Stanbridge D, Fried GM. A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 2005;190 (1):107-13.
52. Vassiliou MC, Feldman LS, Fraser SA, Charlebois P, Chaudhury P, Stanbridge DD, Fried GM. Evaluating intraoperative laparoscopic skill: direct observation versus blinded videotaped performances. Surg Innov 2007;14 (3):211-6.
53. Vassiliou MC, Kaneva PA, Poulose BK, et al. Global assessment of gastrointestinal endoscopic skills (GAGES): a valid measurement tool for technical skills in flexible endoscopy. Surg Endosc 2010;24 (8):1834-41.
54. Winckel CP, Reznick RK, Cohen R, Taylor B. Reliability and construct validity of a structured technical skills assessment form. Am J Surg 1994;167 (4):423-7.
55. Sarker SK, Albrani T, Zaman A, Kumar I. Procedural performance in gastrointestinal endoscopy: live and simulated. World J Surg 2010;34 (8):1764-70.
56. Sarker SK, Kumar I, Delaney C. Assessing operative performance in advanced laparoscopic colorectal surgery. World J Surg 2010;34 (7):1594-603.
57. Cremers SL, Ciolino JB, Ferrufino-Ponce ZK, Henderson BA. Objective assessment of skills in intraocular surgery (OASIS). Ophthalmology 2005;112 (7):1236-41.
58. Gauba V, Saleh GM, Goel S. Ophthalmic plastic surgical skills assessment tool. Ophthal Plast Reconstr Surg 2008;24 (1):43-6.
59. Laeeq K, Infusino S, Lin SY, Reh DD, Ishii M, Kim J, Lane AP, Bhatti NI. Video-based assessment of operative competency in endoscopic sinus surgery. Am J Rhinol Allergy 2010;24 (3):234-7.
60. Larson JL, Williams RG, Ketchum J, Boehler ML, Dunnington GL. Feasibility, reliability and validity of an operative performance rating system for evaluating surgery residents. Surgery 2005;138 (4):640-7; discussion 647-9.
61. Neo EL, Patkin M, Watson DI. Suturing efficiency during hiatal repair for laparoscopic fundoplication. ANZ J Surg 2004;2:13-7.
62. Datta V, Bann S, Aggarwal R, Mandalia M, Hance J, Darzi A. Technical skills examination for general surgical trainees. Br J Surg 2006;93 (9):1139-46.
63. Anderson CI, Jentz AB, Harkema JM, Kareti LR, Apelgren KN, Slomski CA. Assessing the competencies in general surgery residency training. Am J Surg 2005;189 (3):288-92.
64. Eubanks TR, Clements RH, Pohl D, Williams N, Schaad DC, Horgan S, Pellegrini C. An objective scoring system for laparoscopic cholecystectomy. J Am Coll Surg 1999;189 (6):566-74.
65. Roach PB, Roggin KK, Selkov G Jr, Posner MC, Silverstein JC. Continuous, data-rich appraisal of surgical trainees' operative abilities: a novel approach for measuring performance and providing feedback. J Surg Educ 2009;66 (5):255-63.
66. Maizels M, Yerkes EB, Macejko A et al. A new computer-enhanced visual learning method to train urology residents in paediatric orchiopexy: a prototype for Accreditation Council for Graduate Medical Education documentation. J Urol 2008;180 (Suppl 4):1814-8; discussion 1818.
67. McQuiston L, Macneily A, Liu D, Mickelson J, Yerkes E, Chaviano A, Roth D, Stoltz RS, Herz DB, Maizels M. Computer-enhanced visual learning method to train urology residents in paediatric orchiopexy provided a consistent learning experience in a multi-institutional trial. J Urol 2010;184 (Suppl 4):1748-53.
68. Moorthy K, Munz Y, Sarker SK, Darzi A. Objective assessment of technical skills in surgery. BMJ 2003;327 (7422):1032-7.
69. Aggarwal R, Moorthy K, Darzi A. Laparoscopic skills training and assessment. Br J Surg 2004;91 (12):1549-58.
70. Memon MA, Brigden D, Subramanya MS, Memon B. Assessing the surgeon's technical skills: analysis of the available tools. Acad Med 2010;85 (5):869-80.
71. van Hove PD, Tuijthof GJ, Verdaasdonk EG, Stassen LP, Dankelman J. Objective assessment of technical surgical skills. Br J Surg 2010;97 (7):972-87.