In-training gastrointestinal endoscopy competency assessment tools: Types of tools, validation and impact

Best Practice and Research: Clinical Gastroenterology

Volumn 30, Issue 3, 2016, Pages 357-374

  Walsh, Catharine M ^a,b,c  

^a HOSPITAL FOR SICK CHILDREN   (Canada)

^b UNIVERSITY OF TORONTO   (Canada)

^c UNIVERSITY OF TORONTO   (Canada)

Author keywords

Assessment; Clinical Competence; Education, Medical, Graduate standards; Educational Measurement; Endoscopy, Gastrointestinal education; Endoscopy, Gastrointestinal standards; Patient Simulation

Indexed keywords

EDUCATION; GASTROENTEROLOGY; GASTROINTESTINAL ENDOSCOPY; HUMAN; HUMAN EXPERIMENT; INSTRUMENT VALIDATION; MONITORING; SKILL; VALIDATION PROCESS; CLINICAL COMPETENCE; MEDICAL EDUCATION; PATIENT SIMULATION; STANDARDS;

CLINICAL COMPETENCE; EDUCATION, MEDICAL, GRADUATE; EDUCATIONAL MEASUREMENT; ENDOSCOPY, GASTROINTESTINAL; GASTROENTEROLOGY; HUMANS; PATIENT SIMULATION;

EID: 84964692162     PISSN: 15216918     EISSN: 15321916     Source Type: Journal    
DOI: 10.1016/j.bpg.2016.04.001     Document Type: Review

References (100)

1. [1] Leung, W., Competency based medical training: review. BMJ Br Med J 325:7366 (2002), 693–696.
2. [2] Long, D.M., Competency-based residency training: the next advance in graduate medical education. Acad Med 75:12 (2000), 1178–1183.
3. [3] Epstein, R.M., Assessment in medical education. N Engl J Med 356:4 (2007), 387–396.
4. [4] Eisen, G.M., Baron, T.H., Dominitz, J.A., Faigel, D.O., Goldstein, J.L., Johanson, J.F., et al. Methods of granting hospital privileges to perform gastrointestinal endoscopy. Gastrointest Endosc 55:7 (2002 Jun), 780–783.
5. [5] Sedlack, R.E., Colonoscopy. Cohen, J., (eds.) Successful training in gastrointestinal endoscopy, 1st ed., 2011, Wiley-Blackwell, Oxford, 42–72.
6. [6] Leichtner, A.M., Gillis, L.A., Gupta, S., Heubi, J., Kay, M., Narkewicz, M.R., et al. NASPGHAN guidelines for training in pediatric gastroenterology. J Pediatr Gastroenterol Nutr 56:Suppl. 1 (2013), S1–S8.
7. [7] Swing, S.R., The ACGME outcome project: retrospective and prospective. Med Teach 29:7 (2007), 648–654.
8. [8] Frank, J.R., Snell, L., Sherbino, J., (eds.) The draft CanMEDS 2015 physician competency framework – series IV, 2015 March, The Royal College of Physicians and Surgeons of Canada, Ottawa.
9. [9] Faigel, D.O., Baron, T.H., Lewis, B., Petersen, B., Petrini, J., Ensuring competence in endoscopy. 2006, American Society for Gastrointestinal Endoscopy Website Available from: http://www.asge.org/uploadedFiles/Publications_and_Products/Practice_Guidelines/competence.pdf [accessed 01.03.16].
10. [10] Romagnuolo, J., Enns, R., Ponich, T., Springer, J., Armstrong, D., Barkun, A.N., Canadian credentialing guidelines for colonoscopy. Can J Gastroenterol 22:1 (2008), 17–22.
11. [11] Yule, S., Flin, R., Paterson-Brown, S., Maran, N., Non-technical skills for surgeons in the operating room: a review of the literature. Surgery 139:2 (2006), 140–149.
12. [12] Hull, L., Arora, S., Aggarwal, R., Darzi, A., Vincent, C., Sevdalis, N., The impact of nontechnical skills on technical performance in surgery: a systematic review. J Am Coll Surg 214:2 (2012), 214–230.
13. [13] Cullinane, M., Gray, A., Hargraves, C., Lucas, S., Schubert, M., Sherry, K., et al. Scoping our practice. The 2004 report of the national confidential Enquiry into patient outcome and death. London., 2005 Available from: http://www.ncepod.org.uk/2004report/ [accessed 01.03.16].
14. [14] Walsh, C.M., Development and validation of the gastrointestinal endoscopy competency assessment tools for adult and pediatric colonoscopy. [dissertation], 2014, University of Toronto, 221.
15. Walsh, C.M., Ling, S.C., Walters, T.D., Mamula, P., Lightdale, J.R., Carnahan, H., Development of the gastrointestinal endoscopy competency assessment tool for pediatric colonoscopy (GiECATKIDS). J Pediatr Gastroenterol Nutr 59:4 (2014), 480–486.
16. [16] Shute, V.J., Focus on formative feedback. Rev Educ Res 78:1 (2008), 153–189.
17. [17] Govaerts, M.J.B., van der Vleuten, C.P.M., Schuwirth, L.W.T., Muijtjens, A.M.M., Broadening perspectives on clinical performance assessment: rethinking the nature of in-training assessment. Adv Health Sci Educ Theory Pract 12:2 (2007), 239–260.
18. [18] Wass, V., Van Der Vleuten, C., Shatzer, J., Jones, R., Assessment of clinical competence. Lancet 357:9260 (2001), 945–949.
19. [19] Miller, G.E., The assessment of clinical skills/competence/performance. Acad Med 65:9 Suppl (1990), S63–S67.
20. [20] Downing, S.M., Reliability: on the reproducibility of assessment data. Med Educ 38:9 (2004), 1006–1012.
21. [21] Cook, D.A., Beckman, T.J., Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med, 119(2), 2006 166.e7–16.
22. [22] Cass, O.W., Freeman, M.L., Peine, C.J., Zera, R.T., Onstad, G.R., Objective evaluation of endoscopy skills during training. Ann Intern Med 118:1 (1993), 40–44.
23. [23] Chak, A., Cooper, G.S., Blades, E.W., Canto, M., Sivak, M.V., Prospective assessment of colonoscopic intubation skills in trainees. Gastrointest Endosc 44:1 (1996), 54–57.
24. [24] Parry, B.R., Williams, S.M., Competency and the colonoscopist: a learning curve. Aust N Z J Surg 61:6 (1991), 419–422.
25. [25] Ward, S.T., Mohammed, M.A., Walt, R., Valori, R., Ismail, T., Dunckley, P., An analysis of the learning curve to achieve competency at colonoscopy using the JETS database. Gut 63:11 (2014), 1746–1754.
26. [26] Dafnis, G., Granath, F., Påhlman, L., Hannuksela, H., Ekbom, A., Blomqvist, P., The impact of endoscopists’ experience and learning curves and interendoscopist variation on colonoscopy completion rates. Endoscopy 33:6 (2001), 511–517.
27. [27] Jorgensen, J.E., Elta, G.H., Stalburg, C.M., Kolars, J.C., Stout, J.M., Korsnes, S.J., et al. Do breaks in gastroenterology fellow endoscopy training result in a decrement in competency in colonoscopy?. Gastrointest Endosc 78:3 (2013), 503–509.
28. [28] Shah, S.G., Brooker, J.C., Williams, C.B., Thapar, C., Saunders, B.P., Effect of magnetic endoscope imaging on colonoscopy performance: a randomised controlled trial. Lancet 356:9243 (2000), 1718–1722.
29. [29] Klasko, S.K., Cummings, R.V., Glazerman, L.R., Education Resident data collection: do the numbers add up?. Am J Obstet Gynecol 172:4 Pt 1 (1995), 1312–1316.
30. [30] Adler, D.G., Bakis, G., Coyle, W.J., DeGregorio, B., Dua, K.S., Lee, L.S., et al. Principles of training in GI endoscopy. Gastrointest Endosc 75:2 (2012), 231–235.
31. [31] Accreditation Council for Graduate Medical Education, ACGME program requirements for graduate medical education in colon and rectal surgery. 2016 ACGME Website. Available from: https://www.acgme.org/Portals/0/PFAssets/ProgramRequirements/060_colon_rectal_surgery_2016.pdf [accessed 01.04.20].
32. [32] Accreditation Council for Graduate MedicalEducation, ACGME program requirements for graduate medical education in general surgery. 2016 ACGME Website. Available from: http://www.acgme.org/portals/0/pfassets/programrequirements/440_general_surgery_2016.pdf [accessed 01.04.20].
33. [33] Hori, Y., Granting of privilege for gastrointestinal endoscopy. Surg Endosc 22:5 (2008), 1349–1352.
34. [34] Conjoint Committee for the Recognition of Training in Gastrointestinal Endoscopy, Requirements for CCRTGE recognition. 2014 Available from: http://www.conjoint.org.au/applicants.html#procedural [accessed 01.04.20].
35. [35] The European Section and Board of Gastroenterology and Hepatology, The blue book. 2012 Available from: http://www.eubogh.org/documents/the-blue-book-2012.pdf [accessed 01.04.20].
36. [36] Joint Advisory Group on GI Endoscopy (UK), JAG trainee certification process - colonoscopy. 2011 Available from: http://www.thejag.org.uk/Portals/0/file/Training and Certification in Endoscopy - Guidance for Colonoscopy 07_02_11.pdf [accessed 01.03.16].
37. [37] BSPGHAN Endoscopy Working Group, JAG paediatric endoscopy certification, version 2.1. 2014 Available at: http://www.thejag.org.uk/downloads%5CJAG%20certification%20for%20paediatric%20trainees%5CJAG%20Paediatric%20Certification%202.1%20300513.pdf [accessed 01.03.16].
38. [38] Worthington, D., American Academy of family physicians. Colonoscopy: procedural skills (position paper). Am Fam Physician 62:5 (2000), 1177–1182.
39. [39] Shahidi, N., Ou, G., Telford, J., Enns, R., Establishing the learning curve for achieving competency in performing colonoscopy: a systematic review. Gastrointest Endosc 80:3 (2014), 410–416.
40. [40] Cass, O.W., Training to competence in gastrointestinal endoscopy: a plea for continuous measuring of objective end points. Endoscopy 31:9 (1999), 751–754.
41. [41] Spier, B.J., Benson, M., Pfau, P.R., Nelligan, G., Lucey, M.R., Gaumnitz, E.A., Colonoscopy training in gastroenterology fellowships: determining competence. Gastrointest Endosc 71:2 (2010), 319–324.
42. Sedlack, R.E., Training to competency in colonoscopy: assessing and defining competency standards. Gastrointest Endosc 74:2 (2011), 355–366.
43. [43] Koch, A.D., Haringsma, J., Schoon, E.J., de Man, R.A., Kuipers, E.J., Competence measurement during colonoscopy training: the use of self-assessment of performance measures. Am J Gastroenterol 107:7 (2012), 971–975.
44. [44] Walsh, C., Sherlock, M., Ling, S., Carnahan, H., Virtual reality simulation training for health professions trainees in gastrointestinal endoscopy. Cochrane Database Syst Rev, 6, 2012 CD008237.
45. [45] Singh, S., Sedlack, R.E., Cook, D.A., Effects of simulation-based training in gastrointestinal endoscopy: a systematic review and meta-analysis. Clin Gastroenterol Hepatol 12:10 (2014), 1611–1623.
46. [46] Kneebone, R.L., Nestel, D., Moorthy, K., Taylor, P., Bann, S., Munz, Y., et al. Learning the skills of flexible sigmoidoscopy - the wider perspective. Med Educ 37:Suppl. 1 (2003), 50–58.
47. [47] Cohen, J., Thompson, C.C., The next generation of endoscopic simulation. Am J Gastroenterol 108:7 (2013), 1036–1039.
48. [48] Sedlack, R.E., Competency assessment: it's time to expect more from our simulator. Dig Liver Dis 44:7 (2012), 537–538.
49. [49] Elvevi, A., Cantù, P., Maconi, G., Conte, D., Penagini, R., Evaluation of hands-on training in colonoscopy: is a computer-based simulator useful?. Dig Liver Dis 44:7 (2012), 580–584.
50. [50] McConnell, R.A., Kim, S., Ahmad, N.A., Falk, G.W., Forde, K.A., Ginsberg, G.G., et al. Poor discriminatory function for endoscopic skills on a computer-based simulator. Gastrointest Endosc 76:5 (2012), 993–1002.
51. [51] Plooy, A.M., Hill, A., Horswill, M.S., Cresp, A.S.G., Watson, M.O., Ooi, S.-Y., et al. Construct validation of a physical model colonoscopy simulator. Gastrointest Endosc 76:1 (2012), 144–150.
52. [52] Verdaasdonk, E.G.G., Stassen, L.P.S., Schijven, M.P., Dankelman, J., Construct validity and assessment of the learning curve for the SIMENDO endoscopic simulator. Surg Endosc 21:8 (2007), 1406–1412.
53. [53] Felsher, J.J., Olesevich, M., Farres, H., Rosen, M., Fanning, A., Dunkin, B.J., et al. Validation of a flexible endoscopy simulator. Am J Surg 189:4 (2005), 497–500.
54. [54] Fayez, R., Feldman, L.S., Kaneva, P., Fried, G.M., Testing the construct validity of the Simbionix GI Mentor II virtual reality colonoscopy simulator metrics: module matters. Surg Endosc 24:5 (2010), 1060–1065.
55. [55] Koch, A.D., Buzink, S.N., Heemskerk, J., Botden, S.M.B.I., Veenendaal, R., Jakimowicz, J.J., et al. Expert and construct validity of the Simbionix GI Mentor II endoscopy simulator for colonoscopy. Surg Endosc 22:1 (2008), 158–162.
56. [56] Sedlack, R.E., Kolars, J.C., Validation of a computer-based colonoscopy simulator. Gastrointest Endosc 57:2 (2003), 214–218.
57. [57] Mahmood, T., Darzi a. A study to validate the colonoscopy simulator. Surg Endosc 17:10 (2003), 1583–1589.
58. [58] Grantcharov, T.P., Carstensen, L., Schulze, S., Objective assessment of gastrointestinal endoscopy skills using a virtual reality simulator. JSLS 9:2 (2005), 130–133.
59. [59] Sedlack, R.E., Coyle, W.J., Obstein, K.L., Al-Haddad, M.A., Bakis, G., Christie, J.A., et al. ASGE's assessment of competency in endoscopy evaluation tools for colonoscopy and EGD. Gastrointest Endosc 79:1 (2014), 1–7.
60. [60] Sedlack, R.E., Baron, T.H., Downing, S.M., Schwartz, A.J., Validation of a colonoscopy simulation model for skills assessment. Am J Gastroenterol 102:1 (2007), 64–74.
61. [61] Haycock, A.V., Bassett, P., Bladen, J., Thomas-Gibson, S., Validation of the second-generation Olympus colonoscopy simulator for skills assessment. Endoscopy 41:11 (2009), 952–958.
62. [62] Moorthy, K., Munz, Y., Orchard, T.R., Gould, S., Rockall, T., Darzi, A., An innovative method for the assessment of skills in lower gastrointestinal endoscopy. Surg Endosc 18:11 (2004), 1613–1619.
63. [63] Sarker, S.K., Albrani, T., Zaman, A., Kumar, I., Procedural performance in gastrointestinal endoscopy: live and simulated. World J Surg 34:8 (2010), 1764–1770.
64. [64] Thompson, C.C., Jirapinyo, P., Kumar, N., Ou, A., Camacho, A., Lengyel, B., et al. Validation of an endoscopic part-task training box as a skill assessment tool. Gastrointest Endosc 81:4 (2015), 967–973.
65. [65] Mason, J.D., Ansell, J., Warren, N., Torkington, J., Is motion analysis a valid tool for assessing laparoscopic skill?. Surg Endosc 27:5 (2013), 1468–1477.
66. [66] Mohankumar, D., Garner, H., Ruff, K., Ramirez, F.C., Fleischer, D., Wu, Q., et al. Characterization of right wrist posture during simulated colonoscopy: an application of kinematic analysis to the study of endoscopic maneuvers. Gastrointest Endosc 79:3 (2014), 480–489.
67. [67] Appleyard, M.N., Mosse, C.A., Mills, T.N., Bell, G.D., Castillo, F.D., Swain, C.P., The measurement of forces exerted during colonoscopy. Gastrointest Endosc 52:2 (2000), 237–240.
68. [68] Shergill, A.K., Asundi, K.R., Barr, A., Shah, J.N., Ryan, J.C., McQuaid, K.R., et al. Pinch force and forearm-muscle load during routine colonoscopy: a pilot study. Gastrointest Endosc 69:1 (2009), 142–146.
69. [69] Obstein, K.L., Patil, V.D., Jayender, J., San José Estépar, R., Spofford, I.S., Lengyel, B.I., et al. Evaluation of colonoscopy technical skill levels by use of an objective kinematic-based system. Gastrointest Endosc 73:2 (2011), 315–321.
70. [70] Ende, A.R., Shah, P.M., Chandrasekhara, V., Egorov, V., Pasechnik, A., Korman, L.Y., et al. Quantitative force application during simulated colonoscopy is significantly different between novice and expert endoscopists. Gastrointest Endosc, 79(5S), 2014 AB217 [abstract Su1568].
71. [71] Haycock, A., Koch, A.D., Familiari, P., van Delft, F., Dekker, E., Petruzziello, L., et al. Training and transfer of colonoscopy skills: a multinational, randomized, blinded, controlled trial of simulator versus bedside training. Gastrointest Endosc 71:2 (2010), 298–307.
72. [72] Park, J., MacRae, H., Musselman, L.J., Rossos, P., Hamstra, S.J., Wolman, S., et al. Randomized controlled trial of virtual reality simulator training: transfer to live patients. Am J Surg 194:2 (2007), 205–211.
73. [73] Vassiliou, M.C., Dunkin, B.J., Fried, G.M., Mellinger, J.D., Trus, T., Kaneva, P., et al. Fundamentals of endoscopic surgery: creation and validation of the hands-on test. Surg Endosc 28:3 (2014), 704–711.
74. TM manual skills test. Surg Endosc 28:11 (2014), 3081–3085.
75. [75] Tinmouth, J., Kennedy, E., Baron, D., Burke, M., Feinberg, S., Gould, M., et al. Guideline for colonoscopy quality assurance in Ontario. 2013 Sept 9, Cancer Care Ontario, Toronto (ON) Program in Evidence-based Care Evidence-based Series No.: 15–5 Version 2.
76. Armstrong, D., Barkun, A., Bridges, R., Carter, R., de Gara, C., Dube, C., et al. Canadian association of gastroenterology consensus guidelines on safety and quality indicators in endoscopy. Can J Gastroenterol 26:1 (2012), 17–31.
77. [77] Lightdale, J.R., Acosta, R., Shergill, A.K., Chandrasekhara, V., Chathadi, K., Early, D., et al. Modifications in endoscopic practice for pediatric patients. Gastrointest Endosc 79:5 (2014), 699–710.
78. [78] Forget, S., Walsh, C.M., Pediatric endoscopy: need for a tailored approach to guidelines on quality and safety. Can J Gastroenterol, 26(10), 2012, 735.
79. [79] Ten Cate, O., Entrustability of professional activities and competency-based training. Med Educ 39:12 (2005), 1176–1177.
80. [80] Rose, S., Fix, O.K., Shah, B.J., Jones, T.N., Szyjkowski, R.D., Bosworth, B.P., et al. Entrustable professional activities for gastroenterology fellowship training. Gastrointest Endosc 80:1 (2014), 16–27.
81. [81] NASPGHAN EPA Taskforce, Entrustable professional activities. 2016 NASPGHAN Website. Available from: http://www.naspghan.org/content/132/en/training/opportunities/Entrustable-Professional-Activities [accessed 01.03.16].
82. [82] Messick, S., Validity. Linn, R.L., (eds.) Educational measurement, 3rd ed., 1989, American Council on Education and Macmillan, New York, 13–104.
83. Walsh, C.M., Ling, S.C., Khanna, N., Cooper, M.A., Grover, S.C., May, G., et al. Gastrointestinal Endoscopy Competency Assessment Tool: development of a procedure-specific assessment tool for colonoscopy. Gastrointest Endosc 79:5 (2014), 798–807.
84. Sedlack, R.E., The Mayo Colonoscopy Skills Assessment Tool: validation of a unique instrument to assess colonoscopy skills in trainees. Gastrointest Endosc 72:6 (2010), 1125–1133.
85. [85] Joint Advisory Group on GI Endoscopy, Formative DOPS assessment form - colonoscopy and flexible sigmoidoscopy. The Joint advisory Group on GI endoscopy Website. 2010 Available from: http://www.thejag.org.uk/downloads/DOPS Forms For International and reference use only/Formative DOPS Assessment Form - Colonoscopy and FS.pdf [accessed 01.03.16].
86. [86] Beard, J.D., Marriott, J., Purdie, H., Crossley, J., Assessing the surgical skills of trainees in the operating theatre: a prospective observational study of the methodology. Health Technol Assess 15:1 (2011), 1–162.
87. Walsh, C.M., Ling, S.C., Khanna, N., Grover, S.C., Yu, J.J., Cooper, M.A., et al. Gastrointestinal endoscopy competency assessment tool: reliability and validity evidence. Gastrointest Endosc 81:6 (2015), 1417–1424.
88. Barton, J.R., Corbett, S., van der Vleuten, C.P., The validity and reliability of a Direct Observation of Procedural Skills assessment tool: assessing colonoscopic skills of senior endoscopists. Gastrointest Endosc 75:3 (2012), 591–597.
89. Walsh, C.M., Ling, S.C., Mamula, P., Lightdale, J.R., Walters, T.D., Yu, J.J., et al. The gastrointestinal endoscopy competency assessment tool for pediatric colonoscopy. J Pediatr Gastroenterol Nutr 60:4 (2015), 474–480.
90. [90] Kromann, C.B., Jensen, M.L., Ringsted, C., The effect of testing on skills learning. Med Educ 43:1 (2009), 21–27.
91. Sedlack, R.E., Coyle, W.J., Assessment of competency in endoscopy: establishing and validating generalizable competency benchmarks for colonoscopy. Gastrointest Endosc 83:8 (2016), 516–523.
92. [92] Boyle, E., Al-Akash, M., Patchett, S., Traynor, O., McNamara, D., Towards continuous improvement of endoscopy standards: validation of a colonoscopy assessment form. Colorectal Dis 14:9 (2012), 1126–1131.
93. [93] American Association for the Study of Liver Diseases, American College of gastroenterology, Aerican gastroenterological association, American society for gastrointestinal endoscopy. The gastroenterology core curriculum, third edition. Gastroenterology 132:5 (2007), 2012–2018.
94. [94] Sarker, S.K., Albrani, T., Zaman, A., Patel, B., Procedural performance in gastrointestinal endoscopy: an assessment and self-appraisal tool. Am J Surg 196:3 (2008), 450–455.
95. [95] Vassiliou, M.C., Kaneva, P.A., Poulose, B.K., Dunkin, B.J., Marks, J.M., Sadik, R., et al. Global Assessment of Gastrointestinal Endoscopic Skills (GAGES): a valid measurement tool for technical skills in flexible endoscopy. Surg Endosc 24:8 (2010), 1834–1841.
96. [96] Vassiliou, M.C., Kaneva, P.A., Poulose, B.K., Dunkin, B.J., Marks, J.M., Sadik, R., et al. How should we establish the clinical case numbers required to achieve proficiency in flexible endoscopy?. Am J Surg 199:1 (2010), 121–125.
97. [97] Shah, S.G., Thomas-Gibson, S., Brooker, J.C., Suzuki, N., Williams, C.B., Thapar, C., et al. Use of video and magnetic endoscope imaging for rating competence at colonoscopy: validation of a measurement tool. Gastrointest Endosc 56:4 (2002), 568–573.
98. [98] Sullivan, M.E., Ortega, A., Wasserberg, N., Kaufman, H., Nyquist, J., Clark, R., Assessing the teaching of procedural skills: can cognitive task analysis add to our traditional teaching methods?. Am J Surg 195:1 (2008), 20–23.
99. [99] Martin, J.A., Regehr, G., Reznick, R., MacRae, H., Murnaghan, J., Hutchison, C., et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:2 (1997), 273–278.
100. [100] Vassiliou, M.C., Feldman, L.S., Andrew, C.G., Bergman, S., Leffondré, K., Stanbridge, D., et al. A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190:1 (2005), 107–113.