Reliability and validity of procedure-based assessments in otolaryngology training

Laryngoscope

Volumn 125, Issue 6, 2015, Pages 1328-1335

  Awad, Zaid ^a,b   Hayden, Lindsay ^c   Robson, Andrew K ^d   Muthuswamy, Keerthini ^c   Tolley, Neil S ^a  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

^b ST MARY S HOSPITAL   (United Kingdom)

^c IMPERIAL COLLEGE SCHOOL OF MEDICINE   (United Kingdom)

^d CUMBERLAND INFIRMARY   (United Kingdom)

Author keywords

assessment; postgraduate education; Training

Indexed keywords

ARTICLE; ASSESSMENT OF HUMANS; CONSTRUCT VALIDITY; HUMAN; MEDICAL EDUCATION; MEDICAL STUDENT; OTORHINOLARYNGOLOGY; POSTOPERATIVE CARE; PRIORITY JOURNAL; PROCEDURE BASED ASSESSMENT; RECEIVER OPERATING CHARACTERISTIC; RELIABILITY; SENSITIVITY AND SPECIFICITY; UNITED KINGDOM; CLINICAL COMPETENCE; EDUCATION; REPRODUCIBILITY; RETROSPECTIVE STUDY;

CLINICAL COMPETENCE; OTOLARYNGOLOGY; REPRODUCIBILITY OF RESULTS; RETROSPECTIVE STUDIES;

EID: 84929689268     PISSN: 0023852X     EISSN: 15314995     Source Type: Journal    
DOI: 10.1002/lary.24983     Document Type: Article

References (24)

1. Joint Committee on Surgical Training. Quality indicators surgical training-otolaryngology. Available at: http://www.jcst.org/quality-assurance/documents/qis/otolaryngology-qis. Accessed June 8, 2014.
2. Intercollegiate Surgical Curriculum Programme. ISCP/GMP blueprint. Available at: https://www.iscp.ac.uk/static/public/overarching-blueprint2012.pdf. Accessed June 8, 2014.
3. Intercollegiate Surgical Curriculum Programme. Changes to the intercollegiate surgical curriculum for August 2010 (2010). Available at: https://www.iscp.ac.uk/static/public/s-changes-2010.pdf. Accessed June 8, 2014.
4. Allum W. ISCP Management Group. ISCP evaluation report June 2012. Available at: http://www.jcst.org/ISCP%20Evaluation%20/full-evaluation-report. Accessed June 8, 2014.
5. Fishpool S, Stew B, Roberts C,. Otolaryngology WBAs in the Wales Deanery: the first six years. Bull R Coll Surg Engl 2014; 96: 164-166.
6. Fu B, Amin K, Hili S, Davis J,. Assessing ISCP users? Satisfaction in ENT trainers and trainees. Clin Otolaryngol 2011; 36: 594-596.
7. Pereira EAC, Dean BJF,. British surgeons' experiences of a mandatory online workplace based assessment portfolio resurveyed three years on. J SurgEduc 2013; 70: 59-67.
8. Intercollegiate Surgical Curriculum Programme. PBA assessment. Available at: https://www.iscp.ac.uk/surgical/assessment-pba.aspx. Accessed June 8, 2014.
9. Intercollegiate Surgical Curriculum Programme. Tips for using procedure-based assessments (PBAs). Available at: https://www.iscp.ac.uk/static/public/pba-tips.pdf. Accessed June 8, 2014.
10. Intercollegiate Surgical Curriculum Programme. Guidance for using procedure based assessments (PBAs). Available at: https://www.iscp.ac.uk/static/public/pba-guidance.pdf. Accessed June 8, 2014.
11. Awad Z, Hayden L, Muthuswamy K, Ziprin P, Darzi A, Tolley NS,. Does direct observation of procedural skills reflect trainee's progress in otolaryngology? Clin Otolaryngol 2014; 39: 169-173.
12. Gallagher AG, Seymour NE, Jordan-Black JA, Bunting BP, McGlade K, Satava RM,. Prospective, randomized assessment of transfer of training (ToT) and transfer effectiveness ratio (TER) of virtual reality simulation training for laparoscopic skill acquisition. Ann Surg 2013; 257: 1025-1031.
13. Laeeq K, Bhatti NI, Carey JP, et al., Pilot testing of an assessment tool for competency in mastoidectomy. Laryngoscope 2009; 119: 2402-2410.
14. Laeeq K, Waseem R, Weatherly RA, et al., In-training assessment and predictors of competency in endoscopic sinus surgery. Laryngoscope 2010; 120: 2540-2545.
15. Lin SY, Laeeq K, Ishii M, et al., Development and pilot-testing of a feasible, reliable, and valid operative competency assessment tool for endoscopic sinus surgery. Am J Rhinol Allergy 2009; 23: 354-359.
16. Masood A, Phillips J, Bath A,. Surgical competency of ENT trainees and the ISCP. Bull R Coll Surg Engl 2014; 96: e9-e13.
17. Beard J,. Workplace-based assessment: the need for continued evaluation and refinement. Surgeon 2011; 9 (suppl 1): S12-S13.
18. Eng J,. Receiver operating characteristic analysis: a primer. Acad Radiol 2005; 12: 909-916.
19. Zhou XH,. Empirical Bayes combination of estimated areas under ROC curves using estimating equations. Med Decis Making 1996; 16: 24-28.
20. Intercollegiate Surgical Curriculum Programme. ISCP assessment framework 2013. Available at: https://www.iscp.ac.uk/static/public/assessment-framework2013.pdf. Accessed June 8, 2014.
21. Mitchell C, Bhat S, Herbert A, Baker P,. Workplace-based assessments of junior doctors: do scores predict training difficulties?. Med Educ 2011; 45: 1190-1198.
22. McGaghie WC, Issenberg SB, Barsuk JH, Wayne DB,. A critical review of simulation-based mastery learning with translational outcomes. Med Educ 2014; 48: 375-385.
23. Powell H, DiMarco A, Saeed S,. Trainee and trainer opinion on increasing numbers of workplace-based assessments. Bull R Coll Surg Engl 2014; 96: 160-162.
24. Bowles PFD, Harries M, Young P, Das P, Saunders N, Fleming JC,. A validation study on the use of intra-operative video recording as an objective assessment tool for core ENT surgery. Clin Otolaryngol 2014; 39: 102-107.