Evaluating the utility of workplace-based assessment tools for speciality training

Best Practice and Research: Clinical Obstetrics and Gynaecology

Volumn 24, Issue 6, 2010, Pages 767-782

  Setna, Z ^a   Jha, V ^a   Boursicot, K A M ^b   Roberts, T E ^a  

^a UNIVERSITY OF LEEDS   (United Kingdom)

^b ST GEORGE S HOSPITAL   (United Kingdom)

Author keywords

assessment; postgraduate education; speciality training; workplace assessment

Indexed keywords

CASE BASED DISCUSSION; CLINICAL COMPETENCE; CLINICAL EVALUATION; CONCURRENT VALIDITY; CONSTRUCT VALIDITY; CONTENT VALIDITY; GYNECOLOGY; INTERNAL CONSISTENCY; INTERRATER RELIABILITY; MEDICAL EDUCATION; MINI CLINICAL EVALUATION EXERCISE; NAMED INVENTORIES, QUESTIONNAIRES AND RATING SCALES; NATIONAL HEALTH SERVICE; OBJECTIVE STRUCTURED ASSESSMENT OF TECHNICAL SKILLS; OBSTETRICS; PREDICTIVE VALIDITY; PRIORITY JOURNAL; RELIABILITY; REVIEW; SPECIALIZATION; TEST RETEST RELIABILITY; VALIDITY; WORKPLACE;

CLINICAL COMPETENCE; EDUCATION, MEDICAL, GRADUATE; EDUCATIONAL MEASUREMENT; EMPLOYEE PERFORMANCE APPRAISAL; GREAT BRITAIN; GYNECOLOGY; HUMANS; OBSTETRICS; REPRODUCIBILITY OF RESULTS; TASK PERFORMANCE AND ANALYSIS; WORKPLACE;

EID: 77957753162     PISSN: 15216934     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.bpobgyn.2010.04.003     Document Type: Review

References (68)

1. (PMETB) PMEaTB Workplace based assessment: a paper from the PMETB workplace based assessment subcommittee 2005 PMETB UK
2. S. Meadows Medical staffing-the Calman report: implications and cost Health Manpow Manage 22 1996 23 26
3. G. Morris-Stiff, E. Ball, and J. Torkington Registrar operating experience over a 15-year period: more, less or more or less the same? Surg J Royal Coll Surg Edinburgh & Ireland 2 2004 161 164
4. J. Chikwe, A.C. de Souza, and J.R. Pepper No time to train surgeons Br Med J 328 2004 418 419
5. S.C. Day, L.G. Grosso, and J.J. Norcini Residents' perceptions of evaluation procedures used during their training program J Gen Int Med 5 1990 421 426
6. J. Smith Shipman enquiry-6th report 2005
7. G. Australian Department of education employment and workplace relations [cited; Available from: http://www.dest.gov.au/sectors/trainingskills/ policyissuesreviews/keyissues/nts/glo/utoz.htm
8. A. Wragg, W. Wade, and G. Fuller Assessing the performance of specialist registrars Clin Med 3 2 2003 131 134
9. G.E. Miller The assessment of clinical skills/competence/performance Acad Med 65 9 Suppl. 1990 Sep S63 S67
10. J.J. Norcini Work based assessment Br Med J 326 7392 2003 Apr 5 753 755
11. J.J. Norcini Workplace-based assessment in clinical training 2007 ASME Edinburgh
12. L.K.J. Baartman, T.J. Bastiaens, and P.A. Kirschner The wheel of competency assessment: presenting quality criteria for competency assessment programs Stud Educ Eval 32 2006 153 170
13. L. Schuwirth, and C.P.M. Van der Vleuten How to design a useful test: the principles of assessment 2006
14. L.J. Cronbach, R. Nageswari, and G.C. Gleser Theory of generalizability: a liberation of reliability theory Br J Stat Psychol 16 1963 137 163
15. D.J. Sheskin Handbook of parametric and nonparametric statistical procedures 2004 Chapman and Hall/CRC Washington DC
16. Y. Benett The validity and reliability of assessments and self-assessments of work-based learning Assess Eval High Educ 18 2 1993 83 94
17. C.P.M. Van der Vleuten The assessment of professional competence: developments, research and practical im plications Adv Health Sci Educ 1 1996 41 67
18. J. Norcini The mini clinical evaluation exercise (mini CEX) Clin Teach 2 1 2005 25 30
19. J. Norcini, L. Blank, and G. Arnold The mini-CEX (Clinical Evaluation Exercise): A Preliminary Investigation Ann Intern Med 123 no. 10 November 15, 1995 795 799
20. A. Alves de Lima, C. Barrero, and S. Baratta Validity, reliability, feasibility and satisfaction of the mini-clinical evaluation exercise (mini-CEX) for cardiology residency training Med Teach 29 8 2007 Oct 785 790
21. J.J. Norcini, L.L. Blank, and F.D. Duffy The mini-CEX: a method for assessing clinical skills Ann Intern Med 138 6 2003 Mar 18 476 481
22. J. Hance, R. Aggarwal, and R. Stanbridge Objective assessment of technical skills in cardiac surgery Eur J Cardio-Thoracic Surg 28 1 2005 Jul 157 162
23. J.L. Maatsch, R. Huang, and S. Downing Predictive validity of medical speciality examinations 1983 Office of Medical Education and Research and Development, Michigan state University East Lansing MI Final report for Grant HS 02038-04, National Centre of Health Services Research. Office of Medical Education and Research and Development, Michigan state University, East Lansing MI. East Lansing MI. National Centre of Health Services Research
24. ISCP Guidance for using the case-based discussion (CBD) 2006-2009 [cited; Available from: https://www.iscp.ac.uk/Documents/CBD-Guidance.pdf
25. D.M. Torre, D.E. Simpson, and D.M. Elnicki Feasibility, reliability and user satisfaction with a PDA-based mini-CEX to evaluate the clinical skills of third-year medical students Teach Learn Med 19 3 2007 271 277
26. J.R. Kogan, L.M. Bellini, and J.A. Shea Feasibility, reliability, and validity of the mini-clinical evaluation exercise (mCEX) in a medicine core clerkship Acad Med 78 10 Suppl. 2003 Oct S33 S35
27. J.R. Boulet, D.W. McKinley, and J.J. Norcini Assessing the comparability of standardized patient and physician evaluations of clinical skills Adv Health Sci Educ 7 2 2002 85 97
28. J.J. Norcini, L.L. Blank, and G.K. Arnold Examiner differences in the mini-CEX Adv Health Sci Educ 2 1 1997 27 33
29. M. Margolis, B. Clauser, and M. Cuddy Use of the mini-clinical evaluation exercise to rate examinee performance on a multiple-station clinical skills examination: a validity study Acad Med 81 10 2006 S56 S60
30. D.A. Cook, D.M. Dupras, and T.J. Beckman Effect of rater training on reliability and accuracy of mini-CEX scores: a randomized, controlled trial J Gen Int Med 24 1 2009 Jan 74 79
31. G.L. Noel, J.E. Herbers Jr., and M.P. Caplow How well do internal medicine faculty members evaluate the clinical skills of residents? Ann Intern Med 117 9 1992 Nov 1 757 765
32. E.S. Holmboe, R.E. Hawkins, and S.J. Huot Effects of training in direct observation of medical residents' clinical competence: a randomized trial Ann Intern Med 140 11 2004 Jun 1 874 881
33. M. Fialkow, L. Mandel, and A. VanBlaricom A curriculum for Burch colposuspension and diagnostic cystoscopy evaluated by an objective structured assessment of technical skills Am J Obstet Gynecol 197 5 2007 Nov 544.e1 544.e6
34. B. Goff, L. Mandel, and G. Lentz Assessment of resident surgical skills: is testing feasible? Am J Obstet Gynecol 192 4 2005 Apr 1331 1338 discussion 8-40
35. B.A. Goff, G.M. Lentz, and D. Lee Development of an objective structured assessment of technical skills for obstetric and gynecology residents Obstet Gynecol 96 1 2000 Jul 146 150
36. B.A. Goff, P.E. Nielsen, and G.M. Lentz Surgical skills assessment: a blinded examination of obstetrics and gynecology residents Am J Obstet Gynecol 186 4 2002 Apr 613 617
37. S. Siddighi, S.D. Kleeman, and M.S. Baggish Effects of an educational workshop on performance of fourth-degree perineal laceration repair Obstet Gynecol 109 2 Pt 1 2007 Feb 289 294
38. N.Y. Siddiqui, K.J. Stepp, and S.J. Lasch Objective structured assessment of technical skills for repair of fourth-degree perineal lacerations Am J Obstet Gynecol 199 6 2008 Dec 676.e1 676.e6
39. G.R. Norman, D.A. Davis, and S. Lamb Competency assessment of primary care physicians as part of a peer review program J Am Med Assoc 270 9 1993 Sep 1 1046 1051 [Erratum appears in JAMA 1994 Jan 12;271(2):106]
40. D.J. Solomon, M.A. Reinhart, and R.G. Bridgham An assessment of an oral examination format for evaluating clinical competence in emergency medicine Acad Med 65 9 Suppl. 1990 Sep S43 S44
41. R. Hatala, M. Ainslie, and B. Kassen Assessing the mini-clinical evaluation exercise in comparison to a national speciality examination Med Educ 40 2006 950 956
42. S. Durning, J. Cation, and R. Markert Assessing the reliability and validity of the mini-CEX evaluation exercise for internal medicine residency training Acad Med 77 9 September 2002 900 904
43. A.L. VanBlaricom, B.A. Goff, and M. Chinn A new curriculum for hysteroscopy training as demonstrated by an objective structured assessment of technical skills (OSATS) Am J Obstet Gynecol 193 5 2005 Nov 1856 1865
44. B. Chou, C.W. Bowen, and V.L. Handa Evaluating the competency of gynecology residents in the operating room: validation of a new assessment tool Am J Obstet Gynecol 199 5 2008 Nov 571.e1 571.e5
45. G. Ault, R. Reznick, and H. MacRae Exporting a technical skills evaluation technology to other sites Am J Surg 182 3 2001 Sep 254 256
46. R. Reznick, G. Regehr, and H. MacRae Testing technical skill via an innovative "bench station" examination Am J Surg 173 3 1997 Mar 226 230
47. V. Datta, S. Bann, and M. Mandalia The surgical efficiency score: a feasible, reliable, and valid method of skills assessment Am J Surg 192 3 2006 Sep 372 378
48. S. Bann, M. Khan, and V. Datta Surgical skill is predicted by the ability to detect errors Am J Surg 189 4 2005 Apr 412 415
49. J.G. Chipman, and C.C. Schmitz Using objective structured assessment of technical skills to evaluate a basic skills simulation curriculum for first-year surgical residents J Am Coll Surg 209 3 2009 Sep 364 370.e2
50. J.A. Martin, G. Regehr, and R. Reznick Objective structured assessment of technical skill (OSATS) for surgical residents Br J Surg 84 2 1997 Feb 273 278
51. H. MacRae, G. Regehr, and W. Leadbetter A comprehensive examination for senior surgical residents Am J Surg 179 3 2000 Mar 190 193
52. B.R. Nair, H.G. Alexander, and B.P. McGrath The mini clinical evaluation exercise (mini-CEX) for assessing clinical performance of international medical graduates Med J Aust 189 3 2008 Aug 4 159 161
53. M. Kane Validating measures of performance Educ Measures: Iss Pract 18 2 1999 5 17
54. E.S. Holmboe, S. Huot, and J. Chung Construct validity of the miniclinical evaluation exercise (miniCEX) Acad Med 78 8 2003 Aug 826 830
55. J.D. Beard, S. Choksy, S. Khan Vascular Society of Great Britain Assessment of operative competence during carotid endarterectomy Br J Surg 94 6 2007 Jun 726 730
56. A.M. Collins, P.F. Ridgway, and M.S.U. Hassan Surgical instruction for general practitioners: how, who and how often? J Plast Reconstr Aesthet Surg 2009 1 7
57. P.E. Nielsen, L.M. Foglia, and L.S. Mandel Objective structured assessment of technical skills for episiotomy repair Am J Obstet Gynecol 189 5 2003 Nov 1257 1260
58. S.E. Swift, and J.F. Carter Institution and validation of an observed structured assessment of technical skills (OSATS) for obstetrics and gynecology residents and faculty Am J Obstet Gynecol 195 2 2006 Aug 617 621 discussion 21-23
59. J.F. Bodle, S.J. Kaufmann, and D. Bisson Value and face validity of objective structured assessment of technical skills (OSATS) for work based assessment of surgical skills in obstetrics and gynaecology Med Teach 30 2 2008 212 216
60. H. Faulkner, G. Regehr, and J. Martin Validation of an objective structured assessment of technical skill for surgical residents Acad Med 71 12 1996 Dec 1363 1365
61. J.R. Wilkinson, J.G. Crossley, and A. Wragg Implementing workplace-based assessment across the medical specialties in the United Kingdom Med Educ 42 4 2008 Apr 364 373
62. S.N. Cohen, P.B. Farrant, and S.M. Taibjee Assessing the assessments: U.K. dermatology trainees' views of the workplace assessment tools Br J Dermatol 161 1 2009 Jul 34 39
63. C.M. Wiles, K. Dawson, and T.A. Hughes Clinical skills evaluation of trainees in a neurology department Clin Med 7 4 2007 Aug 365 369
64. J.M. Weller, B. Jolly, and M.P. Misur Mini-clinical evaluation exercise in anaesthesia training Br J Anaesth 102 5 2009 May 633 641
65. A.R. Jensen, A.S. Wright, and K.E. Calhoun Validity of the use of objective structured assessment of technical skills (OSATS) with videorecorded surgical task performance J Am Coll Surg 209 3S 2009 S110 S111
66. A. Alves de Lima, R. Henquin, and J. Thierer A qualitative study of the impact on learning of the mini clinical evaluation exercise in postgraduate training Med Teach 27 1 2005 Jan 46 52
67. E.S. Holmboe, M. Yepes, and F. Williams Feedback and the mini clinical evaluation exercise J Gen Int Med 19 5 Pt 2 2004 May 558 561
68. A.S. Elstein, L.S. Shulman, and S.A. Sprafka Medical problem-solving: an analysis of clinical reasoning 1978 Harvard University Press Cambridge, MA