Assessing competency in surgery: Where to begin?

Surgery

Volumn 135, Issue 1, 2004, Pages 6-20

  Sidhu, R S ^a   Grober, E D ^b   Musselman, L J ^b   Reznick, R K ^b  

^a UNIVERSITY HEALTH NETWORK   (Canada)

^b NONE

Author keywords

[No Author keywords available]

Indexed keywords

COMPETENCE; HEALTH CARE SYSTEM; INTERPERSONAL COMMUNICATION; LEARNING; MEDICAL EDUCATION; MEDICAL INFORMATION; MEDICAL PRACTICE; PATIENT CARE; PRIORITY JOURNAL; PROFESSIONAL PRACTICE; RESIDENCY EDUCATION; REVIEW; SCIENCE; SURGEON; SURGERY;

EID: 0346786212     PISSN: 00396060     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0039-6060(03)00154-5     Document Type: Editorial

References (80)

1. Accreditation Council for Graduate Medical Education (ACGME) General Competencies. Version 1.3. September 28, 1999. Chicago (IL): ACGME; 2000.
2. Societal Needs Working Group, CanMEDs 2000 Project. Skills for the new millennium. Ann R Coll Phys Surg Can 1996;29:206-16.
3. Martin JA, Regehr G, Reznick RK, MacRae H, Murnaghan J, Hutchinson C, Brown M. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 1997;84:273-8.
4. Datta V, Mackey S, Mandalia M, Darzi A. The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model. J Am Coll Surg 2001;193:479-85.
5. Wanzel KR, Ward M, Reznick RK. Teaching the surgical craft: from selection to certification. Curr Prob Surg 2002; 39:574-659.
6. Darzi A, Mackay S. Assessment of surgical competence. Qual Health Care 2001;10:64-9.
7. Harvill LM. An NCME instructional module on standard error of measurement. Educ Measure Issues Pract. 1991; 10:33-41.
8. Traub RE, Rowley GL. An NCME instructional module on understanding reliability. Educ Measure Issues Pract 1991; 10:37-45.
9. Norcini JJ. Standards and reliability in evaluation: when rules of thumb don't apply. Acad Med 1999;74:1088-90.
10. Messick S. Standards of validity and the validity of standards in performance assessment. Educ Measure Issues Pract 1995;14:5-8.
11. Turnbull J, Gray J, MacFadyen J. Improving in-training evaluation programs. J Gen Intern Med 1998;13:317-23.
12. Gray JD. Global rating scales in residency education. Acad Med 1996;71:S55-63.
13. Cranton PA, Dauphinee WD, McQueen MM, Smith LP. The reliability and validity of int raining evaluation reports in obstetrics and gynecology. Proc Annu Conf Res Med Educ 1984;23:59-64.
14. Gray JD. Primer on resident evaluation. Ann R Coll Phys Surg Can 1996;29:91-4.
15. Phelan S. Evaluation of the non-cognitive professional traits of medical students. Acad Med 1990;65:S63-7.
16. Regehr G, MacRae H, Reznick R, Szalay D. Comparing the psychometric properties of checklists and global rating scales for assessing performance on an OSCE-format Examination. Acad Med 1998;73:993-7.
17. Norman GR, van der Vleuten CPM, de Graaff E. Pitfalls in the pursuit of objectivity: issues of validity, efficiency and acceptability. Med Educ 1992;26:504-11.
18. van der Vleuten CPM, Norman GR, De Graaff E. Pitfalls in the pursuit of objectivity: issues of reliability. Med Educ 1991;25:I10-8.
19. Hunt DD. Functional and dysfunctional characteristics of the prevailing model of clinical evaluation systems in North American medical schools. Acad Med 1992;67:254-9.
20. Paisley AM, Baldwin P, Paterson-Brown S. Feasibility, reliability and validity of a new assessment form for use with basic surgical trainees. Am J Surg 2001;182:24-9.
21. Turnbull J, MacFadyen J, Van Barneveld C, Norman G. Clinical work sampling: a new approach to the problem of in-training evaluation. J Gen Intern Med 2000;15: 556-61.
22. Stillman PF. Positive effects of a clinical performance assessment program. Acad Med 1991;6:481-3.
23. MacRae HM, Cohen R, Regehr G, Reznick R, Burnstein M. A new assessment tool: the patient assessment and management examination. Surgery 1997;122:335-44.
24. Gallagher R, Donnelly M, Scalzi PM, Deighton M. Toward a comprehensive methodology for resident evaluation. In: Proceedings of the 16th Annual Conference on the Research in Medical Education; 1977 Nov; Washington (DC). Association of American Medical Colleges, 1977.
25. Streiner DL, Global rating scales. In: Neufeld VR, Norman GR, editors. Assessing clinical competence. New York: Springer; 1985. p. 119-41.
26. Irby DM, Milam S. The legal context for evaluating and dismissing medical students and residents. Acad Med 1989;64:639-43.
27. Hammond EJ, McIndoe AK, Sansome AJ, Spargo PM. Multiple-choice examinations: adopting an evidence-based approach to exam technique. Anaesth 1998;53:1105-8.
28. Holsgrove G, Elzubeir M. Imprecise terms in UK medical multiple-choice questions: what examiners think they mean. Med Educ 1998;32:343-50.
29. American Board of Medical Specialties. Policy statements. Available from: http://www.abms.org/policy.asp
30. United States Medical Licensing Examination. Available from: http://www.usmle.org
31. Hammoud MM, Barclay ML. Development of a Web-based question database for students' self-assessment. Acad Med 2002;77:925.
32. Veloski JJ, Rabinowitz HK, Robeson MR, Young PR. Patients don't present with five choices: an alternative to multiple-choice tests in assessing physicians' competence. Acad Med 1999;74:539-46.
33. Newble D, Jolly B, Wakeford R. The Certification and recertification of doctors: issues in the assessment of clinical competence. Cambridge (UK): Cambridge University Press; 1994.
34. Anastakis DJ, Cohen R, Reznick RK. The structured oral examination as a method for assessing surgical residents. Am J Surg 1991;162:67-70.
35. Schubert A, Tetzlaff JE, Tan M, Ryckman JV, Mascha E. Consistency, inter-rater reliability, and validity of 441 consecutive mock oral examinations in anesthesiology: implications for use as a tool for assessment of residents. Anesthesiology 1999;91:288-98.
36. Muller S. Physicians for the twenty-first century: report of the project panel of the general professional education of the physician and college preparation for medicine. J Med Educ 1984;59:1-208.
37. Muzzin LJ, Hart L. Oral Examinations. In: Neufeld VR, Norman GR, editors. Assessing clinical competence. New York: Springer; 1985. p. 71-93.
38. Thomas CS, Mellsop G, Callender J, Crawshaw J, Ellis PM, Hall A, et al. The oral examination: a study of academic and non-academic factors. Med Educ 1993;27:433-9.
39. Stawski WS. Evolution of a mock oral board examination program in surgery. Am J Surg 1994;60:603-5.
40. Wakeford R, Southgate L, Wass V. Improving oral examinations: selecting, training, and monitoring examiners for the MRCGP. BMJ 1995;311:931-5.
41. Harden R, Stevenson M, Downie W, Wilson G. Assessment of clinical competence using an objective structured examination. BMJ 1975;1:447-51.
42. Information pamphlet on the Medical Council of Canada Qualifying Examination Part II 2003. The Medical Council of Canada. Available from: http://www.mcc.ca/pdf/ 2003qii/IP_II.pdf
43. The Educational Commission for Foreign Medical Graduates. ECFMG 2003 Information Booklet: Clinical Skills Assessment. Available from: http://www.ecfmg.org/2003ib/ ibcsa.html#description_of_csa
44. NBME. A public statement from the Federation of State Medical Boards and the National Board of Medical Examiners regarding the American Medical Association's decision to oppose the use of a test of clinical skills for licensure purposes. Available from: http://www.fsmb.org/ cse_info/AMA_Response. html
45. MacRae HM, Regehr G, Leadbetter W, Reznick RK. A comprehensive examination for senior surgical residents. Am J Surg 2000;179:190-3.
46. Sloan DA, Donnelly MB, Schwartz RW, Strodel WE. The objective structured clinical examination the new gold standard for evaluating postgraduate clinical performance. Ann Surg 1995;222:735-42.
47. Schwartz RW, Witzke DB, Donnelly MB, Stratton T, Blue AV, Sloan DA. Assessing residents' clinical performance: cumulative results of a four-year study with the Objective Structured Clinical Examination. Surgery 1998;124: 307-12.
48. Hodges B, Regehr G, McNaughton N, Tiberius R, Hanson M. OSCE checklists do not capture increasing levels of expertise. Acad Med 1999;74:1129-34.
49. Cerilli GJ, Merrick HW, Staren ED. Surgical educator preferences regarding key objective structured clinical examination topics. J Surg Res 2001;101:124-9.
50. Merrick HW, Nowacek GA, Boyer J, Padgett B, Francis P, Gohara SF, et al. Ability of the objective structured clinical examination to differentiate surgical residents, medical students, and physician assistant students. J Surg Res 2002; 106:319-22.
51. Carraccio C, Englander R. The Objective Structured Clinical Examination: a step in the direction of competency-based evaluation. Arch Pediatr Adolesc Med 2000; 154:736-41.
52. Cherry NA, Wanzel KR, Shapiro H. Personal digital assistants: a survey of how useful residents think they would be in the clinical environment. Ann R Coll Phys Surg Can 2002;35:231-3.
53. Malan TK, Haffner WH, Armstrong AY, Satin AJ. Hand-held computer operating system program for collection of resident experience data. Obstet Gynecol 2000;96:792-4.
54. Klasko SK, Cummings RV, Glazerman LR. Resident data collection: do the numbers add up? Am J Obstet Gynecol 1995;1721312-6.
55. American Board of Internal Medicine. Available from: http://www.abim.org
56. Matsumoto ED, Hamstra S, Radomski S, Cusimano M. The effect of bench model fidelity on endourologic skills: a randomized controlled study. J Urol 2002;167:1243-7.
57. Anastakis DJ, Regehr G, Reznick RK, Cusimano M, Murnaghan J, Brown M, Hutchinson C. Assessment of technical skills transfer from the bench training model to the human model. Am J Surg 1999;177:167-70.
58. Derossis AM, Bothwell J, Sigman HH, Fried GM. The effect of practice on performance in a laparoscopic simulator. Surg Endoscop 1998;12:1117-20.
59. Reznick RK, Regehr G, MacRae H, Martin J, McCulloch W. Testing technical skill via innovative "bench station" examination. Am J Surg 1997;173:226-30.
60. Ault G, Reznick RK, MacRae H, Leadbetter W, DaRosa D, Joehl R, et al. Exporting a technical skills evaluation technology to other sites. Am J Surg 2001;182:254-6.
61. Faulkner H, Regehr G, Martin J, Reznick RK. Validation of an objective assessment of technical skill for surgical residents. Acad Med 1996;71:1363-5.
62. Naik VN, Matsumoto ED, Houston PL, Hamstra SJ, Yeung RY, Mallon JS, et al. Fiberoptic orotracheal intubation on anesthetized patients: do manipulation skills learned on a simple model transfer into the operating room? Anesth 2001;95:343-8.
63. Scott DJ, Bergen PC, Rege RV, Laycock R, Tesfay ST, Valentine RJ, et al. Laparoscopic training on bench models: better and more cost effective than operating room experience? J Am Coll Surg 2000;191:272-83.
64. '64. Cusimano MD, Cohen R, Tucker W, Murnaghan J, Kodama R, Reznick RK. A comparative analysis of the costs of administration of an OSCE (objective structured clinical examination). Acad Med 1994;69:571-6.
65. Goff BA, Nielsen PE, Lentz GM, Chow GE, Chalmers RW, Fenner D, et al. Surgical skills assessment: a blinded examination of obstetrics and gynecology residents. Am J Obstet Gynecol 2002;186:613-7.
66. '66. Scott DJ, Rege RV, Bergen PC, Guo WA, Laycock R, Tesfay ST, Valentine RJ, Jones DB. Measuring operative performance after laparoscopic skills training: edited videotape versus direct observation. J Laparoendosc Adv Surg Tech A 2000;10:183-90.
67. Gallagher AG, Satava RM. Virtual reality as a metric for assessment of laparoscopic psychomotor skills. Surg Endosc 2002;16:1746-52.
68. Seymour NE, Gallagher AG, Roman SA, O'Brien MK, Bansal VK, Andersen DK, Satava RM. Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 2002;236:458-63.
69. Cosman PH, Cregan PC, Martin CJ, Cartmill JA. Virtual reality simulators: current status in acquisition and assessment of surgical skills. ANZ J Surg 2002;72:30-4.
70. Taffinder N, Sutton C, Fishwick RJ, McManus IC, Darzi A. Validation of virtual reality to teach and assess psychomotor skills in laparoscopic surgery: results from randomized controlled studies using the MIST VR laparoscopic simulator. Stud Health Technol Inform 1998;50:124-30.
71. Chaudhry A, Sutton C, Wood J, Stone R, McCloy R. Learning rate for laparoscopic surgical skills on MIST VR, a virtual reality simulator: quality of human-computer interface. Ann R Coll Surg Engl 1999;81:281-6.
72. Weghorst S, Airola C, Oppenheimer P, Edmond CV, Patience T, Heskamp D. Validation of the Madigan ESS simulator. Stud Health Technol Inform 1998;50:399-405.
73. Gallagher AG, Richie K, McClure N, McGuigan J. Objective psychomotor skills assessment of experienced, junior, and novice laparoscopists with virtual reality. World J. Surg 2001;25:1478-83.
74. Haluck RS, Marshall RL, Krummel TM, Melkonian MG. Are surgery training programs ready for virtual reality? A survey of program directors in general surgery. J Am Coll Surg 2001;193:660-5.
75. van Ruijven LJ, Beek M, Donker E, van Eijden Tm. The accuracy of joint surface models constructed from data obtained with an electromagnetic tracking device. J Biomech 2000;33:1023-8.
76. Woodburn J, Turner ED, Helliwell PS, Barker S. A preliminary study determining the feasibility of electromagnetic tracking for kinematics at the ankle joint complex. Rheumatology 1999;38:1260-8.
77. Mackay S, Datta V, Mandalia M, Bassett P, Darzi A. Electromagnetic motion analysis in the assessment of surgical skill: relationship between time and movement. ANZ J Surg 2002;72:632-4.
78. Datta V, Chang A, Mackay S, Darzi A. The relationship between motion analysis and surgical technical assessments. Am J Surg 2002;184:70-3.
79. Torkington J, Smith SG, Rees BI, Darzi A. Skill transfers from virtual reality to a real laparoscopic task. Surg Endosc 2001;15:1076-9.
80. Datta V, Mandalia M, Mackay S, Chang A, Cheshire N, Darzi A. Relationship between skill and outcome in the laboratory-based model. Surgery 2002;131:318-23.