Methods for evaluating the clinical competence of residents in internal medicine: A review

Annals of Internal Medicine

Volumn 129, Issue 1, 1998, Pages 42-48

  Holmboe, Eric S ^a   Hawkins, Richard E ^b  

^a WALTER REED ARMY MEDICAL CENTER   (United States)

^b Uniformed Services University of the Health Sciences and the Walter Reed National Military Medical Center   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPETENCE; INTERNAL MEDICINE; MEDICAL AUDIT; MEDICAL RECORD; MEDICAL SOCIETY; PRIORITY JOURNAL; RESIDENT; REVIEW;

EID: 0032124811     PISSN: 00034819     EISSN: None     Source Type: Journal    
DOI: 10.7326/0003-4819-129-1-199807010-00011     Document Type: Review

References (81)

1. American Board of Internal Medicine. Guide to the Evaluation of Residents in Internal Medicine 1990-92. Philadelphia: American Board of Internal Medicine; 1992.
2. Neufeld V, Norman GR, eds. Assessing Clinical Competence. New York: Springer; 1985.
3. Eisenberg JM. Evaluating internists' clinical competence. J Gen Intern Med. 1989;4:139-43.
4. Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(9 Suppl):S63-7.
5. American Board of Internal Medicine. Project Professionalism. Philadelphia: American Board of Internal Medicine; 1994.
6. American Medical Association. Directory of Graduate Medical Education Programs. Chicago: American Med Assoc; 1990.
7. Tugwell P, Dok C. Medical record review. In: Neufeld V, Norman GR, eds. Assessing Clinical Competence. New York: Springer; 1985:142-81.
8. Holmboe ES, Gross RM, Hawkins RE. Chart review for residents [Letter]. Acad Med. 1996;71:1139-40.
9. Scranton R, Holmboe ES, Sumption KF, Hawkins RE. Impact of feedback from medical record audit on resident compliance with preventive health measures. Acad Med. 1998; [In press].
10. Winickoff RN, Coltin KL, Morgan MM, Buxbaum RC, Barnett GO. Improving physician performance through peer comparison feedback. Med Care. 1984;22:527-34.
11. Wayland MT, Culik D, Goldsby C, Carion W, Orzame N. Screening by family practice and internal medicine residents and the effect of an intervention program. J Med Educ. 1987;62:519-22.
12. Kern DE, Harris WL, Boekeloo BO, Barker LR, Hogeland P. Use of an outpatient medical record audit to achieve educational objectives: changes in residents' performances over six years. J Gen Intern Med. 1990;5:218-24.
13. Ramsdell JW, Berry CC. Evaluation of general and traditional internal medicine residents utilizing a medical records audit based on educational objectives. Med Care. 1983;21:1144-53.
14. Martin AR, Wolf MA, Thibodeau LA, Dzau V, Braunwald E. A trial of two strategies to modify the test-ordering behavior of medical residents. N Engl J Med. 1980;303:1330-6.
15. Ognibene AJ, Jarjoura DG, Illera VA, Blend DA, Cugino AE, Whittier FC. Using chart reviews to assess residents' performances of components of physical examinations: a pilot study. Acad Med. 1994;69:583-7.
16. Opila DA. The impact of feedback to medical housestaff on chart documentation and quality of care in the outpatient setting. J Gen Intern Med. 1997; 12:352-6.
17. Engebretsen B. Peer review in graduate education. N Engl J Med. 1977;296: 1230-1.
18. Ashton CM. "Invisible" doctors: making a case for involving medical residents in hospital quality improvement programs. Acad Med. 1993;68:823-4.
19. Norman GR, Neufeld VR, Walsh A, Woodward CA, McConvey GA. Measuring physicians' performances by using simulated patients. J Med Educ. 1985;60:925-34.
20. Leaf DA, Neighbor WE, Schaad D, Scott CS. A comparison of self-report and chart audit in studying resident physician assessment of cardiac risk factors. J Gen Intern Med. 1995;10:194-8.
21. Gennis VM, Gennis MA. Supervision in the outpatient clinic: effects on teaching and patient care. J Gen Intern Med. 1993;9:116-7.
22. Brook RH, Lohr KN. Monitoring quality of care in the Medicare program. Two proposed systems. JAMA. 1987;258:3138-41.
23. Garibaldi RA, Trontell MC, Waxman H, Holbrook JH, Kanya DT, Khoshbin S, et al. The In-Training Examination in internal medicine. Ann Intern Med. 1994;121:117-23.
24. Waxman H, Braunstein G, Dantzker D, Goldberg S, Lefrak S, Lichstein E, et al. Performance on the internal medicine second-year residency In-Training Examination predicts the outcome of the ABIM certifying examination. J Gen Intern Med. 1994;9:692-4.
25. Grossman RS, Fincher RM, Layne RD, Seelig CB, Berkowitz LR, Levine MA. Validity of the In-Training Examination for predicting American Board of Internal Medicine certifying examination scores. J Gen Intern Med. 1992;7: 63-7.
26. Wolfsthal SD, Hooper FJ. In-Training Examination in internal medicine [Letter]. Ann Intern Med. 1995;122:73.
27. O'Dell JR. In-Training Examination in internal medicine [Letter]. Ann Intern Med. 1995;122:73-4.
28. Brailovsky CA, Grand'Maison P, Lescop J. Residency directors' predictions of candidates' performances on a licensing examination. Acad Med. 1995;70: 410-4.
29. Yindra KJ, Rosenfeld PS, Donnelly MB. Medical school achievements as predictors of residency performance. J Med Educ. 1988;63:356-63.
30. Norcini JJ, Grosso LJ, Shea JA, Webster GD. The relationship between features of residency training and ABIM certifying examination performance. J Gen Intern Med. 1987;2:330-6.
31. Streiner DL. Global rating scales. In: Neufeld V, Norman GR, eds. Assessing Clinical Competence. New York: Springer; 1985:119-41.
32. Van Rosendaal GM, Jennett PA. Comparing peer and faculty evaluations in an internal medicine residency. Acad Med. 1994;69:299-303.
33. Matthews DA, Feinstein AR. A new instrument for patients' ratings of physician performance in the hospital setting. J Gen Intern Med. 1989;4:14-22.
34. Klessig J, Robbins AS, Wieland D, Rubenstein L. Evaluating humanistic attributes of internal medicine residents. J Gen Intern Med. 1989;4:514-21.
35. Tamblyn R, Benaroya S, Snell L, McLeod P, Schnarch B, Abrahamowicz M. The feasibility and value of using patient satisfaction ratings to evaluate internal medicine residents. J Gen Intern Med. 1994;9:146-52.
36. Thompson WG, Lipkin M Jr, Gilbert DA, Guzzo RA, Roberson L. Evaluating evaluation: assessment of the American Board of Internal Medicine Resident Evaluation Form. J Gen Intern Med. 1990;5:214-7.
37. Haber RJ, Avins AL. Do ratings on the American Board of Internal Medicine Resident Evaluation Form detect differences in clinical competence? J Gen Intern Med. 1994;9:140-5.
38. Borman WC. Effects of instructions to avoid halo error on reliability and validity of performance evaluation ratings. J Appl Psychol. 1975;60:556-60.
39. Gray JD. Global rating scales in residency education. Acad Med. 1996;71(1 Suppl): S55-63.
40. Wray NP, Friedland JA. Detection and correction of house staff error in physical diagnosis. JAMA. 1983;249:1035-7.
41. Woolliscroft JO, Stross JK, Silva J Jr. Clinical competence certification: a critical appraisal. J Med Educ. 1984;59:799-805.
42. Norcini JJ, Blank LL, Arnold GK, Kimball HR. The mini-CEX (clinical evaluation exercise): a preliminary investigation. Ann Intern Med. 1995;123: 795-9.
43. Fletcher SW, O'Malley MS, Bunce LA. Physicians' abilities to detect lumps in silicone breast models. JAMA. 1985;253:2224-8.
44. Mangione S, Nieman LZ. Cardiac auscultatory skills of internal medicine and family practice trainees. JAMA. 1997;278:717-22.
45. Blank LL, Grosso LJ, Benson JA Jr. A survey of clinical skills evaluation practices in internal medicine residency programs. J Med Educ. 1984;59: 401-6.
46. Noel GL, Herbers JE Jr, Caplow MP, Cooper GS, Pangaro LN, Harvey J. How well do internal medicine faculty members evaluate the clinical skills of residents? Ann Intern Med. 1992;117:757-65.
47. Herbers JE Jr, Noel GL, Cooper GS, Harvey J, Pangaro LN, Weaver MJ. How accurate are faculty evaluations of clinical competence? J Gen Intern Med. 1989;4:202-8.
48. Kroboth FJ, Hanusa BH, Parker S, Coulehan JL, Kapoor WN, Brown FH, et al. The inter-rater reliability and internal consistency of a clinical evaluation exercise. J Gen Intern Med. 1992;7:174-9.
49. Kroboth FJ, Kapoor W, Brown FH, Karpf M, Levey GS. A comparative trial of the clinical evaluation exercise. Arch Intern Med. 1985; 145:1121-3.
50. Barrows HS. An overview of the uses of standardized patients for teaching and evaluating clinical skills. Acad Med. 1993;68:443-53.
51. Stillman P, Swanson D, Regan MB, Philbin MM, Nelson V, Ebert T, et al. Assessment of clinical skills of residents utilizing standardized patients. A follow-up study and recommendations for application. Ann Intern Med. 1991;114:393-401.
52. Stillman PL, Swanson DB, Smee S, Stillman AE, Ebert TH, Emmel VS, et al. Assessing clinical skills of residents with standardized patients. Ann Intern Med. 1986;105:762-71.
53. Van der Vleuten CP, Swanson DB. Assessment of clinical skills with standardized patients: state of the art. Teaching and Learning in Internal Medicine. 1990;2:58-76.
54. Colliver JA, Williams RG. Technical issues: test application. MMC. Acad Med. 1993;68:454-60.
55. Anderson MB, Stillman PL, Wang Y. Growing use of standardized patients in teaching and evaluation in medical education. Teach Learn Med. 1994;6: 15-22.
56. Day SC, Grosso LJ, Norcini JJ Jr, Blank LL, Swanson DB, Horne MH. Residents' perception of evaluation procedures used by their training program. J Gen Intern Med. 1990;5:421-6.
57. Ainsworth MA, Rogers LP, Markus JF, Dorsey NK, Blackwell TA, Petrusa ER. Standardized patient encounters. A method for teaching and evaluation. JAMA. 1991;266:1390-6.
58. Petrusa ER, Blackwell TA, Ainsworth MA. Reliability and validity of an objective structured clinical examination for assessing the clinical performance of residents. Arch Intern Med. 1990;150:573-7.
59. Martin JA, Reznick RK, Rothman A, Tamblyn RM, Regehr G. Who should rate candidates in an objective structured clinical examination? Acad Med. 1996;71:170-5.
60. Schwarte RW, Donnelly MB, Sloan DA, Johnson SB, Strodel WE. The relationship between faculty ward evaluations, OSCE, and ABSITE as measures of surgical intern performance. Am J Surg. 1995;169:414-7.
61. Joorabchi B, Devries JM. Evaluation of clinical competence: the gap between expectation and performance. Pediatrics. 1996;97:179-84.
62. Kokotailo PK, Langhough R, Neary EJ, Matson SC, Fleming MF. Improving pediatric residents' alcohol and other drug use clinical skills: use of an experiential curriculum. Pediatrics. 1995;96(1 pt 1):99-104.
63. Singer PA, Cohen R, Robb A, Rothman A. The ethics objective structured clinical examination. J Gen Intern Med. 1993;8:23-8.
64. Pololi LH, Potter SP. Behavioral change in preventive medicine. An efficacy assessment of a physician education module. J Gen Intern Med. 1996;11: 545-7.
65. Burdick WP, Ben-David MF, Swisher L, Becher J, Magee D, McNamara R, et al. Reliability of performance-based clinical skill assessment of emergency medicine residents. Acad Emerg Med. 1996;3:1119-23.
66. Eggly S, Afonso N, Rojas G, Baker M, Cardozo L, Robertson RS. An assessment of residents' competence in the delivery of bad news to patients. Acad Med. 1997;72:397-9.
67. Boudreau D, Tamblyn R, Dufresne L. Evaluation of consultative skills in respiratory medicine using a structured medical consultation. Am J Respir Crit Care Med. 1994;150(5 pt 1):1298-304.
68. Sloan PA, Donnelly MB, Schwartz RW, Sloan DA. Cancer pain assessment and management by housestaff. Pain. 1996;67:475-81.
69. Chalabian J, Garman K, Wallace P, Dunnington G. Clinical breast evaluation skills of house officers and students. Am Surg. 1996;62:840-5.
70. Day RP, Hewson MG, Kindy P Jr, Van Kirk J. Evaluation of resident performance in an outpatient internal medicine clinic using standardized patients. J Gen Intern Med. 1993;8:193-8.
71. Reznick RK, Blackmore D, Dauphinee WD, Rothman AI, Smee S. Large-scale high-stakes testing with an OSCE: report from the Medical Council of Canada. Acad Med. 1996;71:519-21.
72. Cohen R, Rothman AI, Bilan S, Ross J. Analysis of the psychometric properties of eight administrations of an objective structured clinical examination used to assess international medical graduates. Acad Med. 1996;71: S22-4.
73. Sutnick AI, Stillman PL, Norcini JJ, Friedman M, Williams RG. Trace DA, et al. Pilot study of the use of the ECFMG clinical competence assessment to provide profiles of clinical competencies of graduates of foreign medical schools for residency directors. Educational Commission for Foreign Medical Graduates. Acad Med. 1994;69:65-7.
74. Rothman AI, Cohen R. Bilan S. A comparison of short- and long-case stations in a multiple-station test of clinical skills. Acad Med. 1996;71(1 Suppl):S110-2.
75. Shatzer JH, Wardrop JL, Williams RG, Hatch TF. Generalizability of performance on different-station-length standardized patient cases. Teach Learn Med. 1994;6:54-8.
76. Stillman PL, Regan MB, Haley HL, Norcini JJ, Friedman M, Sutnick AI. The use of a patient note to evaluate clinical skills of first-year residents who are graduates of foreign medical schools. Acad Med. 1992;76(10 Suppl): S57-9.
77. Solomon DJ, Speer AJ, Perkowski LC, DiPette D. Evaluating problem solving based on the use of history findings in a standardized-patient examination. Acad Med. 1994;69:754-7.
78. Rosebraugh CJ, Speer AJ, Ainsworth MA, Solomon DJ, Callaway MR, Holden MD. Developing a presentation and problem-solving station in a multistation standardized-patient examination. Acad Med. 1996;71(1 Suppl): S102-4.
79. Norman GR, Tugwell P, Feightner JW. A comparison of resident performance on real and simulated patients. J Med Educ. 1982;57:708-15.
80. Hull AL, Hodder S, Berger B, Ginsberg D, Lindheim N, Quan J, et al. Validity of three clinical performance assessments of internal medicine clerks. Acad Med. 1995;70:517-22.
81. Dupras DM, Li JT. Use of an objective structured clinical examination to determine clinical competence. Acad Med. 1995;70:1029-34.