Direct observation in medical education: A review of the literature and evidence for validity

Mount Sinai Journal of Medicine

Volumn 76, Issue 4, 2009, Pages 365-371

  Fromme, H Barrett ^a   Karani, Reena ^b   Downing, Steven M ^c  

^a UNIVERSITY OF CHICAGO   (United States)

^b MOUNT SINAI SCHOOL OF MEDICINE   (United States)

^c UNIVERSITY OF ILLINOIS   (United States)

Author keywords

Clinical competence; Direct observation; Internship and residency; Medical education; Medical students; Performance assessment; Performance evaluation; Performance observation; Review; Validation; Validity

Indexed keywords

CLINICAL COMPETENCE; CLINICAL OBSERVATION; CONTENT VALIDITY; EVIDENCE BASED PRACTICE; INTERNAL VALIDITY; MEDICAL EDUCATION; MEDICAL STUDENT; PERFORMANCE MEASUREMENT SYSTEM; PROFESSIONAL STANDARD; RELIABILITY; RESIDENT; RESPONSE VARIABLE; REVIEW; VALIDATION PROCESS;

CLINICAL COMPETENCE; EDUCATION, MEDICAL; EDUCATIONAL MEASUREMENT; HUMANS; INTERNSHIP AND RESIDENCY; REPRODUCIBILITY OF RESULTS; STUDENTS, MEDICAL; TEACHING;

EID: 70049095108     PISSN: 00272507     EISSN: 19317581     Source Type: Journal    
DOI: 10.1002/msj.20123     Document Type: Review

References (27)

1. Accreditation Council for Graduate Medical Education. Outcomes project (2000). Available at: http://www.acgme.org/outcome. Accessed February 2009.
2. Liaison Committee on Medical Education. Accreditation standards (2007). Available at: http://www.lcme.org/ functionslist.htm#teaching%20and%20evaluation. Accessed February 2009.
3. Howley LD, William WG. Direct observation of students during clerkship rotations: a multiyear descriptive study. Acad Med 2004; 79: 276-280.
4. Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990; 65: s63-s67.
5. Shea SC, Green R, Barney C, et al. Designing clinical interviewing training courses for psychiatric residents: a practical primer for interviewing mentors. Psychiatr Clin N Am 2007; 31: 283-314.
6. Lee AG, Carter KD. Managing the new mandate in resident education: a blueprint for translating a national mandate into local compliance. Ophthalmology 2004; 111: 1807-1812.
7. Wendling AL. Assessing resident competency in an outpatient setting. Fam Med 2004; 36: 178-184.
8. Jouriles N, Burdick W, Hobgood C. Clinical assessment in emergency medicine. Acad Emerg Med 2002; 9: 1289-1294.
9. Chaudry SI, Holmboe E, Beasley BW. The state of evaluation in internal medicine residency. J Gen Intern Med 2008; 23: 1010-1015.
10. Kumar A, Gera R, Shah G, et al. Student evaluation practices in pediatric clerkships: a survey of the medical schools in the United States and Canada. Clin Pediatr (Phila) 2004; 43: 729-735.
11. Holmboe ES. Faculty and the observation of trainee's clinical skills: problems and opportunities. Acad Med 2004; 79: 16-22.
12. Downing SM. Validity: on the meaningful interpretation of assessment data. Med Educ 2003; 37: 830-837.
13. Williams RG, Klamen DA, McGaghie WC. Cognitive, social and environmental sources of bias in clinical performance ratings. Teach Learn Med 2003; 15: 270-292.
14. Hamdy H, Prasad K, Williams R, Salih FA. Reliability and validity of the direct observation clinical encounter examination (DOCEE). Med Educ 2003; 37: 205-212.
15. Wiles CM, Dawson K, Hughes TA, et al. Clinical skills evaluation of trainees in a neurology department. Clin Med 2007; 7: 265-269.
16. Lane JL, Gottlieb RP. Structured clinical observations: a method to teach clinical skills with limited time and financial resources. Pediatrics 2000; 105: 973-977.
17. Holmboe ES, Hawkins RE, Huot SJ. Effect of training in direct observation of medical residents' clinical competence: a randomized trial. Ann Intern Med 2004; 140: 874-881.
18. De Haes JCJM, Oort FG, Hulsman RL. Summative assessment of medical students' communication skills and professional attitudes through observation in practice. Med Teach 2005; 27: 583-589.
19. Kogan JR, Bellini LM, Shea JA. Feasibility, reliability, and validity of the mini-clinical evaluation exercise (mCEX) in a medicine core clerkship. Acad Med 2003; 78: s33-s35.
20. Wass V, Jolly B. Does observation add to the validity of the long case? Med Educ 2001; 35: 729-734.
21. Shayne P, Gallahue F, Rinnert S, et al. Reliability of a core competency checklist assessment in the emergency department: the standardized direct observation assessment tool. Acad Emerg Med 2006; 13: 727-732.
22. Hasnain M, Connell KJ, Downing SM, et al. Toward meaningful evaluation of clinical competence: the role of direct observation in clerkship ratings. Acad Med 2004; 79: S21-S24.
23. Woolliscroft JO, Calhoun JG, Beauchamp C, et al. Evaluating the medical history: observation versus write-up review. J Med Educ 1984; 59: 19-23.
24. Friedman Z, Katznelson R, Devito I, et al. Objective assessment of manual skills and proficiency in performing epiduralanesthesia-video-assisted validation. Reg Anesth Pain Med 2006; 31: 304-310.
25. Beard JD, Jolly BC, Newble DI, et al. Assessing the technical skills of surgical trainees. Br J Surg 2005; 92: 778-782.
26. Scott DJ, Rege RV, Bergen PC, et al. Measuring operative performance after laparoscopic skills training: edited videotape versus direct observation. J Laparoendosc Adv Surg Tech A 2000; 10: 183-190.
27. Dunnington G, Reisner L, Witzke D, Fulginiti J. Structured single-observer methods of evaluation for the assessment of ward performance on the surgical clerkship. Am J Surg 1990; 159: 423-426.