Opening the black box of clinical skills assessment via observation: A conceptual model

Medical Education

Volumn 45, Issue 10, 2011, Pages 1048-1060

  Kogan, Jennifer R ^a   Conforti, Lisa ^b   Bernabeo, Elizabeth ^b   Iobst, William ^b   Holmboe, Eric ^b  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b AMERICAN BOARD OF INTERNAL MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ARTICLE; CLINICAL COMPETENCE; DOCTOR PATIENT RELATION; EDUCATION; FEMALE; HUMAN; MALE; MEDICAL EDUCATION; METHODOLOGY; MIDDLE AGED; OBSERVATION; QUESTIONNAIRE; STANDARD; THEORETICAL MODEL; VIDEORECORDING;

ADULT; CLINICAL COMPETENCE; EDUCATION, MEDICAL, GRADUATE; EDUCATIONAL MEASUREMENT; FEMALE; HUMANS; INTERNSHIP AND RESIDENCY; MALE; MIDDLE AGED; MODELS, THEORETICAL; OBSERVATION; PHYSICIAN-PATIENT RELATIONS; QUESTIONNAIRES; VIDEOTAPE RECORDING;

EID: 80052822905     PISSN: 03080110     EISSN: 13652923     Source Type: Journal    
DOI: 10.1111/j.1365-2923.2011.04025.x     Document Type: Article

References (45)

1. Kalet AL, Gillespie CC, Schwarz MD etal. New measures to establish the evidence base for medical education: identifying educationally sensitive patient outcomes. Acad Med 2010;85 (5):844-51.
2. Accreditation Council for Graduate Medical Education. ACGME Outcomes Project. . [Accessed 24 October 2010.]
3. Frank JR. The CanMEDS 2005 Physician Competency Framework. Ottawa, ON: Royal College of Physicians and Surgeons of Canada 2005.
4. Ericson KA. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 2004;79 (10 Suppl):70-81.
5. Accreditation Council for Graduate Medical Education. ACGME Program Requirements for Graduate Medical Education in Internal Medicine. . [Accessed 24 October 2010.]
6. Training and Assessment. The Foundation Programme. . [Accessed 24 October 2010.]
7. Miller G. The assessment of clinical skills/competence/performance. Acad Med 1990;65 (9 Suppl):S63-7.
8. van der Vleuten C, Schuwirth LW. Assessing professional competence: from methods to programmes. Med Educ 2005;39 (3):309-17.
9. Southgate L, Cox J, David T etal. The General Medical Council's performance procedures: peer review of performance in the workplace. Med Educ 2001;35 (1 Suppl):9-19.
10. Hodges BD. A tea-steeping or i-Doc model for medical education? Acad Med 2010;85 (9 Suppl):34-44.
11. ten Cate O, Scheele F. Competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med 2007;82 (6):542-7.
12. Kogan JR, Holmboe ES, Hauer KE. Tools for direct observation and assessment of clinical skills of medical trainees: a systematic review. JAMA 2009;302 (12):1316-26.
13. Williams RG, Klamen DA, McGaghie WC. Cognitive, social, and environmental sources of bias in clinical performance ratings. Teach Learn Med 2003;15 (4):270-92.
14. Woehr DJ, Huffcutt AI. Rater training for performance appraisal: a quantitative review. J Occup Organ Psychol 1994;67 (3):189-205.
15. Downing SM, Haladyna TM. Validity threats: overcoming interference with proposed interpretations of assessment data. Med Educ 2004;38 (3):327-33.
16. Downing SM. Threats to the validity of clinical teaching assessments: what about rater error? Med Educ 2005;39 (4):353-5.
17. McManus IC, Thompson M, Mollon J. Assessment of examiner leniency and stringency ('hawk-dove effect') in the MRCP (UK) clinical examination (PACES) using multi-facet Rasch modelling. BMC Med Educ 2006;6:42.
18. Carline JD, Paauw DS, Thiede KW, Ramsey PG. Factors affecting the reliability of ratings of students' clinical skills in a medicine clerkship. J Gen Intern Med 1992;7 (5):506-10.
19. Weingarten MA, Polliack MR, Tabenkin H, Kahan E. Variations among examiners in family medicine residency board oral examinations. Med Educ 2000;34 (1):13-7.
20. Kogan JR, Hess BJ, Conforti LN, Holmboe ES. What drives faculty ratings of residents' clinical skills? The impact of faculty's own clinical skills. Acad Med 2010;85 (Suppl):25-8.
21. Holmboe ES, Hawkins RE, Huot SJ. Effects of training in direct observation of medical residents' clinical competence. A randomised trial. Ann Intern Med 2004;140 (11):874-81.
22. Cook DA, Dupras DM, Beckman TJ, Thomas KG, Pankratz VS. Effect of rater training on reliability and accuracy of mini-CEX scores: a randomised controlled trial. J Gen Intern Med 2009;24 (1):74-9.
23. Shumway JM, Harden RM, Association for Medical Education in Europe. AMEE guide no. 25: the assessment of learning outcomes for the competent and reflective physician. Med Teach 2003;25 (6):569-84.
24. Crossley J, Humphris G, Jolly B. Assessing health professionals. Med Educ 2002;36 (9):800-4.
25. Norcini JJ, Blank LL, Arnold GK, Kimball HR. The mini-CEX (clinical evaluation exercise): a preliminary investigation. Ann Intern Med 1995;123:795-9.
26. Durning SJ, Cation LJ, Markert RJ, Pangaro LN. Assessing the reliability and validity of the mini-clinical evaluation exercise for internal medicine residency training. Acad Med 2002;77:900-4.
27. Strauss A, Corbin J. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Newbury Park, CA: Sage Publications 1998.
28. Patton MQ. Qualitative Research and Evaluation Methods. Thousand Oaks, CA: Sage Publications 2001.
29. Durning SJ, Artino AR, Pangaro LN, van der Vleuten C, Schuwirth L. Redefining context in the clinical encounter: implications for research and training in medical education. Acad Med 2010;85:894-901.
30. Koens F, Mann KV, Custers EJFM, ten Cate OTJ. Analysing the concept of context in medical education. Med Educ 2005;39:1243-9.
31. Bredo E. Reconstructing educational psychology: situated cognition and Deweyian pragmatism. Educ Psychol 1994;29:23-5.
32. Kirshner J, Whitson JA. Situated Cognition: Social, Semiotic and Psychological Perspectives. Mahwah, NJ: Lawrence Erlbaum Associates 1997.
33. Durning SJ, Artino AR, Holmboe E, Beckman TJ, van der Vleuten C, Schuwirth L. Ageing and cognitive performance: challenges and implications for physicians practising in the 21st century. J Cont Educ Health Prof 2010;30:153-60.
34. Ramsey PG, Curtis JR, Paauw DS, Carline JD, Weinrich MD. History-taking and preventive medicine skills among primary care physicians: an assessment using standardised patients. Am J Med 1998;104 (2):152-8.
35. Paauw DS, Weinrich MD, Curtis JR, Carline JD, Ramsey PG. Ability of primary care physicians to recognise physical findings associated with HIV infection. JAMA 1995;274 (17):1380-2.
36. Vukanovick-Criley JM, Criley S, Warde CM, Boker JR, Guevara-Matheus L, Churchill WH, Nelson WP, Criley JM. Competency in cardiac examination skills in medical students, trainees, physicians and faculty: a multi-centre study. Arch Intern Med 2006;166 (6):610-6.
37. Braddock CH III, Edwards KA, Hasenberg NM, Laidley TL, Levinson W. Informed decision making in out-patient practice: time to get back to basics. JAMA 1999;282 (24):2313-20.
38. Makoul G. Essential elements of communication in medical encounters: the Kalamazoo consensus statement. Acad Med 2001;76 (4):390-3.
39. Deber RB. Physicians in health care management: 7. The patient-physician partnership: changing roles and the desire for information. CMAJ 1994;151 (2):171-6.
40. Swann WB Jr. Self-verification: bringing social reality into harmony with the self. In: Suls J, Greenwald AG, eds. Social Psychological Perspectives on the Self, Vol.2. Hillsdale, NJ: Lawrence Erlbaum Associates 1983;2.
41. Senge PM, Kleiner C, Roberts C, Ross R, Smith B. The Fifth Discipline Fieldbook: Strategies for Building a Learning Organization. New York, NY: Doubleday 1994.
42. Ginsburg S, McIlroy J, Oulanova O, Eva K, Regehr G. Toward authentic clinical evaluation: pitfalls in the pursuit of competency. Acad Med 2010;85 (5):780-6.
43. Hodges B, Regehr G, McNaughton N, Tiberius R, Hanson M. OSCE checklists do not capture increasing levels of expertise. Acad Med 1999;74:1129-34.
44. Govaerts MJ, van der Vleuten CP, Schuwirth LW, Muijtjens AM. Broadening perspectives on clinical performance assessment: rethinking the nature of in-training assessment. Adv Health Sci Educ 2007;12 (2):239-60.
45. American Board of Internal Medicine. Milestones Framework. [Accessed 24 October 2010.]