Realizing the Promise and Importance of Performance-Based Assessment

Teaching and Learning in Medicine

Volumn 25, Issue SUPPL.1, 2013, Pages

  Kogan, Jennifer R ^a   Holmboe, Eric ^b  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

^b AMERICAN BOARD OF INTERNAL MEDICINE   (United States)

Author keywords

competence; work based assessment

Indexed keywords

ARTICLE; CLINICAL COMPETENCE; CURRICULUM; EDUCATION; HUMAN; MEDICAL EDUCATION; METHODOLOGY; STANDARD;

CLINICAL COMPETENCE; COMPETENCY-BASED EDUCATION; EDUCATION, MEDICAL; EDUCATIONAL MEASUREMENT; HUMANS;

EID: 84888034685     PISSN: 10401334     EISSN: None     Source Type: Journal    
DOI: 10.1080/10401334.2013.842912     Document Type: Article

References (81)

1. Epstein, R M. 2007. Assessment in medical education. New England Journal of Medicine, 356: 387 - 96.
2. Miller, G E. 1990. The assessment of clinical skills/competence/performance. Academic Medicine, 65: S63 - 7.
3. Swanwick, T and Chana, N. 2009. Workplace-based assessment. British Journal of Hospital Medicine, 70: 290 - 3. (London)
4. Van der vleuten, C and Verhoeven, B. 2013. In-training assessment developments in postgraduate education in Europe. ANZ Journal of Surgery, doi::10.1111/ans.12190
5. Batalden, P, Leach, D, Swing, S, Dreyfus, H and Dreyfus, S. 2002. General competencies and accreditation in graduate medical education. Health Affairs, 21: 103 - 11.
6. CanMEDS. http://www.royalcollege.ca/portal/page/portal/rc/canmeds
7. General Medical Council's Good Medical Practice. http://www.gmc-uk.org/guidance/good_medical_practice.asp
8. Carraccio, C, Wolfsthal, S D, Englander, R, Ferentz, K and Martin, C. 2002. Shifting paradigms: From Flexner to competencies. Academic Medicine, 77: 361 - 7.
9. Carraccio, C L and Englander, R. 2013. From Flexner to competencies: Reflections on a decade and the journey ahead. Academic Medicine, 88
10. Govaerts, M JB, van der Vleuten, C PM, Schuwirth, L WT and Muijtjens, A MM. 2007. Broadening perspectives on clinical performance assessment: Rethinking the nature of in-training assessment. Advances in Health Sciences Education: Theory and Practice, 12: 239 - 60.
11. Rethans, J, Sturmans, F, Drop, R, van der Vleuten, C and Hobus, P. 1991. Does competence of general practitioners predict their performance? Comparison between examination setting and actual practice. British Medical Journal, 303: 1377 - 80.
12. Kopelow, M, Schnabl, G, Hassard, T, Klass, D, Beazley, G Hechter, F. 1992. Assessing practicing physicians in two settings using standardized patients. Academic Medicine, 67 (SUPPL. 10.): S19 - 21.
13. Frenk, J, Chen, L, Bhutta, Z A, Cohen, J, Crisp, N Evans, T. 2010. Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. Lancet, 376: 1923 - 58.
14. December 2008. Institute of Medicine. Resident duty hours: Enhancing sleep, supervision, and safety. http://www.iom.edu/Reports/2008/Resident-Duty-Hours-Enhancing-Sleep-Supervision-and-Safety.aspx. Accessed July 8, 2013.
15. March 2001. Institute of Medicine. Crossing the quality chasm: A new health system for the 21st century. http://www.iom.edu/~/media/Files/Report%20Files/2001/Crossing-the-Quality-Chasm/Quality%20Chasm%202001%20%20report%20brief.pdf. Accessed July 8, 2013
16. Kennedy, T J, Lingard, L, Baker, G R, Kitchen, L and Regehr, G. 2007. Clinical oversight: Conceptualizing the relationship between supervision and safety. Journal of General Internal Medicine, 22: 1080 - 5.
17. Ten cate, O, Snell, L and Carraccio, C. 2010. Medical competence: The interplay between individual ability and the healthcare environment. Medical Teacher, 32: 669 - 75.
18. Ten Cate, O. 2006. Trust, competence, and the supervisor's role in postgraduate training. British Medical Journal, 333: 748 - 51.
19. Kroboth, F J, Kapoor, W, Brown, F H, Karpf, M and Levey, G S. 1985. A comparative trial of the clinical evaluation exercise. Archives of Internal Medicine, 145: 1121 - 3.
20. Schuwirth, L W and van der vleuten, C PM. 2011. General overview of the theories used in assessment: AMEE Guide No. 57. Medical Teacher, 33: 783 - 97.
21. Martinez, M E and Lipson, J L. 1989. Assessment for learning. Education Leader, 47: 73 - 5.
22. Norcini, J, Anderson, B, Bollela, V, Burch, V, Costa, M J Duvivier, R. 2011. Criteria for good assessment: Consensus statement and recommendations from the Ottawa 2010 Conference. Medical Teacher, 33: 206 - 14.
23. Van der vleuten, C PM. 1996. The assessment of professional competence: Developments, research, and practical implications. Advances in Health Sciences Education, 1: 41 - 67.
24. Norcini, J and Burch, V. 2007. Workplace-based assessment as an educational tool: AMEE Guide No 31. Medical Teacher, 29: 855 - 71.
25. Kogan, J R, Conforti, L N, Iobst, W and Holmboe, E. 2013. Rater cognition as a patient care problem for medical education. Academic Medicine, In press
26. Schuwirth, L W and van der vleuten, C PM. 2011. Programmatic assessment: From assessment of learning to assessment for learning. Medical Teacher, 33: 478 - 85.
27. Kogan, J R, Holmboe, E S and Hauer, K E. 2009. Tools for direct observation and assessment of clinical skills of medical trainees: A systematic review. Journal of the American Medical Association, 302: 1316 - 26.
28. Norcini, J J, Blank, L L, Arnold, G K and Kimball, H R. 1995. The mini-CEX (clinical evaluation exercise): a preliminary investigation. Annals of Intern Medicine, 123: 795 - 99.
29. Norcini, J J, Blank, L L, Duffy, F D and Fortna, G S. 2003. The mini-CEX: A method for assessing clinical skills. Annals of Internal Medicine, 138: 476 - 81.
30. Wilkinson, J R, Crossley, J G, Wragg, A, Mills, P, Cowan, G and Wade, W. 2008. Implementing workplace-based assessment across the medical specialties in the United Kingdom. Medical Education, 42: 364 - 73.
31. Reznick, R, Regehr, G, MacRae, H, Martin, J and McCulloch, W. 1997. Testing technical skill via an innovative "bench station" examination. American Journal of Surgery, 173: 226 - 30.
32. Cruess, R, McIlroy, J H, Cruess, S, Ginsburg, S and Steinert, Y. 2006. The professionalism mini-evaluation exercise: A preliminary investigation. Academic Medicine, 81 (10 SUPPL.): S74 - 8.
33. Miller, A and Archer, J. 2010. Impact of workplace based assessment on doctors' education and performance: a systematic review. British Medical Journal, 341: c5064
34. Jennett, P A, Scott, S M and Atkinson, M A. 1995. Patient charts and office management decisions: Chart audit and chart stimulated recall. Journal of Continuing Education in the Health Professions, 15: 31 - 9.
35. Carr, S. 2006. The Foundation Programme assessment tools: An opportunity to enhance feedback to trainees?. Postgraduate Medical Journal, 82: 576 - 9.
36. Davies, H, Archer, J, Southgate, L and Norcini, J. 2009. Initial evaluation of the of the Foundation assessment programme. Medical Education, 43: 73 - 81.
37. Benson, S, Munger, B S, Krome, R L, Maatsch, J C and Podgorny, G. 1982. The certification examination in emergency medicine: An update. Annals of Emergency Medicine, 11: 91 - 6.
38. Jennett, P and Affleck, L. 1998. Chart audit and chart stimulated recall as methods of needs assessment in continuing professional health education. Journal of Continuing Education in the Health Professions, 18: 163 - 71.
39. Green, M L, Reddy, S G and Holmboe, E S. 2009. Teaching and evaluating point of care learning with an internet-based clinical question portfolio. Journal of Continuing Education in the Health Professions, 29: 209 - 19.
40. Pelgrim, E AM, Kramer, A WM, Mokkink, H GA, van den Elsen, L, Grol, RPTM and van der Vleuten, C PM. 2011. In-training assessment using direct observation of single-patient encounters: A literature review. Advances in Health Sciences Education: Theory and Practice, 16: 131 - 42.
41. Norcini, J J. 2005. Current perspectives in assessment: The assessment of performance at work. Medical Education., 39: 880 - 9.
42. Gingerich, A, Regehr, G and Eva, K W. 2011. Rater-based assessments as social judgments: Rethinking the etiology of rater errors. Academic Medicine, 86 (10 SUPPL.): S1 - 7.
43. Yeates, P, O'Neill, P, Mann, K and Eva, K. 2013. Seeing the same thing differently: Mechanisms that contribute to assessor differences in directly-observed performance assessments. Advances in Health Sciences Education: Theory and Practice, 18: 325 - 41.
44. Crossley, J and Jolly, B. 2012. Making sense of work-based assessment: Ask the right questions, in the right way, about the right things, of the right people. Medical Education, 46: 28 - 37.
45. Redfern, S, Norman, I, Calman, L, Watson, R and Murrells, T. 2002. Assessing competence to practise in nursing: a review of the literature. Research Papers in Education, 17: 51 - 77.
46. Lockyer, J. 2013. Multi-source feedback: Can it meet criteria for good assessment?. Journal of Continuing Education in the Health Professions, 33: 89 - 98.
47. Lockyer, J and Clyman, S. 2008. " Multisource Feedback (360 degree evaluation) ". In Practical guide to the evaluation of clinical competence, Edited by: Holmboe, E S and Hawkins, R E. Philadelphia, PA: Mosby-Elsevier.
48. Baker, D P, Salas, E, King, H, Battles, J and Barach, P. 2005. The role of teamwork in the professional education of physicians: Current status and assessment recommendations. Joint Commission Journal on Quality and Patient Safety, 31: 185 - 202.
49. Interprofessional Education Collaborative. Core competencies for interprofessional collaborative practice. Available at: https://www.aamc.org/download/186750/data/. Accessed July 30, 2013
50. Baker, D P, Day, R and Salas, E. 2006. Teamwork as an essential component of high-reliability organizations. Health Services Research, 41: 1576 - 98.
51. Agency for Healthcare Research and Quality. TeamSTEPPS. Available at: http://www.ahrq.gov/professionals/education/curriculum-tools/teamstepps/. Accessed July 30, 2013
52. Chesluk, B, Bernabeo, E, Hess, B, Lynn, L, Reddy, S and Holmboe, E. 2012. A new assessment tool is designed to give hospitalists feedback to improve interprofessional teamwork and produce better patient care. Health Affairs, 31: 2485 - 92.
53. Wood, L, Hassell, Whitehouse, A, Bullock, A and Wall, D. 2006. A literature review of multi-source feedback systems within and without health services, leading to 10 tips for their successful design. Medical Teacher, 28: e185 - 91.
54. Hess, B J, Lynn, L A, Holmboe, E S and Lipner, R S. 2009. Toward better care coordination through improved communication with referring physicians. Academic Medicine, 84 (SUPPL. 10.): S109 - 12.
55. Lipner, R S, Blank, L L, Leas, B F and Fortna, G S. 2002. The value of patient and peer ratings in recertification. Academic Medicine, 77 (10 SUPPL.): S64 - 6.
56. Levinson, W, Lesser, C S and Epstein, R M. 2010. Developing physician communication skills for patient centered care. Health Affairs, 29: 1310 - 18.
57. Agency for Healthcare Research and Quality. Consumer Assessment of Healthcare Providers and Systems (CAHPS). Available at: http://cahps.ahrq.gov/.Accessed July 30, 2013
58. Solomon, L S, Hays, R D, Zaslavsky, A M, Alan, M, Ding, L and Cleary, P D. 2005. Psychometric properties of a group-level Consumer Assessment of Health Plans Study (CAHPS) instrument. Medical Care, 43: 53 - 60.
59. Hays, R D, Chong, K, Brown, J, Spitzer, K L and Horne, K. 2003. Patient reports and ratings of individual physicians: An evaluation of the DoctorGuide and Consumer Assessment of Health Plans Study provider-level surveys. American Journal of Medical Quality, 18: 190 - 6.
60. Wright, C, Richards, S H, Hill, J J, Roberts, M J, Norman, G R Greco, M. 2012. Multisource feedback in medical regulation: The example of the UK GMC Patient and Colleague Questionnaires. Academic Medicine, 87: 1668 - 78.
61. Physician Achievement Review. Available at: http://www.par-program.org/information/. Accessed July 30, 2013
62. Gray, B, Weng, W and Holmboe, E S. 2012. An assessment of patient based and practice infrastructure based measures of the patient centered medical home. Do we need to ask the patient?. Health Services Research, 47 (1 Pt 1): 4 - 21.
63. Concato, J and Feinstein, A R. 1997. Asking patients what they like: Overlooked attributes of patient satisfaction with primary care. American Journal of Medicine, 102: 399 - 406.
64. Agency for Healthcare Research and Quality. National measures clearinghouse. Available at: http://www.qualitymeasures.ahrq.gov/. Accessed July 30, 2013
65. American Medical Association. Physician Consortium for Performance Improvement. Available at http://www.ama-assn.org/ama/pub/physician-resources/physician-consortium-performance-improvement.page. Accessed July 30, 2013
66. Berenson, R A, Pronovost, P J and Krumholz, H M. Achieving the potential of health care performance measures. Timely analysis of immediate health policy issues. Available at: http://www.rwjf.org/en/research-publications/find-rwjf-research/2013/05/achieving-the-potential-of-health-care-performance-measures.html. Accessed July 30, 2013
67. Ivers, N, Jamtvedt, G, Flottorp, S, Young, J M, Odgaard-Jensen, J French, S D. 2012. Audit and feedback: Effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews, doi: CD000259. doi:10.1002/14651858.CD000259.pub3
68. Willett, L L, Heudebert, G R, Palonen, K P, Massie, F S, Kiefe, C I Allison, J J. 2009. The importance of measuring competency-based outcomes: Standard evaluation measures are not surrogates for clinical performance of internal medicine residents. Teaching and Learning in Medicine, 21: 87 - 93.
69. olmboe, E S, Hess, B J, Conforti, L N and Lynn, L A. 201. Comparative trial of a web-based tool to improve the quality of care provided to older adults in residency clinics: Modest success and a tough road ahead. Academic Medicine, 87: 627 - 34.
70. The Society for Thoracic Surgery. STS national database. Available at: http://www.sts.org/national-database. Accessed July 30, 2013
71. American College of Cardiology. National cardiovascular data registry. Available at: https://www.ncdr.com/webncdr/. Accessed July 30, 2013
72. American Board of Family Medicine. Initial certification/residency. Available at: https://www.theabfm.org/cert/index.aspx. Accessed July 30, 2013
73. Fitzpatrick, R, Bowling, A, Gibbons, E, Haywood, K, Jenkinson, C Mackintosh, A. A structured review of patient-reported measures in relation to selected chronic conditions, perceptions of quality of care and career impact. Available at: http://phi.uhce.ox.ac.uk/. Accessed July 30, 2013
74. National Health Service (UK). Health and Social Care Information Center. Patient reported outcome measures. Available at: http://www.hscic.gov.uk/proms. Accessed July 30, 2013
75. Buckley, S, Coleman, J, Davison, I, Khan, K S, Zamora, J Malick, S. 2009. The educational effects of portfolios on undergraduate student learning; a Best Evidence Medical Education (BEME) systematic review. BEME Guide 11. Medical Teacher, 31: 282 - 98.
76. Holmboe, E S, Ward, D S, Reznick, R K, Katsufrakis, P J, Leslie, K M Patel, V L. 2011. Faculty development in assessment: The missing link in competency-based medical education. Academic Medicine, 86: 460 - 7.
77. Kogan, J R, Conforti, L, Berbaneo, E, Iobst, W and Holmboe, E. 2011. Opening the black box of clinical skills assessment via observation: A conceptual model. Medical Education, 45: 1048 - 60.
78. Dijkstra, J, Galbraith, R, Hodges, B D, McAvoy, P A, McCrorie, P Southgate, L J. 2012. Expert validation of fit-for-purpose guidelines for designing programmes of assessment. BMC Medical Education, 12: 20 doi: 10.1186/1472-6920-12-20
79. Dijkstra, J, van der vleuten, C MP and Schuwirth, L WT. 2009. A new framework for designing programmes of assessment. Advances in Health Sciences Education: Theory and Practice, 15: 379 - 93.
80. Van der Vleuten, C PM, Schuwirth, L WT, Scheele, F, Driessen, E W and Hodges, B. 2010. The assessment of professional competence: Building blocks for theory development. Best Practice & Research Clinical Obstetrics and Gynaecology, 24: 703 - 19.
81. Saedon, H, Salleh, S, Balakrishnan, A, Imray, C HE and Saedon, M. 2012. The role of feedback in improving the effectiveness of workplace based assessments: A systematic review. BMC Medical Education, 12: 25 doi: 10.1186/1472-6920-12-25