Using Contribution Analysis to Evaluate Competency-Based Medical Education Programs: It's All about Rigor in Thinking

Academic Medicine

Volumn 92, Issue 6, 2017, Pages 752-758

  Van Melle, Elaine ^a,d   Gruppen, Larry ^b   Holmboe, Eric S ^c   Flynn, Leslie ^a,d   Oandasan, Ivy ^e,f   Frank, Jason R ^g  

^a ROYAL COLLEGE OF PHYSICIANS AND SURGEONS OF CANADA   (Canada)

^b UNIVERSITY OF MICHIGAN MEDICAL SCHOOL   (United States)

^c ACCREDITATION COUNCIL FOR GRADUATE MEDICAL EDUCATION   (United States)

^d QUEEN S UNIVERSITY   (Canada)

^e College of Family Physicians of Canada ^*  (Canada)

^f UNIVERSITY OF TORONTO   (Canada)

^g Royal College of Physicians and Surgeons of   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; CLINICAL COMPETENCE; CURRICULUM; EVALUATION STUDY; FEMALE; HUMAN; MALE; MEDICAL EDUCATION; STANDARDS; THINKING; YOUNG ADULT;

ADULT; CLINICAL COMPETENCE; COMPETENCY-BASED EDUCATION; CURRICULUM; EDUCATION, MEDICAL, UNDERGRADUATE; FEMALE; HUMANS; MALE; THINKING; YOUNG ADULT;

EID: 85002501610     PISSN: 10402446     EISSN: 1938808X     Source Type: Journal    
DOI: 10.1097/ACM.0000000000001479     Document Type: Article

References (53)

1. Carraccio C, Wolfsthal SD, Englander R, Ferentz K, Martin C. Shifting paradigms: From Flexner to competencies. Acad Med. 2002;77:361-367.
2. Frank JR, Snell LS, Cate OT, et al. Competencybased medical education: Theory to practice. Med Teach. 2010;32:638-645.
3. Hodges BD. A tea-steeping or i-Doc model for medical education Acad Med. 2010;85(9 suppl):S34-S44.
4. McGaghie WC, Miller GE, Sajid AW, Telder TV. Competency-Based Curriculum Development in Medical Education: An Introduction. Geneva, Switzerland: World Health Organization; 1978. Public health papers no. 68.
5. Carraccio CL, Englander R. From Flexner to competencies: Reflections on a decade and the journey ahead. Acad Med. 2013;88:1067-1073.
6. Baker GR, Norton PG, Flintoft V, et al. The Canadian Adverse Events Study: The incidence of adverse events among hospital patients in Canada. CMAJ. 2004;170:1678-1686.
7. Choudhry NK, Fletcher RH, Soumerai SB. Systematic review: The relationship between clinical experience and quality of health care. Ann Intern Med. 2005;142:260-273.
8. Frenk J, Chen L, Bhutta ZA, et al. Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. Lancet. 2010;376:1923-1958.
9. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001.
10. Shaw R, Drever F, Hughes H, Osborn S, Williams S. Adverse events and near miss reporting in the NHS. Qual Saf Health Care. 2005;14:279-283.
11. Frank JR, Danoff D. The CanMEDS initiative: Implementing an outcomes-based framework of physician competencies. Med Teach. 2007;29:642-647.
12. Batalden P, Leach D, Swing S, Dreyfus H, Dreyfus S. General competencies and accreditation in graduate medical education. Health Aff (Millwood). 2002;21:103-111.
13. Simpson JG, Furnace J, Crosby J, et al. The Scottish doctor-Learning outcomes for the medical undergraduate in Scotland: A foundation for competent and reflective practitioners. Med Teach. 2002;24:136-143.
14. Confederation of Postgraduate Medical Education Councils. Australian curriculum framework for junior doctors. http:// curriculum. cpmec. org. Au/. Accessed August 22, 2016.
15. Swing SR; International CBME Collaborators. Perspectives on competencybased medical education from the learning sciences. Med Teach. 2010;32:663-668.
16. Medical Research Council. Developing and Evaluating Complex Interventions: New Guidance. London, UK: Medical Research Council; 2008.
17. Gruppen LD, Mangrulkar RS, Kolars JC. The promise of competency-based education in the health professions for improving global health. Hum Resour Health. 2012;10:43.
18. Kirkpatrick DL, Kirkpatrick JD. Evaluating Training Programs. 3rd ed. San Francisco, CA: Berrett-Koehler Publishers; 2006.
19. Westley F, Zimmerman B, Patton MQ. Getting to Maybe: How the World Is Changed. Toronto, Ontario, Canada: Vintage Canada; 2006.
20. Artino AR, Brydges R, Gruppen LD. Selfregulated learning in healthcare profession education: Theoretical perspectives and research methods. In: Cleland J, Durning SJ, eds. Researching Medical Education. Oxford, UK: John Wiley & Sons; 2015:155-166.
21. Schunk DH, Zimmerman BJ, eds. Motivation and Self-Regulated Learning: Theory, Research, and Applications. Hillsdale, NJ: Lawrence Erlbaum Associates; 2012.
22. Cooper H, Geyer R. Using "complexity" for improving educational research in health care. Soc Sci Med. 2008;67:177-182.
23. North American Primary Care Research Group. A complexity science primer: What is complexity science and why should I learn about it NAPCRG Resources. August 2009. https://www. napcrg. org/Portals/51/ Documents/Beginner%20Complexity%20 Science%20Module. pdf. Accessed August 22, 2016.
24. Capra F. The Hidden Connections: A Science for Sustainable Living. New York, NY: Random House; 2004.
25. Regehr G. It's NOT rocket science: Rethinking our metaphors for research in health professions education. Med Educ. 2010;44:31-39.
26. Patton MQ. A utilization-focused approach to contribution analysis. Evaluation. 2012;18:364-377.
27. Dauphinee WD. The role of theory-based outcome frameworks in program evaluation: Considering the case of contribution analysis. Med Teach. 2015;37:979-982.
28. Sturmberg JP, Martin CM. Complexity in health: An introduction. In: Sturmberg JP, Martin CM, eds. Handbook of Systems and Complexity in Health. New York, NY: Springer; 2013:1-17.
29. Patton MQ. Essentials of Utilization-Focused Evaluation. Thousand Oaks, CA: Sage Publications; 2012.
30. Rogers PJ. Using programme theory to evaluate complicated and complex aspects of interventions. Evaluation. 2008;14:29-48.
31. Mayne J. Contribution analysis: Addressing cause and effect. In: Forss K, Marra M, Schwartz R, eds. Evaluating the Complex: Attribution, Contribution and Beyond. New Brunswick, NJ: Transaction Publishers; 2011.
32. Haji F, Morin MP, Parker K. Rethinking programme evaluation in health professions education: Beyond "did it work" Med Educ. 2013;47:342-351.
33. Funnell SC, Rogers PJ. Purposeful Program Theory: Effective Use of Theories of Change and Logic Models. San Francisco, CA: Jossey-Bass; 2011.
34. Weiss CH. Which links in which theories shall we evaluate New Dir Eval. 2000;87:35-45.
35. Moreau KA, Eady K. Connecting medical education to patient outcomes: The promise of contribution analysis. Med Teach. 2015;37:1060-1062.
36. Delehais T, Toulemonde J. Applying contribution analysis: Lessons from five years of practice. Evaluation. 2012;18:281-293.
37. Mayne J. Addressing contribution analysis through contribution measures: Using performance measures sensibly. Can J Program Eval. 2001;18:1-24.
38. Chen HT. Theory-Driven Evaluations. Newbury Park, CA: Sage Publications; 1990.
39. Mayne J. Contribution analysis: Coming of age Evaluation. 2012;18:270-280.
40. Biggs JS, Farrell L, Lawrence G, Johnson JK. A practical example of contribution analysis to a public health intervention. Evaluation. 2014;20:214-229.
41. Rotem A, Zinovieff MA, Goubarev A. A framework for evaluating the impact of the United Nations fellowship programmes. Hum Resour Health. 2010;8:7.
42. Wimbush E, Montague S, Mulherin T. Applications of contribution analysis to outcome planning and impact evaluation. 2012;18:310-329.
43. Lemire ST, Neilsen SB, Dybdal L. Making contribution analysis work: A practical framework for handling influencing factors and alternative explanations. Evaluaion. 2012;18:294-309.
44. Mayne J. ILAC brief 26: Making causal claims. http://www. managingforimpact. org/ resource/ilac-brief-26-making-causal-claims. Published October 2012. Accessed August 22, 2016.
45. Hodges BD, Kuper A. Theory and practice in the design and conduct of graduate medical education. Acad Med. 2012;87:25-33.
46. Pusic MV, Boutis K, Hatala R, Cook DA. Learning curves in health professions education. Acad Med. 2015;90:1034-1042.
47. Meade LB, Borden SH, McArdle P, Rosenblum MJ, Picchioni MS, Hinchey KT. From theory to actual practice: Creation and application of milestones in an internal medicine residency program, 2004-2010. Med Teach. 2012;34:717-723.
48. Mayne J. Contribution analysis: An approach to exploring cause and effect. ILAC brief 16. betterevaluation. org/resources/guides/ contribution-analysis/ilac-brief. Published May 2008. Accessed August 22, 2016.
49. Bordage G. Conceptual frameworks to illuminate and magnify. Med Educ. 2009;43:312-319.
50. Cook DA, West CP. Reconsidering the focus on "outcomes research" in medical education: A cautionary note. Acad Med. 2013;88:1-6.
51. Altschuld JW, Engle M. The inexorable historical press of the developing evaluation profession. New Dir Eval. 2015;145:5-19.
52. Scriven M. On the differences between evaluation and social science research. Eval Exchange. 2004;9:1-19.
53. Rycroft-Malone J, McCormack B, Hutchinson AM, et al. Realist synthesis: Illustrating the method for implementation research. Implement Sci. 2012;7:1-10.