Developing end-of-training entrustable professional activities for psychiatry: Results and methodological lessons

Academic Medicine

Volumn 93, Issue 7, 2018, Pages 1048-1054

  Young, John Q ^a   Hasser, Caitlin ^b   Hung, Erick K ^b   Kusz, Martin ^a   O'Sullivan, Patricia S ^b   Stewart, Colin ^c   Weiss, Andrea ^d   Williams, Nancy ^e  

^a DONALD AND BARBARA ZUCKER SCHOOL OF MEDICINE AT HOFSTRA NORTHWELL   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c GEORGETOWN UNIVERSITY SCHOOL OF MEDICINE   (United States)

^d ALBERT EINSTEIN COLLEGE OF MEDICINE   (United States)

^e UNIVERSITY OF IOWA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CURRICULUM; DELPHI STUDY; EDUCATION; HUMAN; MEDICAL EDUCATION; PROCEDURES; PSYCHIATRY; QUESTIONNAIRE;

COMPETENCY-BASED EDUCATION; CURRICULUM; DELPHI TECHNIQUE; EDUCATIONAL MEASUREMENT; HUMANS; INTERNSHIP AND RESIDENCY; PSYCHIATRY; SURVEYS AND QUESTIONNAIRES;

EID: 85057484786     PISSN: 10402446     EISSN: 1938808X     Source Type: Journal    
DOI: 10.1097/ACM.0000000000002058     Document Type: Article

References (65)

1. Cooke M, Irby DM, O'Brien BC; Carnegie Foundation for the Advancement of Teaching. Educating Physicians: A Call for Reform of Medical School and Residency. San Francisco, CA: Jossey-Bass; 2010.
2. Eden J, Berwick DM, Wilensky GR; Institute of Medicine (U. S.). Committee on the Governance and Financing of Graduate Medical Education. Graduate Medical Education That Meets the Nation's Health Needs. Washington, DC: National Academies Press; 2014.
3. Frenk J, Chen L, Bhutta ZA, et al. Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. Lancet. 2010; 376: 1923-1958.
4. Skochelak SE. Commentary: A century of progress in medical education: What about the next 10 years? Acad Med. 2010; 85: 197-200.
5. Frank JR, Snell LS, ten Cate O, et al. Competency-based medical education: Theory to practice. Med Teach. 2010; 32: 638-645.
6. Carraccio C, Wolfsthal SD, Englander R, Ferentz K, Martin C. Shifting paradigms: From Flexner to competencies. Acad Med. 2002; 77: 361-367.
7. Leach DC. A model for GME: Shifting from process to outcomes. A progress report from the Accreditation Council for Graduate Medical Education. Med Educ. 2004; 38: 12-14.
8. Caverzagie KJ, Cooney TG, Hemmer PA, Berkowitz L. The development of entrustable professional activities for internal medicine residency training: A report from the Education Redesign Committee of the Alliance for Academic Internal Medicine. Acad Med. 2015; 90: 479-484.
9. ten Cate O, Scheele F. Competency-based postgraduate training: Can we bridge the gap between theory and clinical practice? Acad Med. 2007; 82: 542-547.
10. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system-Rationale and benefits. N Engl J Med. 2012; 366: 1051-1056.
11. Lurie SJ. History and practice of competencybased assessment. Med Educ. 2012; 46: 49-57.
12. Hawkins RE, Welcher CM, Holmboe ES, et al. Implementation of competency-based medical education: Are we addressing the concerns and challenges? Med Educ. 2015; 49: 1086-1102.
13. Rekman J, Gofton W, Dudek N, Gofton T, Hamstra SJ. Entrustability scales: Outlining their usefulness for competencybased clinical assessment. Acad Med. 2016; 91: 186-190.
14. Malone K, Supri S. A critical time for medical education: The perils of competence-based reform of the curriculum. Adv Health Sci Educ Theory Pract. 2012; 17: 241-246.
15. ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005; 39: 1176-1177.
16. Englander R, Flynn T, Call S, et al. Toward defining the foundation of the MD degree: Core entrustable professional activities for entering residency. Acad Med. 2016; 91: 1352-1358.
17. ten Cate O, Young JQ. The patient handover as an entrustable professional activity: Adding meaning in teaching and practice. BMJ Qual Saf. 2012; 21 (suppl 1): i9-i12.
18. Ten Cate O, Chen HC, Hoff RG, Peters H, Bok H, van der Schaaf M. Curriculum development for the workplace using entrustable professional activities (EPAs): AMEE guide no. 99. Med Teach. 2015; 37: 983-1002.
19. ten Cate O. Competency-based education, entrustable professional activities, and the power of language. J Grad Med Educ. 2013; 5: 6-7.
20. Leipzig RM, Sauvigné K, Granville LJ, et al. What is a geriatrician? American Geriatrics Society and Association of Directors of Geriatric Academic Programs end-oftraining entrustable professional activities for geriatric medicine. J Am Geriatr Soc. 2014; 62: 924-929.
21. Wisman-Zwarter N, van der Schaaf M, ten Cate O, Jonker G, van Klei WA, Hoff RG. Transforming the learning outcomes of anaesthesiology training into entrustable professional activities: A Delphi study. Eur J Anaesthesiol. 2016; 33: 559-567.
22. Brown CR Jr, Criscione-Schreiber L, O'Rourke KS, et al. What is a rheumatologist and how do we make one? Arthritis Care Res (Hoboken). 2016; 68: 1166-1172.
23. Rose S, Fix OK, Shah BJ, et al. Entrustable professional activities for gastroenterology fellowship training. Gastrointest Endosc. 2014; 80: 16-27.
24. Weiss A, Ozdoba A, Carroll V, DeJesus F. Entrustable professional activities: Enhancing meaningful use of evaluations and milestones in a psychiatry residency program. Acad Psychiatry. 2016; 40: 850-854.
25. Boyce P, Spratt C, Davies M, McEvoy P. Using entrustable professional activities to guide curriculum development in psychiatry training. BMC Med Educ. 2011; 11: 96.
26. Royal Australian & New Zealand College of Psychiatrists. EPA handbook. https://www. ranzcp. org/Files/PreFellowship/2012-Fellowship-Program/EPA-forms/EPAhandbook. aspx. Accessed October 10, 2017.
27. Aylward M, Nixon J, Gladding S. An entrustable professional activity (EPA) for handoffs as a model for EPA assessment development. Acad Med. 2014; 89: 1335-1340.
28. Beeson MS, Warrington S, Bradford-Saffles A, Hart D. Entrustable professional activities: Making sense of the emergency medicine milestones. J Emerg Med. 2014; 47: 441-452.
29. Chan B, Englander H, Kent K, et al. Transitioning toward competency: A resident-faculty collaborative approach to developing a transitions of care EPA in an internal medicine residency program. J Grad Med Educ. 2014; 6: 760-764.
30. Chang A, Bowen JL, Buranosky RA, et al. Transforming primary care training-Patient-centered medical home entrustable professional activities for internal medicine residents. J Gen Intern Med. 2013; 28: 801-809.
31. Chen HC, McNamara M, Teherani A, ten Cate O, O'Sullivan P. Developing entrustable professional activities for entry into clerkship. Acad Med. 2016; 91: 247-255.
32. Chen HC, van den Broek WE, ten Cate O. The case for use of entrustable professional activities in undergraduate medical education. Acad Med. 2015; 90: 431-436.
33. Englander R, Carraccio C. From theory to practice: Making entrustable professional activities come to life in the context of milestones. Acad Med. 2014; 89: 1321-1323.
34. Hauer KE, Kohlwes J, Cornett P, et al. Identifying entrustable professional activities in internal medicine training. J Grad Med Educ. 2013; 5: 54-59.
35. ten Cate O, Snell L, Carraccio C. Medical competence: The interplay between individual ability and the health care environment. Med Teach. 2010; 32: 669-675.
36. Hauer KE, ten Cate O, Boscardin C, Irby DM, Iobst W, O'Sullivan PS. Understanding trust as an essential element of trainee supervision and learning in the workplace. Adv Health Sci Educ Theory Pract. 2014; 19: 435-456.
37. Ten Cate O. Nuts and bolts of entrustable professional activities. J Grad Med Educ. 2013; 5: 157-158.
38. Carraccio CL, Englander R. From Flexner to competencies: Reflections on a decade and the journey ahead. Acad Med. 2013; 88: 1067-1073.
39. Kogan JR, Conforti LN, Bernabeo E, Iobst W, Holmboe E. How faculty members experience workplace-based assessment rater training: A qualitative study. Med Educ. 2015; 49: 692-708.
40. Long DM. Competency-based residency training: The next advance in graduate medical education. Acad Med. 2000; 75: 1178-1183.
41. Norcini JJ. Work based assessment. BMJ. 2003; 326: 753-755.
42. Landeta J, Barrutia J, Lertxundi A. Hybrid Delphi: A methodology to facilitate contribution from experts in professional contexts. Technol Forecast Soc. 2011; 78: 1629-1641.
43. Humphrey-Murto S, Varpio L, Gonsalves C, Wood TJ. Using consensus group methods such as Delphi and nominal group in medical education research. Med Teach. 2017; 39: 14-19.
44. Lynn MR. Determination and quantification of content validity. Nurs Res. 1986; 35: 382-385.
45. Penfield RD. A score method of constructing asymmetric confidence intervals for the mean of a rating scale item. Psychol Methods. 2003; 8: 149-163.
46. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007; 30: 459-467.
47. Sanchez R, Sloan SR, Josephson CD, Ambruso DR, Hillyer CD, O'Sullivan P. Consensus recommendations of pediatric transfusion medicine objectives for clinical pathology residency training programs. Transfusion. 2010; 50: 1071-1078.
48. Thrush CR, Putten JV, Rapp CG, Pearson LC, Berry KS, O'Sullivan PS. Content validation of the Organizational Climate for Research Integrity (OCRI) survey. J Empir Res Hum Res Ethics. 2007; 2: 35-52.
49. Penfield RD, Miller JM. Improving content validation studies using an asymmetric confidence interval for the mean of expert ratings. Appl Meas Educ. 2004; 17: 359-370.
50. Shaughnessy AF, Sparks J, Cohen-Osher M, Goodell KH, Sawin GL, Gravel J Jr. Entrustable professional activities in family medicine. J Grad Med Educ. 2013; 5: 112-118.
51. Spenkelink-Schut G, ten Cate TJ, Kort HSM. Application of EPA concept as connection between professional activities and CanMEDS competency areas: Pilot study for physician assistants in urology [in Dutch]. Tijdschrift voor Medisch Onderwijs. 2008; 27: 230-238.
52. Fessler HE, Addrizzo-Harris D, Beck JM, et al. Entrustable professional activities and curricular milestones for fellowship training in pulmonary and critical care medicine: Report of a multisociety working group. Chest. 2014; 146: 813-834.
53. Teherani A, Chen HC. The next steps in competency-based medical education: Milestones, entrustable professional activities and observable practice activities. J Gen Intern Med. 2014; 29: 1090-1092.
54. Warm EJ, Held JD, Hellmann M, et al. Entrusting observable practice activities and milestones over the 36 months of an internal medicine residency. Acad Med. 2016; 91: 1398-1405.
55. Warm EJ, Mathis BR, Held JD, et al. Entrustment and mapping of observable practice activities for resident assessment. J Gen Intern Med. 2014; 29: 1177-1182.
56. Sunderji N, Waddell A, Gupta M, Soklaridis S, Steinberg R. An expert consensus on core competencies in integrated care for psychiatrists. Gen Hosp Psychiatry. 2016; 41: 45-52.
57. ten Cate O. Trusting graduates to enter residency: What does it take? J Grad Med Educ. 2014; 6: 7-10.
58. Mojtabai R, Olfson M. National trends in psychotherapy by office-based psychiatrists. Arch Gen Psychiatry. 2008; 65: 962-970.
59. Yager J, Bienenfeld D. How competent are we to assess psychotherapeutic competence in psychiatric residents? Acad Psychiatry. 2003; 27: 174-181.
60. Carraccio C, Englander R, Gilhooly J, et al. Building a framework of entrustable professional activities, supported by competencies and milestones, to bridge the educational continuum. Acad Med. 2017; 92: 324-330.
61. Downing SM. Validity: On meaningful interpretation of assessment data. Med Educ. 2003; 37: 830-837.
62. Crossley J, Jolly B. Making sense of workbased assessment: Ask the right questions, in the right way, about the right things, of the right people. Med Educ. 2012; 46: 28-37.
63. ten Cate O, Hart D, Ankel F, et al; International Competency-Based Medical Education Collaborators. Entrustment decision making in clinical training. Acad Med. 2016; 91: 191-198.
64. Carraccio C, Englander R, Holmboe ES, Kogan JR. Driving care quality: Aligning trainee assessment and supervision through practical application of entrustable professional activities, competencies, and milestones. Acad Med. 2016; 91: 199-203.
65. Regehr G, Ginsburg S, Herold J, Hatala R, Eva K, Oulanova O. Using "standardized narratives" to explore new ways to represent faculty opinions of resident performance. Acad Med. 2012; 87: 419-427.