An argument-based approach to the validation of UHTRUST: Can we measure how recent graduates can be trusted with unfamiliar tasks?

Advances in Health Sciences Education

Volumn 18, Issue 5, 2013, Pages 1009-1027

  Wijnen Meijer, M ^a   Van der Schaaf, M ^b   Booij, E ^a   Harendza, S ^c   Boscardin, C ^d   Van Wijngaarden, J ^a   Ten Cate, Th J ^a,d  

^a UNIVERSITY MEDICAL CENTER UTRECHT   (Netherlands)

^b UTRECHT UNIVERSITY   (Netherlands)

^c UNIVERSITY MEDICAL CENTER HAMBURG EPPENDORF   (Germany)

^d UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Argument based approach; Assessment; Authentic simulation; Coping with unfamiliar clinical situations; Entrustment decisions; Readiness for clinical practice; Validity

Indexed keywords

ARTICLE; CLINICAL COMPETENCE; EDUCATION; FEMALE; GERMANY; HUMAN; MALE; MEDICAL EDUCATION; NETHERLANDS; PSYCHOMETRY; REPRODUCIBILITY; STANDARD; TEACHING; VALIDATION STUDY;

CLINICAL COMPETENCE; EDUCATION, MEDICAL; EDUCATIONAL MEASUREMENT; FEMALE; GERMANY; HUMANS; MALE; NETHERLANDS; PATIENT SIMULATION; PSYCHOMETRICS; REPRODUCIBILITY OF RESULTS;

EID: 84888018850     PISSN: 13824996     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10459-013-9444-x     Document Type: Article

References (54)

1. Arnold, L. (2002). Assessing professional behavior: Yesterday, today and tomorrow. Academic Medicine, 77, 502-515.
2. Bakker, M. (2008). Design and evaluation of video portfolios. Reliability, generalizability, and validity of an authentic performance assessment for teachers. Leiden: Mostert & van Onderen.
3. Barton, J. R., Corbett, S., & van der Vleuten, C. P. (2012). The validity and reliability of a direct observation of procedural skills assessment tool: Assessing colonoscopic skills of senior endoscopists. Gastrointestinal Endoscopy, 75(3), 591-597.
4. Birenbaum, M., & Dochy, F. (Eds). (1996). Alternatives in assessment of achievement, learning processes and prior knowledge. Boston: Kluwer.
5. Boursicot, K., & Roberts, T. (2005). How to set up an OSCE. The Clinical Teacher, 2, 16-20.
6. Boyce, P., Spratt, C., Davies, M., & McEvoy, P. (2011). Using entrustable professional activities to guide curriculum development in psychiatry training. BMC Medical Education, 11, 96.
7. Brennan, R. L. (2006). Perspectives on the evolution and future of educational measurement. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 1-16). Westport, CT: American Council on Education and Praeger Publishers.
8. Chang, A., Bowen, J. L., Buranosky, R. A., Frankel, R. M., Gosh, N., Rosenblum, M. J., Thompson, S., & Green, M. L. (2012). Transforming primary care training-patient-centered medical home entrustable professional activities for internal medicine residents. Journal of General Internal Medicine (early online).
9. Chapelle, C. A., Enright, M. K., & Jamieson, J. (2010). Does an argument-based approach to validity make a difference? Educational Measurement Issues and practice, 29, 3-13.
10. Cleland, J. A., Abe, K., & Rethans, J. (2009). The use of simulated patients in medical education: AMEE Guide no. 42. Medical Teacher, 31, 477-486.
11. Cohen, A. S., & Wollack, J. A. (2006). Test administration, security, scoring and reporting. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 17-64). Westport, CT: American Council on Education and Praeger Publishers.
12. Cronbach, L. J. (1971). Test validation. In R. L. Thorndike (Ed.), Educational measurement (2nd ed., pp. 443-507). Washington, DC: American Council on Education.
13. Crossley, J., Johnson, G., Booth, J., & Wade, W. (2011). Good questions, good answers: Construct alignment improves the performance of workplace-based assessment scales. Medical Education, 45, 560-569.
14. Cureton, E. E. (1951). Validity. In E. F. Lindquist (Ed.), Educational measurement (pp. 621-694). Washington, DC: American Council on Education.
15. Dijksterhuis, M. G. K., Teunissen, P. W., Voorhuis, M., Schuwirth, L. W. T., Ten Cate, Th. J., Braat, D. D. M., et al. (2009). Determining competence and progressive independence in postgraduate clinical training. Medical Education, 43, 1156-1165.
16. Durning, S. J., Artino, A., Boulet, J., La Rochelle, J., van der Vleuten, C., Arze, B., et al. (2012). The feasibility, reliability and validity of a post-encounter form for evaluating clinical reasoning. Medical Teacher, 34, 30-37.
17. Dwyer, C. A. (1995). Criteria for performance-based teacher assessments: Validity, standards and issues. In A. J. Shinkfield & D. Stufflebeam (Eds.), Teacher evaluation guide to effective practice (pp. 62-80). Boston: Kluwer.
18. Epstein, R. M. (2007). Assessment in medical education. The New England journal of medicine, 356, 387-396.
19. Fraser, S. W., & Greenhalgh, T. (2001). Coping with complexity: Educating for capability. BMJ, 323, 799-803.
20. Freidson, E. (1970). Profession of medicine: A study of the sociology of applied knowledge. New York: Dodd, Mead & Company.
21. Ginsburg, S. (2011). Respecting the expertise of clinician assessors: construct alignment is one good answer. Medical Education, 45, 546-548.
22. Ginsburg, S., McIlroy, J., Oulanova, O., Eva, K., & Regehr, G. (2010). Toward authentic clinical evaluation: Pitfalls in the pursuit of competency. Academic Medicine, 85, 780-786.
23. Gipps, C. V. (1994). Beyond testing. Towards a theory of educational assessment. London: RoutledgeFalmer.
24. Govaerts, M. J. B., van der Vleuten, C. P. M., Schuwirth, L. W. T., & Muijtjens, A. M. (2007). Broadening perspectives on clinical performance assessment: Rethinking the nature of in-training assessment. Advances in Health Sciences Education, 12, 239-260.
25. Harden, R. M., & Gleeson, F. A. (1979). Assessment of clinical competence using an objective structured clinical examination (OSCE). Medical Education, 13(1), 41-54.
26. Harendza, S. (2011). "HUS" diary of a German nephrologist during the current EHEC outbreak in Europe. Kidney International, 80, 687-689.
27. Hawkins, R. E., Katsufrakis, P. J., Holtman, M. C., & Clauser, B. E. (2009). Assessment of medical professionalism: Who, what, when, where, how, and... why? Medical Teacher, 31, 348-361.
28. Holmboe, E. S., & Hawkins, R. E. (Eds.). (2008). Practical guide to the evaluation of clinical competence. Philadelphia: Mosby-Elsevier.
29. Howley, L. D. (2004). Performance assessment in medical education: Where we've been and where we're going. Evaluation and the Health Professions, 27, 285-301.
30. Kane, M. T. (1992). An argument-based approach to validity. Psychological Bulletin, 112(3), 527-535.
31. Kane, M. (2004). Certification testing as an illustration of argument-based validation. Measurement: Interdisciplinary Research & Perspective, 2, 135-170.
32. Kane, M. (2006). Validation. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 17-64). Westport, CT: American Council on Education and Praeger Publishers.
33. Knight, P. T. (2002). The Achilles' heel of quality: The assessment of student learning. Quality in Higher Education, 8, 107-115.
34. Kreiter, C. D., & Bergus, G. (2008). The validity of performance-based measures of clinical reasoning and alternative approaches. Medical Education, 43, 320-325.
35. Lane, S., & Stone, C. A. (2006). Performance assessment. In R. L. Brennan (Ed.), Educational measurement (4th ed., pp. 387-432). Westport, CT: American Council on Education and Praeger Publishers.
36. Mercer, S. W., Maxwell, M., Heaney, D., & Watt, G. C. M. (2004). The consultation and relational empathy (CARE) measure: Development and preliminary validation and reliability of an empathy-based consultation process measure. Family Practice, 21, 699-705.
37. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-104). New York: American Council on Education and Macmillan.
38. Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741-749.
39. Miller, G. E. (1990). The assessment of clinical skills/competence/performance. Academic Medicine, 65(9), S63-S67.
40. Newble, D. (2004). Techniques for measuring clinical competence: Objective structured clinical examinations. Medical Education, 38, 199-203.
41. Nijveldt, M. (2007). Validity in teacher assessment. An exploration of the judgement processes of assessors. Enschede: Gildeprint.
42. Norcini, J. J., Blank, L. L., Arnold, G. K., & Kimball, H. R. (1995). The Mini-CEX (Clinical Evaluation Exercise): A preliminary investigation. Annals of Internal Medicine, 123, 795-799.
43. Sterkenburg, A., Barach, P., Kalkman, C., Gielen, M., & Ten Cate, O. T. J. (2010). When do supervising physicians decide to entrust residents with unsupervised tasks? Academic Medicine, 85, 1408-1417.
44. Tavares, W., & Eva, K. W. (2012). Exploring the impact of mental workload on rater-based assessments. Advances in Health Sciences Education. doi: 10. 1007/s10459-012-9370-3.
45. Ten Cate, O. (2005). Entrustability of professional activities and competency-based training. Medical Education, 39, 1176-1177.
46. Ten Cate, O. & Scheele, F. (2007). Competence-based postgraduate training: Can we bridge the gap between educational theory and clinical practice? Academic Medicine, 82, 542-547.
47. Ten Cate, O., Snell, L., & Carraccio, C. (2010). Medical competence: The interplay between individual ability and the health care environment. Medical Teacher, 32, 669-675.
48. van der Vleuten, C. P. M. (1996). The assessment of professional competence: Developments, research and practical implications. Advances in Health Sciences Education, 1, 41-67.
49. Wass, V., & Archer, J. (2011). Assessing learners. In T. Dornan, K. Mann, A. Scherpbier, & J. Spencer (Eds.), Medical education: Theory and practice (pp. 229-255). Toronto: Churchill Livingstone Elsevier.
50. Wass, V., van der Vleuten, C., Shatzer, J., & Jones, R. (2001). Assessment of clinical competence. The Lancet, 357, 945-949.
51. Wetzel, A. P. (2012). Analysis methods and validity evidence: A review of instrument development across the medical education continuum. Academic Medicine, 87(8), 2012.
52. Wittert, G. A., & Nelson, A. J. (2009). Medical Education: Revolution, devolution and evolution in curriculum philosophy and design. Medical Journal of Australia, 191, 35-37.
53. Wijnen-Meijer, M., van der Schaaf, M., Nillesen, K., Harendza, S. & Ten Cate, O. Essential FOCs that enable trust in graduates: A Delphi study among physician educators in the Netherlands. Journal of Graduate Medical Education (Accepted for publication).
54. Wijnen-Meijer, M., van der Schaaf, M., Nillesen, K., Harendza, S. & Ten Cate, O. Essential facets of competence that enable trust in medical graduates: A ranking study among physician educators in two countries. (Submitted).