Do in-training evaluation reports deserve their bad reputations? a study of the reliability and predictive ability of iter scores and narrative comments

Academic Medicine

Volumn 88, Issue 10, 2013, Pages 1539-1544

  Ginsburg, Shiphra ^a   Eva, Kevin ^b   Regehr, Glenn ^b  

^a UNIVERSITY OF TORONTO   (Canada)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84885171869     PISSN: 10402446     EISSN: 1938808X     Source Type: Journal    
DOI: 10.1097/ACM.0b013e3182a36c3d     Document Type: Article

References (23)

1. Chaudhry SI, Holmboe E, Beasley BW. The state of evaluation in internal medicine residency. J Gen Intern Med. 2008;23:1010-1015.
2. Cleland JA, Knight LV, Rees CE, Tracey S, Bond CM. Is it me or is it them? Factors that influenc. The passing of underperforming students. Med Educ. 2008;42:800-809.
3. Bullock G, Sherbino J, Hodder R, Watling C. Assessment: A White Paper Prepared be the Royal College of Physicians and Surgeons of Canada, Future of Medical Education in Canada. Ottawa, Ontario, Canada: Royal College of Physicians and Surgeons of Canada; 2011.
4. Cohen G, Blumberg P, Ryan N, Sullivan P. Do final grades reflect written qualitative evaluations of student performance? Teach Learn Med. 1993;5:10-15.
5. Richards SH, Campbell JL, Walshaw E, Dickens A, Greco M. A multi-method analysis of free-text comments fro. The UK General Medical Council Colleague Questionnaires. Med Educ. 2009;43:757-766.
6. Durning SJ, Hanson J, Gilliland W, McManigle JM, Waechter D, Pangaro LN. Using qualitative data from a program director's evaluation form as an outcome measurement for medical school. Mil Med. 2010;175:448-452.
7. Frohna A, Stern D. The nature of qualitative comments in evaluating professionalism. Med Educ. 2005;39:763-768.
8. Ginsburg S, Gold W, Cavalcanti RB, Kurabi B, McDonald-Blumer H. Competencies "plus": The nature of written comments on internal medicine residents' evaluation forms. Acad Med. 2011;86(10 suppl):S30-S34.
9. Regehr G, Ginsburg S, Herold J, Hatala R, Eva K, Oulanova O. Using "standardized narratives" to explore new ways to represent faculty opinions of resident performance. Acad Med. 2012;87:419-427.
10. Frank JR. The CanMEDS 2005 Physician Competency Framework. Ottawa, Ontario, Canada: Royal College of Physicians and Surgeons of Canada; 2005.
11. Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychol Bull. 1979;86:420-428.
12. Gray JD. Global rating scales in residency education. Acad Med. 1996;71(1 suppl):S55-S63.
13. Williams RG, Dunnington GL, Klamen DL. Forecasting residents' performance-partly cloudy. Acad Med. 2005;80:415-422.
14. Carline JD, Paauw DS, Thiede KW, Ramsey PG. Factors affectin. The reliability of ratings of students' clinical skills in a medicine clerkship. J Gen Intern Med. 1992;7:506-510.
15. Kreiter CD, Ferguson K, Lee WC, Brennan RL, Densen P. A generalizability study of a new standardized rating form used to evaluate students' clinical clerkship performances. Acad Med. 1998;73:1294-1298.
16. Littlefield J, Paukert J, Schoolfield J. Quality assurance data for residents' global performance ratings. Acad Med. 2001;76(10 suppl):S102-S104.
17. Epstein RM. Assessment in medical education. N Engl J Med. 2007;356:387-396.
18. Eva KW, Hodges BD. Scylla or Charybdis? Can we navigate between objectification and judgement in assessment? Med Educ. 2012;46:914-919.
19. Silber CG, Nasca TJ, Paskin DL, Eiger G, Robeson M, Veloski JJ. Do global rating forms enable program directors to asses. The ACGME competencies? Acad Med. 2004;79:549-556.
20. Haber RJ, Avins AL. Do ratings of the American Board of Internal Medicine Resident Evaluation Form detect differences in clinical competence? J Gen Intern Med. 1994;9:140-145.
21. Lurie SJ, Mooney CJ, Lyness JM. Measurement of the general competencies of the Accreditation Council for Graduate Medical Education: A systematic review. Acad Med. 2009;84:301-309.
22. Crossley J, Johnson G, Booth J, Wade W. Good questions, good answers: Construct alignment improve. The performance of workplace-based assessment scales. Med Educ. 2011;45:560-569.
23. Ginsburg S. Respectin. The expertise of clinician assessors: Construct alignment is one good answer. Med Educ. 2011;45:546-548.