Factor instability of clinical teaching assessment scores among general internists and cardiologists

Medical Education

Volumn 40, Issue 12, 2006, Pages 1209-1216

  Beckman, Thomas J ^a   Cook, David A ^a   Mandrekar, Jayawant N ^a  

^a MAYO CLINIC   (United States)

Author keywords

*Internship and residency; Cardiology *education; Factor analysis, statistical; Minnesota; Observer variation; Teaching *standards methods

Indexed keywords

ACCURACY; ARTICLE; CALCULATION; CARDIOLOGIST; CLINICAL COMPETENCE; CLINICAL EDUCATION; CRONBACH ALPHA COEFFICIENT; FACTORIAL ANALYSIS; HUMAN; HUMAN RELATION; INTERRATER RELIABILITY; MEDICAL EDUCATION; MEDICAL SPECIALIST; OBSERVER VARIATION; RESIDENCY EDUCATION; SCORING SYSTEM; STANDARD; UNITED STATES; VALIDATION PROCESS;

CARDIOLOGY; FACTOR ANALYSIS, STATISTICAL; INTERNSHIP AND RESIDENCY; MINNESOTA; OBSERVER VARIATION; TEACHING;

EID: 33751352124     PISSN: 03080110     EISSN: 13652923     Source Type: Journal    
DOI: 10.1111/j.1365-2929.2006.02632.x     Document Type: Article

References (37)

1. Messick S. Validity. In: Linn RL, ed. Educational Measurement, 3rd edn. Phoenix, AZ: Oryx Press 1993;13-103.
2. Downing SM. Validity: on the meaningful interpretation of assessment data. Med Educ 2003;37:830-7.
3. American Education Research Association, American Psychological Association. Standards for Educational and Psychological Testing. Washington, DC: American Education Research Association 1999;9-24.
4. Beckman TJ, Cook DA, Mandrekar JN. What is the validity evidence for assessments of clinical teaching? J Gen Intern Med 2005;20:1159-64.
5. Beckman TJ, Ghosh AK, Cook DA, Erwin PJ, Mandrekar JN. How reliable are assessments of clinical teaching? A review of the published instruments. J Gen Intern Med 2004;19:971-7.
6. Beckman TJ, Mandrekar JN. The interpersonal, cognitive and efficiency domains of clinical teaching: construct validity of a multidimensional scale. Med Educ 2005;39:1221-9.
7. Guyatt GH, Nishikawa J, Willan A et al. A measurement process for evaluating clinical teachers in internal medicine. CMAJ 1993;149:1097-102.
8. Litzelman DK, Stratos GA, Marriott DJ, Skeff KM. Factorial validation of a widely disseminated educational framework for evaluating clinical teachers. Acad Med 1998;73:688-95.
9. Benbassat J, Bachar E. Validity of students' ratings of clinical instructors. Med Educ 1981;15:373-6.
10. Cohen R, McRae H, Jamieson C. Teaching effectiveness of surgeons. Am J Surg 1996;171:612-4.
11. Copeland HL, Hewson MG. Developing and testing an instrument to measure the effectiveness of clinical teaching in an academic medical centre. Acad Med 2000;75:161-6.
12. Donnelly MB, Woolliscroft JO. Evaluation of clinical instructors by third year medical students. Acad Med 1989;64:159-64.
13. Irby DM, Rakestraw P. Evaluating clinical teaching in medicine. J Med Educ 1981;56:181-6.
14. James PA, Osborne JW. A measure of medical instructional quality in ambulatory settings: the Med-IQ. Fam Med 1999;31:263-9.
15. McLeod PJ, James CA, Abrahamowicz M. Clinical tutor evaluation: a 5-year study by students on an inpatient service and residents in an ambulatory care clinic. Med Educ 1993;27:48-54.
16. Shellenberger S, Mahan JM. A factor analytic study of teaching in off-campus general practice clerkships. Med Educ 1982;16:151-5.
17. Solomon DJ, Speer AJ, Rosenbraugh CJ, DiPette DJ. The reliability of medical student ratings of teaching. Eval Health Prof 1997;20:343-52.
18. Donner-Banzhoff N, Merle H, Baum E, Basler HD. Feedback for general practice trainers: developing and testing a standardised instrument using the importance-quality-score method. Med Educ 2003;37:772-7.
19. Hayward RA, Williams BC, Gruppen LD, Rosenbaum D. Measuring attending physician performance in a general medicine outpatient clinic. J Gen Intern Med 1995;10:504-10.
20. Smith CA, Varkey AB, Evans AT, Reilly BM. Evaluating the performance of inpatient attending physicians: a new instrument for today's teaching hospitals. J Gen Intern Med 2004;19:766-71.
21. Beckman TJ, Lee MC, Rohren CH. Evaluating an instrument for the peer review of inpatient teaching. Med Teach 2003;25:131-5.
22. Beckman TJ, Lee MC, Mandrekar JN. A comparison of clinical teaching evaluations by resident and peer physicians. Med Teach 2004;26:321-5.
23. Litzelman DK, Westmorland GR, Skeff KM, Stratos GA. Student and resident evaluations of faculty - how reliable are they? Acad Med 1999;74 (Suppl): 25-7.
24. Ramsbottom-Lucier MT, Gillmore GM, Irby DM, Ramsey PG. Evaluation of clinical teaching by general internal medicine faculty in outpatient and inpatient settings. Acad Med 1994;69:152-4.
25. Magill MK, McLure C, Commerford K. A system for evaluating teaching in the ambulatory setting. Fam Med 1986;18:173-4.
26. Steiner IP, Franc-Law J, Kelly KD, Rowe BH. Faculty evaluation by residents in an emergency medicine programme: a new evaluation instrument. Acad Emerg Med 2000;7:1015-21.
27. Tortolani AJ, Risucci DA, Rosati RJ. Resident evaluation of surgical faculty. J Surg Res 1991;51:186-91.
28. Risucci DA, Lutsky L, Rosati RJ, Tortolani AJ. Reliability and accuracy of resident evaluations of surgical faculty. Eval Health Prof 1992;15:313-24.
29. Beckman TJ. Lessons learned from a peer review of bedside teaching. Acad Med 2004;79:343-6.
30. DeVillis RF. Scale Development: Theory and Applications. London: Sage Publications 1991;60-137.
31. Fleiss JL, Leven B, Paik MC. Statistical Methods for Rates and Proportions, 3rd edn. Hoboken, NJ: John Wiley & Sons 2003;604-611.
32. Fleiss JL, Cohen J. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Measurement 1973;33:613-9.
33. Landis JR, Koch GG. The measure of observer agreement for categorical data. Biometrics 1977;33:159-74.
34. Silber CG, Nasca TJ, Paskin DL, Eiger G, Robeson M, Voleski J. Do global rating forms enable program directors to assess the ACGME competencies? Acad Med 2004;79:549-56.
35. van der Hem-Stokroos HH, van der Vleuten CPM, Daelmans HEM, Haarman HJTM, Scherpbier AJJA. Reliability of the Clinical Teaching Effectiveness Instrument. Med Educ 2005;39:904-10.
36. Wilkinson L, the Task Force on Statistical Inference. Statistical methods in psychology journals: guidelines and explanations. Am Psychol 1999;54:594-604.
37. Cook DA, Beckman TJ. Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med 2006;119:7-16.