Course evaluation in medical education

Teaching and Teacher Education

Volumn 23, Issue 3, 2007, Pages 251-264

  Kogan, Jennifer R ^a   Shea, Judy A ^a  

^a HOSPITAL OF THE UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

Curriculum; Medical school; Program evaluation; Reliability; Student ratings; Validity

Indexed keywords

EID: 33947111340     PISSN: 0742051X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tate.2006.12.020     Document Type: Article

References (103)

1. Abrami P.C., Cohen P.A., and d'Apollonia S. Implementation problems in meta-analysis. Review of Educational Research 58 (1998) 151-179
2. Afonso N.M., Cardoza L.J., Mascarensas O.A., Aranha A.N., and Shah C. Are anonymous evaluations a better assessment of faculty teaching performance? A comparative analysis of open and anonymous evaluation processes. Family Medicine 37 (2005) 43-47
3. Albanese M.A. Challenges in using rater judgments in medical education. Journal of Evaluation in Clinical Practice 6 3 (2000) 305-319
4. Albanese M.A., Prucha C., and Barnet J.H. Labeling each response option and the direction of the positive options impacts students' course ratings. Academic Medicine 72 (1997) S4-S6
5. Albanese M.A., Prucha C., Barnet J.H., and Gjerde C. The effect of right or left placement of the positive response on Likert type scales used by medical students for rating instruction. Academic Medicine 72 7 (1997) 627-630
6. Albanese M.A., Schroeder J., and Barnes H.B. Student perceptions of instruction: Assessment in a multiple instructor course which maximizes student response. Evaluation and the Health Professions 2 2 (1979) 231-239
7. Albanese M.A., Schuldt S.S., Case D.E., and Brown D. The validity of lecturer ratings by students and trained observers. Academic Medicine 66 1 (1991) 26-28
8. Anderson W.A., Carline J.D., Ambrozy D.M., and Irby D.M. Faculty development for ambulatory care education. Academic Medicine 72 12 (1997) 1072-1075
9. Beckman T.J., Ghosh A.K., Cook D.A., Erwin P.J., and Mandrekar J.N. How reliable are assessments of clinical teaching? A review of published instruments. Journal of General Internal Medicine 19 9 (2004) 971-977
10. Beckman T.J., Lee M.C., Rohren C.H., and Pankratz V.S. Evaluating an instrument for the peer review of inpatient teaching. Medical Teacher 25 2 (2003) 131-135
11. Beckman T.J., Lee M.C., and Mandrekar J.N. A comparison of clinical teaching evaluations by resident and peer physicians. Medical Teacher 26 (2004) 231-235
12. Benbassatt J., and Bachar E. Validity of students' ratings of clinical instructors. Medical Education 15 (1981) 373-376
13. Billings-Gagliardi S., Barrett S.V., and Mazor K.M. Interpreting course evaluation results: Insights from think aloud interviews with medical students. Medical Education 38 (2004) 1061-1070
14. Blue A.V., Griffith C.H., Wilson J.F., Sloan D.A., and Schwartz R.W. Surgical teaching quality makes a difference. American Journal of Surgery 177 (1999) 86-89
15. Blumberg P. Multidimensional outcome considerations in assessing the efficacy of medical educational programs. Teaching and Learning in Medicine 15 3 (2003) 210-214
16. Brennan R.L. Generalizability theory (2001), Springer, New York
17. Broomfield D., and Bligh J. An evaluation of the 'short form' course experience questionnaire with medical students. Medical Education 32 4 (1998) 367
18. Burke M.J., Bonaminio G., and Walling A. Instituting a systematic course/clerkship peer review process. Academic Medicine 77 9 (2002) 930-931
19. Carline J.D., and Scher M. Comparison of course evaluations by random and volunteer student samples. Journal of Medical Education 56 (1981) 122-127
20. Chen F.M., Bauchner H., and Burstin H. A call for outcomes research in medical education. Academic Medicine 79 10 (2004) 955-960
21. Chen F.M., Burstin H., and Huntingdon J. The importance of clinical outcomes in medical education research. Medical Education 39 4 (2004) 350
22. Copeland H.L., and Hewson M.G. Developing and testing an instrument to measure the effectiveness of clinical teaching in an academic medical center. Academic Medicine 75 2 (2000) 161-166
23. Crossley J., Humphris G., and Jolly B. Assessing health professionals. Medical Education 36 (2002) 800-804
24. D'Apollonia S., and Abrami P.C. Navigating student rating of instruction. American Psychologist 52 11 (1997) 1198-1208
25. DaRosa D.A., Prystowsky J.B., and Nahrwold D.L. Evaluating a clerkship curriculum: Description and results. Teaching and Learning in Medicine 13 1 (2001) 21-26
26. Dolmans D.H., De Grave W., Wolfhagen I.H., and van der Vleuten C.P. Problem-based learning: Future challenges for educational practice and research. Medical Education 39 7 (2005) 732-741
27. Downing S.M. Threats to the validity of clinical teaching assessments. What about rater error?. Medical Education 39 4 (2005) 353-355
28. Elzubeir M., and Rizk D. Evaluating the quality of teaching in medical education: Are we using the evidence for both formative and summative purposes. Medical Teacher 24 3 (2002) 313-319
29. Eva K.W., and Regehr G. Self-Assessment in the health professions: A reformulation and research agenda. Academic Medicine 80 Suppl. 10 (2005) S46-S54
30. Feinstein E., and Levine H.G. Impact of student ratings on basic science portion of the medical school curriculum. Journal of Medical Education 55 (1980) 502-512
31. Feldman K.A. Instructional effectiveness of college teachers as judged by teachers, themselves, current and former students, colleagues, administrators, and external (neutral) observers. Research in Higher Education 30 (1989) 137-194
32. Fraenkel J.R., and Wallen N.E. How to design and evaluate research in education. 4th ed (2000), McGraw Hill, Boston
33. Frasier P.Y., Slatt L., Kowlowitz V., Kollisch D.O., and Mintzer M. Focus groups: A useful tool for curriculum evaluation. Family Medicine 29 7 (1997) 500-507
34. Gall M.D., Borg W.R., and Gall J.P. Educational research: An introduction. 6th ed (1996), Allyn and Bacon, Boston, MA
35. Gartland J.J., Hojat M., Christian E.B., Callahan C.A., and Nasca T.J. African American and white physicians: A comparison of satisfaction with medical education, professional careers, and research activities. Teaching and Learning in Medicine 15 2 (2003) 106-112
36. Gilbert S., and Davidson J.S. Using the world-wide web to obtain feedback on the quality of surgical residency training. American Journal of Surgery 179 1 (2000) 74-75
37. Greenwald A.G. Validity concerns and usefulness of student ratings of instruction. American Psychologist 52 11 (1997) 1182-1186
38. Greenwald A.G., and Gillmore G.M. Grading leniency is a removable contaminant of student ratings. American Psychologist 52 11 (1997) 1209-1217
39. Griffith C.H., Wilson J.F., Haist S.A., and Ramsbottom M. Do students who work with better housestaff in their medicine clerkship learn more?. Academic Medicine 73 10 suppl (1998) S57-S59
40. Griffith C.H., Wilson J.F., Haist S.A., and Ramsbottom-Lucier M. Relationships of how well attending physicians teach to their students' performance and residency choice. Academic Medicine 72 10 suppl (1997) S118-S126
41. Hafferty F.W. Beyond curriculum reform: Confronting medicine's hidden curriculum. Academic Medicine 73 (1998) 403-407
42. Haidet P., Kelly P.A., and Chou C. Communication, curriculum and culture study group. Characterizing the patient centeredness of hidden curricula in medical schools: Development and validation of a new measure. Academic Medicine 80 1 (2005) 44-50
43. Hayward R.A., Williams B.C., Gruppen L.D., and Rosenbaum D. Measuring attending physician performance in a general medicine outpatient clinic. Journal of General Internal Medicine 10 (1995) 504-510
44. Hewson M.G., and Copeland H.L. What's the use of faculty development? Program evaluation using retrospective self assessments and independent performance ratings. Teaching and Learning in Medicine 13 (2001) 153-160
45. Hojat M., Brigham T.P., Gottheil E., Xu G., Glaser K., and Veloski J.J. Medical students' personal values and their career choices a quarter-century later. Psychological Reports 83 1 (1998) 243-248
46. Horowitz S., Van Eyck S., and Albanese M. Successful peer review of courses: A case study. Academic Medicine 73 3 (1998) 266-271
47. Howard G.S., Conway C.G., and Maxwell S.E. Construct validity of measures of college teaching effectiveness. Journal of Educational Psychology 77 (1985) 810-820
48. Institute for Improvement of Medical Education. Educating doctors to provide high quality medical care, A vision for medical education in the United States (2003), Association of American Medical Colleges, Washington DC
49. Irby D., and Rakestraw P. Evaluating clinical teaching in medicine. Journal of Medical Education 56 (1981) 181-186
50. James P.A., Kreiter C.D., Shipengrover J., and Crosson J. Identifying the attributes of instructional quality in ambulatory teaching sites: A validation study of the MedEd IQ. Family Medicine 34 4 (2002) 268-273
51. James P.A., and Osborne J.W. A measure of medical instructional quality in ambulatory settings: The MEDQI. Family Medicine 31 (1999) 263-269
52. Jones R.G., and Froom J.D. Faculty and administration view of problems in faculty evaluation. Academic Medicine 69 (1994) 476-483
53. Kalet A., Schwartz M., Capponi L., et al. Ambulatory versus inpatient rotations in teaching third year students in internal medicine. Journal of General Internal Medicine 13 (1998) 327-330
54. Kernan W.N., Holmboe E., and O'Conner P.J. Assessing the teaching behaviors of ambulatory care preceptors. Academic Medicine 79 11 (2004) 1088-1094
55. Kreiter C.D., and Lakshman V. Investigating the use of sampling for maximizing the efficiency of student-generated faculty teaching evaluations. Medical Education 39 (2005) 171-175
56. Lam T.P., Irwin M., Chow L.W., and Chan P. Early introduction of clinical skills teaching in a medical curriculum-Factors affecting students' learning. Medical Education 36 3 (2002) 233-240
57. Leamon M.H., and Fields L. Measuring teaching effectiveness in a preclinical multi-instructor course: A case study in the development and application of a brief instructor rating scale. Teaching and Learning in Medicine 17 2 (2005) 119-129
58. Lempp H., and Seale C. The hidden curriculum in undergraduate medical education: Qualitative study of medical students perceptions of teaching. British Medical Journal 329 7469 (2004) 770-773
59. Levine H., Vanek E., Lefferts G., Michener W., and Weiker G. A peer review process to assess the quality of graduate medical education. Journal of Medical Education 63 4 (1988) 288-293
60. Lewis B.S., and Pace W.D. Qualitative and quantitative methods for the assessment of clinical preceptors. Family Medicine 22 (1990) 356-360
61. Lin C.S. Medical student perceptions of good PBL tutors in Tawain. Teaching and Learning in Medicine 17 2 (2005) 179-183
62. Linn R.L., and Gronlund N.E. Measurement and assessment in teaching. 8th ed (2000), Prentice-Hall, Inc, Upper Saddle River NJ
63. Litzelman, D.K., Shea, J.A., Wales, P.S., Kogan, J.R. (2005) Evaluation of the clerkship: Clinical teachers and programs. Guidebook for Clerkship Directors. 3rd ed., Omaha NE, Alliance for Clinical Education (Chapter 7). The electronic reference is http://familymed.uthscsa.edu/ACE/chapter7.htm
64. Litzelman D.K., Stratos G.A., Marriot D.J., and Skeff K.M. Factorial validation of a widely disseminated educational framework for evaluating clinical teachers. Academic Medicine 73 6 (1998) 688-695
65. Lloyd-Jones G., Fowell S., and Bligh J.G. The use of nominal group technique as an evaluative tool in medical undergraduate education. Medical Education 33 (1999) 8-13
66. Marsh H.W., and Roche L.A. Making students' evaluations of teaching effectiveness effective: The critical issues of validity, bias and utility. American Psychologist 52 11 (1997) 1187-1197
67. Mazor K.M., Stone S.L., Carlin M., and Alper E. What do medicine clerkship preceptors do best?. Academic Medicine 77 8 (2002) 837-840
68. McKeachie W.J. Student ratings: The validity of use. American Psychologist 52 11 (1997) 1218-1225
69. Myers K. Evaluating clinical teachers: Does the learning environment matter. Academic Medicine 76 (2001) 286
70. Pabst R., Nave H., Rothkotter H.J., and Tschernig T. Evaluation of the medical curriculum: Why, when, by whom and for whom should questionnaires be used. European Journal of Morphology 39 4 (2001) 237-239
71. Pandachuck K., Harley D., and Cook D. Effectiveness of a brief workshop designed to improve teaching performance at the University of Alberta. Academic Medicine 79 8 (2004) 798-804
72. Paolo A.M., Bonaminio G.A., Gibson C., Partridge T., and Kallail K. Response rate comparisons to email and mail distributed student evaluations. Teaching and Learning in Medicine 12 2 (2000) 81-84
73. Parrino T.A., and Kern D.C. The alumni survey as an instrument for program evaluation in internal medicine. Journal of General Internal Medicine 9 2 (1994) 92-95
74. Paukert J.L., and Richards B.F. How medical students and residents describe the roles and characteristics of their influential teachers. Academic Medicine 75 (2000) 843-845
75. Pelsang R.E., and Smith W.L. Comparison of anonymous student ballots with student debriefing for faculty evaluations. Medical Education 34 6 (2000) 465-467
76. Peluso M.A., Tavares H., and D'Elia G. Assessment of medical courses in Brazil using student-completed questionnaires. Is it reliable?. Revista do Hospital das Clinicas; Faculdade de Medicina Da Universidade de Sao Paulo 55 (2000) 55-60
77. Powers C.L., Allen R.M., Johnson V.A., and Cooper-Witt C.M. Evaluating immediate and long-range effect of a geriatric clerkship using reflections and ratings from participants as students and as residents. Journal of the American Geriatric Society 53 2 (2005) 331-335
78. Ramsbottom-Lucier M.T., Gillmore G.M., Irby D.M., and Ramsey P.G. Evaluation of clinical teaching by general internal medicine faculty in outpatient and inpatient settings. Academic Medicine 69 (1994) 152-154
79. Sarker S.K., Vincent C., and Darzi A.W. Assessing the teaching of technical skills. American Journal of Surgery 189 4 (2005) 416-418
80. Schum T.R., Yindra K.J., Koss R., and Nelson D.B. Students' and residents' ratings of teaching effectiveness in departments of pediatrics. Teaching and Learning in Medicine 5 (1993) 128-132
81. Scriven M. The validity of student ratings. Instructional Evaluation (1988) 5-18
82. Shavelson R.J., and Webb N. Generalizability theory: A primer (measurement methods for the social science) (1991), Sage Publications, Thousand Oaks, CA
83. Shea J.A., Arnold L., and Mann K.V. A RIME perspective on the quality and relevance of current and future education research. Academic Medicine 79 10 (2004) 931-938
84. Shea J.A., and Bellini L.M. Evaluations of clinical faculty: The impact of level of learner and time of year. Teaching and Learning in Medicine 14 2 (2002) 87-91
85. Shea J.A., Bridge P.D., Gould B.E., and Harris I.B. UME-21 Local evaluation initiatives: Contributions and challenges. Family Medicine 36 (2004) S133-S137
86. Shea J.A., and Fortna G.S. Psychometric methods. In: Norman G., van der Vleuten C., and Newble D. (Eds). International handbook for research in medical education (2002), Kluwer Publishing, Boston
87. Shores J.H., Clearfield M., and Alexander J. An index of students' satisfaction with instruction. Academic Medicine 75 10 (2000) S106-S108
88. Sierles F.S. Evaluation of the clerkship: Its components and its faculty. In: Fincher R.E. (Ed). Handbook for clerkship directors. 1st ed (1996), Washington DC, AAMC
89. Snell L., Tallett S., Haist S., Hays R., et al. A review of the evaluation of clinical teaching: New perspectives and challenges. Medical Education 34 10 (2000) 862-870
90. Solomon D.J., Speer A.J., Rosenbraugh C.J., and DiPette D.J. The reliability of medical students ratings of clinical teaching. Evaluation and the Health Professions 20 3 (1997) 343-352
91. Speer A.J., and Elnicki D.M. Assessing the quality of teaching. American Journal of Medicine 106 (1999) 381-384
92. Steiner I.P., Franc-Law J., Kelly K.D., and Rowe B.H. Faculty evaluation by residents in an emergency room program: A new evaluation instrument. Academic Emergency Medicine 7 (2000) 1015-1021
93. Steiner I.P., Yoon P.W., Kelly K.D., Diner B.M., Blitz S., Donof M.G., et al. The influence of residents' training level on their evaluation of clinical teaching faculty. Teaching and Learning in Medicine 17 1 (2005) 42-48
94. Steiner I.P., Yoon P.W., Kelly K.D., Diner B.M., Donoff M.G., Mackey D.S., et al. Resident evaluation of clinical teachers based on teachers' certification. Academic Emergency Medicine 10 7 (2003) 731-737
95. Stern D.T., Williams B.C., Gill A., Gruppen L.D., Woolliscroft J.O., and Grum C.M. Is there a relationship between attending physicians' and residents' teaching skills and students' examination scores?. Academic Medicine 75 11 (2000) 1144-1146
96. Stewart T.J., and Frye A.W. Investigating the use of negatively phrased survey items in medical education settings: Common wisdom or common mistake?. Academic Medicine 79 10 Suppl (2004) S18-S20
97. Stone S.L., and Qualters D.M. Course-based assessment: Implementing outcome assessment in medical education. Academic Medicine 73 4 (1998) 397-401
98. Ullian J.A., Bland C.J., and Simpson D.E. The alternative approach to defining the role of the clinical teacher. Academic Medicine 69 (1994) 832-838
99. Vu T.R., Marriott D.J., Skeff K.M., Stratos G.A., and Litzelman D.K. Prioritizing areas for faculty development of clinical teachers by using student evaluations for evidence-based decisions. Academic Medicine 72 10 (1997) S7-S9
100. West R.F. The short-term stability of student ratings of instruction in medical school. Medical Education 22 2 (1988) 104-112
101. Williams B.C., Litzelman D.K., Babbott S.F., Lubitz R.M., and Hofer T.P. Validation of a global measure of a faculty's clinical teaching performance. Academic Medicine 77 (2002) 177-180
102. Williams B.C., Pillsbury M.S., Stern D.T., and Grum C.M. Comparison of resident and medical student evaluation of faculty teaching. Evaluation and the Health Professions 24 1 (2001) 53-60
103. Windish D.M., Knight A.M., and Wright S.M. Clinician-teachers' self assessments versus learners perceptions. Journal of General Internal Medicine 19 5 Pt2 (2004) 554-557