Student evaluations of teaching: Are they related to what students learn?: A meta-analysis and review of the literature

Journal of Marketing Education

Volumn 31, Issue 1, 2009, Pages 16-30

  Clayson, Dennis E ^a  

^a UNIVERSITY OF NORTHERN IOWA   (United States)

Author keywords

Business education; Grades; Learning; Rigor; Student evaluation of teaching

Indexed keywords

EID: 62749182288     PISSN: 02734753     EISSN: 15526550     Source Type: Journal    
DOI: 10.1177/0273475308324086     Document Type: Review

References (108)

1. Abrami, P.C., Cohen, P.A., & d' Apollonia, S. (1988). Implementation problems in meta-analysis. Review of Educational Research, 58(2), 151-179.
2. Abrami, P.C., d'Appollonia, S., & Cohen, P.A. (1990). Validity of student ratings of instruction: What we know and what we do not. Journal of Educational Psychology, 82, 219-231.
3. Adams, J.B. (2005). What makes the grade? Faculty and student perceptions. Teaching of Psychology, 32(1), 21-24.
4. Adams, J.V. (1997). Student evaluations: The rating game. Inquiry, 1(2), 10-16.
5. Aleamoni, L.M. (1999). Student ratings myths versus research facts from 1924 to 1998. Journal of Personnel Evaluation in Education, 13(2), 153-166.
6. Attiyeh, R., & Lumsden, K.G. (1972). Some modern myths in teaching economics: The U. K. experience. American Economic Review, 62, 429-433.
7. Bacon, D.R., & Novotny, J. (2002). Exploring achievement striving as a moderator of the grading leniency effect. Journal of Marketing Education, 24, 4-14.
8. Baird, J.S. (1987). Perceived learning in relation to student evaluation of university instruction. Journal of Educational Psychology, 79, 9091.
9. Bendig, A.W. (1953). The relationship of level of course achievement to students' instructor and course ratings in introductory psychology. Educational and Psychological Measurement, 13, 437-448.
10. Bharadwaj, S., Futrell, C.M., & Kantak, D.M. (1993). Using student evaluations to improve learning. Marketing Education Review, 3(2), 16-21.
11. Birnbaum, M.H. (2000). A survey of faculty opinions concerning student evaluation of teaching. Retrieved June 21, 2008, from http://psych.fullerton.edu/mbirnbaum/faculty3.htm
12. Boex, L.F.J. (2000). Attributes of effective economics instructors: An analysis of student evaluations. Research in Economic Education, 31, 211-227.
13. Braskamp, L.A., Caulley, D., & Costin, F. (1979). Student ratings and instructor self-rating and their relationship to student achievement. American Educational Research Journal, 16, 295-306.
14. Braskamp, L.A., & Ory, J.C. (1994). Assessing faculty work: Enhancing individual and institutional performances. San Francisco: Jossey-Bass.
15. Cashin, W.E. (1988). Student ratings of teaching: A summary of the research (IDEA Paper No. 20). Manhattan: Center for Faculty Evaluation & Development, Division of Continuing Education, Kansas State University.
16. Cashin, W.E. (1995). Student ratings of teaching: The research revisited (IDEA Paper No. 32). Manhattan: Center for Faculty Evaluation & Development, Division of Continuing Education, Kansas State University.
17. Centra, J.A. (1977). Student ratings of instruction and their relationship to student learning. American Educational Research Journal, 14, 17-24.
18. Centra, J.A. (2003). Will teachers receive higher student evaluations by giving higher grades and less course work? Research in Higher Education, 44, 495-518.
19. Chacko, T.I. (1983). Student ratings of instruction: A function of grading standards. Educational Research Quarterly, 8(2), 19-25.
20. Chonko, L.B., Tanner, J.F., & Davis, R. (2002). What are they thinking? Students' expectations and self-assessments. Journal of Education for Business, 77, 271-281.
21. Clayson, D.E. (1994). Contrasting results of three methodological approaches on the interpretation of a student evaluation of instruction. In E. W. Chandler (Ed.), Proceedings of the Midwest Marketing Association (pp. 209-214). Chicago: Midwest Marketing Association.
22. Clayson, D.E. (2004). A test of the reciprocity effect in the student evaluation of instructors in marketing classes. Marketing Education Review, 14(2), 11-21.
23. Clayson, D.E. (2005 a). Performance overconfidence: Metacognitive effects or misplaced student experience. Journal of Marketing Education, 27, 122-129.
24. Clayson, D.E. (2005 b). Within-class variability in student-teacher evaluations: Example and problems. Decision Sciences Journal of Innovative Education, 3(1), 109-124.
25. Clayson, D.E. (2007). Conceptual and statistical problems of using between-class data in educational research. Journal of Marketing Education, 29, 34-38.
26. Clayson, D.E., Frost, T.F., & Sheffet, M.J. (2005). Grades and the student evaluation of instruction: A test of the reciprocity effect. Academy of Management Learning & Education, 5(1), 52-65.
27. Clayson, D.E., & Haley, D.A. (1990). Student evaluations in marketing: What is actually being measured ? Journal of Marketing Education, 12, 9-17.
28. Clayson, D.E., & Sheffet, M.J. (2006). Personality and the student evaluation of teaching. Journal of Marketing Education, 28, 149-160.
29. Cohen, P.A. (1981). Student ratings of instruction and student achievement: A meta-analysis of multi-section validity studies. Review of Educational Research, 51, 281-309.
30. Comm, C.L., & Manthaisel, D.F.X. (1998). Evaluating teaching effectiveness in America's business schools: Implications for service marketers. Journal of Professional Service Marketing, 16(2), 163-170.
31. Costin, F. (1978). Do student ratings of college teachers predict student achievement ? Teaching of Psychology, 5(2), 86-88.
32. Cruse, D.B. (1987). Student evaluations of the university professor: Caveat professor. Higher Education, 16, 723-737.
33. Dowell, D.A., & Neal, J.A. (1982). A selective review of the validity of student ratings of teaching. Journal of Higher Education, 53, 51-62.
34. Doyle, K.O., & Whitely, S.E. (1974). Student ratings as criteria for effective teaching. American Educational Research Journal, 11, 259-274.
35. Faranda, W.T., & Clarke, I., III. (2004). Student observations of outstanding teaching: Implications for marketing educators. Journal of Marketing Education, 26, 271-281.
36. Flowers, L., Osterlind, S.J., Pascarella, E.T., & Pierson, C.T. (2001). How much do students learn in college? Journal of Higher Education, 72, 565-583.
37. Frey, P.W. (1973). Student ratings of teaching: Validity of several rating factors. Science, 182(4107), 83-85.
38. Frey, P.W., Leonard, D.W., & Beatty, W.W. (1975). Student ratings of instruction: Validation research. American Educational Research Journal, 12, 435-447.
39. Gaski, J.F. (1987). On "Construct validity of measures of college teaching effectiveness." Journal of Educational Psychology, 79, 326-330.
40. Gaultney, J.F., & Cann, A. (2001). Grade expectations. Teaching of Psychology, 28(2), 84-87.
41. Gillmore, G.M., & Greenwald, A.G. (1999). Using statistical adjustment to reduce biases in student ratings. American Psychologist, 54, 518-519.
42. Goldberg, G., & Callahan, J. (1991). Objectivity of student evaluations of instructors. Journal of Education for Business, 66, 377-378.
43. Goldman, L. (1985). The betrayal of the gatekeepers: Grade inflation. Journal of General Education, 37(2), 97-121.
44. Gramlich, E.M., & Greenlee, G.A. (1993). Measuring teaching performance. Research in Economic Education, 24(1), 3-13.
45. Greenwald, A.G. (1997). Validity concerns and usefulness of student ratings of instruction. American Psychologist, 52, 1182-1186.
46. Greenwald, A.G., & Gillmore, G.M. (1997 a). Grading leniency is a removable contaminant of student ratings. American Psychologist, 52, 1209-1217.
47. Greenwald, A.G., & Gillmore, G.M. (1997 b). No pain, no gain? The importance of measuring course workload in student ratings of instruction. Journal of Educational Psychology, 89, 743-751.
48. Gremler, S.D., & McCollough, M.A. (2002). Student satisfaction guarantees: An empirical examination of attitudes, antecedents, and consequences. Journal of Marketing Education, 24, 150-160.
49. Grimes, P.W. (2002). The overconfident principles of economics students: An examination of metacognitive skill. Journal of Economic Education, 33(1), 15-30.
50. Guthrie, E.R. (1954). The evaluation of teaching: A progress report. Seattle: University of Washington.
51. Hake, R.R. (2002). Problems with student evaluations: Is assessment the remedy? Retrieved June 19, 2007, and February 13, 2008, from http://physics.indiana.edu/~hake/assesstherem1.pdf
52. Howard, G.S., & Maxwell, S.E. (1980). Correlation between student satisfaction and grades: A case of mistaken causation? Journal of Educational Psychology, 72, 810-820.
53. Hunter, J.E., & Schmidt, F.L. (2004). Methods of meta-analysis: Correcting error and bias in research findings. Thousands Oaks, CA: Sage.
54. Johnson, V.E. (2003). Grade inflation: A crisis in college education. New York: Springer.
55. Kaplan, M., Mets, L.A., & Cook, C.E. (2000). Questions frequently asked about student ratings forms: Summary of research findings. Retrieved March 31, 2006, from http://www.crlt.umich.edu/tstrategies/ studentratingfaq.html
56. Kennedy, E.J., Lawton, L., & Plumlee, E.L. (2002). Bliss ignorance: The problem of unrecognized incompetence and academic performance. Journal of Marketing Education, 24, 243-252.
57. Kolevzon, M.S. (1981). Grade inflation in higher education: A comparative study. Research in Higher Education, 15, 195-212.
58. Kulik, J.A. (2001). Student ratings: Validity, utility, and controversy. New Directions for Institutional Research, 109, 9-25.
59. Laverie, D.A. (2002). Improving teaching through improving evaluation: A guide to course portfolios. Journal of Marketing Education, 24, 104-113.
60. Lundsten, N.L. (1986). Student evaluations in a business administration curriculum: A marketing viewpoint. AMA Developments in Marketing Science, 9, 169-173.
61. Lyons, L.C. (1997). Meta-analysis: Methods of accumulating results across domains. Retrieved February 6, 2008, from http://www.lyons-morris.com/MetaA/index.htm
62. Machina, K. (1987). Evaluating student evaluations. Academe, 73(3), 19-22.
63. Marks, R.B. (2000). Determinants of student evaluations of global measures of instructor and course value. Journal of Marketing Education, 22, 108-119.
64. Marlin, J.W., & Niss, J.F. (1980). End-of-course evaluations as indicators of student learning and instructor effectiveness. Journal of Economic Education, 11(2), 16-27.
65. Marsh, H.W. (1987). Students' evaluations of university teaching: Research findings, methodological issues, and directions for future research. International Journal of Educational Research, 11, 253-388.
66. Marsh, H.W., & Dunkin, M. (1992). Students' evaluations of university teaching: A multidimensional perspective. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. 8, pp. 143-233). New York: Agathon.
67. Marsh, H.W., Hau, K., Chung, C., & Siu, T.L. (1997). Students' evaluations of university teaching: Chinese version of the Students' Evaluations of Educational Quality instrument. Journal of Educational Psychology, 89, 568-572.
68. Marsh, H.W., & Roche, L.A. (1997). Making students' evaluations of teaching effectiveness effective. American Psychologist, 52, 1187-1197.
69. Marsh, H.W., & Roche, L.A. (1999). Reply upon SET research. American Psychologist, 54, 517-518.
70. Marsh, H.W., & Roche, L.A. (2000). Effects of grading leniency and low workload on students' evaluations of teaching: Popular myth, bias, validity, or innocent bystanders? Journal of Educational Psychology, 92, 202-228.
71. McKeachie, W.J. (1987). Commentary: Instructional evaluation: Current issues and possible improvements. Journal of Higher Education, 58, 344-350.
72. Moore, M., & Trahan, R. (1998). Tenure status and grading practices. Sociological Perspectives, 41, 775-781.
73. Moreland, R., Miller, J., & Laucka, F. (1981). Academic achievement and self-evaluations of academic performances. Journal of Educational Psychology, 73, 335-344.
74. Morley, C., & Evertt, L. D. (Eds). (1965). Bartlett's familiar quotations. New York: Pocket Books.
75. Morsh, J.E., Burgess, G.G., & Smith, P.N. (1956). Student achievement as a measure of instructor effectiveness. Journal of Educational Psychology, 47(2), 79-88.
76. Palmer, J., Carliner, G., & Romer, T. (1978). Leniency, learning, and evaluations. Journal of Educational Psychology, 70, 855-863.
77. Paswan, A.K., & Young, J.A. (2002). Student evaluation of instructors: A nomological investigation using structural equation modeling. Journal of Marketing Education, 24, 193-202.
78. Pollio, H.R., & Beck, H.P. (2000). When the tail wags the dog. Journal of Higher Education, 71, 84-102.
79. Powell, R.W. (1977). Grades, learning, and student evaluation of instructors. Research in Higher Education, 7, 193-205.
80. Redding, R.E. (1998). Students' evaluation of teaching fuel grade inflation. American Psychologist, 53, 1227-1228.
81. Remmers, H.H., & Brandenburg, G.C. (1927). Experimental data on the Purdue Rating Scale for Instruction. Educational Administration and Supervision, 13, 519-527.
82. Rodin, M., & Rodin, B. (1972). Student evaluation of teachers. Science, 177, 1164-1166.
83. Rosenthal, R., Rosnow, R.I., & Rubin, D.B. (2000). Contrasts and effect sizes in behavioral research. Cambridge, UK: Cambridge University Press.
84. Ryan, J.J., Anderson, J.A., & Birchler, A.B. (1980). Student evaluation: The faculty responds. Research in Higher Education, 12, 317-333.
85. Schlee, R.P. (2005). Social styles of students and professors: Do students' social styles influence their preferences for professors? Journal of Marketing Education, 27, 130-142.
86. Schmidt, T.A., Houston, M.B., Bettencourt, L.A., & Boughton, P.D. (2003). The impact of voice and justification on students' perceptions of professors' fairness. Journal of Marketing Education, 25, 177-186.
87. Schulze, R. (2007). Current methods for meta-analysis: Approaches, issues, and developments. Zeitschrift für Psychologie/Journal of Psychology, 215(2), 90-103.
88. Schwab, D.P. (1976). Manual for the Course Evaluation Instrument. Madison: University of Wisconsin, School of Business.
89. Scriven, M. (1983). Summative teacher evaluations. In J. Milman (Ed.), Handbook of teacher evaluation (pp. 244-271). Thousand Oaks, CA: Sage.
90. Seiver, D.A. (1983). Evaluations and grades: A simultaneous framework. Journal of Economic Education, 14(3), 32-38.
91. Seldin, P. (1993, July 21). The use and abuse of student ratings of professors. Chronicles of Higher Education, 39(46), A40.
92. Seldin, P. (1999). Changing practices in evaluating teaching: A practical guide to improving faculty performance and promotion/tenure decisions. Bolton, MA: Anker.
93. Sheehan, E.P., & DuPrey, T. (1999). Student evaluations of university teaching. Journal of Instructional Psychology, 26, 188-193.
94. Sheets, D.F., Topping, E.E., & Hoftyzer, J. (1995). The relationship of student evaluations of faculty to student performance on a common final examination in the principles of economics courses. Journal of Economics, 21(2), 55-64.
95. Shmanske, S. (1988). On the measurement of teacher effectiveness. Research in Economic Education, 19, 307-314.
96. Simpson, P.M., & Siguaw, J.A. (2000). Student evaluations of teaching: An exploratory study of the faculty response. Journal of Marketing Education, 22, 199-213.
97. Sixbury, G.R., & Cashin, W.E. (1995). Description of database for the IDEA Diagnostic Form (IDEA Technical Report No. 9). Manhattan: Center for Faculty Evaluation & Development, Division of Continuing Education, Kansas State University.
98. Soper, J.C. (1973). Soft research on a hard subject: Student evaluations reconsidered. Journal of Economic Research, 5(1), 22-26.
99. Stapleton, R.J., & Murkison, G. (2001). Optimizing the fairness of student evaluations: A study of correlations between instructor excellence, study production, learning production, and expected grades. Journal of Management Education, 25, 269-291.
100. Steiner, S., Holley, L.C., Gerdes, K., & Campbell, H.E. (2006). Evaluating teaching: Listening to students while acknowledging bias. Journal of Social Work Education, 42, 355-376.
101. Stumpf, S.A., & Freedman, R.D. (1979). Expected grade covariation with student ratings of instruction: Individual versus class effects. Journal of Educational Psychology, 71, 293-302.
102. Sullivan, A.M., & Skanes, G.R. (1974). Validity of student evaluation of teaching and the characteristics of successful instructors. Journal of Educational Psychology, 66, 584-590.
103. Theall, M., & Franklin, J. (2001). Looking for bias in all the wrong places: A search for truth or a witch hunt in student ratings of instruction? New Directions for Institutional Research, 27(5), 45-56.
104. Weinberg, B.A., Fleisher, B.M., & Hashimoto, M. (2007). Evaluating methods for evaluating instruction: The case of higher education (NBER Working Paper No. 12844). Retrieved June 21, 2008, from http://www.nber.org/papers/w12844
105. Wilhelm, W.B. (2004). The relative influence of published teaching evaluations and other instructor attributes on course choice. Journal of Marketing Education, 26, 17-30.
106. Williams, W.M., & Ceci, S.J. (1997). "How'm I Doing?": Problems with student ratings of instructors and courses. Change, 29(5), 13-23.
107. Wilson, R. (1998). New research casts doubt on value of student evaluations of professors. Chronicle of Higher Education, 44(19), A12-A14.
108. Yunker, P. J., & Yunker, J. (2003). Are student evaluations of teaching valid? Evidence from an analytical business core course. Journal of Education for Business, 78, 313-317.