Bias in grading: A meta-analysis of experimental research findings

Australian Journal of Education

Volumn 60, Issue 3, 2016, Pages 245-256

  Malouff, John M ^a   Thorsteinsson, Einar B ^a  

^a UNIVERSITY OF NEW ENGLAND   (Australia)

Author keywords

assessment; bias; Ethnicity; experiment; grading; marking (scholastic); meta analysis

Indexed keywords

EID: 84994499970     PISSN: 00049441     EISSN: 20505884     Source Type: Journal    
DOI: 10.1177/0004944116664618     Document Type: Article

References (48)

1. References marked with an asterisk indicate studies included in the meta-analysis.
2. Babad E. Y., (1980) Expectancy bias in scoring as a function of ability and ethnic labels. Psychological Reports 46: 625-626. doi: 10.2466/pr0.1980.46.2.625.
3. Babad E. Y., (1985) Some correlates of teachers' expectancy bias. American Educational Research Journal 22: 175-183.
4. Babad E. Y., Mann M., Mar-Hayim M., (1975) Bias in scoring the WISC subtests. Journal of Consulting and Clinical Psychology 43: 268,. doi: 10.1037/h0076368.
5. Baird J., (1998) What's in a name? Experiments with blind marking in A-level examinations. Educational Research 40: 191-202. doi: 10.1080/0013188980400207.
6. Batten J., Batey J., Shafe L., Gubby L., Birch P., (2013) The influence of reputation information on the assessment of undergraduate student work. Assessment & Evaluation in Higher Education 38: 417-435. doi: 10.1080/02602938.2011.640928.
7. Belsey, C. (1988). Marking by numbers. AUT women, No. 15 (pp. 1-2). London, UK: Association of University Teachers.
8. Borenstein M., Hedges L. V., Higgins J. P. T., Rothstein H. R., (2005) Comprehensive Meta-Analysis Version 2, Englewood, NJ: Biostat.
9. Bradley C., (1984) Sex bias in the evaluation of students. British Journal of Social Psychology 23: 147-153. doi: 10.1111/j.2044-8309.1984.tb00623.x.
10. Brennan D. J., (2008) University student anonymity in the summative assessment of written work. Higher Education Research & Development 27: 43-54. doi: 10.1080/07294360701658724.
11. Chase C. I., (1986) Essay test scoring: Interaction of relevant variables. Journal of Educational Measurement 23: 33-41. doi: 10.1111/j.1745-3984.1986.tb00232.x.
12. Cooper W. H., (1981) Ubiquitous halo. Psychological Bulletin 90: 218-244. doi: 10.1037/0033-2909.90.2.218.
13. Cornett-Ruiz S., Hendricks B., (1993) Effects of labeling and ADHD behaviors on peer and teacher judgments. The Journal of Educational Research 86: 349-355.
14. Darley, J. M., & Gross, P. H. (1983). A hypothesis-confirming bias in labeling effects. Journal of Personality and Social Psychology, 44, 20-33. doi: 10.1037/0022-3514.44.1.20.
15. Dennis I., Newstead S. E., (1994) The strange case of the disappearing sex bias. Assessment & Evaluation in Higher Education 19: 49-56. doi: 10.1080/0260293940190105.
16. Erwin P. G., Calev A., (1984) The influence of Christian name stereotypes on the marking of children's essays. British Journal of Educational Psychology 54: 223-227. doi: 10.1111/j.2044-8279.1984.tb02583.x.
17. Fogel L. S., Nelson R. O., (1983) The effects of special education labels on teachers' behavioral observations, checklist scores, and grading of academic work. Journal of School Psychology 21: 241-251. doi: 10.1016/0022-4405(83)90019-5.
18. ∗Gerritson, M. (2013). Rubrics as a mitigating instrument for bias in the grading of student writing (Doctoral dissertation, Walden University).
19. Graham S., Dwyer A., (1987) Effects of the learning disability label, quality of writing performance, and examiner's level of expertise on the evaluation of written products. Journal of Learning Disabilities 20: 317-318.
20. Graham S., Leone P., (1987) Effects of behavioral disability labels, writing performance, and examiner's expertise on the evaluation of written products. Journal of Experimental Education 55: 89-94.
21. Greenwald A. G., Banaji M. R., (1995) Implicit social cognition: Attitudes, self-esteem, and stereotypes. Psychological Review 102: 4-27. doi: 10.1037/0033-295X.102.1.4.
22. Guttmann J., Boudo M., (1988) Teachers' evaluations of pupils' performance as a function of pupils' sex, family type and past school performance. Educational Review 40: 105-113. doi: 10.1080/0013191880400108.
23. Haeran J., Cowling J., (2010) Objectivity in grading: The promise of bar codes. College Teaching 57: 51-55.
24. ∗Hanna, R., & Linden, L. (2009). Measuring discrimination in education (No. w15057). Cambridge, MA: National Bureau of Economic Research.
25. Harari H., McDavid J. W., (1973) Name stereotypes and teachers' expectations. Journal of Educational Psychology 65: 222-224. doi: 10.1037/h0034978.
26. Jonsson A., Svingby G., (2007) The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review 2: 130-144.
27. Kahneman D., (2011) Thinking, fast and slow, New York, NY: Farrar, Straus and Giroux.
28. Kaplan R. M., (1978) Is beauty talent? Sex interaction in the attractiveness halo effect. Sex Roles 4: 195-204.
29. Kayaplnar U., (2014) Measuring essay assessment: Intra-rater and inter-rater reliability. Eurasian Journal of Educational Research 57: 113-136. http://dx.doi.org/10.14689/ejer.2014.57.2.
30. ∗Kehle, T. J. (1976). Effect of physical attractiveness, sex, and intelligence on expectations for students' academic ability and personality: A replication. Retrieved from ERIC database (ED124288).
31. King J. L., (1998) The effects of gender bias and errors in essay grading. Educational Research Quarterly 22: 13-25.
32. Klein, J., & Taub, D. (2005). The effect of variations in handwriting and print on evaluation of student essays. Assessing Writing, 10, 134-148.
33. Landy D., Sigall H., (1974) Beauty is talent: Task evaluation as a function of the performer's physical attractiveness. Journal of Personality and Social Psychology 29: 299-304. doi: 10.1037/h0036018.
34. Lebuda I., Karwowski M., (2013) Tell me your name and I'll tell you how creative your work is: Author's name and gender as factors influencing assessment of products' creativity in four different domains. Creativity Research Journal 25: 137-142. doi: 10.1080/10400419.2013.752297.
35. Lipsey M. W., Wilson D. B., (2001) Practical meta-analysis, Thousand Oaks, CA: Sage.
36. Malouff J. M., (2008) Bias in grading. College Teaching 56: 191-192. doi: 10.3200/CTCH.56.3.191-192.
37. Malouff J. M., Emmerton A. J., Schutte N. S., (2013) The risk of a halo bias as a reason to keep students anonymous during grading. Teaching of Psychology 40: 233-237. doi: 10.1177/0098628313487425.
38. ∗Malouff, J. M., Stein, S. J., Bothma, L. N., Coulter, K., & Emmerton, A. J. (2014). Preventing halo bias in grading the work of university students. Cogent Psychology, 1 (1). Published online at http://cogentoa.tandfonline.com/doi/full/10.1080/23311908.2014.988937.
39. Middlesex Students' Union (undated). Mark my work, not my name: Introduce anonymous marking wherever possible. Retrieved from http://www.change.org/en-GB/petitions/middlesex-university-mark-my-work-not-my-name-introduce-anonymous-marking-wherever-possible.
40. Perry-Langdon, N. (1990). Marking by numbers: Evaluation of the marking of final degree examinations in the Faculty of Humanities and Social Studies. Retrieved from British Library Document Supply Centre-DSC: q94/21236.
41. Pirog M. A., Buffardi A. L., Chrisinger C. K., Singh P., Briney J., (2009) Are the alternatives to randomized assignment nearly as good? Statistical corrections to nonrandomized evaluations. Journal of Policy Analysis and Management 28 (1): 169-172. http://www.jstor.org/stable/29738995.
42. Rosenthal R., (1979) The "file drawer problem" and tolerance for null results. Psychological Bulletin 86: 638-641. doi: 10.1037/0033-2909.86.3.638.
43. Seraydarian L., Busse T. V., (1981) First-name stereotypes and essay grading. The Journal of Psychology 108: 253-257. doi: 10.1080/00223980.1981.9915271.
44. Shadish W. R., Ragsdale K., (1996) Random versus nonrandom assignment in controlled experiments: Do you get the same answer? Journal of Consulting and Clinical Psychology 64: 1290.
45. Sprietsma M., (2013) Discrimination in grading: Experimental evidence from primary school teachers. Empirical Economics 45: 523-538.
46. Van Ewijk R., (2011) Same work, lower grade? Student ethnicity and teachers' subjective assessments. Economics of Education Review 30: 1045-1058.
47. Warren Piper D., McNulty D. D., O'Grady G., (1996) Examination practices and procedures in Australian universities, Canberra, Australia: Australian Government Publishing Service.
48. Wen S. S., (1979) Racial halo on evaluative rating: General or differential? Contemporary Educational Psychology 4: 15-19. doi: 10.1016/0361-476X(79)90022-5.