A critical review of the arguments against the use of rubrics

Educational Research Review

Volumn 30, Issue , 2020, Pages

  Panadero, Ernesto ^a,b   Jonsson, Anders ^c  

^a UNIVERSITY OF DEUSTO   (Spain)

^b BASQUE FOUNDATION FOR SCIENCE   (Spain)

^c KRISTIANSTAD UNIVERSITY   (Sweden)

Author keywords

Criteria compliance; Explicit criteria; Formative assessment; Rubrics; Self regulated learning

Indexed keywords

EID: 85083576422     PISSN: 1747938X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.edurev.2020.100329     Document Type: Review

References (89)

1. The publications indicated with ∗ were included as empirical evidence in this review.
2. Ajjawi, R., Bearman, M., Problematizing “standards”: Representation or performance?. Boud, D., Ajjawi, R., Dawson, P., Tai, J., (eds.) Developing evaluative judgement in higher education: Assessment for knowing and producing quality work, 2018, Routledge, Oxon; New York, NY, 41–50.
3. Andrade, H., Teaching with rubrics: The good, the bad, and the ugly. College Teaching 53:1 (2005), 27–32, 10.3200/CTCH.53.1.27-31.
4. Andrade, H.L., The trouble with a narrow view of rubrics. English Journal, 2006 9-9.
5. Andrade, H., Du, Y., Student perspectives on rubric-referenced assessment. Practical Assessment, Research and Evaluation 10:3 (2005), 1–11 Retrieved from http://pareonline.net/getvn.asp?v==10=n==3.
6. Andrade, H., Valtcheva, A., Promoting learning and achievement through self-assessment. Theory Into Practice 48:1 (2009), 12–19, 10.1080/00405840802577544.
7. Arter, J., McTighe, J., Scoring rubrics in the classroom: Using performance criteria for assessing and improving student performance. 2000, Corwin Press.
8. Au, W., High-stakes testing and curricular control: A qualitative metasynthesis. Educational Researcher 36 (2007), 258–267, 10.3102/0013189X07306523.
9. Bacchus, R., Colvin, E., Knight, E.B., Ritter, L., When rubrics aren't enough: Exploring exemplars and student rubric co-construction. Journal of Curriculum and Pedagogy, 2019, 1–14, 10.1080/15505170.2019.1627617.
10. Balloo, K., Evans, C., Hughes, A., Zhu, X., Winstone, N.E., Transparency isn't spoon-feeding: How a transformative approach to the use of explicit assessment criteria can support student self-regulation. Frontiers in Education, 3(69), 2018, 10.3389/feduc.2018.00069.
11. Bearman, M., Ajjawi, R., From “seeing through” to “seeing with”: Assessment criteria and the myths of transparency. Frontiers in Education, 3(96), 2018, 10.3389/feduc.2018.00096.
12. Bell, A., Mladenovic, R., Price, M., Students' perceptions of the usefulness of marking guides, grade descriptors and annotated exemplars. Assessment = Evaluation in Higher Education 38:7 (2013), 769–788, 10.1080/02602938.2012.714738.
13. Biesta, G., Good education in an age of measurement: On the need to reconnect with the question of purpose in education. Educational Assessment, Evaluation and Accountability 21 (2008), 33–46.
14. Biggs, J., Student approaches to learning. 1987, Australian Council for Educational Research, Melbourne.
15. Biggs, J., What the student does: Teaching for enhanced learning. Higher Education Research and Development 18:1 (1999), 57–75, 10.1080/0729436990180105.
16. Black, P., Wiliam, D., Assessment and classroom learning. Assessment in Education: Principles, Policy = Practice 5:1 (1998), 7–73, 10.1080/0969595980050102.
17. Bloxham, S., Boyd, P., Orr, S., Mark my words: The role of assessment criteria in UK higher education grading practices. Studies in Higher Education 36:6 (2011), 655–670, 10.1080/03075071003777716.
18. Bouwer, R., Lesterhuis, M., Bonne, P., De Maeyer, S., Applying criteria to examples or learning by comparison: Effects on students' evaluative judgment and performance in writing. Frontiers in Education, 3(86), 2018, 10.3389/feduc.2018.00086.
19. Brookhart, S.M., How to create and use rubrics for formative assessment and grading. 2013, ASCD, Virginia, USA.
20. Brookhart, S.M., Appropriate criteria: Key to effective rubrics. Frontiers in Education, 3(22), 2018, 10.3389/feduc.2018.00022.
21. Brookhart, S.M., Chen, F., The quality and effectiveness of descriptive rubrics. Educational Review 67:3 (2015), 343–368, 10.1080/00131911.2014.929565.
22. Brookhart, S.M., Guskey, T.R., Bowers, A.J., McMillan, J.H., Smith, J.K., Smith, L.F., Welsh, M.E., A century of grading research. Review of Educational Research 86:4 (2016), 803–848, 10.3102/0034654316672069.
23. Brown, G.T.L., Assessment of student achievement. 2018, Routledge, New York.
24. Chapman, V.G., Inman, M.D., A conundrum: Rubrics or creativity/metacognitive development?. Educational Horizons, 2009, 198–202.
25. Cheng, M.W.T., Chan, C.K.Y., An experimental test: Using rubrics for reflective writing to develop reflection. Studies In Educational Evaluation 61 (2019), 176–182, 10.1016/j.stueduc.2019.04.001.
26. Dawson, P., Assessment rubrics: Towards clearer and more replicable design, research and practice. Assessment = Evaluation in Higher Education, 2017, 1–14, 10.1080/02602938.2015.1111294.
27. Entwistle, N.J., Ramsden, P., Understanding student learning. 1983, Croom Helm, London = Canberra.
28. Fraile, J., Panadero, E., Pardo, R., Co-creating rubrics: Useful or waste of time? The effects of establishing assessment criteria with students on self- regulation, self-efficacy and performance. Studies In Educational Evaluation 53 (2017), 69–76, 10.1016/j.stueduc.2017.03.003.
29. Greenberg, K.P., Rubric use in formative assessment: A detailed behavioral rubric helps students improve their scientific writing skills. Teaching of Psychology 42:3 (2015), 211–217, 10.1177/0098628315587618.
30. Halonen, J.S., Bosack, T., Clay, S., McCarthy, M., Dunn, D.S., Hill Iv, G.W., Whitlock, K., A rubric for learning, teaching, and assessing scientific inquiry in psychology. Teaching of Psychology 30:3 (2003), 196–208, 10.1207/s15328023top3003_01.
31. Helvoort, J. van, A scoring rubric for performance assessment of information literacy in Dutch Higher Education. Journal of Information Literacy 4:1 (2010), 22–39, 10.11645/4.1.1256.
32. Holmstedt, P., Jonsson, A., Aspelin, J., Learning to see new things: Using criteria to support pre-service teachers' discernment in the context of teachers' relational work. Frontiers in Education: Assessment, Testing and Applied Measurement, 3(54), 2018, 10.3389/feduc.2018.00054.
33. Hudson, J., Bloxham, S., den Outer, B., Price, M., Conceptual acrobatics: Talking about assessment standards in the transparency era. Studies in Higher Education 42:7 (2017), 1309–1323, 10.1080/03075079.2015.1092130.
34. Humphry, S.M., Heldsinger, S.A., Common structural design features of rubrics may represent a threat to validity. Educational Researcher 43:5 (2014), 253–263, 10.3102/0013189x14542154.
35. Ito, H., ∗Is a rubric worth the time and effort? Conditions for its success. International Journal of Learning, Teaching and Educational Research, 10(2), 2015.
36. Jones, L., Allen, B., Dunn, P., Brooker, L., Demystifying the rubric: A five-step pedagogy to improve student understanding and utilisation of marking criteria. Higher Education Research and Development, 2016, 1–14, 10.1080/07294360.2016.1177000.
37. Jonsson, A., Rubrics as a way of providing transparency in assessment. Assessment = Evaluation in Higher Education, 2014, 1–13, 10.1080/02602938.2013.875117.
38. Jonsson, A., Rubrics as a tool for self-regulated learning. Grainger, P., Weir, K., (eds.) Assessment rubrics in higher education, 2020, Cambridge Scholars Publishing, Newcastle upon Tyne, 25–40.
39. Jonsson, A., Balan, A., Analytic or holistic: A study of agreement between different grading models. Practical Assessment, Research and Evaluation, 23(12), 2018 Retrieved from http://pareonline.net/getvn.asp?v==23=n==12.
40. Jonsson, A., Eriksson, U., Formative assessment in higher education: An example from astronomy. Andrade, H.L., Bennett, R.E., Cizek, G.J., (eds.) Handbook of formative assessment in the disciplines, 2019, Routledge, New York, NY, = London, 146–169.
41. Jönsson, A., Panadero, E., The use and design of rubrics to support AfL. Carless, D., (eds.) Scaling up assessment for learning in higher education, 2016, Springer, Dordrecht, 99–111.
42. Jonsson, A., Svingby, G., The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review 2 (2007), 130–144.
43. Joseph, S., Rickett, C., Northcote, M., Christian, B.J., Who are you to judge my writing?’: Student collaboration in the co-construction of assessment rubrics. New Writing. 2019, 10.1080/14790726.2019.1566368.
44. Kane, M., Crooks, T., Cohen, A., Validating measures of performance. Educational Measurement: Issues and Practice 18 (1999), 5–17.
45. Kilgour, P., Northcote, M., Williams, A., Kilgour, A., A plan for the co-construction and collaborative use of rubrics for student learning. Assessment = Evaluation In Higher Education, 2019, 1–14, 10.1080/02602938.2019.1614523.
46. Kocakulah, M.S., Development and application of a rubric for evaluating students' performance on Newton's laws of motion. Journal of Science Education and Technology 19:2 (2010), 146–164, 10.1007/s10956-009-9188-9.
47. Kohn, A., ∗The trouble with rubrics. English Journal 95:4 (2006), 12–15.
48. Lindström, L., Creativity: What is it? Can you assess it? Can it be taught?. Journal of Art = Design Education 25:1 (2006), 53–66, 10.1111/j.1476-8070.2006.00468.x.
49. Lindström, L., Assessing craft and design. Conceptions of expertise in education and work. Havnes, A., McDowell, L., (eds.) Balancing dilemmas in assessment and learning in contemporary education, 2008, Routledge, New York, NY, 61–72.
50. Lipnevich, A.A., McCallen, L.N., Miles, K.P., Smith, J.K., Mind the gap! Students' use of exemplars and detailed rubrics as formative assessment. Instructional Science 42:4 (2014), 539–559, 10.1007/s11251-013-9299-9.
51. Lovorn, M.G., Rezaei, A.R., Assessing the assessment: Rubrics training for pre-service and new in-service teacher. Practical Assessment, Research and Evaluation, 16(16), 2011.
52. Mabry, L., Writing to the rubric: Lingering effects of traditional standardized testing on direct writing assessment. Phi Delta Kappan 80:9 (1999), 673–679.
53. Messick, S., Validity of performance assessments. Phillips, G.W., (eds.) Technical issues in large-scale performance assessment, 1996, National Center for Education Statistics, Washington, DC, 1–18.
54. Minarechová, M., Negative impacts of high-stakes testing. Journal of Pedagogy 3:1 (2012), 82–100, 10.2478/v10159-012-0004-x.
55. Moskal, B.M., Leydens, J.A., Scoring rubric development: Validity and reliability. Practical assessment, research = evaluation, Vol. 7, 2000 10). Retrieved from http://PAREonline.net/getvn.asp?v==7=n==10.
56. Nichols, S.L., Harris, L.R., Accountability assessment's effects on teachers and schools. Brown, G.T.L., Harris, L.R., (eds.) Handbook of human and social conditions in assessment, 2016, Routledge, New York, 40–56.
57. Norton, L., Using assessment criteria as learning criteria: A case study in psychology. Assessment = Evaluation in Higher Education 29:6 (2004), 687–702, 10.1080/0260293042000227236.
58. O'Donovan, B., Price, M., Rust, C., Know what I mean? Enhancing student understanding of assessment standards and criteria. Teaching in Higher Education 9:3 (2004), 325–335, 10.1080/1356251042000216642.
59. Panadero, E., Alonso-Tapia, J., Huertas, J.A., Rubrics and self-assessment scripts effects on self-regulation, learning and self-efficacy in secondary education. Learning and Individual Differences 22:6 (2012), 806–813, 10.1016/j.lindif.2012.04.007.
60. Panadero, E., Alonso-Tapia, J., Huertas, J.A., Rubrics vs. self-assessment scripts: Effects on first year university students' self-regulation and performance. Infancia Y Aprendizaje: Journal for the Study of Education and Development 37:1 (2014), 149–183, 10.1080/02103702.2014.881655.
61. Panadero, E., Jonsson, A., The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review 9 (2013), 129–144, 10.1016/j.edurev.2013.01.002 0.
62. Panadero, E., Jonsson, A., Botella, J., Effects of self-assessment on self-regulated learning and self-efficacy: Four meta-analyses. Educational Research Review 22 (2017), 74–98, 10.1016/j.edurev.2017.08.004.
63. Panadero, E., Romero, M., To rubric or not to rubric? The effects of self-assessment on self-regulation, performance and self-efficacy. Assessment in Education: Principles, Policy = Practice 21:2 (2014), 133–148, 10.1080/0969594X.2013.877872.
64. Parkes, J., Reliability in classroom assessment. McMillan, J.H., (eds.) SAGE handbook of research on classroom Assessment, 2013, SAGE, Los Angeles, CA, London, New Dehli, Singapore, Washington DC, 107–123.
65. Popham, W.J., The instructional consequences of criterion referenced clarity. Educational Measurement: Issues and Practice 13:4 (1994), 15–18, 10.1111/j.1745-3992.1994.tb00565.x 30.
66. Popham, J., What's wrong and what's right with rubrics. Educational Leadership 55:2 (1997), 72–75.
67. Reddy, Y.M., Andrade, H., A review of rubric use in higher education. Assessment = Evaluation in Higher Education 35:4 (2010), 435–448, 10.1080/02602930902862859.
68. Reynolds-Keefer, L., Rubric-referenced assessment in teacher preparation: An opportunity to learn by using. Practical Assessment, Research and Evaluation, 15(8), 2010 Retrieved from http://pareonline.net/getvn.asp?v==15=n==8.
69. Rezaei, A.R., Lovorn, M.G., ∗Reliability and validity of rubrics for assessment through writing. Assessing Writing 15:1 (2010), 18–39, 10.1016/j.asw.2010.01.003.
70. Ritchie, S.M., Self-assessment of video-recorded presentations: Does it improve skills?. Active Learning in Higher Education 17:3 (2016), 207–221, 10.1177/1469787416654807.
71. Rochford, L., Borchert, P.S., Assessing higher level learning: Developing rubrics for case analysis. The Journal of Education for Business 86 (2011), 258–265, 10.1080/08832323.2010.512319.
72. Rust, C., Price, M., O'Donovan, B., Improving students' learning by developing their understanding of assessment criteria and processes. Assessment = Evaluation in Higher Education 28:2 (2003), 147–164.
73. Sadler, D.R., Indeterminacy in the use of preset criteria for assessment and grading. Assessment = Evaluation in Higher Education 34:2 (2009), 159–179, 10.1080/02602930801956059.
74. Sadler, D.R., The futility of attempting to codify academic achievement standards. Higher Education 67 (2014), 273–288, 10.1007/s10734-013-9649-1.
75. Säljö, R., Qualitative differences in learning as a function of the learner's conception of the task. Doctoral dissertation, 1975, University of Gothenburg, Gothenburg.
76. Shay, S.B., The assessment of complex performance: A socially situated interpretive act. Harvard Business Review 74:3 (2004), 307–329.
77. Shipman, D., Roa, M., Hooten, J., Wang, Z.J., ∗Using the analytic rubric as an evaluation tool in nursing education: The positive and the negative. Nurse Education Today 32:3 (2012), 246–249, 10.1016/j.nedt.2011.04.007.
78. Soderstrom, N.C., Bjork, R.A., Learning versus performance an integrative review. Perspectives on Psychological Science 10:2 (2015), 176–199.
79. Stake, R.E., Standards-based = responsive evaluation. 2004, Sage, Thousand Oaks, CA.
80. Struyven, K., Dochy, F., Janssens, S., Students' perceptions about evaluation and assessment in higher education: A review. Assessment = Evaluation in Higher Education 30:4 (2005), 331–347.
81. Taylor, F.W., The principles of scientific management. 1911, Harper = Brothers, New York.
82. Tomas, C., Whitt, E., Lavelle-Hill, R., Severn, K., Modeling holistic marks with analytic rubrics. Frontiers in Education: Assessment, Testing and Applied Measurement, 4(89), 2019, 10.3389/feduc.2019.00089.
83. Torrance, H., Assessment as learning? How the use of explicit learning objectives, assessment criteria and feedback in post-secondary education and training can come to dominate learning. Assessment in Education: Principles, Policy = Practice 14:3 (2007), 281–294, 10.1080/09695940701591867.
84. Torrance, H., Formative assessment at the crossroads: Conformative, deformative and transformative assessment. Oxford Review of Education 38:3 (2012), 323–342, 10.1080/03054985.2012.689693.
85. Turley, E.D., Gallagher, C.W., On the “uses” of rubrics: Reframing the great rubric debate. English Journal 97:4 (2008), 87–92, 10.2307/30047253.
86. Wiggins, G., Educative assessment. 1998, Jossey-Bass, San Francisco, CA.
87. Wilson, M., Rethinking rubrics in writing assessment. 2006, Heinemann, Porthsmouth, NH.
88. Wilson, M., Why I won't be using rubrics to respond to students' writing. English Journal 96:4 (2007), 62–66, 10.2307/30047167.
89. Wollenschläger, M., Hattie, J., Machts, N., Möller, J., Harms, U., What makes rubrics effective in teacher-feedback? Transparency of learning goals is not enough. Contemporary Educational Psychology, 2016, 10.1016/j.cedpsych.2015.11.003.