Rubrics enhance accuracy and reduce cognitive load in self-assessment

Metacognition and Learning

Volumn 17, Issue 2, 2022, Pages 627-650

  Krebs, Rebecca ^a   Rothstein, Björn ^a   Roelle, Julian ^a  

^a RUHR UNIVERSITY BOCHUM   (Germany)

Author keywords

Cognitive load; Judgment accuracy; Metacognition; Rubrics; Self assessment; Self regulated learning

Indexed keywords

EID: 85130491984     PISSN: 15561623     EISSN: 15561631     Source Type: Journal    
DOI: 10.1007/s11409-022-09302-1     Document Type: Article

References (78)

1. Andrade, H. G. (2001). The effects of instructional rubrics on learning to write. Current Issues in Education, 4(4).
2. Andrade Goodrich, H., & Boulay, B. (2003). Role of rubric-referenced self-assessment in learning to write. The Journal of Educational Research, 9(1), 21–30. 10.1080/00220670309596625 DOI: 10.1080/00220670309596625
3. Andrade, H., & Du, Y. (2005). Student perspectives on rubric-referenced assessment. Practical Assessment, Research, and Evaluation, 10(1), 3.
4. Andrade, H., Buff, C., Terry, J., Erano, M., & Paolino, S. (2009). Assessment-driven improvements in middle school students’ writing. Middle School Journal, 40(4), 4–12. 10.1080/00940771.2009.11461675 DOI: 10.1080/00940771.2009.11461675
5. Andrade, H. L., Du, Y., & Mycek, K. (2010). Rubric-referenced self-assessment and middle school students’ writing. Assessment in Education: Principles, Policy & Practice, 17(2), 199–214. 10.1080/09695941003696172 DOI: 10.1080/09695941003696172
6. Ashton, S., & Davies, R. S. (2015). Using scaffolded rubrics to improve peer assessment in a MOOC writing course. Distance Education, 36(3), 312–334. 10.1080/01587919.2015.1081733 DOI: 10.1080/01587919.2015.1081733
7. Baars, M., Vink, S., van Gog, T., de Bruin, A., & Paas, F. (2014). Effects of training self-assessment and using assessment standards on retrospective and prospective monitoring of problem solving. Learning and Instruction, 33, 92–107. 10.1016/j.learninstruc.2014.04.004 DOI: 10.1016/j.learninstruc.2014.04.004
8. Baker, J. M. C., Dunlosky, J., & Hertzog, C. (2010). How accurately can older adults evaluate the quality of their text recall? The effect of providing standards on judgment accuracy. Applied Cognitive Psychology, 24, 134–147. 10.1002/acp.1553 DOI: 10.1002/acp.1553
9. Bradford, K. L., Newland, A. C., Rule, A. C., & Montgomery, S. E. (2016). Rubrics as a tool in writing instruction: Effects on the opinion essays of first and second graders. Early Childhood Education Journal, 44(5), 463–472. 10.1007/s10643-015-0727-0 DOI: 10.1007/s10643-015-0727-0
10. Busch-Lauer, I. (2012). Abstracts - eine facettenreiche Textsorte der Wissenschaft [Abstracts - a multifaceted text type of science]. Linguistik Online, 52(2), https://doi.org/10.13092/lo.52.293
11. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates, Publishers.
12. De Bruin, A. B., Thiede, K. W., Camp, G., & Redford, J. (2011). Generating keywords improves metacomprehension and self-regulation in elementary and middle school children. Journal of Experimental Child Psychology, 109(3), 294–310. 10.1016/j.jecp.2011.02.005 DOI: 10.1016/j.jecp.2011.02.005
13. De Bruin, A. B., & van Gog, T. (2012). Improving self-monitoring and self-regulation: From cognitive psychology to the classroom. Learning and Instruction, 22(4), 245–252. 10.1016/j.learninstruc.2012.01.003 DOI: 10.1016/j.learninstruc.2012.01.003
14. De Bruin, A. B., & van Merriënboer, J. J. (2017). Bridging cognitive load and self-regulated learning research: A complementary approach to contemporary issues in educational research. Learning and Instruction, 51(22), 1–9. 10.1016/j.learninstruc.2017.06.001 DOI: 10.1016/j.learninstruc.2017.06.001
15. De Bruin, A. B., Roelle, J., Carpenter, S. K., & Baars, M. (2020). Synthesizing cognitive load and self-regulation theory: A theoretical framework and research agenda. Educational Psychology Review, 32, 903–915. 10.1007/s10648-020-09576-4 DOI: 10.1007/s10648-020-09576-4
16. Dunlosky, J., Hartwig, M. K., Rawson, K. A., & Lipko, A. R. (2011). Improving college students’ evaluation of text learning using idea-unit standards. Quarterly Journal of Experimental Psychology, 64(3), 467–484. DOI: 10.1080/17470218.2010.502239
17. Dunlosky, J., & Lipko, A. R. (2007). Metacomprehension: A brief history and how to improve its accuracy. Current Directions in Psychological Science, 16(4), 228–232. 10.1111/j.1467-8721.2007.00509.x DOI: 10.1111/j.1467-8721.2007.00509.x
18. Dunlosky, J., & Rawson, K. A. (2012). Overconfidence produces underachievement: Inaccurate self evaluations undermine students’ learning and retention. Learning and Instruction, 22(4), 271–280. 10.1016/j.learninstruc.2011.08.003 DOI: 10.1016/j.learninstruc.2011.08.003
19. Dunning, D., Johnson, K., Ehrlinger, J., & Kruger, J. (2003). Why people fail to recognize their own incompetence. Current Directions in Psychological Science, 12(3), 83–87. 10.1111/1467-8721.01235 DOI: 10.1111/1467-8721.01235
20. Froese, L., & Roelle, J. (2022). Expert example standards but not idea unit standards help learners accurately evaluate the quality of self-generated examples. Metacognition and Learning. 10.1007/s11409-022-09293-z DOI: 10.1007/s11409-022-09293-z
21. Furze, J., Black, L., Hoffman, J., Barr, J. B., Cochran, T. M., & Jensen, G. M. (2015). Exploration of students’ clinical reasoning development in professional physical therapy education. Journal of Physical Therapy Education, 29(3), 22–33. 10.1016/j.hpe.2020.06.002 DOI: 10.1016/j.hpe.2020.06.002
22. Gentner, D. (2010). Bootstrapping the mind: Analogical processes and symbol systems. Cognitive Science, 34(5), 752–775. 10.1111/j.1551-6709.2010.01114.x DOI: 10.1111/j.1551-6709.2010.01114.x
23. Goodman, L. A., & Kruskal, W. H. (1954). Measures of association for cross classifications. Journal of the American Statistical Association, 49, 732–764.
24. Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. The Guilford Press.
25. Hafner, J., & Hafner, P. (2003). Quantitative analysis of the rubric as an assessment tool: An empirical study of student peer-group rating. International Journal of Science Education, 25(12), 1509–1528. 10.1080/0950069022000038268 DOI: 10.1080/0950069022000038268
26. Hübner, S., Nückles, M., & Renkl, A. (2010). Writing learning journals: Instructional support to overcome learning-strategy deficits. Learning and Instruction, 20(1), 18–29. 10.1016/j.learninstruc.2008.12.001 DOI: 10.1016/j.learninstruc.2008.12.001
27. Jensen, R. (2013). Clinical reasoning during simulation: Comparison of student and faculty ratings. Nurse Education in Practice, 13(1), 23–28. 10.1016/j.nepr.2012.07.001 DOI: 10.1016/j.nepr.2012.07.001
28. Jonsson, A. (2014). Rubrics as a way of providing transparency in assessment. Assessment and Evaluation in Higher Education, 39(7), 840–852. 10.1080/02602938.2013.875117 DOI: 10.1080/02602938.2013.875117
29. Kant, J. M., Scheiter, K., & Oschatz, K. (2017). How to sequence video modeling examples and inquiry tasks to foster scientific reasoning. Learning and Instruction, 52, 46–58. 10.1016/j.learninstruc.2017.04.005 DOI: 10.1016/j.learninstruc.2017.04.005
30. Koriat, A. (1997). Monitoring one’s own knowledge during study: A cue-utilization approach to judgments of learning. Journal of Experimental Psychology: General, 126(4), 349–370. 10.1037/0096-3445.126.4.349 DOI: 10.1037/0096-3445.126.4.349
31. Kostons, D., van Gog, T., & Paas, F. (2009). How do I do? Investigating effectsof expertise and performance-process records on self-assessment. Applied Cognitive Psychology, 23, 1256–1265. 10.1002/acp.1528 DOI: 10.1002/acp.1528
32. Kostons, D., Van Gog, T., & Paas, F. (2012). Training self-assessment and task-selection skills: A cognitive approach to improving self-regulated learning. Learning and Instruction, 22(2), 121–132. 10.1016/j.learninstruc.2011.08.004 DOI: 10.1016/j.learninstruc.2011.08.004
33. Lachner, A., Backfisch, I., Hoogerheide, V., Van Gog, T., & Renkl, A. (2020). Timing matters! Explaining between study phases enhances students’ learning. Journal of Educational Psychology, 112(4), 841–853. 10.1037/edu0000396 DOI: 10.1037/edu0000396
34. Lerdpornkulrat, T., Poondej, C., Koul, R., Khiawrod, G., & Prasertsirikul, P. (2019). The positive effect of intrinsic feedback on motivational engagement and self-efficacy in information literacy. Journal of Psychoeducational Assessment, 37(4), 421–434. 10.1177/0734282917747423 DOI: 10.1177/0734282917747423
35. Lipko, A. R., Dunlosky, J., Hartwig, M., Rawson, K. A., Swan, K., & Cook, D. (2009). Using standards to improve middle school students’ accuracy at evaluating the quality of their recall. Journal of Experimental Psychology: Applied, 15(4), 307–318. 10.1037/a0017599 DOI: 10.1037/a0017599
36. Lipnevich, A. A., McCallen, L. N., Miles, K. P., & Smith, J. K. (2014). Mind the gap! Students’ use of exemplars and detailed rubrics as formative assessment. Instructional Science, 42(4), 539–559. 10.1007/s11251-013-9299-9 DOI: 10.1007/s11251-013-9299-9
37. Moning, J., & Roelle, J. (2021). Self-regulated learning by writing learning protocols: Do goal structures matter? Learning and Instruction, 75, 101486. 10.1016/j.learninstruc.2021.101486 DOI: 10.1016/j.learninstruc.2021.101486
38. Murillo-Zamorano, L., & Montanero, M. (2018). Oral presentations in higher education: A comparison of the impact of peer and teacher feedback. Assessment & Evaluation in Higher Education, 43(1), 138–150. 10.1080/02602938.2017.1303032 DOI: 10.1080/02602938.2017.1303032
39. Nederhand, N., Tabbers, H., Abrahimi, H., & Rikers, R. (2018). Improving calibration over texts by providing standards both with and without idea-units. Journal of Cognitive Psychology, 30(7), 689–700. 10.1080/20445911.2018.1513005 DOI: 10.1080/20445911.2018.1513005
40. Nelson, T. O. (1984). A comparison of current measures of the accuracy of feeling-of-knowing predictions. Psychological Bulletin, 95(1), 109–133. 10.1037/0033-2909.95.1.109 DOI: 10.1037/0033-2909.95.1.109
41. Nelson, T. O., & Narens, L. (1994). Why investigate metacognition. Metacognition: Knowing about Knowing, 13, 1–25.
42. Nordrum, L., Evans, K., & Gustafsson, M. (2013). Comparing student learning experiences of in-text commentary and rubric-articulated feedback: Strategies for formative assessment. Assessment & Evaluation in Higher Education, 38(8), 919–940. 10.1080/02602938.2012.758229 DOI: 10.1080/02602938.2012.758229
43. Nückles, M., Roelle, J., Glogger-Frey, I., Waldeyer, J., & Renkl, A. (2020). The self-regulation-view in writing-to-learn: Using journal writing to optimize cognitive load in self-regulated learning. Educational Psychology Review, 32, 1089–1126. 10.1007/s10648-020-09541-1 DOI: 10.1007/s10648-020-09541-1
44. Paas, F. G. (1992). Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of Educational Psychology, 84(4), 429–434. 10.1037/0022-0663.84.4.429 DOI: 10.1037/0022-0663.84.4.429
45. Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38(1), 63–71. 10.1207/S15326985EP3801_8 DOI: 10.1207/S15326985EP3801_8
46. Panadero, E., Tapia, J. A., & Huertas, J. A. (2012). Rubrics and self-assessment scripts effects on self-regulation, learning and self-efficacy in secondary education. Learning and Individual Differences, 22(6), 806–813. 10.1016/j.lindif.2012.04.007 DOI: 10.1016/j.lindif.2012.04.007
47. Panadero, E., & Jonsson, A. (2013). The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review, 9, 129–144. 10.1016/j.edurev.2013.01.002 DOI: 10.1016/j.edurev.2013.01.002
48. Panadero, E., & Romero, M. (2014). To rubric or not to rubric? The effects of self-assessment on self-regulation, performance and self-efficacy. Assessment in Education: Principles, Policy & Practice, 21(2), 133–148. 10.1016/j.edurev.2013.01.002 DOI: 10.1016/j.edurev.2013.01.002
49. Panadero, E., Jonsson, A., & Botella, J. (2017). Effects of self-assessment on self-regulated learning and self-efficacy: Four meta-analyses. Educational Research Review, 22, 74–98. 10.1016/j.edurev.2017.08.004 DOI: 10.1016/j.edurev.2017.08.004
50. Panadero, E., Brown, G. T., & Strijbos, J. W. (2016). The future of student self-assessment: A review of known unknowns and potential directions. Educational Psychology Review, 28(4), 803–830. DOI: 10.1007/s10648-015-9350-2
51. Panadero, E., & Jonsson, A. (2020). A critical review of the arguments against the use of rubrics. Educational Research Review, 30, 100329. 10.1016/j.edurev.2020.100329 DOI: 10.1016/j.edurev.2020.100329
52. Prinz, A., Golke, S., & Wittwer, J. (2018). The double curse of misconceptions: Misconceptions impair not only text comprehension but also metacomprehension in the domain of statistics. Instructional Science, 46, 723–765. 10.1007/s11251-018-9452-6 DOI: 10.1007/s11251-018-9452-6
53. Prinz, A., Golke, S., & Wittwer, J. (2020a). How accurately can learners discriminate their comprehension of texts? A comprehensive meta-analysis on relative metacomprehension accuracy and influencing factors. Educational Research Review, 31, 100358. 10.1016/j.edurev.2020.100358 DOI: 10.1016/j.edurev.2020.100358
54. Prinz, A., Golke, S., & Wittwer, J. (2020b). To what extent do situation-model-approach interventions improve relative metacomprehension accuracy? Meta-analytic insights. Educational Psychology Review, 32, 917–949. 10.1007/s10648-020-09558-6 DOI: 10.1007/s10648-020-09558-6
55. Quilici, J. L., & Mayer, R. E. (1996). Role of examples in how students learn to categorize statistics word problems. Journal of Educational Psychology, 88(1), 144–161. 10.1037/0022-0663.88.1.144 DOI: 10.1037/0022-0663.88.1.144
56. Rawson, K. A., & Dunlosky, J. (2007). Improving students’ self-evaluation of learning for key concepts in textbook materials. European Journal of Cognitive Psychology, 19(4/5), 559–579. 10.1080/09541440701326022 DOI: 10.1080/09541440701326022
57. Reddy, Y. M., & Andrade, H. (2010). A review of rubric use in higher education. Assessment & Evaluation in Higher Education, 35(4), 435–448. 10.1080/02602930902862859 DOI: 10.1080/02602930902862859
58. Renkl, A. (2014). Toward an instructionally oriented theory of example-based learning. Cognitive Science, 38(1), 1–37. 10.1111/cogs.12086 DOI: 10.1111/cogs.12086
59. Roelle, J., & Berthold, K. (2013). The expertise reversal effect in prompting focused processing of instructional explanations. Instructional Science, 41(4), 635–656. 10.1111/cogs.12086 DOI: 10.1111/cogs.12086
60. Roelle, J., Krüger, S., Jansen, C., & Berthold, K. (2012). The use of solved example problems for fostering strategies of self-regulated learning in journal writing. Education Research International. 10.1155/2012/751625 DOI: 10.1155/2012/751625
61. Roelle, J., Schmidt, E. M., Buchau, A., & Berthold, K. (2017). Effects of informing learners about the dangers of making overconfident judgments of learning. Journal of Educational Psychology, 109, 99–117. 10.1037/edu0000132 DOI: 10.1037/edu0000132
62. Rost, D. H., Wirthwein, L., Frey, K., & Becker, E. (2010). Steigert Kaugummikauen das kognitive Leistungsvermögen? [Does chewing gum enhance cognitive abilities? Two rather extraordinary experiments]. Zeitschrift Für Pädagogische Psychologie, 24(1), 39–49. 10.1024/1010-0652.a000003 DOI: 10.1024/1010-0652.a000003
63. Sadler, P. M., & Good, E. (2006). The impact of self-and peer-grading on student learning. Educational Assessment, 11(1), 1–31. 10.1207/s15326977ea1101_1 DOI: 10.1207/s15326977ea1101_1
64. Schalk, L., Roelle, J., Saalbach, H., Berthold, K., Stern, E., & Renkl, A. (2020). Providing worked examples for learning multiple principles. Applied Cognitive Psychology, 34(4), 813–824. 10.1002/acp.3653 DOI: 10.1002/acp.3653
65. Schamber, J. F., & Mahoney, S. L. (2006). Assessing and improving the quality of group critical thinking exhibited in the final projects of collaborative learning groups. The Journal of General Education, 55(2), 103–137. 10.1353/jge.2006.0025 DOI: 10.1353/jge.2006.0025
66. Scheiter, K., Ackerman, R., & Hoogerheide, V. (2020). Looking at mental effort appraisals through a metacognitive lens: Are they biased? Educational Psychology Review, 32, 1003–1027. 10.1007/s10648-020-09555-9 DOI: 10.1007/s10648-020-09555-9
67. Schmeck, A., Opfermann, M., Van Gog, T., Paas, F., & Leutner, D. (2015). Measuring cognitive load with subjective rating scales during problem solving: Differences between immediate and delayed ratings. Instructional Science, 43(1), 93–114. 10.1007/s11251-014-9328-3 DOI: 10.1007/s11251-014-9328-3
68. Schnotz, W., Fries, S., & Horz, H. (2009). Motivational aspects of cognitive load theory. In M. Wosnitza, S. A. Karabenick, A. Efklides, & P. Nenniger (Eds.), Contemporary motivation research: From global to local perspectives (pp. 69–96). Hogrefe & Huber Publishers.
69. Schraw, G. (2009). A conceptual analysis of five measures of metacognitive monitoring. Metacognition and learning, 4(1), 33–45. DOI: 10.1007/s11409-008-9031-3
70. Schworm, S., & Renkl, A. (2007). Learning argumentation skills through the use of prompts for self-explaining examples. Journal of Educational Psychology, 99(2), 285–296. 10.1037/0022-0663.99.2.285 DOI: 10.1037/0022-0663.99.2.285
71. Smit, R., Bachmann, P., Blum, V., Birri, T., & Hess, K. (2017). Effects of a rubric for mathematical reasoning on teaching and learning in primary school. Instructional Science, 45(5), 603–622. 10.1007/s11251-017-9416-2 DOI: 10.1007/s11251-017-9416-2
72. Swales, J. (1990). Genre analysis: English in academic and research settings. Cambridge University Press.
73. Swales, J. M. & Feak, C. B. (2009). Abstracts and the writing of abstracts (Vol. 2). University of Michigan Press ELT.
74. Sweller, J., van Merriënboer, J. J., & Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review, 31(2), 261–292. 10.1007/s10648-019-09465-5 DOI: 10.1007/s10648-019-09465-5
75. Thiede, K. W., Anderson, M., & Therriault, D. (2003). Accuracy of metacognitive monitoring affects learning of texts. Journal of Educational Psychology, 95(1), 66–73. 10.1037/0022-0663.95.1.66 DOI: 10.1037/0022-0663.95.1.66
76. Van Gog, T., & Paas, F. (2008). Instructional efficiency: Revisiting the original construct in educational research. Educational Psychologist, 43(1), 16–26. 10.1080/00461520701756248 DOI: 10.1080/00461520701756248
77. Waldeyer, J., & Roelle, J. (2021). The keyword effect: A conceptual replication, effects on bias, and an optimization. Metacognition and Learning, 16(1), 37–56. 10.1007/s11409-020-09235-7 DOI: 10.1007/s11409-020-09235-7
78. Zamary, A., Rawson, K. A., & Dunlosky, J. (2016). How accurately can students evaluate the quality of self-generated examples of declarative concepts? Not well, and feedback does not help. Learning and Instruction, 46, 12–20. 10.1016/j.learninstruc.2016.08.002 DOI: 10.1016/j.learninstruc.2016.08.002