The Influence of Delaying Judgments of Learning on Metacognitive Accuracy: A Meta-Analytic Review

Psychological Bulletin

Volumn 137, Issue 1, 2011, Pages 131-148

  Rhodes, Matthew G ^a   Tauber, Sarah K ^b  

^a Department of Environmental and Radiological Health Sciences   (United States)

^b KENT STATE UNIVERSITY   (United States)

Author keywords

Delayed judgments of learning; Judgments of learning; Memory; Metacognition; Metamemory

Indexed keywords

ARTICLE; ASSOCIATION; COGNITION; DECISION MAKING; HUMAN; LEARNING; LONG TERM MEMORY; META ANALYSIS; PSYCHOLOGICAL MODEL; SHORT TERM MEMORY; TIME;

COGNITION; CUES; HUMANS; JUDGMENT; LEARNING; MEMORY, LONG-TERM; MEMORY, SHORT-TERM; MODELS, PSYCHOLOGICAL; TIME FACTORS;

EID: 78751498925     PISSN: 00332909     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0021705     Document Type: Article

References (88)

1. Anderson M., Thiede K. Why do delayed summaries improve metacomprehension accuracy?. Acta Psychologica 2008, 128:110-118. doi: 10.1016/j.actpsy.2007.10.006.
2. Arbuckle T.Y., Cuddy L.L. Discrimination of item strength at time of presentation. Journal of Experimental Psychology 1969, 81:126-131. doi: 10.1037/h0027455.
3. Begg I., Duft S., Lalonde P., Melnick R., Sanvito J. Memory predictions are based on ease of processing. Journal of Memory and Language 1989, 28:610-632. doi: 10.1016/0749-596X(89)90016-8.
4. Benjamin A.S., Diaz M. Measure of relative metamnemonic accuracy. Handbook of memory and metamemory 2008, 73-94. Psychology Press, New York, NY. J. Dunlosky, R.A. Bjork (Eds.).
5. Borenstein M. Effect sizes for continuous data. The handbook of research synthesis 2009, 221-235. Russell Sage Foundation, New York, NY. H. Cooper, L. Hedges, J.C. Valentine (Eds.).
6. Carroll M., Nelson T.O., Kirwan A. Tradeoff of semantic relatedness and degree of overlearning: Differential effects on metamemory and on long-term retention. Acta Psychologica 1997, 95:239-253. doi: 10.1016/S0001-6918(96)00040-6.
7. Cepeda N.J., Pashler H., Vul E., Wixted J.T., Rohrer D. Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin 2006, 132:354-380. doi: 10.1037/0033-2909.132.3.354.
8. Connor L.T., Dunlosky J., Hertzog C. Age-related differences in absolute but not relative metamemory accuracy. Psychology and Aging 1997, 12:50-71. doi: 10.1037/0882-7974.12.1.50.
9. Cowan N. An embedded-processes model of working memory. Models of working memory: Mechanisms of active maintenance and executive control 1999, 62-101. Cambridge University Press, Cambridge, United Kingdom. A. Miyake, P. Shah (Eds.).
10. Dunlap W.P., Cortina J.M., Vaslow J.B., Burke M.J. Meta-analysis of experiments with matched groups or repeated measures designs. Psychological Methods 1996, 1:170-177. doi: 10.1037/1082-989X.1.2.170.
11. Dunlosky J., Domoto P.K., Wang M.-L., Ishikawa T., Roberson I., Nelson T.O., Ramsay D.S. Inhalation of 30% nitrous oxide impairs people's learning without impairing people's judgments of what will be remembered. Experimental and Clinical Psychopharmacology 1998, 6:77-86. doi: 10.1037/1064-1297.6.1.77.
12. Dunlosky J., Nelson T.O. Importance of the kind of cue for judgments of learning (JOL) and the delayed-JOL effect. Memory & Cognition 1992, 20:374-380.
13. Dunlosky J., Nelson T.O. Does the sensitivity of judgments of learning (JOLs) to the effects of various study activities depend on when the JOLs occur?. Journal of Memory and Language 1994, 33:545-565. doi: 10.1006/jmla.1994.1026.
14. Dunlosky J., Nelson T.O. Similarity between the cue for judgments of learning (JOL) and the cue for test is not the primary determinant of JOL accuracy. Journal of Memory and Language 1997, 36:34-49. doi: 10.1006/jmla.1996.2476.
15. Dunlosky J., Rawson K.A., McDonald S.L. Influence of practice tests on the accuracy of predicting memory performance for paired associates, sentences, and text material. Applied metacognition 2002, 68-92. Cambridge University Press, Cambridge, United Kingdom. T.J. Perfect, B.L. Schwartz (Eds.).
16. Dunlosky J., Serra M., Matvey G., Rawson K. Second-order judgments about judgments of learning. Journal of General Psychology 2005, 132:335-346. doi: 10.3200/GENP.132.4.335-346.
17. Eakin, D. K., & Hertzog, C. (2010). Immediate judgments of learning are insensitive to implicit interference effects at retrieval. Manuscript submitted for publication.
18. Finn B. Framing effects on metacognitive monitoring and control. Memory & Cognition 2008, 36:813-821. doi: 10.3758/MC.36.4.813.
19. Finn B., Metcalfe J. The role of memory for past test in the underconfidence with practice effect. Journal of Experimental Psychology Learning, Memory, and Cognition 2007, 33:238-244. doi: 10.1037/0278-7393.33.1.238.
20. Gonzalez R., Nelson T.O. Measuring ordinal association in measures that contain tied scores. Psychological Bulletin 1996, 119:159-165. doi: 10.1037/0033-2909.119.1.159.
21. Hedges L.V. Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics 1981, 6:107-128. doi: 10.2307/1164588.
22. Hedges L.V. Fixed effects models. The handbook of research synthesis 1994, 286-299. Russell Sage Foundation, New York, NY. H. Cooper, L. Hedges (Eds.).
23. Hedges L.V., Olkin I. Statistical methods for meta-analysis 1985, Academic Press, Orlando, FL.
24. Jang Y. The effect of practice with test on the relative accuracy of judgments of learning (Doctoral dissertation) 2007, Retrieved from ProQuest Dissertations and Theses database. (No. 1196415301).
25. Jang Y., Nelson T.O. How many dimensions underlie judgments of learning and recall? Evidence from state-trace methodology. Journal of Experimental Psychology: General 2005, 134:308-326. doi: 10.1037/0096-3445.134.3.308.
26. Kelemen W.L. Metamemory for related information is determined by the type of mnemonic cue (Doctoral dissertation) 1999, Retrieved from ProQuest Dissertations and Theses database. (No. 732910361).
27. Kelemen W.L. Metamemory cues and monitoring accuracy: Judging what you know and what you will know. Journal of Educational Psychology 2000, 92:800-810. doi: 10.1037/0022-0663.92.4.800.
28. Kelemen W.L., Frost P.J., Weaver C. Individual differences in metacognition: Evidence against a general metacognitive ability. Memory & Cognition 2000, 28:92-107.
29. Kelemen W.L., Weaver C. Enhanced memory at delays: Why do judgments of learning improve over time?. Journal of Experimental Psychology: Learning, Memory, and Cognition 1997, 23:1394-1409. doi: 10.1037/0278-7393.23.6.1394.
30. Kennedy M.R.T. Self-monitoring recall during two tasks after traumatic brain injury: A preliminary study. American Journal of Speech-Language Pathology 2004, 13:142-154. doi: 10.1044/1058-0360(2004/015).
31. Kennedy M.R.T., Carney E., Peters S.M. Predictions of recall and study strategy decisions after diffuse brain injury. Brain Injury 2003, 17:1043-1064. doi: 10.1080/0269905031000110436.
32. Kennedy M.R.T., Yorkston K.M. The effects of frontal injury on " on-line" self-monitoring during verbal learning by adults with diffuse brain injury. Neuropsychological Rehabilitation 2004, 14:449-465. doi: 10.1080/09602010443000038.
33. Kimball D.R., Metcalfe J. Delaying judgments of learning affects memory, not metamemory. Memory & Cognition 2003, 31:918-929.
34. Knouse L.E., Paradise M.J., Dunlosky J. Does ADHD in adults affect the relative accuracy of metamemory judgments?. Journal of Attention Disorders 2006, 10:160-170. doi: 10.1177/1087054706288116.
35. Koriat A. Monitoring one's own knowledge during study: A cue-utilization approach to judgments of learning. Journal of Experimental Psychology: General 1997, 126:349-370. doi: 10.1037/0096-3445.126.4.349.
36. Koriat A. Metacognition and consciousness. The Cambridge handbook of consciousness 2007, 289-325. Cambridge University Press, New York, NY. P.D. Zelazo, M. Moscovitch, E. Thompson (Eds.).
37. Koriat A., Bjork R.A. Illusions of competence during study can be remedied by manipulations that enhance learners' sensitivity to retrieval conditions at test. Memory & Cognition 2006, 34:959-972.
38. Koriat A., Ma'ayan H. The effects of encoding fluency and retrieval fluency on judgments of learning. Journal of Memory and Language 2005, 52:478-492. doi: 10.1016/j.jml.2005.01.001.
39. Koriat A., Ma'ayan H., Nussinson R. The intricate relationships between monitoring and control in metacognition: Lessons for the cause-and-effect relation between subjective experience and behavior. Journal of Experimental Psychology General 2006, 135:36-49. doi: 10.1037/0096-3445.135.1.36.
40. Koriat A., Ma'ayan H., Sheffer L., Bjork R.A. Exploring a mnemonic debiasing account of the underconfidence-with-practice effect. Journal of Experimental Psychology: Learning, Memory, and Cognition 2006, 32:595-608. doi: 10.1037/0278-7393.32.3.595.
41. Koriat A., Shitzer-Reichert R. Metacognitive judgments and their accuracy: Insights from the processes underlying judgments of learning in children. Metacognition: Process, function, and use 2002, 1-17. Kluwer, New York, NY. P. Chambres, M. Izaute, P.-J. Marescaux (Eds.).
42. Masson M.E.J., Rotello C.M. Sources of bias in the Goodman-Kruskal gamma coefficient measure of association: Implications for studies of metacognitive processes. Journal of Experimental Psychology: Learning, Memory, and Cognition 2009, 35:509-527. doi: 10.1037/a0014876.
43. Matvey G., Dunlosky J., Schwartz B. The effects of categorical relatedness on judgments of learning (JOLs). Memory 2006, 14:253-261. doi: 10.1080/09658210500216844.
44. Meeter M., Nelson T.O. Multiple study trials and judgments of learning. Acta Psychologica 2003, 113:123-132. doi: 10.1016/S0001-6918(03)00023-4.
45. Metcalfe J. Metamemory: Theory and data. The Oxford handbook of memory 2000, 197-211. Oxford University Press, New York, NY. E. Tulving, F.I.M. Craik (Eds.).
46. Metcalfe J., Finn B. Evidence that judgments of learning are causally related to study choice. Psychonomic Bulletin and Review 2008, 15:174-179. doi: 10.3758/PBR.15.1.174.
47. Metcalfe J., Finn B. Familiarity and retrieval processes in delayed judgments of learning. Journal of Experimental Psychology 2008, 34:1084-1097. doi: 10.1037/a0012580.
48. Metcalfe J., Kornell N. A region of proximal learning model of study time allocation. Journal of Memory and Language 2005, 52:463-477. doi: 10.1016/j.jml.2004.12.001.
49. Mintzer M.Z., Griffiths R.R. Drugs, memory, and metamemory: A dose-effect study with lorazepam and scopolamine. Experimental and Clinical Psychopharmacology 2005, 4:336-347. doi: 10.1037/1064-1297.13.4.336.
50. Narens L., Nelson T.O., Scheck P. Memory monitoring and the delayed JOL effect. Handbook of memory and metamemory 2008, 137-153. Psychology Press, New York, NY. J. Dunlosky, R.A. Bjork (Eds.).
51. Nelson T.O. A comparison of current measures of the accuracy of feeling-of-knowing predictions. Psychological Bulletin 1984, 95:109-133. doi: 10.1037/0033-2909.95.1.109.
52. Nelson T.O. Consciousness and metacognition. American Psychologist 1996, 51:102-116. 10.1037/0003-066X.51.2.102.
53. Nelson T.O., Dunlosky J. When people's judgments of learning (JOLs) are extremely accurate at predicting subsequent recall: The " delayed-JOL effect. " Psychological Science 1991, 2:267-270. doi: 10.1111/j.1467-9280.1991.tb00147.x.
54. Nelson T.O., Leonesio R.J. Allocation of self-paced study time and the " labor-in vain" effect. Journal of Experimental Psychology: Learning, Memory, and Cognition 1988, 14:676-686. doi: 10.1037/0278-7393.14.4.676.
55. Nelson T.O., Narens L. Metamemory: A theoretical framework and new findings. The psychology of learning and motivation 1990, 26:125-173. Academic Press, New York, NY. G.H. Bower (Ed.).
56. Nelson T.O., Narens L. Why investigate metacognition?. Metacognition: Knowing about knowing 1994, 1-25. MIT Press, Cambridge, MA. J. Metcalfe, A.P. Shimamura (Eds.).
57. Nelson T.O., Narens L., Dunlosky J. A revised methodology for research on metamemory: Pre-judgment recall and monitoring (PRAM). Psychological Methods 2004, 9:53-69. doi: 10.1037/1082-989X.9.1.53.
58. Oberauer K. Access to information in working memory: Exploring the focus of attention. Journal of Experimental Psychology: Learning, Memory, and Cognition 2002, 28:411-421. doi: 10.1037/0278-7393.28.3.411.
59. Oberauer K. Control of the contents of working memory-A comparison of two paradigms and two age groups. Journal of Experimental Psychology: Learning, Memory, and Cognition 2005, 31:714-728. doi: 10.1037/0278-7393.31.4.714.
60. Rast P., Zimprich D. Age differences in the underconfidence-with-practice effect. Experimental Aging Research 2009, 35:400-431.
61. Rhodes M.G., Castel A.D. Memory predictions are influenced by perceptual information: Evidence for metacognitive illusions. Journal of Experimental Psychology: General 2008, 137:615-625. doi: 10.1037/a0013684.
62. Rhodes M.G., Castel A.D. Metacognitive illusions for auditory information: Effects on monitoring and control. Psychonomic Bulletin and Review 2009, 16:550-554. doi: 10.3758/PBR.16.3.550.
63. Rhodes, M. G., & Tauber, S. K. (2010). Monitoring memory errors: The influence of the veracity of retrieved information in the accuracy of judgments of learning (JOLs). Manuscript submitted for publication.
64. Roebers C.M., von der Linden N., Schneider W., Howie P. Children's metamemorial judgments in an event recall task. Journal of Experimental Child Psychology 2007, 97:117-137. doi: 10.1016/j.jecp.2006.12.006.
65. Rosenthal R. The " file-drawer problem" and tolerance for null results. Psychological Bulletin 1979, 86:638-641. doi: 10.1037/0033-2909.86.3.638.
66. Scheck P.A. How do judgments of intention (JOIs) differ from judgments of learning (JOLs)? (Doctoral dissertation) 2006, Retrieved from ProQuest Dissertations and Theses database. (No. 1044404181).
67. Scheck P., Meeter M., Nelson T.O. Anchoring effects in the absolute accuracy of immediate versus delayed judgments of learning. Journal of Memory and Language 2004, 51:71-79. doi: 10.1016/j.jml.2004.03.004.
68. Schneider W., Visé M., Lockl K., Nelson T.O. Developmental trends in children's memory monitoring: Evidence from a judgment-of-learning task. Cognitive Development 2000, 15:115-134.
69. Serra, M. J., & England, B. D. (2009). [A between-subjects replication of the delayed-JOL effect]. Unpublished data set.
70. Serra M.J., Dunlosky J. Does retrieval fluency contribute to the underconfidence-with-practice effect?. Journal of Experimental Psychology: Learning, Memory, and Cognition 2005, 31:1258-1266. doi: 10.1037/0278-7393.31.6.1258.
71. Serra M.J., Dunlosky J., Hertzog C. Do older adults show less confidence in their monitoring of learning?. Experimental Aging Research 2008, 34:379-391. doi: 10.1080/03610730802271898.
72. Shadish W.R., Haddock C.K. Combining estimates of effect size. The handbook of research synthesis 1994, 261-281. Russell Sage Foundation, New York, NY. H. Cooper, L. Hedges (Eds.).
73. Sikström S., Jönsson F. A model for stochastic drift in memory strength to account for judgments of learning. Psychological Review 2005, 112:932-950. doi: 10.1037/0033-295X.112.4.932.
74. Soderstrom, N. C., & McCabe, D. P. (2010). [Failure to find the delayed-JOL effect when recognition is used at test]. Unpublished data set.
75. Son L., Metcalfe J. Judgments of learning: Evidence for a two-stage process. Memory & Cognition 2005, 33:1116-1129.
76. Spellman B.A., Bjork R.A. When predictions create reality: Judgments of learning may alter what they are intended to assess. Psychological Science 1992, 5:315-316. doi: 10.1111/j.1467-9280.1992.tb00680.x.
77. Spellman B.A., Bloomfield A., Bjork R.A. Measuring memory and metamemory: Theoretical and statistical problems with assessing learning (in general) and using gamma (in particular) to do so. Handbook of metamemory and memory 2008, 95-114. Psychology Press, New York, NY. J. Dunlosky, R.A. Bjork (Eds.).
78. Sutton A.J. Publication bias. The handbook of research synthesis 2009, 435-452. Russell Sage Foundation, New York, NY. H. Cooper, L. Hedges, J.C. Valentine (Eds.).
79. Tauber S.K., Rhodes M.G. Are judgments of learning (JOLs) sensitive to the amount of material to-be-remembered?. Memory 2010, 18:351-362. doi: 10.1080/09658211003662755.
80. Thiede K.W., Anderson M.C.M., Therriault D. Accuracy of metacognitive monitoring affects learning of texts. Journal of Educational Psychology 2003, 95:66-73. doi: 10.1037/0022-0663.95.1.66.
81. Thiede K.W., Dunlosky J. Delaying students' metacognitive monitoring improves their accuracy in predicting their recognition performance. Journal of Educational Psychology 1994, 86:290-302. doi: 10.1037/0022-0663.86.2.290.
82. Thiede K.W., Dunlosky J., Griffin T.D., Wiley J. Understanding the delayed-keyword effect on metacomprehension accuracy. Journal of Experimental Psychology: Learning, Memory, and Cognition 2005, 31:1267-1280. doi: 10.1037/0278-7393.31.6.1267.
83. Thomas A.K., McDaniel M.A. Metacomprehension for educationally relevant materials: Dramatic effects of encoding-retrieval interactions. Psychonomic Bulletin and Review 2007, 14:212-218.
84. Thomson D.M., Tulving E. Associative encoding and retrieval: Weak and strong cues. Journal of Experimental Psychology 1970, 86:255-262. doi: 10.1037/h0029997.
85. van Overschelde J., Nelson T.O. Delayed judgments of learning cause both a decrease in absolute accuracy (calibration) and an increase in relative accuracy (resolution). Memory & Cognition 2006, 34:1527-1538.
86. Wahlheim, C. N. (2010). Predicting memory performance under conditions of proactive interference: Immediate and delayed judgments of learning. Manuscript submitted for publication.
87. Weaver C., Kelemen W. Judgments of learning at delays: Shifts in response patterns or increased metamemory accuracy?. Psychological Science 1997, 8:318-321. doi: 10.1111/j.1467-9280.1997.tb00445.x.
88. Weaver C., Kelemen W. Processing similarity does not improve metamemory: Evidence against transfer-appropriate monitoring. Journal of Experimental Psychology: Learning, Memory, and Cognition 2003, 29:1058-1065. doi: 10.1037/0278-7393.29.6.1058.