Exploring a three-level model of calibration accuracy

Journal of Educational Psychology

Volumn 106, Issue 4, 2014, Pages 1192-1202

  Schraw, Gregory ^a   Kuch, Fred ^a   Gutierrez, Antonio P ^a   Richmond, Aaron S ^b  

^a UNIVERSITY OF NEVADA   (United States)

^b METROPOLITAN STATE UNIVERSITY OF DENVER   (United States)

Author keywords

Calibration accuracy; Measurement; Metacognition

Indexed keywords

EID: 84925424428     PISSN: 00220663     EISSN: 19392176     Source Type: Journal    
DOI: 10.1037/a0036653     Document Type: Article

References (48)

1. Agresti, A., & Franklin, C. (2007). Statistics: The art of learning from data. Upper Saddle River, NJ: Pearson Education.
2. Allwood, C. M., Jonsson, A. C., & Granhag, P. A. (2005). The effects of source and type of feedback on child witnesses' metamemory accuracy. Applied Cognitive Psychology, 19, 331-344. doi:10.1002/acp.1071
3. Benjamin, A. S., & Diaz, M. (2008). Measurement of relative metamnemonic accuracy. In J. Dunlosky & R. A. Bjork (Eds.), Handbook of memory and metamemory (pp. 73-94). New York, NY: Psychology Press.
4. Bentler, P. M. (2005). EQS 6 structural equations program manual. Encino, CA: Multivariate Software. Retrieved from http://www.econ .upf.edu/~satorra/CourseSEMVienna2010/EQSManual.pdf
5. Boekaerts, M., & Rozendaal, J. S. (2010). Using multiple calibration indices in order to capture the complex picture of what affects students' accuracy of feeling of confidence. Learning and Instruction, 20, 372- 382. doi:10.1016/j.learninstruc.2009.03.002
6. Brody, N. (1992). Intelligence (2nd ed.). San Diego, CA: Academic Press.
7. Byrne, B. M. (2006). Structural equation modeling with EQS (2nd ed.). Mahwah, NJ: Erlbaum.
8. Carroll, J. B. (1993). Human cognitive abilities: A survey of factor-analytic studies. Cambridge, England: Cambridge University Press. doi:10.1017/ CBO9780511571312
9. Cattell, R. B. (1987). Intelligence: Its structure, growth, and action. Amsterdam, the Netherlands: North-Holland.
10. Cheetham, A. H., & Hazel, J. E. (1969). Binary (presence-absence) similarity coefficients. Journal of Paleontology, 43, 1130-1136.
11. Connor, L. T., Dunlosky, J., & Hertzog, C. (1997). Age-related differences in absolute but not relative metamemory accuracy. Psychology and Aging, 12, 50-71. doi:10.1037/0882-7974.12.1.50
12. Ekstrom, R. B., French, J. W., & Harman, H. H. (1976). Manual for kit of factor-referenced cognitive tests. Princeton, NJ: Educational Testing Service.
13. Feuerman, M., & Miller, A. R. (2008). Relationships between statistical measures of agreement: Sensitivity, specificity and kappa. Journal of Evaluation in Clinical Practice, 14, 930-933. doi:10.1111/j.1365-2753 .2008.00984.x
14. Goodman, L. A., & Kruskal, W. H. (1954). Measures of association for cross classification. Journal of the American Statistical Association, 49, 732-764.
15. Hart, J. T. (1965). Memory and the feeling-of-knowing experience. Journal of Educational Psychology, 56, 208-216. doi:10.1037/h0022263
16. Hart, J. T. (1967). Second-try recall, recognition, and the memorymonitoring process. Journal of Educational Psychology, 58, 193-197. doi:10.1037/h0024908
17. Horn, J. L., & Cattell, R. B. (1966). Refinement and test of the theory of fluid and crystallized general intelligence. Journal of Educational Psychology, 57, 253-270. doi:10.1037/h0023816
18. Kadlec, H. (1999). Statistical properties of d' and estimates of signal detection theory. Psychological Methods, 4, 22-43. doi:10.1037/1082-989X.4.1.22
19. Kennedy, M. R. T. (2001). Retrospective confidence judgements made by adults with traumatic brain injury: Relative and absolute accuracy. Brain Injury, 15, 469-487. doi:10.1080/02699050010007380
20. Keren, G. (1991). Calibration and probability judgments: Conceptual and methodological issues. Acta Psychologica, 77, 217-273. doi:10.1016/ 0001-6918(91)90036-Y
21. Kline, R. (2005). Principles and practice of structural equation modeling. New York, NY: Guilford Press.
22. Koriat, A. (2012). The self-consistency model of subjective confidence. Psychological Review, 119, 80-113. doi:10.1037/a0025648
23. Leonesio, R. J., & Nelson, T. O. (1990). Do different metamemory judgments tap the same underlying aspects of memory? Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 464-470. doi:10.1037/0278-7393.16.3.464
24. Macmillan, N. A., & Creelman, C. D. (2005). Detection theory: A user's guide. New York, NY: Erlbaum.
25. Maki, R. H., Shields, M., Wheeler, A. E., & Zacchilli, T. L. (2005). Individual differences in absolute and relative metacomprehension accuracy. Journal of Educational Psychology, 97, 723-731. doi:10.1037/ 0022-0663.97.4.723
26. Marshalek, B., Lohman, D. F., & Snow, R. E. (1983). The complexity continuum in the radex and hierarchical models of intelligence. Intelligence, 7, 107-127. doi:10.1016/0160-2896(83)90023-5
27. Masson, M. E. J., & Rotello, C. M. (2009). Sources of bias in the Goodman-Kruskal gamma coefficient measure of association: Implications for studies of metacognitive processes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35, 509-527. doi:10.1037/a0014876
28. Mathieu, M. T., & Gosling, S. D. (2012). The accuracy or inaccuracy of affective forecasts depends on how accuracy is indexed: A meta-analysis of past studies. Psychological Science, 23, 161-162. doi:10.1177/ 0956797611427044
29. Mayer, D. (2010). Essential evidence-based medicine (2nd ed.). Cambridge, England: Cambridge University Press.
30. Meisner, M. T., & Maki, R. H. (2007). The role of test anxiety in absolute and relative metacomprehension accuracy. European Journal of Cognitive Psychology, 19, 650-670. doi:10.1080/09541440701326196
31. Nelson, T. O. (1984). A comparison of current measures of accuracy of feeling-of-knowing predictions. Psychological Bulletin, 95, 109-133. doi:10.1037/0033-2909.95.1.109
32. Nelson, T. O., & Narens, L. (1990). Metamemory: A theoretical framework and new findings. In G. Bower (Ed.), The psychology of learning and motivation (Vol. 26, pp. 125-173). New York, NY: Academic Press. doi:10.1016/S0079-7421(08)60053-5
33. Nietfeld, J. L., Enders, C. K., & Schraw, G. (2006). A Monte Carlo comparison of measures of relative and absolute monitoring accuracy. Educational and Psychological Measurement, 66, 258 -271. doi: 10.1177/0013164404273945
34. Pagano, M., & Gauvreau, K. (2000). Principles of biostatistics (2nd ed.). Pacific Grove, CA: Duxbury.
35. Romesburg, H. C. (1984). Cluster analysis for researchers. Belmont, CA: Lifetime Learning.
36. Schraw, G. (1995). Measures of feeling-of-knowing accuracy: A new look at an old problem. Applied Cognitive Psychology, 9, 321-332. doi: 10.1002/acp.2350090405
37. Schraw, G. (2009). Measuring metacognitive judgments. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Handbook of metacognition in education (pp. 415-429). Mahwah, NJ: Erlbaum.
38. Schraw, G., Kuch, F., & Gutierrez, A. P. (2013). Measure for measure: Calibrating ten commonly used calibration scores. Learning and Instruction, 24, 48-57. doi:10.1016/j.learninstruc.2012.08.007
39. Schwartz, A., & Bergus, G. (2008). Medical decision making: A physician's guide. New York, NY: Cambridge University Press. doi:10.1017/ CBO9780511722080
40. Serra, M. J., & Metcalfe, J. (2009). Effective implementation of metacognition. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Handbook of metacognition in education (pp. 278 -298). Mahwah, NJ: Erlbaum.
41. Simon, G. A. (1978). Efficacies of measures of association for ordinal contingency tables. Journal of the American Statistical Association, 73, 545-551. doi:10.1080/01621459.1978.10480052
42. Spellman, B. A., Bloomfield, A., & Bjork, R. A. (2008). Measuring memory and metamemory. In J. Dunlosky & R. A. Bjork (Eds.), Handbook of metamemory and memory (pp. 95-114). New York, NY: Psychology Press.
43. Tabachnick, B. G., & Fidell, L. S. (2011). Cleaning up your act: Screening data prior to analysis. Using multivariate statistics (6th ed., pp. 60- 116). New York, NY: Pearson.
44. Van Overschelde, J. P., & Nelson, T. O. (2006). Delayed judgments of learning cause both a decrease in absolute accuracy (calibration) and an increase in relative accuracy (resolution). Memory & Cognition, 34, 1527-1538. doi:10.3758/BF03195916
45. van Stralen, K. J., Stel, V. S., Reitsma, J. B., Dekker, F. W., Zoccali, C., & Jager, K. J. (2009). Diagnostic methods I: Sensitivity, specificity, and other measures of accuracy. Kidney International, 75, 1257-1263. doi: 10.1038/ki.2009.92
46. Warnick, E. M., Bracken, M. B., & Kasl, S. (2008). Screening efficiency of the Child Behavior Checklist and Strengths and Difficulties Questionnaire: A systematic review. Child and Adolescent Mental Health, 13, 140-147. doi:10.1111/j.1475-3588.2007.00461.x
47. Warrens, M. J. (2008). On similarity coefficients for 2 2 tables and corrections for chance. Psychometrika, 73, 487-502. doi:10.1007/ s11336-008-9059-y
48. Winne, P., & Hadwin, A. (2008). The weave of motivation and selfregulated learning. In D. Schunk & B. Zimmerman (Eds.), Motivation and self-regulated learning: Theory, research, and applications (pp. 297-314). Mahwah, NJ: Erlbaum.