Regulating accuracy on university tests with the plurality option

Learning and Instruction

Volumn 24, Issue 1, 2013, Pages 26-36

  Higham, Philip A ^a  

^a UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

Author keywords

Accuracy regulation; Base rate fallacy; Calibration; Metacognition; Underconfidence

Indexed keywords

EID: 84871028668     PISSN: 09594752     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.learninstruc.2012.08.001     Document Type: Article

References (49)

1. Ackerman R., Goldsmith M. Control over grain size in memory reporting - with and without satisficing knowledge. Journal of Experimental Psychology: Learning, Memory and Cognition 2008, 34:1224-1245. 10.1037/a0012938.
2. Allwood C.M., Granhag P.A., Johansson M. Increased realism in eyewitness confidence judgements: the effect of dyadic collaboration. Applied Cognitive Psychology 2003, 17:545-561. 10.1002/acp.888.
3. Bar-Hillel M. The base-rate fallacy in probability judgments. Acta Psychologica 1980, 44:211-233. 10.1016/0001-6918(80)90046-3.
4. Bliss L.B. A test of Lord's assumption regarding examinee guessing behavior on multiple-choice tests using elementary school students. Journal of Educational Measurement 1980, 17:147-153. 10.1111/j.1745-3984.1980.tb00823.x.
5. Coombs C.H. On the use of objective examinations. Educational and Psychological Measurement 1953, 13:308-310. 10.1177/001316445301300214.
6. Coombs C.H., Milholland J.E., Womer F.B. The assessment of partial knowledge. Educational and Psychological Measurement 1956, 16:13-37. 10.1177/001316445601600102.
7. Cross L.H., Frary R.B. An empirical test of Lord's theoretical results regarding formula scoring of multiple-choice tests. Journal of Educational Measurement 1977, 14:313-321. 10.1111/j.1745-3984.1977.tb00047.x.
8. Dressel P.L., Schmid J. Some modifications of multiple-choice items. Educational and Psychological Measurement 1953, 13:574-595. 10.1177/001316445301300404.
9. Dube C., Rotello C.M., Heit E. Assessing the belief bias effect with ROCs: it's a response bias effect. Psychological Review 2010, 117:831-863. 10.1037/a0019634.
10. Dunlosky J., Rawson K.A. Overconfidence produces underachievement: inaccurate self evaluations undermine students' learning and retention. Learning and Instruction 2012, 22:271-280. 10.1016/j.learninstruc.2011.08.003.
11. Ebel R.L. Blind guessing on objective achievement tests. Journal of Educational Measurement 1968, 5:321-325. 10.1111/j.1745-3984.1968.tb00646.x.
12. Erev I., Wallsten T.S., Budescu D.V. Simultaneous over- and underconfidence: the role of error in judgment processes. Psychological Review 1994, 101:519-527. 10.1037/0033-295X.101.3.519.
13. Evans J.R., Fisher R.P. Eyewitness memory: balancing the accuracy, precision and quantity of information through metacognitive monitoring and control. Applied Cognitive Psychology 2011, 25:501-508. 10.1002/acp.1722.
14. Frary R.B. Partial-credit scoring methods for multiple-choice tests. Applied Measurement in Education 1989, 2:79-96. 10.1207/s15324818ame0201_5.
15. Gigerenzer G., Hoffrage U., Kleinbölting H. Probabilistic mental models: a Brunswikian theory of confidence. Psychological Review 1991, 98:506-528. 10.1037//0033-295X.98.4.506.
16. Goldsmith G., Koriat A. The strategic regulation of memory accuracy and informativeness. Memory use as skilled cognition 2008, Vol. 48:307-324. Elsevier, San Diego, CA. A. Benjamin, B. Ross (Eds.).
17. Goldsmith M., Koriat A., Pansky A. Strategic regulation of grain size in memory reporting over time. Journal of Memory and Language 2005, 52:505-525. 10.1016/j.jml.2005.01.010.
18. Goldsmith M., Koriat A., Weinberg-Eliezer A. Strategic regulation of grain size memory reporting. Journal of Experimental Psychology: General 2002, 131:73-95. 10.1037/0096-3445.131.1.73.
19. Griffin T., Tversky A. The weighing of evidence and the determinants of confidence. Cognitive Psychology 1992, 24:411-435. 10.1016/0010-0285(92)90013-R.
20. Higham P.A. Strong cues are not necessarily weak: Thomson and Tulving (1970) and the encoding specificity principle revisited. Memory and Cognition 2002, 30:67-80. 10.3758/BF03195266.
21. Higham P.A. No Special K! A signal detection framework for the strategic regulation of memory accuracy. Journal of Experimental Psychology: General 2007, 136:1-22. 10.1037/0096-3445.136.1.1.
22. Higham P.A. Accuracy discrimination and type-2 signal detection theory: clarifications, extensions, and an analysis of bias. Constructions of remembering and metacognition: Essays in honour of Bruce Whittlesea 2011, 109-127. Palgrave MacMillan, Basingstoke. P.A. Higham, J.P. Leboe (Eds.).
23. Higham P.A., Arnold M.M. How many questions should I answer? Using bias profiles to estimate optimal bias and maximum score on formula-scored tests. European Journal of Cognitive Psychology 2007, 19:718-742. 10.1080/09541440701326121.
24. Higham P.A., Gerrard C. Not all errors are created equal: metacognition and changing answers on multiple-choice tests. Canadian Journal of Experimental Psychology 2005, 59:28-34. 10.1037/h0087457.
25. Juslin P., Olsson H., Björkman M. Brunswikian and Thurstonian origins of bias in probability assessment: on the interpretation of stochastic components of judgment. Journal of Behavioral Decision Making 1997, 10:189-209. 10.1002/(SICI)1099-0771(199709)10:3<189::AID-BDM258>3.0.CO;2-4.
26. Kahneman D., Tversky A. On the psychology of prediction. Psychological Review 1973, 80:237-251. 10.1037/h0034747.
27. Koriat A., Goldsmith M. Monitoring and control processes in the strategic regulation of memory. Psychological Review 1996, 103:490-517. 10.1037/0033-295X.103.3.490.
28. Koriat A., Lichtenstein S., Fischhoff B. Reasons for confidence. Journal of Experimental Psychology: Human Learning and Memory 1980, 6:107-118. 10.1037//0278-7393.6.2.107.
29. Liberman V. Local and global judgments of confidence. Journal of Experimental Psychology: Learning, Memory and Cognition 2004, 30:729-732. 10.1037/0278-7393.30.3.729.
30. Lichtenstein S., Fischhoff B., Phillips L.D. Calibration of probabilities: the state of the art to 1980. Judgement under uncertainty: heuristics and biases 1982, 306-334. Cambridge University Press, Cambridge. D. Kahneman, P. Slovic, A. Tversky (Eds.).
31. Luna K., Higham P.A., Martin-Luengo B. The regulation of memory accuracy with multiple answers: the plurality option. Journal of Experimental Psychology: Applied 2011, 17:148-158. 10.1037/a0023276.
32. Luna, K., & Martin-Luengo, B. Improving the accuracy of eyewitnesses in the presence of misinformation with the plurality option. Applied Cognitive Psychology, http://dx.doi.org/doi:10.1002/acp.2845.
33. Muijtjens A.M.M., van Mameren H., Hoogenboom R.J.I. The effect of a "don't know" option on test scores: number-right and formula scoring compared. Medical Education 1999, 33:267-275. 10.1046/j.1365-2923.1999.00292.x.
34. Perfect T.J. The role of self-rated ability in the accuracy of confidence judgments in eyewitness memory and general knowledge. Applied Cognitive Psychology 2004, 18:157-168. 10.1002/acp.952.
35. Perfect T.J., Hollins T.S. Predictive feeling of knowing judgements and postdictive confidence judgements in eyewitness memory and general knowledge. Applied Cognitive Psychology 1996, 10:371-382. 10.1002/(SICI)1099-0720(199610)10:5<371::AID-ACP389>3.0.CO;2-O.
36. Perfect T.J., Hollins T.S. Feeling-of-knowing judgments do not predict subsequent recognition performance for eyewitness memory. Journal of Experimental Psychology: Applied 1999, 5:250-264. 10.1037/1076-898X.5.3.250.
37. Perfect T.J., Hollins T.S., Hunt A.L.R. Practice and feedback effects on the confidence-accuracy relation in eyewitness memory. Memory 2000, 8:235-244. 10.1080/096582100406793.
38. Roebers C.M., Schneider W. The strategic regulation of children's memory performance and suggestibility. Journal of Experimental Child Psychology 2005, 91:24-44. 10.1016/j.jecp.2005.01.001.
39. Sax G., Collet L. The effects of differing instructions and guessing formulas on reliability and validity. Educational and Psychological Measurement 1968, 28:1127-1136. 10.1177/001316446802800411.
40. Schraw, G., Kuch, F., & Gutierrez, A.P. Assessing the dimensionality of calibration measures used in monitoring research. Learning and Instruction, in this volume.
41. Sherriffs A.C., Boomer D.S. Who is penalized by the penalty for guessing?. Journal of Educational Psychology 1954, 45:81-90. 10.1037/h0053756.
42. Sieck W.R., Merkle E.C., Van Zandt T. Option fixation: a cognitive contributor to overconfidence. Organizational Behavior and Human Decision Processes 2007, 103:68-83. 10.1016/j.obhdp.2006.11.001.
43. Slakter M.J. The effect of guessing strategy on objective test scores. Journal of Educational Measurement 1968, 5:217-222. 10.1111/j.1745-3984.1968.tb00629.x.
44. Slakter M.J. The penalty for not guessing. Journal of Educational Measurement 1968, 5(2):141-144. 10.1111/j.1745-3984.1968.tb00616.x.
45. Weber N., Brewer N. Eyewitness recall: regulation of grain size and the role of confidence. Journal of Experimental Psychology: Applied 2008, 14:50-60. 10.1037/1076-898X.14.1.50.
46. Windschitl P.D., Chambers J.R. The dud-alternative effect in likelihood judgment. Journal of Experimental Psychology: Learning, Memory, and Cognition 2004, 30:198-215. 10.1037/0278-7393.30.1.198.
47. Yaniv I., Foster D. Graininess of judgment under uncertainty: an accuracy-informativeness trade-off. Journal of Experimental Psychology: General 1995, 124:424-432. 10.1037/0096-3445.124.4.424.
48. Yaniv I., Foster D. Precision and accuracy of judgmental estimation. Journal of Behavioral Decision Making 1997, 10:21-32. 10.1002/(SICI)1099-0771(199703)10:1<21::AID-BDM243>3.0.CO;2-G.
49. Yonelinas A.P. The nature of recollection and familiarity: a review of 30 years of research. Journal of Memory and Language 2002, 46:441-517. 10.1006/jmla.2002.2864.