A little bias goes a long way: The effects of feedback on the strategic regulation of accuracy on formula-scored tests

Journal of Experimental Psychology: Applied

Volumn 19, Issue 4, 2013, Pages 383-402

  Arnold, Michelle M ^a   Higham, Philip A ^b   Mart́ín Luengo, Beatriz ^c  

^a FLINDERS UNIVERSITY   (Australia)

^b UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

^c UNIVERSITY OF THE BASQUE COUNTRY UPV EHU   (Spain)

Author keywords

Bias profiles; Formula scoring; Metacognition; Strategic regulation of accuracy; Type 2 signal detection theory

Indexed keywords

ADAPTIVE BEHAVIOR; EDUCATION; HUMAN; PROCEDURES; PSYCHOLOGY; REPRODUCIBILITY; STANDARDS; STATISTICAL BIAS; STUDENT;

BIAS (EPIDEMIOLOGY); EDUCATIONAL MEASUREMENT; FEEDBACK, PSYCHOLOGICAL; HUMANS; REPRODUCIBILITY OF RESULTS; STUDENTS;

EID: 84890530351     PISSN: 1076898X     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0034833     Document Type: Article

References (42)

1. Bisseret, A. (1981). Application of signal detection theory to decision making in supervisory control. Ergonomics, 24, 81-94.
2. Bliss, L. B. (1980). A test of Lord's assumption regarding examinee guessing behavior on multiple-choice tests using elementary school students. Journal of Educational Measurement, 17, 147-152.
3. Bruno, D., Higham, P. A., & Perfect, T. J. (2009). Global subjective memorability and the strength-based mirror effect in recognition memory. Memory & Cognition, 37, 807-818.
4. Budescu, D., & Bar-Hillel, M. (1993). To guess or not to guess: A decision-theoretic view of formula scoring. Journal of Educational Measurement, 30, 277-291.
5. Cañal-Bruland, R., & Schmidt, M. (2009). Response bias in judging deceptive movements. Acta Psychologica, 130, 235-240.
6. Cross, L. H., & Frary, R. B. (1977). An empirical test of Lord's theoretical results regarding formula scoring of multiple-choice tests. Journal of Educational Measurement, 14, 313-321.
7. Dunlosky, J., & Hertzog, C. (2000). Updating knowledge about encoding strategies: A componential analysis of learning about strategy effectiveness from task experience. Psychology and Aging, 15, 462-474.
8. Dunlosky, J., & Hertzog, C. (2001). Measuring strategy production during associative learning: The relative utility of concurrent versus retrospective reports. Memory & Cognition, 29, 247-253.
9. Durso, F. T., & Shore, W. J. (1991). Partial knowledge of word meanings. Journal of Experimental Psychology: General, 120, 190-202.
10. Ebel, R. L. (1968). Blind guessing on objective achievement tests. Journal of Educational Measurement, 5, 321-325.
11. Goldsmith, G., & Koriat, A. (2008). The strategic regulation of memory accuracy and informativeness. In A. Benjamin & B. Ross (Eds.), The psychology of learning and motivation. Vol. 48: Memory use as skilled cognition (pp. 307-324). San Diego, CA: Elsevier.
12. Goldsmith, M. (2011). Quantity-accuracy profiles or type-2 signal detection measures? Similar methods toward a common goal. In P. A. Higham & J. P. Leboe (Eds.), Constructions of remembering and metacognition: Essays in honour of Bruce Whittlesea (pp. 128-136). Basingstoke, UK: Palgrave-MacMillan.
13. Goodman, L. A., & Kruskal, W. H. (1954). Measures of association for cross classifications. Journal of the American Statistical Association, 49, 732-769.
14. Green, D., & Swets, J. (1966). Signal detection theory and psychophysics. New York, NY: Wiley.
15. Hertzog, C., Price, J., & Dunlosky, J. (2008). How is knowledge generated about memory encoding strategy effectiveness? Learning and Individual Differences, 18, 430-445.
16. Higham, P. A. (1997). Dissociations of grammaticality and specific similarity effects in artificial grammar learning. Journal of Experimental Psychology: Learning Memory & Cognition, 23, 1029-1045.
17. Higham, P. A. (2007). No special K! A signal-detection framework for the strategic regulation of memory accuracy. Journal of Experimental Psychology: General, 136, 1-22.
18. Higham, P. A. (2011). Accuracy discrimination and type-2 signal detection theory: Clarifications, extensions, and an analysis of bias. In P. A. Higham & J. P. Leboe (Eds.), Constructions of remembering and metacognition: Essays in honour of Bruce Whittlesea (pp. 109-127). Basingstoke, UK: Palgrave-MacMillan.
19. Higham, P. A., & Arnold, M. M. (2007a). How many questions should I answer? Using bias profiles to estimate optimal bias and maximum score on formula-scored tests. European Journal of Cognitive Psychology, 19, 718-742.
20. Higham, P. A., & Arnold, M. M. (2007b). Beyond reliability and validity: The role of metacognition in psychological testing. In R. A. DeGregorio (Ed.), New developments in psychological testing (pp. 139-162). Hauppauge, NY: Nova Science.
21. Jang, Y., Wixted, J. T., & Huber, D. E. (2009). Testing signal-detection models of yes/no recognition and two-alternative-forced-choice recognition memory. Journal of Experimental Psychology: General, 138, 291-306.
22. Koriat, A., & Goldsmith, M. (1996). Monitoring and control processes in the strategic regulation of memory accuracy. Psychological Review, 103, 490-517.
23. Lueddeke, S. E., & Higham, P. A. (2011). Expertise and gambling: Using type-2 signal detection theory to investigate differences between regular gamblers and non-gamblers. The Quarterly Journal of Experimental Psychology, 64, 1850-1871.
24. Luna, K., Higham, P. A., & Martin-Luengo, B. (2011). The regulation of memory accuracy with multiple answers: The plurality option. Journal of Experimental Psychology: Applied, 17, 148 -158.
25. Macmillan, N. A., & Creelman, C. D. (2005). Detection theory: A user's guide (2nd ed.). Mahwah, NJ: Erlbaum.
26. Maddox, W. T. (2002). Toward a unified theory of decision criterion learning in perceptual categorization. Journal of the Experimental Analysis of Behavior, 78, 567-595.
27. Masson, M. E. J., & Rotello, C. M. (2009). Sources of bias in the Goodman-Kruskal gamma coefficient measure of association: Implications for studies of metacognitive processes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35, 509-527.
28. Meissner, C. A., & Kassin, S. M. (2002). "He's guilty!": Investigator bias in judgments of truth and deception. Law and Human Behavior, 26, 469-480.
29. Metcalfe, J., & Kornell, N. (2005). A region of proximal learning model of study allocation. Journal of Memory and Language, 52, 463-477.
30. Muijtjens, A. M. M., van Mameren, H., Hoogenboom, R. J. I., Evers, J. L. H., & van der Vleuten, C. P. M. (1999). The effect of a "don't know" option on test scores: Number-right and formula scoring compared. Medical Education, 33, 267-275. http://dx.
31. Nelson, T. O. (1984). A comparison of current measures of the accuracy of feeling-of-knowing predictions. Psychological Bulletin, 95, 109-133.
32. Nelson, T. O., & Narens, L. (1980). Norms of 300 general-information questions: Accuracy of recall, latency of recall, and feeling-of-knowing ratings. Journal of Verbal Learning & Verbal Behavior, 19, 338-368.
33. Parasuraman, R. (1985). Detection and identification of abnormalities in chest x-rays: Effects of reader skill, disease prevalence, and reporting standards. In R. E. Eberts & C. G. Eberts (Eds.), Trends in ergonomics/human factors II (pp. 59-66). Amsterdam, The Netherlands: North-Holland.
34. Rawson, K. A., & Kintsch, W. (2005). Rereading effects depend on time of test. Journal of Educational Psychology, 97, 70-80.
35. Reber, A. S. (1993). Implicit learning and tacit knowledge: An essay on the cognitive unconscious. Oxford, UK: Oxford University Press.
36. Rotello, C. M., Masson, M. E. J., & Verde, M. F. (2008). Type 1 error rates and power analyses for single-point sensitivity measures. Perception & Psychophysics, 70, 389-401.
37. Sax, G., & Collet, L. (1968). The effects of differing instructions and guessing formulas on reliability and validity. Educational and Psychological Measurement, 28, 1127-1136.
38. Sherriffs, A. C., & Boomer, D. S. (1954). Who is penalized by the penalty for guessing? Journal of Educational Psychology, 45, 81-90.
39. Slakter, M. J. (1968a). The penalty for not guessing. Journal of Educational Measurement, 5, 141-144.
40. Slakter, M. J. (1968b). The effect of guessing strategy on objective test scores. Journal of Educational Measurement, 5, 217-222.
41. Stretch, V., & Wixted, J. T. (1998). On the difference between strengthbased and frequency-based mirror effects in recognition memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24, 1379-1396.
42. Thurstone, L. L. (1919). A scoring method for mental tests. Psychological Bulletin, 16, 235-240.