Traditional difference-score analyses of reasoning are flawed

Cognition

Volumn 131, Issue 1, 2014, Pages 75-91

  Heit, Evan ^a   Rotello, Caren M ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF MASSACHUSETTS   (United States)

Author keywords

Belief bias; Conditional reasoning; Deduction; Reasoning; Signal detection theory; Syllogistic reasoning

Indexed keywords

ARTICLE; BELIEF BIAS; COGNITIVE BIAS; CONDITIONAL REASONING; CONTROLLED STUDY; HUMAN; HUMAN EXPERIMENT; PRIORITY JOURNAL; REASONING; RECEIVER OPERATING CHARACTERISTIC; RESPONSE BIAS; SIGNAL DETECTION; SIMULATION; THINKING; COGNITION ASSESSMENT; COMPARATIVE STUDY; LOGIC; NORMAL HUMAN; SOCIAL INTERACTION; STIMULUS RESPONSE;

BELIEF BIAS; CONDITIONAL REASONING; DEDUCTION; REASONING; SIGNAL DETECTION THEORY; SYLLOGISTIC REASONING;

COGNITION; CONCEPT FORMATION; HUMANS; LOGIC; ROC CURVE; SIGNAL DETECTION, PSYCHOLOGICAL;

EID: 84892859177     PISSN: 00100277     EISSN: 18737838     Source Type: Journal    
DOI: 10.1016/j.cognition.2013.12.003     Document Type: Article

References (75)

1. Alloy L., Tabachnik N. Assessment of covariation by humans and animals: The joint influence of prior expectations and current situational information. Psychological Review 1984, 91:112-149.
2. Bartlett F.C. Remembering: A study in experimental and social psychology 1932, Cambridge University Press, Cambridge.
3. Blanchette I., Campbell M. Reasoning about highly emotional topics: Syllogistic reasoning in a group of war veterans. Journal of Cognitive Psychology 2012, 24:157-164.
4. Bransford J.D., Johnson M.K. Contextual prerequisites for understanding: Some investigations of comprehension and recall. Journal of Verbal Learning and Verbal Behavior 1972, 11:717-726.
5. Cleary A.M. ROCs in recognition with and without identification. Memory 2005, 13:472-483.
6. Cohen J. Statistical power for the behavioral sciences 1988, Lawrence Erlbaum Associates, Hillsdale, New Jersey. 2nd ed.
7. Cook T.D., Campbell D.T. Quasi-experimentation: Design and analysis issues for field settings 1979, Rand McNally, Chicago.
8. Dube C., Rotello C.M. Binary ROCs in perception and recognition memory are curved. Journal of Experimental Psychology: Learning, Memory, and Cognition 2012, 38:130-151.
9. Dube C., Rotello C.M., Heit E. Assessing the belief bias effect with ROCs: It's a response bias effect. Psychological Review 2010, 117:831-863.
10. Dube C., Rotello C.M., Heit E. The belief bias effect is aptly named: A reply to Klauer and Kellen (2011). Psychological Review 2011, 118:155-163.
11. Eliades M., Mansell W., Stewart A.J., Blanchette I. An investigation of belief-bias and logicality in reasoning with emotional contents. Thinking & Reasoning 2012, 18:461-479.
12. Evans J.St.B.T. The heuristic-analytic theory of reasoning: Extension and evaluation. Psychonomic Bulletin & Review 2006, 13:378-395.
13. Evans J.St.B.T. Hypothetical thinking: Dual processes in reasoning and judgement 2007, Psychology Press, New York, NY.
14. Evans J.St.B.T. Dual-processing accounts of reasoning, judgement and social cognition. Annual Review of Psychology 2008, 59:255-278.
15. Evans J.St.B.T., Barston J.L., Pollard P. On the conflict between logic and belief in syllogistic reasoning. Memory & Cognition 1983, 11:295-306.
16. Evans J.St.B.T., Curtis-Holmes J. Rapid responding increases belief bias: Evidence for the dual-process theory of reasoning. Thinking & Reasoning 2005, 11:382-389.
17. Evans J.S.B.T., Handley S.J., Bacon A.M. Reasoning under time pressure. Experimental Psychology (formerly Zeitschrift für Experimentelle Psychologie) 2009, 56:77-83.
18. Evans J.St.B.T., Legrenzi P., Girotto V. The influence of linguistic form on reasoning: The case of matching bias. The Quarterly Journal of Experimental Psychology: Section A 1999, 52:185-216.
19. Evans J.St.B.T., Newstead S.E., Allen J.L., Pollard P. Debiasing by instruction: The case of belief bias. European Journal of Cognitive Psychology 1994, 6:263-285.
20. Evans K., Rotello C.M., Li X., Rayner K. Scene perception and memory revealed by eye movements and ROC analyses: Does a cultural difference truly existα. Quarterly Journal of Experimental Psychology 2009, 62:276-285. [PMCID: PMC2668147].
21. Goel V., Vartanian O. Negative emotions can attenuate the influence of beliefs on logical reasoning. Cognition & Emotion 2011, 25:121-131.
22. Griggs R.A. To "see" or not to "see": That is the selection task. The Quarterly Journal of Experimental Psychology 1989, 41:517-529.
23. Guido W., Lu S.-M., Vaughan J.W., Godwin D.W., Sherman S.M. Receiver operating characteristic (ROC) analysis of neurons in the cat's lateral geniculate nucleus during tonic and burst response model. Visual Neuroscience 1995, 12:723-741.
24. Handley S.J., Capon A., Beveridge M., Dennis I., Evans J.St.B.T. Working memory, inhibitory control and the development of children's reasoning. Thinking & Reasoning 2004, 10:175-195.
25. Heit E., Bott L. Knowledge selection in category learning. Psychology of learning and motivation 2000, Vol. 39:163-199. Academic Press, San Diego.
26. Heit E., Hahn U., Feeney A. Defending diversity. Categorization inside and outside of the laboratory: Essays in honor of Douglas L. Medin 2005, 87-99. APA, Washington, DC. W. Ahn, R. Goldstone, B. Love, A. Markman, P. Wolff (Eds.).
27. Heit E., Rotello C.M. Are there two kinds of reasoningα. Proceedings of the twenty-five annual meeting of the cognitive science society 2005, 923-928. Erlbaum, Mahwah, NJ.
28. Heit E., Rotello C.M. Modeling two kinds of reasoning. Proceedings of the thirtieth annual meeting of the cognitive science society 2008, 1831-1836. Cognitive Science Society, Austin, TX.
29. Heit E., Rotello C.M. Relations between inductive reasoning and deductive reasoning. Journal of Experimental Psychology: Learning, Memory, and Cognition 2010, 36:805-812.
30. Heit E., Rotello C.M. The pervasive effects of argument length on inductive reasoning. Thinking & Reasoning 2012, 18:244-277. (Special issue on Reasoning and Argumentation).
31. Heit E., Rotello C.M., Hayes B.K. Relations between memory and reasoning. Psychology of Learning and Motivation 2012, Vol. 52:57-101. Elsevier, Amsterdam.
32. Jaeger T.F. Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language 2008, 59:434-446.
33. Kinchla R.A. Comments on Batchelder and Riefer's multinomial model for source monitoring. Psychological Review 1994, 101:166-171.
34. Klauer K.C., Kellen D. Assessing the belief bias effect with ROCs: Reply to Dube, Rotello, and Heit (2010). Psychological Review 2011, 118:164-173.
35. Klauer K.C., Musch J., Naumer B. On belief bias in syllogistic reasoning. Psychological Review 2000, 107:852-884.
36. Macmillan N.A., Creelman C.D. Detection theory: A user's guide 2005, Lawrence Erlbaum Associates Publishers, Mahwah, NJ, US. 2nd ed.
37. Macmillan N.A., Kaplan H.L., Creelman D. The psychophysics of categorical perception. Psychological Review 1977, 84:452-471.
38. Malmberg K.J. On the form of ROCs constructed from confidence ratings. Journal of Experimental Psychology: Learning, Memory, and Cognition 2002, 28:380-387.
39. Markovits H., Forgues H.L., Brunet M.-L. Conditional reasoning, frequency of counterexamples, and the effect of response modality. Memory & Cognition 2010, 38:485-492.
40. Markovits H., Handley S. Is inferential reasoning just probabilistic reasoning in disguiseα. Memory & Cognition 2005, 33:1315-1323.
41. Masson M.E.J., Rotello C.M. Sources of bias in the Goodman-Kruskal gamma coefficient measure of association: Implications for studies of metacognitive processes. Journal of Experimental Psychology: Learning, Memory, and Cognition 2009, 35:509-527.
42. Mickes L., Johnson E.M., Wixted J.T. Continuous recollection versus unitized familiarity in associative recognition. Journal of Experimental Psychology: Learning, Memory, and Cognition 2010, 36:843-863.
43. Morley N.J., Evans J.St.B.T., Handley S.J. Belief bias and figural bias in syllogistic reasoning. The Quarterly Journal of Experimental Psychology A: Human Experimental Psychology 2004, 57:666-692.
44. Newstead S.E., Pollard P., Evans J.S., Allen J. The source of belief bias effects in syllogistic reasoning. Cognition 1992, 45:257-284.
45. Oakhill J.V., Johnson-Laird P. The effects of belief on the spontaneous production of syllogistic conclusions. The Quarterly Journal of Experimental Psychology A: Human Experimental Psychology 1985, 37:553-569.
46. Pazzaglia A.M., Dube C., Rotello C.M. A critical comparison of discrete-state and continuous models of recognition memory: Implications for recognition and beyond. Psychological Bulletin 2013, 139:1173-1203.
47. Platt R.D., Griggs R.A. Facilitation in the abstract selection task: The effects of attentional and instructional factors. The Quarterly Journal of Experimental Psychology 1993, 46:591-613.
48. Polk T.A., Newell A. Deduction as verbal reasoning. Psychological Review 1995, 102:533-566.
49. Pollard P., Evans J.St.B.T. On the relationship between content and context effects in reasoning. American Journal of Psychology 1987, 100:41-60.
50. Quayle J., Ball L. Working memory, metacognitive uncertainty, and belief bias in syllogistic reasoning. The Quarterly Journal of Experimental Psychology A: Human Experimental Psychology 2000, 53:1202-1223.
51. Rips L.J. Two kinds of reasoning. Psychological Science 2001, 12:129-134.
52. Rotello C.M., Heit E. Modeling the effects of argument length and validity on inductive and deductive reasoning. Journal of Experimental Psychology: Learning, Memory, and Cognition 2009, 35:1317-1330.
53. Rotello C.M., Macmillan N.A., Hicks J.L., Hautus M. Interpreting the effects of response bias on remember-know judgments using signal-detection and threshold models. Memory & Cognition 2006, 34:1598-1614.
54. Rotello C.M., Masson M.E.J., Verde M.F. Type I error rates and power analyses for single-point sensitivity measures. Perception & Psychophysics 2008, 70:389-401.
55. Schyns P.G., Oliva A. Dr. Angry and Mr. Smile: when categorization flexibly modifies the perception of faces in rapid visual presentations. Cognition 1999, 69:243-265.
56. Sellen J.L., Oaksford M., Gray N.S. Schizotypy and conditional reasoning. Schizophrenia Bulletin 2005, 31:105-116.
57. Sewell D.K., Smith P.L. Attentional control in visual signal detection: Effect of abrupt-onset and no-onset stimuli. Journal of Experimental Psychology: Human Perception and Performance 2012, 38:1043-1068.
58. Sherman J.W., Gawronski B., Gonsalkorale K., Hugenberg K., Allen T.J., Groom C.J. The self-regulation of automatic associations and behavioral impulses. Psychological Review 2008, 115:314-335.
59. Shynkaruk J.M., Thompson V.A. Confidence and accuracy in deductive reasoning. Memory & Cognition 2006, 34:619-632.
60. Skyrms B. Choice and chance: An introduction to inductive logic 2000, Wadsworth, Belmont, CA. 4th ed.
61. Sloman S.A. Categorical inference is not a tree: The myth of inheritance hierarchies. Cognitive Psychology 1998, 35:1-33.
62. Slotnick S.D. "Remember" source memory ROCs indicate recollection is a continuous process. Memory 2010, 18:27-39.
63. Snodgrass J.G., Corwin J. Pragmatics of measuring recognition memory: Applications to dementia and amnesia. Journal of Experimental Psychology: General 1988, 117:34-50.
64. Stangor C., Macmillan D. Memory for expectancy-congruent and expectancy-incongruent information: A review of the social and social developmental literatures. Psychological Bulletin 1992, 111:42-61.
65. Stanovich K.E. What intelligence tests miss: The psychology of rational thought 2009, Yale, New Haven.
66. Stanovich K.E., West R.F. On the relative independence of thinking biases and cognitive ability. Journal of Personality and Social Psychology 2008, 94:672.
67. Stollstorff M., Bean S.E., Anderson L.M., Devaney J.M., Vaidya C.J. Rationality and emotionality: Serotonin transporter genotype influences reasoning bias. Social Cognitive and Affective Neuroscience 2013, 8:404-409.
68. Stupple E.J.N., Ball L.J., Evans J.St.B.T., Kamal-Smith E. When logic and belief collide: Individual differences in reasoning times support a selective processing model. Journal of Cognitive Psychology 2011, 23:931-941.
69. Swets J.A. Indices of discrimination or diagnostic accuracy: Their ROCs and implied models. Psychological Bulletin 1986, 99:100-117.
70. Thompson V., Evans J.St.B.T. Belief bias in informal reasoning. Thinking & Reasoning 2012, 18:278-310.
71. Trippas D., Handley S.J., Verde M.J. The SDT model of belief bias: Complexity, time and cognitive ability mediate the effects of believability. Journal of Experimental Psychology: Learning, Memory, and Cognition 2013, 39:1393-1402.
72. Verde M.F., Rotello C.M. Does familiarity change in the revelation effectα. Journal of Experimental Psychology: Learning, Memory, and Cognition 2003, 29:739-746.
73. Wason P.C. Reasoning about a rule. The Quarterly Journal of Experimental Psychology 1968, 20:273-281.
74. Wixted J.T., Mickes L. The field of eyewitness memory should abandon "probative value" and embrace receiver operating characteristic analysis. Perspectives on Psychological Science 2012, 7:275-278.
75. Wixted J.T., Squire L.R. Constructing receiver operating characteristics (ROCs) with experimental animals: Cautionary notes. Learning & Memory 2008, 15:687-690.