Instruction in information structuring improves Bayesian judgment in intelligence analysts

Frontiers in Psychology

Volumn 6, Issue MAR, 2015, Pages

  Mandel, David R ^a  

^a YORK UNIVERSITY   (Canada)

Author keywords

Bayesian judgment; Coherence; Information structuring; Instructional methods; Probability judgment

Indexed keywords

EID: 84926675710     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2015.00387     Document Type: Article

References (70)

1. Adams, B. A., Thomson, M., Derbentseva, N., and Mandel, D. R. (2012). Capability Challenges in the Human Domain for Intelligence Analysis: Report on Community-Wide Discussions with Canadian Intelligence Professionals [Contractor Report CR-2011-182] . Toronto, ON: Defence Research and Development Canada.
2. Ayal, S., and Beyth-Marom, R. (2014). The effects of mental steps and compatibility on Bayesian reasoning. Judgm. Decis. Mak . 9, 226-242.
3. Baratgin, J., and Noveck, I. (2000). Not only base-rates are neglected in the Lawyer-Engineer problem: an investigation of reasoners' underutilization of complementarity. Mem. Cogn . 28, 79-91. doi: 10.3758/BF03211578
4. Baratgin, J., Over, D. E., and Politzer, G. (2014). New psychological paradigm for conditionals and General de Finetti Tables. Mind Lang . 29, 73-84. doi: 10.1111/mila.12042
5. Baratgin, J., and Politzer, G. (2006). Is the mind Bayesian? The case for agnosticism. Mind Soc . 5, 1-38. doi: 10.1007/s11299-006-0007-1
6. Baratgin, J., and Politzer, G. (2010). Updating: a psychologically basic situation of probability revision. Think. Reason . 16, 253-287. doi: 10.1080/13546783.2010.519564
7. Baratgin, J. (2009). Updating our beliefs about inconsistency: the Monty-Hall case. Math. Soc. Sci . 57, 67-95. doi: 10.1016/j.mathsocsci.2008.08.006
8. Barbey, A. K., and Sloman, S. A. (2007). Base-rate respect: from ecological rationality to dual processes. Behav. Brain Sci . 30, 241-297. doi: 10.1017/S0140525X07001653
9. Bar-Hillel, M. (1980). The base rate fallacy in probability judgments. Acta Psychol . 44, 211-233. doi: 10.1016/0001-6918(80)90046-3
10. Brase, G. L. (2008). Pictorial representations in statistical reasoning. Appl. Cogn. Psychol . 23, 369-381. doi: 10.1002/acp.1460
11. Brase, G. L. (2014). The power of representation and interpretation: doubling statistical reasoning performance with icons and frequentist interpretations of ambiguous numbers. J. Cogn. Psychol . 26, 81-97. doi: 10.1080/20445911.2013.861840
12. Casscells, W., Schoenberger, A., and Graboys, T. (1978). Interpretation by physicians of clinical laboratory results. N. Engl. J. Med . 299, 999-1001. doi: 10.1056/NEJM197811022991808
13. Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., and Rohrer, D. (2006). Distributed practice in verbal recall tasks: a review and quantitative synthesis. Psychol. Bull . 132, 354-380. doi: 10.1037/0033-2909.132.3.354
14. Cosmides, L., and Tooby, J. (1996). Are humans good intuitive statisticians after all? Rethinking some conclusions from the literature on judgment under uncertainty. Cognition 58, 1-73. doi: 10.1016/0010-0277(95)00664-8
15. Cozic, M. (2011). Imaging and sleeping beauty: the case for double-halfers. Int. J. Approx. Reason . 52, 147-153. doi: 10.1016/j.ijar.2009.06.010
16. Delaney, P. F., Verkoeijen, P. P. J. L., and Spirgel, A. (2010). Spacing and the testing effects: a deeply critical, lengthy, and at times discursive review of the literature. Psychol. Learn. Motiv . 53, 63-147. doi: 10.1016/S0079-7421(10)53003-2
17. Douven, I., and Schupbach, J. N. (2015). Probabilistic alternatives to Bayesianism: the case of explanationism. Front. Psychol . 6:459. doi: 10.3389/fpsyg.2015.00459
18. Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., and Willingham, D. T. (2013). Improving students' learning with effective learning techniques: promising directions from cognitive and educational psychology. Psychol. Sci. Public Interest 14, 4-58. doi: 10.1177/1529100612453266
19. Eddy, D. M. (1982). "Probabilistic reasoning in clinical medicine: problems and opportunities," in Judgment under Uncertainty: Heuristics and Biases , eds D. Kahneman, P. Slovic and A. Tversky (New York, NY: Cambridge University Press), 249-267.
20. Evans, J. B. T. (2012). Questions and challenges for the new psychology of reasoning. Think. Reason . 18, 5-31. doi: 10.1080/13546783.2011.637674
21. Fischhoff, B., and Bruine de Bruin, W. (1999). Fifty-fifty=50%? J. Behav. Decis. Mak . 12, 149-163. doi: 10.1002/(SICI)1099-0771(199906)12:2<149::AID-BDM314>3.0.CO;2-J
22. Friedman, J. A., and Zeckhauser, R. (2012). Assessing uncertainty in intelligence. Intell. Natl. Secur . 27, 824-847. doi: 10.1080/02684527.2012.708275
23. Garcia-Retamero, R., and Hoffrage, U. (2013). Visual representation of statistical information improves diagnostic inferences in doctors and their patients. Soc. Sci. Med . 83, 27-33. doi: 10.1016/j.socscimed.2013.01.034
24. Gigerenzer, G., and Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction: frequency formats. Psychol. Rev . 102, 684-704. doi: 10.1037/0033-295X.102.4.684
25. Gigerenzer, G., and Hoffrage, U. (2007). The role of representation in Bayesian reasoning: correcting common misconceptions. Behav. Brain Sci . 30, 264-267. doi: 10.1017/S0140525X07001756
26. Gigerenzer, G., Hoffrage, U., and Ebert, A. (1998). AIDS counselling for low-risk clients. AIDS Care 10, 197-211. doi: 10.1080/09540129850124451
27. Hamm, R. (1993). Explanations for common responses to the Blue/Green cab probabilistic inference word problem. Psychol. Rep . 72, 219-242. doi: 10.2466/pr0.1993.72.1.219
28. Hoffrage, U., and Gigerenzer, G. (1998). Using natural frequencies to improve diagnostic inferences. Acad. Med . 73, 538-540. doi: 10.1097/00001888-199805000-00024
29. Juslin, P., Winman, A., and Olsson, H. (2003). Calibration, additivity, and source independence of probability judgments in general knowledge and sensory discrimination tasks. Organ. Behav. Hum. Decis. Process . 92, 34-51. doi: 10.1016/S0749-5978(03)00063-3
30. Juslin, P. (2015). Controlled information integration and Bayesian inference. Front. Psychol . 6:70. doi: 10.3389/fpsyg.2015.00070
31. Kahneman, D., and Tversky, A. (1972). Subjective probability: a judgment of representativeness. Cogn. Psychol . 3, 430-454. doi: 10.1016/0010-0285(72)90016-3
32. Kahneman, D., and Tversky, A. (1973). On the psychology of prediction. Psychol. Rev . 80, 237-251. doi: 10.1037/h0034747
33. Kapler, I. V., Weston, T., and Wiseheart, M. (2014). Spacing in a simulated undergraduate classroom: long-term benefits for factual and higher-level learning. Learn. Instr . 36, 38-45. doi: 10.1016/j.learninstruc.2014.11.001
34. Karvetski, C. W., Olson, K. C., Mandel, D. R., and Twardy, C. R. (2013). Probabilistic coherence weighting for optimizing expert forecasts. Decis. Anal . 10, 305-326. doi: 10.1287/deca.2013.0279
35. Kent, S. (1964). Words of Estimative Probability . Available online at: https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/books-and-monographs/sherman-kent-and-the-board-of-national-estimates-collected-essays/6words.html
36. Kleiter, G. (1994). "Natural sampling: rationality without base rates," in Contributions to Mathematical Psychology, Psychometrics, and Methodology , eds G. H. Fischer and D. Laming (New York, NY: Springer-Verlag), 375-388.
37. Kober, N. (2015). Reaching Students: What Research says about Effective Instruction in Undergraduate Science and Engineering. Board on Science Education, Division of Behavioral and Social Sciences and Education . Washington, DC: The National Academies Press.
38. Koehler, J. J. (1996). The base rate fallacy reconsidered: descriptive, normative and methodological challenges. Behav. Brain Sci . 19, 1-17. doi: 10.1017/S0140525X00041157
39. Kolmogorov, A. N. (1950). Foundations of the Theory of Probability . Oxford, UK: Chelsea Publishing Co.
40. Kurzenhäuser, S., and Hoffrage, U. (2002). Teaching Bayesian reasoning: an evaluation of a classroom tutorial for medical students. Med. Teach . 24, 516-521. doi: 10.1080/0142159021000012540
41. Lewis, D. (1976). Probabilities of conditionals and conditional probabilities. Philos. Rev . 85, 297-315. doi: 10.2307/2184045
42. Lyon, D., and Slovic, P. (1976). Dominance of accuracy information and neglect of base rates in probability estimation. Acta Psychol . 40, 287-298. doi: 10.1016/0001-6918(76)90032-9
43. Macchi, L., Osherson, D., and Krantz, D. H. (1999). A note on superadditive probability judgment. Psychol. Rev . 106, 210-214. doi: 10.1037/0033-295X.106.1.210
44. Macchi, L. (1995). Pragmatic aspects of the base rate fallacy. Q. J. Exp. Psychol . 48A, 188-207. doi: 10.1080/14640749508401384
45. Mandel, D. R., and Barnes, A. (2014). Accuracy of forecasts in strategic intelligence. Proc. Nat. Acad. Sci. U.S.A . 111, 10984-10989. doi: 10.1073/pnas.1406138111
46. Mandel, D. R. (2005). Are risk assessments of a terrorist attack coherent? J. Exp. Psychol. Appl . 11, 277-288. doi: 10.1037/1076-898X.11.4.277
47. Mandel, D. R. (2007). Nested-sets theory, full stop: explaining performance on Bayesian inference tasks without dual-systems assumptions. Behav. Brain Sci . 30, 275-276. doi: 10.1017/S0140525X07001835
48. Mandel, D. R. (2008). Violations of coherence in subjective probability: a representational and assessment processes account. Cognition 106, 130-156. doi: 10.1016/j.cognition.2007.01.001
49. Mandel, D. R. (2014a). The psychology of Bayesian reasoning. Front. Psychol . 5:1144. doi: 10.3389/fpsyg.2014.01144
50. Mandel, D. R. (2014b). Visual representation of rational belief revision: another look at the Sleeping Beauty problem. Front. Psychol . 5:1232. doi: 10.3389/fpsyg.2014.01232
51. McCloy, R., Beaman, C. P., Morgan, B., and Speed, R. (2007). Training conditional and cumulative risk judgements: the role of frequencies, problem-structure and Einstellung. Appl. Cogn. Psychol . 21, 325-344. doi: 10.1002/acp.1273
52. Mellers, B., Ungar, L., Baron, J., Ramos, J., Gurcay, B., Fincher, K., et al. (2014). Psychological strategies for winning a geopolitical forecasting tournament. Psychol. Sci . 25, 1106-1115. doi: 10.1177/0956797614524255
53. Oaksford, M., and Chater, N. (2007). Bayesian Rationality: The Probabilistic approach to Human Reasoning . Oxford, UK: Oxford University Press.
54. Pool, R. (2010). Field Evaluation in the Intelligence and Counterintelligence Context: Workshop Summary . Washington, DC: The National Academies Press.
55. Rottenstreich, Y., and Tversky, A. (1997). Unpacking, repacking, and anchoring: advances in support theory. Psychol. Rev . 104, 406-415. doi: 10.1037/0033-295X.104.2.406
56. Ruscio, J. (2003). Comparing Bayes's theorem to frequency-based approaches to teaching Bayesian reasoning. Teach. Psychol . 30, 325-328.
57. Sedlmeier, P., and Gigerenzer, G. (2001). Teaching Bayesian reasoning in less than two hours. J. Exp. Psychol. Gen . 130, 380-400. doi: 10.1037/0096-3445.130.3.380
58. Sedlmeier, P. (1999). Improving Statistical Reasoning: Theoretical Models and Practical Implications . Mahwah, NJ: Erlbaum.
59. Sirota, M., Kostovičová, L., and Vallée-Tourangeau, F. (2015). How to train your Bayesian: a problem-representation transfer rather than a format-representation shift explains training effects. Q. J. Exp. Psychol . 68, 1-9. doi: 10.1080/17470218.2014.972420
60. Sloman, S., Rottenstreich, Y., Wisniewski, E., Hadjichristidis, C., and Fox, C. R. (2004). Typical versus atypical unpacking and superadditive probability judgment. J. Exp. Psychol. Learn. Mem. Cognit . 30, 573-582. doi: 10.1037/0278-7393.30.3.573
61. Sloman, S. A., Over, D. E., Slovak, L., and Stibel, J. M. (2003). Frequency illusions and other fallacies. Organ. Behav. Hum. Decis. Process . 91, 296-309. doi: 10.1016/S0749-5978(03)00021-9
62. Thagard, P. (1989). Explanatory coherence. Behav. Brain Sci . 12, 435-502. doi: 10.1017/S0140525X00057046
63. Tversky, A., and Koehler, D. J. (1994). Support theory: a nonextensional representation of subjective probability. Psychol. Rev . 101, 547-567. doi: 10.1037/0033-295X.101.4.547
64. Villejoubert, G., and Mandel, D. R. (2002). The inverse fallacy: an account of deviations from Bayes's theorem and the additivity principle. Mem. Cogn . 30, 171-178. doi: 10.3758/BF03195278
65. Walliser, B., and Zwirn, D. (2002). Can Bayes' rule be justified by cognitive rationality principles? Theory Decis . 53, 95-135. doi: 10.1023/A:102122710674
66. Wallsten, T. S., Budescu, D. V., and Zwick, R. (1993). Comparing the calibration and coherence of numerical and verbal probability judgments. Manage. Sci . 39, 176 -190. doi: 10.1287/mnsc.39.2.176
67. Williams, J. J., and Mandel, D. R. (2007). "Do evaluation frames improve the quality of conditional probability judgment?," in Proceedings of the 29th Annual Meeting of the Cognitive Science Society , eds D. S. McNamara and J. G. Trafton (Mahwah, NJ: Erlbaum), 1653-1658.
68. Windschitl, P. D., Kruger, J., and Simms, E. N. (2003). The influence of egocentrism and focalism on people's optimism in competitions: when what affects us equally affects me more. J. Pers. Soc. Psychol . 85, 398-408. doi: 10.1037/0022-3514.85.3.389
69. Wolfe, C. R. (1995). Information seeking on Bayesian conditional probability problems: a fuzzy-trace theory account. J. Behav. Decis. Mak . 8, 85-108. doi: 10.1002/bdm.3960080203
70. Zlotnick, J. (1972). Bayes theorem for intelligence analysis. Stud. Intell . 16, 43-52.