Handling and Mishandling Estimative Probability: Likelihood, Confidence, and the Search for Bin Laden

Intelligence and National Security

Volumn 30, Issue 1, 2015, Pages 77-99

  Friedman, Jeffrey A ^a   Zeckhauser, Richard ^b  

^a Dartmouth's Dickey Center for International Understanding   (United States)

^b HARVARD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84916593986     PISSN: 02684527     EISSN: 17439019     Source Type: Journal    
DOI: 10.1080/02684527.2014.885202     Document Type: Article

References (54)

1. Graham Allison, ‘How It Went Down’, Time, 7 May 2012.
2. Mark Bowden, The Finish: The Killing of Osama bin Laden (NY: Atlantic Monthly 2012) pp.158-60.
3. David Sanger, Confront and Conceal: Obama’s Secret Wars and Surprising Use of American Power (NY: Crown 2012) p.93
4. Peter Bergen, Manhunt: The Ten-Year Search for Bin Laden from 9/11 to Abbottabad (NY: Crown 2012) pp.133.
5. Bowden, The Finish, pp.160-1
6. Bergen, Manhunt, p.198
7. Sanger, Confront and Conceal, p.93.
8. Howard Raiffa, Decision Analysis: Introductory Lectures on Choices under Uncertainty (Reading, MA: Addison-Wesley 1968) p.10: ‘[Decision theory does not] present a descriptive theory of actual behavior. Neither [does it] present a positive theory of behavior for a superintelligent, fictitious being; nowhere in our analysis shall we refer to the behavior of an “idealized, rational, and economic man”, a man who always acts in a perfectly consistent manner as if somehow there were embedded in his nature a coherent set of evaluation patterns that cover any and all eventualities. Rather the approach we take prescribes how an individual who is faced with a problem of choice under uncertainty should go about choosing a course of action that is consistent with his personal basic judgments and preferences. He must consciously police the consistency of his subjective inputs and calculate their implications for action. Such an approach is designed to help us reason and act a bit more systematically-when we choose to do so!’
9. Structured analytic techniques can also mitigate biases. Some relevant biases are intentional, such as the way that some analysts are said to hedge their estimates in order to avoid criticism for mistaken predictions. More generally, individuals encounter a wide range of cognitive constraints when assessing probability and risk. See, for instance, Paul Slovic, The Perception of Risk (London: Earthscan 2000)
10. Reid Hastie and Robyn M. Dawes, Rational Choice in an Uncertain World: The Psychology of Judgment and Decision Making (Thousand Oaks, CA: Sage 2001)
11. Daniel Kahneman, Thinking, Fast and Slow (NY: FSG 2011).
12. Debates about estimative probability and intelligence analysis are long-standing; the seminal article is by Sherman Kent, ‘Words of Estimative Probability’, Studies in Intelligence 8/4 (1964) pp.49-65.
13. Yet those debates remain unresolved, and they comprise a small slice of current literature. Miron Varouhakis, ‘What is Being Published in Intelligence? A Study of Two Scholarly Journals’, International Journal of Intelligence and CounterIntelligence 26/1 (2013) p.183
14. Shows that fewer than 7 per cent of published studies in two prominent intelligence journals focus on analysis. In a recent survey of intelligence scholars, the theoretical foundations for analysis placed among the most under-researched topics in the field: Loch K. Johnson and Allison M. Shelton, ‘Thoughts on the State of Intelligence Studies: A Survey Report’, Intelligence and National Security 28/1 (2013) p.112.
15. Mark Phythian, ‘Policing Uncertainty: Intelligence, Security and Risk’, Intelligence and National Security 27/2 (2012) pp.187-205, for a similar distinction. Ignorance is another important concept, denoting situations where it is not even possible to define all possible answers to an estimative question. This situation often arises in intelligence analysis, but it is beyond the scope of this article.
16. Sometimes subjective probability is called ‘personal probability’ to emphasize that it captures an individual’s beliefs about the world rather than objective frequencies determined via controlled experiments. As Frank Lad describes, the subjectivist approach to probability ‘represents your assessment of your own personal uncertain knowledge about any event that interests you. There is no condition that events be repeatable… In the proper syntax of the subjectivist formulation, you might well ask me and I might well ask you, “What is your probability for a specified event?” It is proposed that there is a distinct (and generally different) correct answer to this question for each person who responds to it. We are each sanctioned to look within ourselves to find our own answer. Your answer can be evaluated as correct or incorrect only in terms of whether or not you answer honestly’. Frank Lad, Operational Subjective Statistical Methods (NY: Wiley 1996) pp.8-9.
17. Robert L. Winkler, Introduction to Bayesian Inference and Decision, 2nd ed. (Sugar Land, TX: Probabilistic Publishing 2010) pp.14-23.
18. To control for time preferences, one would ideally make it so that the potential resolutions of these gambles and their payoffs occurred at the same time. Thus, if the gamble involved the odds of regime change in a foreign country by the end of the year, the experimenter would draw from the urn at year’s end or when the regime change occurred, at which point any payoff would be made.
19. Adam Meirowitz and Joshua A. Tucker, ‘Learning from Terrorism Prediction Markets’, Perspectives on Politics 2/2 (2004) pp.331-6.
20. Kristan J. Wheaton, ‘The Revolution Begins on Page Five: The Changing Nature of NIEs’, International Journal of Intelligence and CounterIntelligence 25/2 (2012) p.336.
21. The separate question of how decision makers should determine when to act versus waiting to gather additional information is a central topic in the section that follows.
22. The discussion below applies broadly to debating any question that has a yes-or-no answer. Matters get more complicated with broader questions that admit many possibilities, for example ‘Where is bin Laden currently living?’ These situations can be addressed by decomposing the issue into binary components, such as ‘Is bin Laden living in location A?’, ‘Is bin Laden living in location B?’, and so on.
23. By extension, this arrangement covers situations where the best estimate could lie anywhere between 40 and 80 per cent, and you do not believe that any options within this range are more likely than others, since the contents of the urn could be combined to construct any intermediate mix of colors.
24. Ellsberg’s seminal example involved deciding whether to choose from an urn with exactly 50 red marbles and 50 black marbles to win a prize versus an urn where the mix of colors was unknown. Subjects will often pay non-trivial amounts to select from the urn with known risk, which makes no rational sense. If people are willing to pay more in order to gamble on drawing a red marble from the urn with a 50/50 distribution, this implies that they believe there is less than a 50 per cent chance of drawing a red marble from the ambiguous urn. But by the same logic, they will also be willing to pay more in order to gamble on drawing a black marble from the 50/50 urn, which implies they believe there is less than a 50 per cent chance of drawing a black marble from the ambiguous urn. These two statements cannot be true simultaneously. This is known as the ‘Ellsberg paradox’. See Daniel Ellsberg, ‘Risk, Ambiguity, and the Savage Axioms’, Quarterly Journal of Economics 75/4 (1961) pp.643-69.
25. Stefan T. Trautmann and Gijs van de Kuilen, ‘Ambiguity Attitudes’ in Gideon Keren and George Wu (eds.) The Blackwell Handbook of Judgment and Decision Making (Malden, MA: Blackwell forthcoming).
26. W. Kip Viscusi and Richard J. Zeckhauser, ‘The Less Than Rational Regulation of Ambiguous Risks’, University of Chicago Law School Conference, 26 April 2013. These contrasting examples show how risk preferences depend on a decision maker’s views of whether it would be worse to make an error of commission or omission-in decision theory more generally, a key concept is balancing the risks of Type I and Type II errors. 21It is especially important to disentangle these questions when it comes to intelligence analysis, a field in which one of the cardinal rules is that it is the analyst’s role to provide information that facilitates decision making, but not to interfere with making the decision itself. Massaging probabilistic estimates in light of potential policy responses inherently blurs this line.
27. Throughout this discussion, the information available to one analyst is assumed to be available to all, hence analysts are basing their estimates on the same evidence. The importance of this point is discussed below.
28. Baruch Fischhoff and Wandi Bruine de Bruin, ‘Fifty-Fifty = 50%?’, Journal of Behavioral Decisionmaking 12/2 (1999) pp.149-63.
29. Paul Lehner et al., ‘Using Inferred Probabilities to Measure the Accuracy of Imprecise Forecasts’ (MITRE 2012) Case #12-4439, show that intelligence estimates predicting that an event will occur with 50 per cent odds tend to be especially inaccurate.
30. In this case, some people believed the analyst fell prey to confirmation bias and thus interpreted the evidence over-optimistically. In other cases, of course, analysts involved with intelligence collection may deserve extra credibility given their knowledge of the relevant material.
31. This is not to deny that there may be instances where the views of ‘Red Teams’ or ‘devil’s advocates’ will turn out to be more accurate than the consensus opinion (and more generally, that these tools are useful for helping to challenge and refine other estimates). It is to say that, on balance, one should expect an analysis to be less credible if it is biased, and Red Teams are explicitly tasked to slant their views.
32. As Morell reflected: ‘People don’t have differences [over estimative probabilities] because they have different intel… We are all looking at the same things. I think it depends more on your past experience’ (Bowden, The Finish, p.161).
33. Dominic D. P. Johnson, Overconfidence in War: The Havoc and Glory of Positive Illusions (Cambridge, MA: Harvard University Press 2004).
34. It is important to note that prior assumptions do not always lead analysts towards the right conclusions. Robert Jervis argues, for instance, that flawed assessments of Iraq’s potential weapons of mass destruction programs were driven by the assumed plausibility that Saddam Hussein would pursue such capabilities. The point of assessing priors is thus not to reify them, but rather to make their role in the analysis explicit (and, where possible, to submit such assumptions to structured critique). See Robert Jervis, ‘Reports, Politics, and Intelligence Failures: The Case of Iraq’, Journal of Strategic Studies 29/1 (2006) pp.3-52.
35. For example, imagine that two analysts are assigned to assess the likelihood that a certain state will attack its neighbor by the end of the year. These analysts share the prior assumption that the odds of this happening are relatively low (say, about 5 per cent). They independently encounter different pieces of information suggesting that these chances are higher than they originally anticipated. Analyst A learns that the country has been secretly importing massive quantities of armaments, and Analyst B learns that the country has been conducting large, unannounced training exercises for its air force. Based on this information, our analysts respectively estimate a 30 per cent (A) and a 40 per cent (B) chance of war breaking out by the end of the year. In this instance, it would be problematic to think that the odds of war are somewhere between 30 and 40 per cent, because if the analysts had been exposed to and properly incorporated each others’ information, both their respective estimates would presumably have been higher. On such processes, see Richard Zeckhauser, ‘Combining Overlapping Information’, Journal of the American Statistical Association 66/333 (1971) pp.91-2.
36. Scholars have shown this to be the case in several fields including medicine, law, and climate change. See, for instance, Elke U. Weber et al., ‘Determinants of Diagnostic Hypothesis Generation: Effects of Information, Base Rates, and Experience’, Journal of Experimental Psychology: Learning, Memory, and Cognition 19/5 (1993) pp.1151-64
37. Jonas Jacobson et al., ‘Predicting Civil Jury Verdicts: How Attorneys Use (and Misuse) a Second Opinion’, Journal of Empirical Legal Studies 8/1 (2011) pp.99-119.
38. This tendency follows from the availability heuristic, one of the most thoroughly-documented biases in behavioral decision, which shows that people inflate the probability of events that are easier to bring to mind; see Amos Tversky and Daniel Kahneman, ‘Availability: A Heuristic for Judging Frequency and Probability’, Cognitive Psychology 5 (1973) pp.207-32. In the context of intelligence analysis, this implies that analysts may conflate the predictive value of a piece of information with how easily they are able to interpret it.
39. An analyst’s past record of making successful predictions may also inform the credibility of her estimates. Recent research demonstrates that some people are systematically better than others at political forecasting. However, contemporary systems for evaluating analyst performance are relatively underdeveloped, especially since analysts tend not to specify probabilistic estimates in a manner that can be rated objectively. Moreover, when decision makers consider the past performance of their advisors, they may be tempted to extrapolate from a handful of experiences that offer little basis for judging analytic skill (or from personal qualities that are not relevant for estimating likelihood). Determining how to draw sound inferences about an analyst’s credibility from their past performance is thus an area where further research can have practical benefit. In the interim, we focus on evaluating the logic of each analyst’s assessment per se.
40. Philip E. Tetlock, Expert Political Judgment: How Good Is It? How Can We Know? (Princeton, NJ: Princeton University Press 2005)
41. Lyle Ungar et al., ‘The Good Judgment Project: A Large Scale Test of Different Methods of Combining Expert Predictions’, AAAI Technical Report FS-12-06 (2012).
42. Some relevant techniques include: asking individuals to rate the credibility of their own predictions and using these ratings as weights; asking individuals to debate among themselves whose estimates seem most credible and thereby determine appropriate weighting by consensus; and denoting a member of the team to assign weights to each estimate after evaluating the reasoning that analysts present. For a review, see Detlof von Winterfeldt and Ward Edwards, Decision Analysis and Behavioral Research (NY: Cambridge University Press 1986) pp.133-6.
43. Office of the Director of National Intelligence [ODNI], US National Intelligence: An Overview (Washington, DC: ODNI 2011) p.60.
44. For example, Bergen prints a quote from Director of National Intelligence James Clapper referring to estimates of the likelihood that bin Laden was living in Abbottabad as ‘percentage [s] of confidence’ (Manhunt, p.197). On the dangers of conflating likelihood and confidence in intelligence analysis more generally, see Jeffrey A. Friedman and Richard Zeckhauser, ‘Assessing Uncertainty in Intelligence’, Intelligence and National Security 27/6 (2012) pp.835-41.
45. On the WEPs and their origins, Wheaton, ‘Revolution Begins on Page Five’, pp.333-5.
46. To repeat, these statements are equivalent only from the standpoint of how decision makers should choose among options right now. In reality, decision makers must weigh immediate action against the potential costs and benefits of delaying to gather new information. In this respect, the estimates ‘between 0 and 100 per cent’ and ‘50 per cent’ may indeed have different interpretations, as the former literally relays no information (and thus a decision maker might be inclined to search for additional intelligence) while an estimate of ‘50 per cent’ could represent a careful weighing of evidence that is unlikely to shift much moving forward. This distinction shows why it is important to assess both likelihood and confidence, and why estimates of confidence should be tied directly to questions about whether decision makers should find it worthwhile to gather additional information. This is the subject of discussion below.
47. This is one of the central subjects of decision theory. Winkler, Introduction to Bayesian Inference and Decision, ch.6, and Raiffa, Decision Analysis, ch.7.
48. This holds constant the idea that the situation on the ground might change within the next month: bin Laden might have learned that he was being watched and might have fled, for instance, and that was something which the president reportedly worried about. This, however, is a matter of how much the state of the world might change, which is different from thinking about how assessments of the current situation might respond to additional information.
49. For broader discussions of this point, see Willis C. Armstrong et al., ‘The Hazards of SingleOutcome Forecasting’, Studies in Intelligence 28/3 (1984) pp.57-70
50. Friedman and Zeckhauser, ‘Assessing Uncertainty in Intelligence’, pp.829-34.
51. Ronald Howard argues that this is perhaps the fundamental takeaway from decision theory: ‘I tell my students that if they learn nothing else about decision analysis from their studies, this distinction will have been worth the price of admission. A good outcome is a future state of the world that we prize relative to other possibilities. A good decision is an action we take that is logically consistent with the alternatives we perceive, the information we have, and the preferences we feel. In an uncertain world, good decisions can lead to bad outcomes, and vice versa. If you listen carefully to ordinary speech, you will see that this distinction is usually not observed. If a bad outcome follows an action, people say that they made a bad decision. Making the distinction allows us to separate action from consequence and hence improve the quality of action.’ Ronald Howard, ‘Decision Analysis: Practice and Promise’, Management Science 34/6 (1988) p.682.
52. Richard K. Betts, Enemies of Intelligence: Knowledge and Power in American National Security (NY: Columbia University Press 2007)
53. Paul D. Pillar, Intelligence and US Foreign Policy: Iraq, 9/11, and Misguided Reform (NY: Columbia University Press 2011)
54. Bowden, The Finish, p.161.