Communicating uncertainty in intelligence and other professions

International Journal of Intelligence and CounterIntelligence

Volumn 21, Issue 1, 2007, Pages 57-85

  Weiss, Charles ^a  

^a GEORGETOWN UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 57749134990     PISSN: 08850607     EISSN: 15210561     Source Type: Journal    
DOI: 10.1080/08850600701649312     Document Type: Article

References (57)

1. Michael Schrage, “What Percent is ‘Slam Dunk’? Give Us Odds on Those Estimates,” The Washington Post, 20 February 2005, p. B1.
2. Steven Marrin and Jonathan D. Clemente, “Improving Intelligence Analysis by Looking to the Medical Profession,” International Journal of Intelligence and Counterintelligence, Vol. 18, No. 4, Winter 2005-2006, pp. 707-729.
3. Jack Davis et al., “A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis,” Tradecraft Review, Vol. 2, pp. 1-43
4. Rob Johnston, Analytic Culture and the US Intelligence Community (Washington: Central Intelligence Agency, 2005), pp. 13, 21
5. Richards, Heuer., Jr., Psychology of Intelligence Analysis (Washington, DC: Center for the Study of Intelligence, Central Intelligence Agency, 1999). Available at https://www.cia.gov/csi/books/19104/index.html;
6. Steven Marrin, “Improving the CIA’s Analysis,” letter to The Washington Times, 21 February 2004. Available at http://www.washingtontimes.com/op-ed/20040221-112044-8356r.htm.
7. Joel Primack and Frank von Hippel, Advice and Dissent: Scientists in the Political Arena (New York: Basic Books, 1974).
8. Sheila Jasanoff, The Fifth Branch: Science Advisers as Policy Makers (Cambridge, MA: Harvard University Press, 1990).
9. Jonathan Koehler, “The Influence of Prior Beliefs on Scientific Judgments of Evidence Quality,” Journal of Organizational Behavior and Human Decision Processes, Vol. 56, 1993, pp. 28-56. Available at http://www.mccombs.utexas.edu/faculty/jonathan.koehler/articles/1993_OBHDP.asp
10. Max Planck, Scientific Autobiography and Other Papers, translated by F. Gaynor (New York: Philosophical Library, 1949), pp. 33-34
11. Steven Marrin and Jonathan D. Clemente, “Improving Analysis by Looking to the Medical Profession.”
12. Simon Shackley, Brian Wynne, Jeroen van der Sluijs, and Josee van Eijndhoven, “Anchoring Devices in Science for Policy: The Case of Consensus Around Climate Sensitivity,” Social Studies of Science, Vol. 28, 1998, pp. 291-323.
13. Woo Suk Hwang et al., “Patient-Specific Embryonic Stem Cells Derived from Human SCNT Blastocysts,” Science, Vol. 308, 17 June 2005, pp. 1777-1783.
14. David Michaels and Celeste Montforton, “Manufacturing Uncertainty: Contested Science and the Protection of the Public’s Health and Environment,” American Journal of Public Health, Vol. 95 (S1), 2005, pp. S39-S48.
15. Jack Davis et al., “A Tradecraft Prime: Structured Analytic Techniques for Improving Intelligence Analysis,” p. 34.
16. Richards Heuer, Psychology of Intelligence Analysis.
17. M. Valtorta, M. Huhns, J. Dang, H. Goradia, and J. Huang, “Extending Heuer’s Analysis of Competing Hypotheses Method to Support Complex Decision Analysis,” USC Technical Report, USC CSE TR-2005-001. Available at http://www.cse.sc.edu/~mgv/reports/IA-05.pdf.
18. National Intelligence Strategy of the United States, Transformation through Integration and Innovation (Washington: Office of the Director of National Intelligence, 2005). Available at http://www.dni.gov/pressreleases/20051025_ release.htm and http://www.dni.gov/publications/NISOctober2005.pdf
19. L. Jonathan Cohen, The Knowable and the Provable (Oxford: Clarendon Press, 1977), p. 74.
20. Alfred S. Evans, “Causation and Disease: the Henle-Koch Postulates Revisited,” Yale Journal of Biology and Medicine, Vol. 175, 1976, pp. 175-195.
21. Donald W. Miller and Clifford G. Miller, “On Evidence, Medical and Legal,” Journal of American Physicians and Surgeons, Vol. 10, 2005, pp. 70-75.
22. D.L. Sackett, S.E. Straus, W.S. Richardson et al., Evidence-Based Medicine: How to Practice and Teach EBM, 2d ed. (Edinburgh: Churchill Livingstone, 2000).
23. Mark Lowenthal, Intelligence: From Secrets to Policy (Washington, D.C.: Congressional Quarterly Press, 2006), p. 139.
24. Sherman Kent, “Words of Estimative Probability,” Studies in Intelligence, 1964. Available at https://www.cia.gov/csi/books/shermankent/6words.html. Kent cites several non-quantitative scales in addition to his own (p. 7).
25. Rob Johnston, Analytic Culture and the U.S. Intelligence Community, p. 42.
26. Rob Johnston, Analytic Culture and the US Intelligence Community, pp. 5-6, and 23;
27. Mark Lowenthal, Intelligence: From Secrets to Policy, pp. 121-122, 129, and 132.
28. Stanley Feder, “Forecasting for Policy Making in the Post Cold War Period,” Annual Review of Political Science, Vol. 5, 2002, pp. 111-125;
29. Richards Heuer, Psychology of Intelligence Analysis.
30. Steven Rieber, “Intelligence Analysis and Judgmental Calibration,” International Journal of Intelligence and CounterIntelligence, Vol. 17, No. 1, Spring 2004, pp. 97-112
31. Charles Weiss, “Expressing Scientific Uncertainty,” Law, Probability and Risk, Vol. 2, 2003, pp. 25-46.
32. Kenneth Green, 1997, “A Plain English Guide to the Science of Climate Change,” Reason Public Policy Institute: Los Angeles. Policy Study, No. 237. Available at http://www.reason.org/ps237.html.
33. R.H. Moss and S.H. Schneider, “Uncertainties in the IPCC: Recommendations to Lead Authors For More Consistent Assessment and Reporting,” in R. Pachauri, T. Taniguchi, and K. Tanaka, eds., Guidance Papers on the Cross Cutting Issues of the Third Assessment Report of the IPCC (Geneva: World Meteorological Organization, 2000), pp. 33-51.
34. David Michaels and Celeste Montforton, “Manufacturing Uncertainty,” pp. S39-S48
35. Al Gore, An Inconvenient Truth: The Planetary Emergency of Global Warming and What We Can Do About It (Emmaus, PA: Rodale Press, 2006), pp. 284-287.
36. C.M.A. McCauliff, “Burdens of Proof: Degrees of Belief, Quanta of Evidence, or Constitutional Guarantees?” Vanderbilt Law Review, Vol. 35, 1982, pp. 1293-1335.
37. Charles Weiss, “Precaution: The Willingness to Accept Costs to Avert Uncertain Danger,” in Kurt Marti, Yuri Ermolev, Marek Makowski, and Georg Pflug, eds., Coping with Uncertainty: Modeling and Policy Issues (Berlin: Springer Verlag, 2006).
38. Robert T. Reagan, Frederick Mosteller and Cleo Youtz, “Quantitative Meanings of Verbal Probability Expressions,” Journal of Applied Psychology, 1989, Vol. 74, pp. 433-442.
39. Ron Suskind, The One Percent Doctrine: Deep Inside America's Pursuit of Its Enemies after 9/11 (New York: Simon and Schuster, 2006), p. 62.
40. Jack Davis, “Tensions in Analyst-Policy Maker Relations: Opinions, Facts and Science” (Sherman Kent School for Intelligence Analysis, Occasional Papers, Vol. 2, No. 2, January, 2003). Available at https://www.cia.gov/cia/publications/Kent_Papers/vol2no2.htm, accessed 18 October 2006.
41. R. McDonald, “Juries and Crime Labs: Correcting the Weak Links in the DNA Chain,” American Journal of Law & Medicine, Vol. 24, 1998, pp. 345-363.
42. David Harney, Medical Malpractice, 3d ed. (Charlottesville, VA: Michie Co., 1993), pp. 25ff.
43. Frank M. McClellan, Medical Malpractice: Law, Tactics and Ethics (Philadelphia: Temple University Press, 1994), p. 23.
44. D. Michael Risinger and Michael J. Ellis, “A House With No Foundation,” Issues in Science and Technology, Fall 2003, pp. 35-39.
45. Bruce Budowle, JoAnn Busaglia, and Rebecca S. Perlman, “Review of the Scientific Basis for Function Ridge Comparisons as a Means of Identification: Committee Findings and Recommendations,” Forensic Science Communications, January 2006, available at http://www.fbi.gov/hq/lab/fsc/backissu/jan2006/research/2006_01_research02.htm (accessed 5/23/2006).
46. Anthony Patt and Suraje Dessai, “Communicating Uncertainty: Lessons Learned and Suggestions for Climate Change Assessment,” Geoscience, Vol. 337, 2005, pp. 425-41.
47. Penny Kloprogge, Jeroen P. van der Sluijs, and A. Wardekker, 2007, “Uncertainty Communication: Issues and Good Practice,” Copernicus Institute for Sustainable Development and Innovation, Utrecht University, pp. 9-10.
48. Anthony Patt and Daniel Schrag, “Using Specific Language to Describe Risk and Probability,” Climatic Change, Vol. 61, 2003, pp. 17-30.
49. Anthony Patt and Suraje Dessai, “Communicating Uncertainty: Lessons Learned and Suggestions for Climate Change Assessment,” Geoscience, Vol. 337, 2005, pp. 425; Vol. 441.
50. Penny Kloprogge and Jeroen van der Sluijs, “Uncertainly Communication,” p. 11.
51. Anthony Patt and Suraje Dessai, “Communicating Uncertainty,” p. 429.
52. Penny Kloprogge and Jeroen van der Sluijs, “Uncertainity Communication,” p. 12.
53. Anthony Patt and Suraje Dessai, “Communicating Uncertainty,” p. 437.
54. Penny Kloprogge and Jeroen van der Sluijs, “Uncertainity Communication,” p. 13.
55. Jeroen P. van der Sluijs, Matthieu Craye, Silvio Funtowicz, Penny Kloprogge, Jerry Ravetz, and James Risbey, “Combining Quantitative and Qualitative Measures of Uncertainty in Model-based Environmental Assessment: The NUSAP System.” Risk Analysis, Vol. 25, 2005, pp. 481-492.
56. M. Granger Morgan and Max Henrion, Uncertainty: A Guide to Dealing with Uncertainty, Quantitative Risk and Policy Analysis (Cambridge, UK: Cambridge University Press, 1990).
57. Harald Ibrekk and M. Granger Morgan, “Graphical Communication of Uncertain Quantities to Non-technical People,” Risk Analysis, Vol. 7, 1987, pp. 519-529.