Problems with scoring methods and ordinal scales in risk assessment

IBM Journal of Research and Development

Volumn 54, Issue 3, 2010, Pages

  Hubbard, D ^a   Evans, D ^b  

^a Hubbard Decision Research   (United States)

^b UNIVERSITY COLLEGE CORK   (Ireland)

Author keywords

[No Author keywords available]

Indexed keywords

DIFFERENT DOMAINS; HUMAN ERRORS; MATHEMATICAL PROBABILITY; ORDINAL SCALE; PERCEIVED BENEFITS; RISK ASSESSMENT METHODS; RISK FACTORS; SCORING METHODS;

ERRORS; RISK ANALYSIS; RISK MANAGEMENT; SAFETY FACTOR;

RISK ASSESSMENT;

EID: 77955064258     PISSN: 00188646     EISSN: 00188646     Source Type: Journal    
DOI: 10.1147/JRD.2010.2042914     Document Type: Article

References (40)

1. J. D. Van Vactor, "Risk mitigation through a composite risk management process: The U.S. army risk assessment," Org. Dev. J., vol. 27, no. 4, pp. 84-97, 2009.
2. United States Department of Health and Human Services, Draft Guidance on Allocating and Targeting Pandemic Influenza Vaccine, 2007. [Online]. Available: http://www.pandemicflu.gov/ vaccine/prioritization.html
3. J. V. Turner, "Risk management on the space shuttle upgrades development program," in Proc. Joint ESA-NASA Space-Flight Safety Conf., B. Battrick and C. Preyssi, Eds., 2002, European Space Agency, ESA SP-486.
4. D. Hubbard, The Failure of Risk Management: Why It's Broken and How to Fix It. Hoboken, NJ: Wiley, 2009, pp. 237-238.
5. Risk Management Guide for Information Technology Systems, NIST 800-830, Nat. Inst. Stand. Technol., Gaithersburg, MD, 2002. [Online]. Available: http://csrc.nist.gov/publications/ nistpubs/800-30/sp800-30.pdf.
6. A Guide to the Project Management Body of Knowledge, Project Manage. Inst., Newtown Square, PA, 2008.
7. D. Ariely, Predictably Irrational: The Hidden Forces That Shape Our Decisions. London, U.K.: Harper Collins, 2008.
8. D. Kahneman, P. Slovic, and A. Tversky, Judgement Under Uncertainty: Heuristics and Biases. Cambridge, U.K.: Cambridge Univ. Press, 1982.
9. D. Kahneman and A. Tversky, "Subjective probability: A judgement of representativeness," Cogn. Psychol., vol. 3, no. 3, pp. 430-454, Jul. 1972.
10. D. Kahneman and A. Tversky, "On the psychology of prediction," Psychol. Rev., vol. 80, no. 4, pp. 237-251, Jul. 1973.
11. E. Brunswick, "Representative design and probabilistic theory in a functional psychology," Psychol. Rev., vol. 62, no. 3, pp. 193-217, May 1955.
12. N. Karelaia and R. M. Hogarth, "Determinants of linear judgement: A meta-analysis of lens studies," Psychol. Bull., vol. 134, no. 3, pp. 404-426, May 2008.
13. D. G. Ullman and M. E. Doherty, "Two determinants of the diagnosis of hyperactivity: The child and the clinician," Adv. Dev. Behav. Pediatr., vol. 5, pp. 201-211, 1984.
14. P. Hoffman, P. Slovic, and L. Rorer, "An analysis of variance model for an assignment of configural cue utilization in clinical judgment," Psychol. Bull., vol. 69, no. 5, pp. 338-349, May 1968.
15. C. P. Bradley, "Can we avoid bias?" Brit. Med. J., vol. 330, no. 7494, p. 784, Apr. 2005.
16. D. V. Budescu, S. Broomell, and H.-H. Por, "Improving communication of uncertainty in the reports of the intergovernmental panel on climate change," Psychol. Sci., vol. 20, no. 3, pp. 299-308, Mar. 2009.
17. S. S. Stevens, "On the theory of scales of measurement," Science, vol. 103, no. 2684, pp. 677-680, Jun. 1946.
18. L. A. Cox, Jr., "What's wrong with risk matrices?" Risk Anal., vol. 28, no. 2, pp. 497-512, Apr. 2008.
19. M. Parker, R. Benson, and H. Trainor, Information Economics: Linking Business Performance to Information Technology. Englewood Cliffs, NJ: Prentice-Hall, 1988.
20. J. V. Neumann and O. Morgenstern, Theory of Games and Economic Behavior. Princeton, NJ: Princeton Univ. Press, 1944.
21. J. S. Dyer, "Remarks on the analytic hierarchy process," Manage. Sci., vol. 36, no. 3, pp. 249-258, Mar. 1990.
22. J. S. Dyer, "A clarification of FRemarks on the analytic hierarchy process," Manage. Sci., vol. 36, no. 3, pp. 274-275, Mar. 1990.
23. R. D. Holder, "Some comment on the analytic hierarchy process," J. Oper. Res. Soc., vol. 41, no. 11, pp. 1073-1076, Nov. 1990.
24. R. D. Holder, "Response to holder's comments on the analytic hierarchy process: Response to the response," J. Oper. Res. Soc., vol. 42, no. 10, pp. 914-918, Oct. 1991.
25. S. Schenkerman, "Inducement of nonexistent order by the analytic hierarchy process," Decis. Sci., vol. 28, no. 2, pp. 475-482, Apr. 1997.
26. J. Pérez, "Some comments on Saaty's AHP," Manage. Sci., vol. 41, no. 6, pp. 1091-1095, Jun. 1995.
27. J. Pérez, J. Jimeno, and E. Mokotoff, "Another potential shortcoming of AHP," TOP, vol. 14, no. 1, pp. 99-111, Jun. 2006.
28. P. J. H. Schoemaker and C. C. Waid, "An experimental comparison of different approaches to determining weights in additive utility models," Manage. Sci., vol. 28, no. 2, pp. 182-196, Feb. 1982.
29. C. Tsai, J. Klayman, and R. Hastie, "Effects of amount of information on judgment accuracy and confidence," Org. Behavior Human Decis. Process., vol. 107, no. 2, pp. 97-105, Nov. 2008.
30. C. Heath and R. Gonzalez, "Interaction with others increases decision confidence but not decision quality: Evidence against information collection views of interactive decision making," Org. Behavior Human Decis. Process., vol. 61, no. 3, pp. 305-326, Mar. 1995.
31. M. L. Williams, A. R. Dennis, A. Stam, and J. E. Aronson, "The impact of DSS use and information load on errors and decision quality," Eur. J. Oper. Res., vol. 176, no. 1, pp. 468-481, Jan. 2007.
32. B. Flyvbjerg, Procedures for Dealing With Optimism Bias in Transport Planning. London, U.K.: Brit. Dept. Transp., 2004.
33. A. H. Murphy and R. L. Winkler, "Can weather forecasters formulate reliable forecasts of precipitation and temperature?" Nat. Weather Dig., vol. 2, no. 2, pp. 2-9, 1977.
34. D. Hubbard, How to Measure Anything: Finding the Value of Intangibles in Business. Hoboken, NJ: Wiley, 2007.
35. S. Lichtenstein, B. Fischhoff, and L. D. Phillips, "Calibration of probabilities: The state of the art to 1980," in Judgement Under Uncertainty: Heuristics and Biases, D. Kahneman, P. Slovic, and A. Tversky, Eds. Cambridge, U.K.: Cambridge Univ. Press, 1982, pp. 306-334.
36. J. M. Hampton, P. G. Moore, and H. Thomas, "Subjective probability and its measurement," J. R. Stat. Soc., Ser. A (General), vol.136, no.1, pp. 21-42, 1973.
37. E. Servan-Schreiber, J. Wolfers, D. M. Pennock, and B. Galebach, "Prediction markets: Does money matter?" Electron. Markets, vol. 14, no. 3, pp. 243-251, Sep. 2004.
38. D. Bearden, C. Freaner, R. Bitten, and D. Emmons, BAn assessment of the inherent optimism in early conceptual designs and its effect on cost and schedule growth,[ in Proc. SSCAG/ SCAF/EACE Joint Int. Conf., Noordwijk, The Netherlands, 2008.
39. G. S. Simpson, F. E. Lamb, J. H. Finch, and N. C. Dinnie, BThe application of probabilistic and qualitative methods to asset management decision making,[ presented at the Proc. SPA Asia Pacific Conf. Integr. Modeling Asset Manage., Yokohama, Japan, 2000, Paper 59455.
40. W. Bailey, B. Couet, F. Lamb, G. Simpson, and P. Rose, "Taking a calculated risk," Oilfield Rev., vol. 12, no. 3, pp. 20-35, 2000.