Eliciting probabilities from experts

Advances in Decision Analysis: From Foundations to Applications

Volumn , Issue , 2007, Pages 129-153

  Hora, Stephen C ^a  

^a UNIVERSITY OF HAWAII AT HILO   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84925663537     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1017/CBO9780511611308.009     Document Type: Chapter

References (62)

1. Alpert, M. & Raiffa, H. (1982). A progress report on the training of probability assessors in judgment under uncertainty: Heuristics and biases. In D. Kahneman, P. Slovic, and A. Tversky, (Eds.) Judgment under uncertainty: Heuristics and biases, Cambridge, UK: Cambridge University Press.
2. Bar-Hillel, M. (2001). Subjective probability judgments, in Smelser, N. J. & Baltes, D. B. (Eds.) International Encyclopedia of the Social & Behavioral Sciences, Amsterdam: Elsevier Science Ltd. 15248–15251.
3. Bar-Hillel, M. & Neter, E. (1993). How alike is it versus how likely is it: Adisjunction fallacy in stereotype judgments. Journal of Personality and Social Psychology65, 1119–1132.
4. Bonano, E. J., Hora S. C., Keeney, R. L., and von Winterfeldt, D. (1990). Elicitation and Use of Expert Judgment in Performance Assessment for High-Level RadioactiveWaste Repositories. NUREG/CR-5411. Washington, DC: U.S. Nuclear Regulatory Commission.
5. Brenner, L. A., Koehler, D. J., Liberman, V. (1996). Overconfidence in probability and frequency judgments: A critical examination. Organizational Behavior and Human Decision Processes, 65, 212–219.
6. Christensen-Szalanski, J. & Bushyhead, J. (1981). Physicians’ Use of Probabilistic Information in a Real Clinical Setting. Journal of Experimental Psychology: Human Perception and Performance, 7, 928–935.
7. Clemen, R. T., Fischer, G.W., & Winkler, R. L. (2000). Assessing dependence: Some experimental results. Management Science, 46, 1100–1115.
8. Clemen, R. T. & Winkler, R. L. (1985). limits for the precision and value of information from dependent sources. Operations Research, 33, 427–442.
9. Clemen, R. T. & Reilly, T. (1999). Correlations and copulas for decision and risk analysis. Management Science, 45, 208–224.
10. Cooke, R. M. (1991). Experts in Uncertainty, Oxford: Oxford University Press.
11. Cover, T. M. & J. A. Thomas (1991). Elements of information theory. New York: Wiley-Interscience.
12. de Finetti, B. (1974). Theory of Probability, Vol. 1. New York: John Wiley and Sons.
13. de Finetti, B. (1977). Probabilities of probabilities: a real problem or a misunderstanding? in A. Aykac and C. Brumat, (Eds.) New Developments in the Application of Bayesian Methods, Amsterdam: North Holland Publishing Company.
14. Druzdel, M.J. (1989).Verbal Uncertainty Expressions: Literature Review, Technical Report CMU-EPP-1990–02–02. Pittsburgh: Department of Engineering, Carnegie Mellon University.
15. Fischoff, B. & MacGregor, D. (1982). Subjective Confidence in Forecasts. Journal of Forecasting, 1, 155–72.
16. Fischhoff, B., Slovic, P., & Lichtenstein, S. (1978). Fault trees: Sensibility of estimated failure probabilities to problem representation. Journal of Experimental Psychology: Human Perception and Performance, 4, 330–344.
17. Fox, C. R. & Clemen, R. T. (2005). Subjective probability assessment in decision analysis: Partition dependence and bias toward the ignorance prior. Management Science, 51, 1417–1432.
18. Fox, C. R. & Tversky, A. (1998). A belief-based account of decision under uncertainty, Management Science, 44, 879–95.
19. Gigerenzer, G. (1991). How to make cognitive illusions to disappear: Beyond heuristics and biases. In W. Stroebe & M. Hewstone (Ed.), European Review of Social Psychology, vol. 2. Chichester: Wiley.
20. Gigerenzer, G., Hoffrage, U., & Kleinbolting, H. (1991). Probabilistic mental models: A Brunswikian theory of confidence. Psychological Review, 98, 506–528.
21. Gilovich, T., Griffin, D., and Kahneman, D. (Eds.). (2002). Heuristics and biases: The psychology of intuitive judgment. Cambridge, UK: Cambridge University Press.
22. Haim, E. (1982). Characterization and Construction of Proper Scoring Rules. Unpublished doctoral dissertation, University of California, Berkeley.
23. Harper, F. T., Hora, S. C., Young, M. L., Miller, L. A., Lui, C. H., McKay, M. D., Helton, J. C., Goossens, L. H. J., Cooke, R. M., Pasler-Sauer, J., Kraan, B. & Jones, J. A. (1994). Probability Accident Consequence Uncertainty Analysis, Vols. 1–3, (NUREG/CR-6244, EUR 15855 EN)Brussels: USNRC and CEC DGXII.
24. Helton, J. C. & Burmaster, D. E., Eds. (1996). Special issue on treatment of aleatory and epistemic uncertainty. Reliability Engineering & System Safety, 54, 2–3.
25. Hora, S. C. (2004). Probability Judgments for Continuous Quantities: Linear Combinations and Calibration. Management Science, 50, 597–604.
26. Hora, S., Dodd, N. G., & Hora, J. (1993). The Use of Decomposition in Probability Assessments of Continuous Variables. The Journal of Behavioral Decision Making, 6, 133–147.
27. Hora, S. C., Hora, J. A., & Dodd, N. G. (1992). Assessment of Probability Distributions for Continuous Random Variables: A Comparison of the Bisection and Fixed Value Methods. Organizational Behavior and Human Decision Processes, 51, 133–155.
28. Hora, S. C. & Iman, R. L. (1989). Expert Opinion in Risk Analysis: The NUREG-1150 Experience. Nuclear Science and Engineering, 102, 323–331.
29. Hora, S. C. & Jensen, M. (2002). Expert Judgement Elicitation, Stockholm:. Swedish Radiation Protection Authority.
30. Juslin, P. (1994). The overconfidence phenomenon as a consequence of informal experimenter guided selection of almanac items. Organizational Behavior and Human Decision Processes, 57, 226–246.
31. Kahneman, D., Slovic, P., & Tversky, A. (Eds.) (1982). Judgment under uncertainty: Heuristics and biases. Cambridge, UK: Cambridge University Press.
32. Keeney, R., & von Winterfeldt, D. (1991). Eliciting probabilities from experts in complex technical problems. IEEE Transactions on Engineering Management, 38, 191–201.
33. Kent, S. (1964).Words of Estimated Probability, in D. P. Steury, ed., Sherman Kent and the Board of National Estimates: Collected Essays. Washington, DC: Center for the Study of Intelligence, 1994.
34. Keren, G. (1988). On the ability of monitoring nonvertical perceptions and uncertainty knowledge: some calibration studies. Acta Psychologica, 67, 95–119.
35. Klayman, J., Soll, J. B., Gonzalez-Vallejo, C., Barlas, S. (1999). Overconfidence: It depends on how, whom you ask. Organizational Behavior and Human Decision Processes, 79, 216–2247.
36. Knight, F. H. (1921). Risk, Uncertainty, and Profit. Boston: Houghton Mifflin Company.
37. Lichtenstein, S., & Fischhoff, B. (1977). Do those who know more also know more about how much they know? Organizational Behavior and Human Performance, 20, 159–183.
38. Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982). Calibration of probabilities: The state of the art to 1980, in D. Kahneman, P. Slovic, and A. Tversky, (Eds.) Judgment under uncertainty: Heuristics and biases, Cambridge, UK: Cambridge University Press.
39. Matheson, J. E. & Winkler, R. L. (1976). Scoring Rules for Continuous Probability Distributions, Management Science, 22, 1087–1096.
40. Merkhofer, M.W. (1987). Quantifying judgmental uncertainty: Methodology, experiences, and insights. IEEE Transactions on Systems, Man, and Cybernetics, 17, 741–752.
41. Morgan, M. G., Henrion, M. (1992). Uncertainty: A guide to dealing with uncertainty in quantitative risk and policy analysis, New York: Cambridge University Press.
42. Murphy, A. H. (1973). A New Vector Partition of the Probability Score. Journal of Applied Meteorology, 12595–6.
43. Murphy, A. H. & Winkler, R. L. (1977). Reliability of Subjective Probability Forecasts of Precipitation and Temperature. Applied Statistics, 26, 41–47.
44. Murphy, A. H. (1972). Scalar and Vector Partitions of the Probability Score (Part I) Twostate situation. Journal of Applied Meteorology, 11, 273–82.
45. Ortiz, N. R., Wheeler, T. A., Breeding, R. J., Hora, S., Meyer, M. A., & Keeney, R. L. (1991). The Use of Expert Judgment in the NUREG-1150. Nuclear Engineering and Design, 126, 313–331.
46. Raiffa, H. & Schlaifer, R. (1961). Applied Statistical Decision Theory, Cambridge: MIT Press.
47. Ravinder, H. V., Kleinmuntz, D. N., & Dyer, J. S. (1988). The reliability of subjective probability assessments obtained through decomposition. Management Science, 34, 186–199.
48. Savage, L. J. (1954). The Foundations of Statistics. New York: John Wiley and Sons (second edition, 1972, New York: Dover).
49. Savage, L. J. (1971). The elicitation of personal probabilities and expectations. Journal of the American Statistical Association, 66, 783–801.
50. Skyrms, B. (1980). “Higher order degrees of belief.” In “Prospects for Pragmatism: Essays in Honor of F. P. Ramsey, ” D. H. Mellor (Ed.), Cambridge University Press, Cambridge, pp. 109–137.
51. Sloman, S., Rottenstreich, Y., Wisniewski, E., Hadjichristidis, C., & Fox, C. R. (2004).Typical versus atypical unpacking and superadditive judgment. Journal of Experimental Psychology, 30, 573–582.
52. Spetzler, C. S. & Stael von Holstein, C. A. S. (1975). Probability Encoding in Decision Analysis. Management Science, 22, 340–358.
53. Stael von Holstein, C.A.S. & Matheson, J. E. (1979). A manual for encoding probability distributions. Menlo Park, CA: SRI International.
54. Stael von Holstein, C.A.S. (1972). Probabilistic forecasting: An experiment related to the stock market. Organizational Behavior and Human Performance, 8, 139–158.
55. Trauth, K. M., Hora S. C. & Guzowski, R.V. (1994).Aformal expert judgment procedure for performance assessments of the waste isolation pilot plant, SAND93–2450. Albuquerque: Sandia National Laboratories.
56. Tversky, A. & Kahneman D. (1973). Availability a Heuristic for Judging Probability, Cognitive Psychology5, 207–232. and in an abbreviated form in D. Kahneman, P. Slovic, and A. Tversky, (Eds.) Judgment under uncertainty: Heuristics and biases. Cambridge: Cambridge University Press.
57. Tversky, A. & Kahneman D. (1974). Judgment under uncertainty: Heuristics and biases. Science185, 1124–1131 and in D. Kahneman, P. Slovic, and A. Tversky, (Eds.) Judgment under uncertainty: Heuristics and biases. Cambridge: Cambridge University Press.
58. Tversky, A. & Koehler, D. J. (1994). Support theory: A nonextensional representation of subjective probability. Psychological Review, 101, 547–567.
59. U. S. Nuclear Regulatory Commission (1975). Reactor safety study: WASH-1400, NUREG- 751014. Washington: U. S. Nuclear Regulatory Commission.
60. Winkler, R. L. (1971). Probabilistic prediction: Some experimental results. Journal of the American Statistical Association, 66, 675–685.
61. Winkler, R. L. (1996). Scoring rules and the evaluation of probabilities. Test, 5, 1–60.
62. Winman, A., Hansson, P., & Juslin, P. (2004). Subjective probability intervals: How to cure overconfidence by interval evaluation. Journal of Experimental Psychology: Learning Memory and Cognition, 30, 1167–1175.