Aggregating probability distributions

Advances in Decision Analysis: From Foundations to Applications

Volumn , Issue , 2007, Pages 154-176

  Clemen, Robert T ^a   Winkler, Robert L ^a  

^a DUKE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84925687318     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1017/CBO9780511611308.010     Document Type: Chapter

References (97)

1. Armstrong, J. S. (2001a). Combining forecasts. In J. S. Armstrong (Ed.), Principles of forecasting: A handbook for researchers and practitioners. Norwell, MA: Kluwer, pp. 417–439.
2. Armstrong, J. S. (Ed.). (2001b). Principles of forecasting: A handbook for researchers and practitioners. Norwell, MA: Kluwer.
3. Armstrong, J. S., Denniston, W.B., and Gordon, M. M. (1975). The use of the decomposition principle in making judgments. Organizational Behavior and Human Performance, 14, 257–263.
4. Arrow, K. J. (1951). Social choice and individual values. New York: Wiley.
5. Bacharach, M. (1979). Normal Bayesian dialogues. Journal of the American Statistical Association, 74, 837–846.
6. Bates, J. M., and Granger, C. W. J. (1969). The combination of forecasts. Operational Research Quarterly, 20, 451–468.
7. Bernreuter, D. L., Savy, J. B., Mensing, R. W., and Chen, J. C. (1989). Seismic hazard characterization of 69 nuclear sites east of the Rocky Mountains. Vol. 1–8, NUREG/CR 5250, UCID-21517.
8. Bunn, D. W. (1975). A Bayesian approach to the linear combination of forecasts. Operational Research Quarterly, 26, 325–329.
9. Bunn, D. W. (1988). Combining forecasts. European Journal of Operational Research, 33, 223–229.
10. Bunn, D., and Wright, G. (1991). Interaction of judgmental and statistical forecasting methods: Issues and analysis. Management Science, 37, 501–518.
11. Chandrasekharan, R., Moriarty, M. M., and Wright, G. P. (1994). Testing for unreliable estimators and insignificant forecasts in combined forecasts. Journal of Forecasting,13, 611–624.
12. Chhibber, S., and Apostolakis, G. (1993). Some approximations useful to the use of dependent information sources. Reliability Engineering and System Safety, 42, 67–86.
13. Clemen, R. T. (1989). Combining forecasts: A review and annotated bibliography. International Journal of Forecasting, 5, 559–583.
14. Clemen, R. T., Fischer, G. W., and Winkler, R. L. (2000). Assessing dependence: Some experimental results. Management Science, 46, 1100–1115.
15. Clemen, R. T., Jones, S. K., andWinkler, R. L. (1996). Aggregating forecasts: An empirical evaluation of some Bayesian methods. In D. Berry, K. Chaloner, and J. Geweke (Eds.), Bayesian statistics and econometrics: Essays in honor of Arnold Zellner. New York: Wiley, pp. 3–13.
16. Clemen, R. T., and Reilly, T. (1999). Correlations and copulas for decision and risk analysis. Management Science, 45, 208–224.
17. Clemen, R. T., andWinkler, R. L. (1985). Limits for the precision and value of information from dependent sources. Operations Research, 33, 427–442.
18. Clemen, R. T., and Winkler, R. L. (1987). Calibrating and combining precipitation probability forecasts. In R. Viertl (Ed.), Probability and Bayesian statistics. New York: Plenum, pp. 97–110.
19. Clemen, R. T., and Winkler, R. L. (1990). Unanimity and compromise among probability forecasters. Management Science, 36, 767–779.
20. Clemen, R. T., andWinkler, R. L. (1993). Aggregating point estimates: A flexible modeling approach. Management Science, 39, 501–515.
21. Clemen, R. T., andWinkler, R. L. (1999). Combining probability distributions from experts in risk analysis. Risk Analysis, 19, 187–203.
22. Cooke, R. M. (1991). Experts in uncertainty: Opinion and subjective probability in science. New York: Oxford University Press.
23. Dall’Aglio, G., Kotz, S., and Salinetti, G. (1991). Advances in probability distributions with given marginals: Beyond the copulas. Dordrecht, The Netherlands: Kluwer.
24. Davis, J. (1992). Some compelling intuitions about group consensus decisions, theoretical and empirical research, and interpersonal aggregation phenomena: Selected examples, 1950–1990. Organizational Behavior and Human Decision Processes,52, 3–38.
25. Dawes, R. M., Faust, D., and Meehl, P. A. (1989). Clinical versus actuarial judgment. Science, 243, 1668–1673.
26. de Finetti, B. (1937). La prévision: Ses lois logiques, ses sources subjectives. Annales de l’Institut Henri Poincaré, 7, 1–68. Translated in 1980 by H. E. Kyburg, Jr., Foresight. Its logical laws, its subjective sources. In H. E. Kyburg, Jr. and H. E. Smokler (Eds.), Studies in subjective probability (2nd ed.). Huntington, New York: Robert E. Krieger, pp. 53–118.
27. Dillon, R., John, R., and von Winterfeldt, D. (2002). Assessment of cost uncertainties for large technology projects: A methodology and an application. Interfaces, 32(Jul/Aug), 52–66.
28. EPRI (1986). Seismic hazard methodology for the central and eastern United States. Vol. 1: Methodology. NP-4/26. Palo Alto, CA: Electric Power Research Institute.
29. Flores, B. E., and White, E. M. (1989). Subjective vs. objective combining of forecasts: An experiment. Journal of Forecasting, 8, 331–341.
30. French, S. (1981). Consensus of opinion. European Journal of Operational Research, 7, 332–340.
31. French, S. (1985). Group consensus probability distributions: A critical survey. In J. M. Bernardo, M. H. DeGroot, D. V. Lindley, and A. F. M. Smith (Eds.), Bayesian statistics2. Amsterdam: North-Holland, pp. 183–197.
32. French, S., and R´ıos Insua, D. (2000). Statistical decision theory. London: Arnold.
33. Garthwaite, P. H., Kadane, J. B., and O’Hagan, A. (2005). Statistical methods for eliciting prior distributions. Journal of the American Statistical Association, 100, 680–700.
34. Gelfand, A. E., Mallick, B. K., and Dey, D. K. (1995). Modeling expert opinion rising as a partial probabilistic specification. Journal of the American Statistical Association, 90, 598–604.
35. Genest, C. (1984). Pooling operators with the marginalization property. Canadian Journal of Statistics, 12, 153–163.
36. Genest, C., and McConway, K. J. (1990). Allocating the weights in the linear opinion pool. Journal of Forecasting, 9, 53–73.
37. Genest, C., and Schervish, M. J. (1985). Modeling expert judgments for Bayesian updating. Annals of Statistics, 13, 1198–1212.
38. Genest, C., and Zidek, J. V. (1986). Combining probability distributions: A critique and annotated bibliography. Statistical Science, 1, 114–148.
39. Gigone, D., and Hastie, R. (1997). Proper analysis of the accuracy of group judgments. Psychological Bulletin, 121, 149–167.
40. Gokhale, D. V., and Press, S. J. (1982). Assessment of a prior distribution for the correlation coefficient in a bivariate normal distribution. Journal of the Royal Statistical Society, Series A, 145, 237–249.
41. Goodman, B. (1972). Action selection and likelihood estimation by individuals and groups. Organizational Behavior and Human Performance, 7, 121–141.
42. Hammitt, J. K., and Shlyakhter, A. I. (1999). The expected value of information and the probability of surprise. Risk Analysis, 19, 135–152.
43. Hastie, R. (1986). Experimental evidence on group accuracy. In B. Grofman and G. Owen (Eds.), Information pooling and group decision making. Greenwich, CT: JAI Press, pp. 129–157.
44. Hill, G. W. (1982). Group vs. individual performance: Are N + 1 heads better than one? Psychological Bulletin, 91, 517–539.
45. Hogarth, R. M. (1987). Judgment and choice. (2nd ed.). Chichester, England: Wiley.
46. Hora, S. C. (1992). Acquisition of expert judgment: Examples from risk assessment. Journal of Energy Engineering, 118, 136–148.
47. Hora, S. C. (2004). Probability judgments for continuous quantitites: Linear combinations and calibration. Management Science, 50, 597–604.
48. Hora, S. C., Dodd, N. G., and Hora, J. A. (1993). The use of decomposition in probability assessments on continuous variables. Journal of Behavioral Decision Making, 6, 133–147.
49. Hora, S. C., and Iman, R. L. (1989). Expert opinion in risk analysis: The NUREG-1150 methodology. Nuclear Science and Engineering, 102, 323–331.
50. Jouini, M. N., and Clemen, R. T. (1996). Copula models for aggregating expert opinions. Operations Research, 44, 444–457.
51. Kahneman, D., and Tversky, A. (1972). Subjective probability: A judgment of representativeness. Cognitive Psychology, 3, 430–454.
52. Keeney, R. L., and von Winterfeldt, D. (1989). On the uses of expert judgment on complex technical problems. IEEE Transactions on Engineering Management, 36, 83–86.
53. Keeney, R. L., and von Winterfeldt, D. (1991). Eliciting probabilities from experts in complex technical problems. IEEE Transactions on Engineering Management, 38, 191–201.
54. Lacke, C. J., and Clemen, R. T. (2001). Analysis of colorectal cancer screening regimens. Health Care Management Science, 4, 257–267.
55. Larrick, R. P., and Soll, J. B. (2006). Intuitions about combining opinions: Misappreciation of the averaging principle. Management Science, 52, 111–127.
56. Lawrence, M. J., Edmundson, R. H., and O’Connor, M. J. (1986). The accuracy of combining judgmental and statistical forecasts. Management Science, 32, 1521–1532.
57. Lichtendahl, K. C. (2005). Bayesian nonparametric combination of multivariate expert forecasts. Ph.D. dissertation, Duke University.
58. Lindley, D. V. (1983). Reconciliation of probability distributions. Operations Research, 31, 866–880.
59. Lindley, D.V. (1985). Reconciliation of discrete probability distributions. In J. M. Bernardo, M. H. DeGroot, D.V. Lindley, and A.F. M. Smith (Eds.), Bayesian statistics2. Amsterdam: North-Holland, pp. 375–390.
60. Lipscomb, J., Parmigiani, G., and Hasselblad, V. (1998). Combining expert judgment by hierarchical modeling: An application to physician staffing. Management Science, 44, 149–161.
61. MacGregor, D., Lichtenstein, S., and Slovic, P. (1988). Structuring knowledge retrieval: An analysis of decomposing quantitative judgments. Organizational Behavior and Human Decision Processes, 42, 303–323.
62. Merkhofer, M. W. (1987). Quantifying judgmental uncertainty: Methodology, experience, and insights. IEEE Transactions on Systems, Man, and Cybernetics, 17, 741–752.
63. Merrick, J. R. W., van Dorp, J. R., and Singh, A. (2005). Analysis of correlated expert judgments from extended pairwise comparisons. Decision Analysis, 2, 17–29.
64. Morgan, M. G., and Henrion, M. (1990). Uncertainty: A guide to dealing with uncertainty in quantitative risk and policy analysis. Cambridge, MA: Cambridge University Press.
65. Morgan, M. G., and Keith, D. W. (1995). Subjective judgments by climate experts. Envirionmental Science and Technology, 29, 468–476.
66. Morris, P. A. (1974). Decision analysis expert use. Management Science, 20, 1233–1241.
67. Morris, P. A. (1977). Combining expert judgments: A Bayesian approach. Management Science, 23, 679–693.
68. Morris, P. A. (1983). An axiomatic approach to expert resolution. Management Science, 29, 24–32.
69. Mosleh, A., Bier, V. M., and Apostolakis, G. (1987). A critique of current practice for the use of expert opinions in probabilistic risk assessment. Reliability Engineering and System Safety, 20, 63–85.
70. Newbold, P., and Granger, C. W. J. (1974). Experience with forecasting univariate time series and the combination of forecasts. Journal of the Royal Statistical Society, Series A, 137, 131–149.
71. Otway, H., and von Winterfeldt, D. (1992). Expert judgment in risk analysis and management: Process, context, and pitfalls. Risk Analysis, 12, 83–93.
72. Ravinder, H. V., Kleinmuntz, D. N., and Dyer, J. S. (1988). The reliability of subjective probabilities obtained through decomposition. Management Science, 34, 186–199.
73. Reagan-Cirincione, P. (1994). Improving the accuracy of group judgment; A process intervention combining group facilitation, social judgment analysis, and information technology. Organizational Behavior and Human Decision Processes, 58, 246–270.
74. Rohrbaugh, J. (1979). Improving the quality of group judgment: Social judgment analysis and the Delphi technique. Organizational Behavior and Human Performance, 24, 73–92.
75. Savage, L. J. (1954). The foundations of statistics. New York: Wiley.
76. Schmittlein, D. C., Kim, J., and Morrison, D. G. (1990). Combining forecasts: Operational adjustments to theoretically optimal rules. Management Science, 36, 1044–1056.
77. Seaver, D. A. (1978). Assessing probability with multiple individuals: Group interaction versus mathematical aggregation. Report No. 78–3, Social Science Research Institute, University of Southern California.
78. Shlyakhter, A. I. (1994). Improved framework for uncertainty analysis: Accounting for unsuspected errors. Risk Analysis, 14, 441–447.
79. Shlyakhter, A. I., Kammen, D. M., Brodio, C. L., and Wilson, R. (1994). Quantifying the credibility of energy projections from trends in past data: The U. S. energy sector. Energy Policy, 22, 119–130.
80. Sniezek, J. A. (1989). An examination of group process in judgmental forecasting. International Journal of Forecasting, 5, 171–178.
81. Spetzler, C. S., and Staël von Holstein, C.-A. S. (1975). Probability encoding in decision analysis. Management Science, 22, 340–358.
82. Staël von Holstein, C.-A. S. (1972). Probabilistic forecasting: An experiment related to the stock market. Organizational Behavior and Human Performance, 8, 139–158.
83. Stone, M. (1961). The opinion pool. Annals of Mathematical Statistics, 32, 1339–1342.
84. West, M. (1988). Modelling expert opinion. In J. M. Bernardo, M. H. DeGroot, D.V. Lindley, and A. F. M. Smith (Eds.), Bayesian statistics, 3, Amsterdam: New Holland, pp. 493–508.
85. West, M. (1992). Modelling agent forecast distributions. Journal of the Royal Statistical Society, Series B, 54, 553–567.
86. West, M., and Crosse, J. (1992). Modelling probabilistic agent opinion. Journal of the Royal Statistical Society, Series B, 54, 285–299.
87. Winkler, R. L. (1968). The consensus of subjective probability distributions. Management Science, 15, 361–375.
88. Winkler, R. L. (1981). Combining probability distributions from dependent information sources. Management Science, 27, 479–488.
89. Winkler, R. L. (1986). Expert resolution. Management Science, 32, 298–303.
90. Winkler, R. L., and Clemen, R. T. (1992). Sensitivity of weights in combining forecasts. Operations Research, 40, 609–614.
91. Winkler, R. L., and Clemen, R. T. (2004). Multiple experts vs. multiple methods: Combining correlation assessments. Decision Analysis, 1, 167–176.
92. Winkler, R. L., and Makridakis, S. (1983). The combination of forecasts. Journal of the Royal Statistical Society, Series A, 146, 150–157.
93. Winkler, R. L., and Poses, R. M. (1993). Evaluating and combining physicians’ probabilities of survival in an intensive care unit. Management Science, 39, 1526–1543.
94. Winkler, R. L., Wallsten, T. S., Whitfield, R. G., Richmond, H. M., Hayes, S. R., and Rosenbaum, A. S. (1995). An assessment of the risk of chronic lung injury attributable to long-term ozone exposure. Operations Research, 43, 19–28.
95. Wright, G., and Ayton, P. (Eds). (1987). Judgmental forecasting. Chichester, UK: Wiley.
96. Wright, G., Saunders, C., and Ayton, P. (1988). The consistency, coherence, and calibration of holistic, decomposed, and recomposed judgmental probability forecasts. Journal of Forecasting,7, 185–199.
97. Wu, J. S., Apostolakis, G., and Okrent, D. (1990). Uncertainties in system analysis: Probabilistic versus nonprobabilistic theories. Reliability Engineering and System Safety, 30, 163–181.