Trimmed opinion pools and the crowd's calibration problem

Management Science

Volumn 60, Issue 2, 2014, Pages 463-475

  Jose, Victor Richmond R ^a   Grushka Cockayne, Yael ^b   Lichtendahl Jr , Kenneth C ^b  

^a GEORGETOWN UNIVERSITY   (United States)

^b UNIVERSITY OF VIRGINIA   (United States)

Author keywords

Diversity; Expert combination; Linear opinion pool; Overconfidence; Probability forecasts; Scoring rules; Trimming; Underconfidence; Wisdom of crowds

Indexed keywords

EID: 84894327484     PISSN: 00251909     EISSN: 15265501     Source Type: Journal    
DOI: 10.1287/mnsc.2013.1781     Document Type: Article

References (24)

1. Armstrong SJ (2001) Combining forecasts. Armstrong SJ, ed. Principles of Forecasting: A Handbook for Researchers and Practitioners (Kluwer Academic Publishers, Norwell, MA), 417-439.
2. Boero G, Smith J, Wallis KF (2008) Uncertainty and disagreement in economic prediction: The Bank of England survey of external forecasters. Econom. J. 118:1107-1127.
3. Bowles C, Friz R, Genre V, Kenny G, Meyler A, Rautanen T (2007) The ECB survey of professional forecasters (SPF): A review after eight years' experience. Eur. Central Bank Occasional Paper Series 59:1-68.
4. Clemen RT (1989) Combining forecasts: A review and annotated. Internat. J. Forecasting 5:559-583.
5. Clemen RT (2008) Comment on Cooke's classical method. Reliability Engrg. System Safety 93:760-765.
6. Clemen RT, Winkler RL (1986) Combining economic forecasts. J. Bus. Econom. Statist. 4:39-46.
7. Clements MP (2010) Explanations of the inconsistencies in survey respondents' forecasts. Eur. Econom. Rev. 54:536-549.
8. Cooke RM (1991) Experts in Uncertainty: Opinion and Subjective Probability in Science (Oxford University Press, Oxford, UK).
9. Dawid AP, DeGroot MH, Mortera J (1995) Coherent combination of experts' opinions. Test 4:263-313.
10. Diebold FX, Tay AS, Wallis KF (1999) Evaluating density forecasts of inflation: The survey of professional forecasters. Engle R, White H, eds. Festschrift in Honor of C.W.J. Granger (Oxford University Press, Oxford, UK), 76-90.
11. Engelberg J, Manski CF, Williams J (2009) Comparing the point predictions and subjective probability distributions of professional forecasters. J. Bus. Econom. Statist. 27:30-41.
12. Giordani P, Söderlind P (2003) Inflation forecast uncertainty. Eur. Econom. Rev. 47:1037-1059.
13. Hora SC (2004) Probability judgments for continuous quantities: Linear combinations and calibration. Management Sci. 50:597-604.
14. Jose VRR, Winkler RL (2008) Simple robust average of forecasts: Some empirical results. Internat. J. Forecasting 24:163-169.
15. Larrick RP, Mannes AE, Soll JB (2011) The social psychology of the wisdom of crowds. Krueger JI, ed. Frontiers in Social Psychology: Social Judgment and Decision Making (Psychology Press, New York), 227-242.
16. Lichtendahl KC Jr, Grushka-Cockayne Y, Winkler RL (2013) Is it better to average probabilities or quantiles? Management Sci. 59:1594-1611.
17. O'Hagan AO, Buck CE, Daneshkhah A, Eiser JR, Garthwaite PH, Jenkinson DJ, Oakley JE, Rakow T (2006) Uncertain Judgements: Eliciting Experts' Probabilities (John Wiley & Sons, Chichester, UK).
18. Prescott P, Hogg RV (1977) Trimmed and outer means and their variances. Amer. Statistician 31:156-157.
19. Ranjan R, Gneiting T (2010) Combining probability forecasts. J. Roy. Statist. Soc. B 72:71-91.
20. Smith J, Wallis KF (2009) A simple explanation of the forecast combination puzzle. Oxford Bull. Econom. Statist. 71:331-355.
21. Soll JB, Mannes AE, Larrick RP (2013) The "wisdom of crowds" effect. Pashler H, ed. Encyclopedia of the Mind, Vol. 2 (Sage Publications, Los Angeles), 776-778.
22. Stone M (1961) The opinion pool. Ann. Math. Statist. 32:1339-1342.
23. Winkler RL (1996) Scoring rules and the evaluation of probabilities. Test 5:1-60.
24. Winkler RL, Jose VRR (2011) Scoring rules. Cohcran J, ed. Wiley Encyclopedia of Operations Research and Management Science, Vol. 7 (John Wiley & Sons, New York), 4733-4744.