Confidence in aggregation of opinions from multiple sources

Information Sampling and Adaptive Cognition

Volumn , Issue , 2005, Pages 327-352

  Budescu, David V ^a  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84926118331     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1017/CBO9780511614576.014     Document Type: Chapter

References (62)

1. Anderson, N. H. (1981). Information integration theory. Hillsdale, NJ: Lawrence Erlbaum.
2. Ariely, D., Au, W. T., Bender, R. H., Budescu, D. V., Dietz, C. B., Gu, H., Wallsten, T. S., & Zauberman, G. (2000). The effects of averaging subjective probability estimates between and within judges. Journal of Experimental Psychology: Applied, 6, 130–147.
3. Armstrong, J. S. (2001). Combining forecasts. In J. S. Armstrong (Ed.), Principles of forecasting: A handbook for researchers and practitioners. Norwell, MA: Kluwer Academic Publishers.
4. Arrow, K. (1963). Social choice and individual values (2nd edition). New Haven, CT: Yale University Press.
5. Ashton, R. H. (1986). Combining the judgments of advisors: How many and which ones? Organizational Behavior and Human Decision Processes, 38, 405–414.
6. Black, D. (1958). The theory of committees and elections. Cambridge, UK: Cambridge University Press.
7. Bordley, R. F. (1982). A multiplicative formula for aggregating probability assessments. Management Science, 28, 1137–1148.
8. Budescu, D. V. (2001). Aggregation of probabilistic forecasts and opinions. Presidential address at the annual meeting of the Society for Judgment and Decision Making (SJDM), Orlando, FL.
9. Budescu, D. V., Erev, I., & Wallsten, T. S. (1997). On the importance of random error in the study of probability judgment, Part I: New theoretical developments. Journal of Behavioral Decision Making, 10, 157–172.
10. Budescu, D. V., & Rantilla, A. K. (2000) Confidence in aggregation of expert opinions. Acta Psychologica, 104, 371–398.
11. Budescu, D. V., Rantilla, A. K, Yu, H., & Karelitz, T. M. (2003). The effects of asymmetry among advisors on the aggregation of their opinions. Organizational Behavior and Human Decision Processes, 90, 178–194.
12. Chapman, L. J., & Chapman, J. P. (1967). Genesis of popular but erroneous psychodiagnostic observations. Journal of Abnormal Psychology, 72, 193–204.
13. Chapman, L. J., & Chapman, J. P. (1969). Illusory correlation as an obstacle to the use of valid psychodiagnostic signs. Journal of Abnormal Psychology, 74, 271–280.
14. Clemen, R. T. (1989). Combining forecasts: A review and annotated bibliography. International Journal of Forecasting, 5, 559–583.
15. Clemen, R. T., & Winkler, R. L. (1993). Aggregating point estimates: A flexible modeling approach. Management Science, 39, 501–515.
16. Clemen, R. T., & Winkler, R. L. (1985). Limits for the precision and value of information from dependent sources. Operation Research, 33, 427–442.
17. Clemen, R. T., & Winkler, R. L. (1986). Combining economic forecasts. Journal of Business and Economic Statistics, 4, 39–46.
18. Davis, J. H. (1973). Group decision and social interaction: A theory of social decision schemes. Psychological Review, 80, 97–125.
19. Davis, J. H. (1996). Group decision making and quantitative judgments: A consensus model. In E. H. Witte & J. H. Davis (Eds.), Understanding group behavior (Vol 1): Small group processes and interpersonal relations. Hillsdale, NJ: Lawrence Erlbaum.
20. Einhorn, H. J., & Hogarth, R. M. (1978). Confidence in judgment: Persistence of the illusion of validity. Psychological Review, 85, 395–416.
21. Erev, I., Wallsten, T. S., & Budescu, D. V. (1994). Simultaneous over- and under-confidence: The role of error in judgment processes. Psychological Review, 101, 519–527.
22. Ferrel, W. R. (1985). Combining individual judgments. In G. Wright (Ed.), Behavioral decision making(pp. 111–145). New York: Plenum.
23. Fiedler, K. (1996). Explaining and simulating judgment biases as an aggregation phenomenon in probabilistic multiple-cue environments. Psychological Review, 103, 193–214.
24. Fischer, I., & Harvey, N. (1999). Combining forecasts: What information do judges need to outperform the simple average? International Journal of Forecasting, 15, 227–246.
25. Flores, B. E., & White, E. M. (1988). A framework for the combination of forecasts. Academy of Marketing Science, 16, 95–103.
26. Gigerenzer, G., & Goldstein, D. G. (1999). Betting on one good reason: The take the best heuristic. In G. Gigerenzer, P. M. Todd, & the ABC Research Group (Eds.), Simple heuristics that make us smart (pp. 75–96). New York: Oxford University Press.
27. Gigerenzer, G., Hoffrage, U., & Kleinbolting, H. (1991). Probabilistic mental models: A Brunswikian theory of confidence. Psychological Review, 98, 506–528.
28. Harvey, N., & Harries, C. (2003). Effects of judges’ forecasting on their later combination of forecasts for the same outcomes. International Journal of Forecasting, in press.
29. Hastie, R. (1993). Inside the juror: The psychology of jury decision-making. Cambridge, UK: Cambridge University Press.
30. Hogarth, R. M. (1977). Methods for aggregating opinions. In H. Jungermann & G. de Zeeuw (Eds.), Decision making and change in human affairs (pp. 231–255). Boston: Riedel.
31. Hogarth, R. M. (1978). A note on aggregating opinions. Organizational Behavior and Human Performance, 21, 40–46.
32. Johnson, T. R., Budescu, D. V., & Wallsten, T. S. (2001). Averaging probability judgments: Monte Carlo analyses of asymptotic diagnostic value. Journal of Behavioral Decision Making, 14, 123–140.
33. Juslin, P., Olsson, H., & Bjorkman, M. (1997). Brunswikian and Thurstonian origins of bias in probability assessment: On the interpretation of stochastic components of judgment. Journal of Behavioral Decision Making, 10,189–210.
34. Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80, 237–251.
35. Kuhn, K. M., & Sniezek, J. A. (1996). Confidence and uncertainty in judgmental forecasting: Differential effects of scenario presentation. Journal of Behavioral Decision Making, 9, 231–247.
36. Larrick, R. B., & Soll, J. B. (2003). Lay intuitions about combining quantitative judgments. Paper presented at the workshop on Information Aggregation in Decision Making, Silver Spring, MD.
37. Laughlin, P. R., & Ellis, A. L. (1986). Demonstrability and social combination processes on mathematical intellective tasks, Journal of Experimental Social Psychology, 22, 177–189.
38. Maines, L. (1996). An experimental examination of subjective forecast combination. International Journal of Forecasting, 12, 223–233.
39. Mojzisch, A., & Schulz-Hardt, S. (2004). Information sampling in group decision making: Sampling biases and their consequences. This volume.
40. Morris, P. A. (1983). An axiomatic approach to expert resolution. Management Science, 29, 24–32.
41. Morris, P. A. (1986). Comment on Genest and Zideck’s –Combining probability distributions: A critique and annotated bibliography.— Statistical Science, 1,141–144.
42. Payne, J. W., Bettman, J. R., & Johnson, E. R. (1993). The adaptive decision maker. New York: Cambridge University Press.
43. Price, P. C., & Stone, E. R. (2004). Intuitive evaluation of likelihood judgment producers: Evidence for a confidence heuristic. Journal of Behavioral Decision Making, 17, 39–57.
44. Rantilla, A. K., & Budescu, D. V. (1999). Aggregation of advisor opinions. In Proceedings of the 32nd Annual Hawai'i International Conference on Systems Sciences.
45. Salvadori, L., van Swol, L. M., & Sniezek, J. A. (2001). Information sampling and confidence within groups and judge advisor systems. Communication Research, 28, 737–771.
46. Sniezek, J. A. (1986). The role of variable labels in cue probability learning tasks. Organizational Behavior and Human Decision Processes, 38,141–161.
47. Sniezek, J. A. (1992). Groups under uncertainty: An examination of confidence in group decision making. Organizational Behavior and Human Decision Processes, 52, 124–155.
48. Sniezek, J. A., & Buckley, T. (1995). Cueing and cognitive conflict in judge–advisor decision making. Organizational Behavior and Human Decision Processes,62, 159–174.
49. Soll, J. B. (1999). Intuitive theories of information: Beliefs about the value of redundancy. Cognitive Psychology, 38, 317–346.
50. Soll, J. B., Larrick, R. (2000). The 80/20 rule and the revision of judgment in light another's opinion: Why fo we believe ourseleves so much? Paper presented at the annual meeting of the Society for Judgment and Decision Making (SJDM), Los Angeles.
51. Trafimow, D., & Sniezek, J. A. (1994). Perceived expertise and its effect on confidence. Organizational Behavior and Human Decision Processes, 57, 290–302.
52. van Swol, L. M., & Sniezek, J. A. (2002). Trust me, I'm an expert: Trust and confidence and acceptance of expert advice. Paper presented at the 8th Conference on Behavioral Decision Research in Management (BDRM), Chicago.
53. Wallsten, T. S., Budescu, D. V., Erev, I., & Diederich, A. (1997). Evaluating and combining subjective probability estimates. Journal of Behavioral Decision Making, 10, 243–268.
54. Wallsten, T. S., & Diederich, A. (2001). Understanding pooled subjective probability estimates. Mathematical Social Sciences, 18, 1–18.
55. Walz, D. T., & Walz, D. D. (1989). Combining forecast: Multiple regression versus a Bayesian approach. Decision Sciences, 20, 77–89.
56. Winkler, R. L. (1971). Probabilistic prediction: Some experimental results. Journal of the American Statistical Association, 66, 675–685.
57. Winkler, R. L., & Makridakis, S. (1983). The combination of forecasts. Journal of the Royal Statistical Society: Series A, 146, 150–157.
58. Winkler, R. L., & Poses, R. M. (1993). Evaluating and combining physician's probabilities of survival in an intensive care unit. Management Science, 39,1526–1543.
59. Wittenbaum, G., & Stasser, G. (1996). Management of information in small groups. In J. L. Nye & A. M. Brower (Eds.), What's social about social cognition? Research on socially shared cognition in small groups. (pp. 3–28). Thousand Oaks, CA: Sage.
60. Yaniv, I. (1997). Weighting and trimming: Heuristics for aggregating judgments under uncertainty. Organizational Behavior and Human Decision Processes, 69, 237–249.
61. Yaniv, I. (2004). Receiving other people's advice: Influence and benefit. Organizational Behavior and Human Decision Processes, 93, 1–13.
62. Yaniv, I., & Kleinberger, E. (2000). Advice taking in decision–making: Egocentric discounting and reputation formation. Organizational Behavior and Human Decision Processes, 83, 260–281.