Aggregation of opinions based on correlated cues and advisors

Journal of Behavioral Decision Making

Volumn 20, Issue 2, 2007, Pages 153-177

  Budescu, David V ^a   Yu, Hsiu Ting ^a  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

Advice taking; Aggregation; Confidence; Correlated advisors; Correlated cues; System neglect

Indexed keywords

EID: 34147220129     PISSN: 08943257     EISSN: 10990771     Source Type: Journal    
DOI: 10.1002/bdm.547     Document Type: Article

References (61)

1. Ariely, D., Au, W. T., Bender, R. H., Budescu, D. V., Dietz, C. B., Gu, H., Wallsten, T. S., & Zauberman, G. (2000). The effects of averaging subjective probability estimates between and within judges. Journal of Experimental Psychology: Applied, 6, 130-147.
2. Armstrong, J. S. (2001). Combining forecasts. In J. S. Armstrong (Ed.), Principles of forecasting: A handbook for researchers and practitioners. Norwell, MA: Kluwer Academic Publishers.
3. Ashton, R. H. (1986). Combining the judgments of advisors: How many and which ones? Organizational Behavior and Human Decision Processes, 38, 405-414.
4. Ashton, A. H., & Ashton, R. H. (1985). Aggregating subjective forecasts: Some empirical results. Management Science, 31, 1499-1508.
5. Azen, R., & Budescu, D. V. (2003). Dominance analysis: A method for comparing predictors in multiple regression. Psychological Methods, 8, 129-148.
6. Budescu, D. V. (1993). Dominance analysis: A new approach to the problem of relative importance of predictors in multiple regression. Psychological Bulletin, 114, 542-551.
7. Budescu, D. V. (2006). Confidence in aggregation of opinions from multiple sources. In K. Fiedler, & P. Juslin (Eds.), Information sampling and adaptive cognition (pp. 327-354). Cambridge: Cambridge University Press.
8. Budescu, D. V., Erev, I., & Wallsten, T. S. (1997). On the importance of random error in the study of probability judgment, Part I: New theoretical developments. Journal of Behavioral Decision Making, 10, 157-172.
9. Budescu, D. V., & Rantilla, A. K. (2000). Confidence in aggregation of expert opinions. Acta Psychologica, 104, 371-398.
10. Budescu, D. V., Rantilla, A. K., Yu, H. -T., & Karelitz, T. M. (2003). The effects of symmetry among advisors on the aggregation of their opinions. Organizational Behavior and Human Decision Processes, 90, 178-194.
11. Budescu, D. V., & Yu, H.-Y. (in press). To Bayes or not to Bayes: A comparison of models of information aggregation. Paper presented at the 46th annual meeting of the Psychonomic Society, Toronto, CA. Decision Analysis, In press.
12. Clemen, R. T., & Winkler, R. L. (1985). Limits for the precision and value of information from dependent sources. Operations Research, 33, 427-442.
13. Clemen, R. T. (1989). Combining forecasts: A review and annotated bibliography. International Journal of Forecasting, 5, 559-583.
14. Clemen, R. T., & Murphy, A. H. (1986). Objective and subjective precipitation probability forecasts: Some methods for improving forecast quality. Weather and Forecasting, 1, 213-218.
15. Clemen, R. T., & Winkler, R. L. (1987). Calibrating and combining precipitation probability forecasts. In R. Viertl (Ed.), Probability and Bayesian statistics. New York, NY: Plenum Press.
16. Einhorn, H. J., & Hogarth, R. M. (1978). Confidence in judgment: Persistence of the illusion of validity. Psychological Review, 85, 395-416.
17. Erev, 1., Wallsten, T. S., & Budescu, D. V. (1994). Simultaneous over- and underconfidence: The role of error in judgment processes. Psychological Review, 101, 519-527.
18. Ferrell, W. R. (1985). Combining individual judgments. InG . Wright (Ed.),ehavioral decision making (pp. 111-145). New York: Plenum Publishing Corporation.
19. Fischer, I., & Budescu, D. V. (2005). When do those who know more also know more about how much they know? The development of confidence in performance and performance in decision-making. Organizational Behavior and Human Performance, 98, 39-53.
20. Fischer, I., & Harvey, N. (1999). Combining forecasts: What information do judges need to outperform the simple average? International Journal of Forecasting, 15, 227-246.
21. Fischer, G. W. (1981). When oracles failA comparison of four procedures for aggregating subjective probability forecasts. Organizational Behavior and Human Performance, 28, 96-110.
22. Flores, B. E., & White, E. M. (1988). A framework for the combination of forecasts. Academy of Marketing Science, 16, 95-103.
23. Gigerenzer, G., Hoffrage, U., & Kleinbölting, H. (1991). Probabilistic mental models: A Brunswikian theory of confidence. Psychological Review, 98, 506-528.
24. Griffin, D., & Tversky, A. (1992). The weighing of evidence and the determinants of confidence. Cognitive Psychology, 24, 411-4435.
25. Harvey, N., Harries, C., & Fischer, I. (2000). Using advice and assessing its quality. Organizational Behavior and Human Decision Performance, 81, 252-273.
26. Hogarth, R. M. (1978). A note on aggregating opinions. Organizational Behavior and Human Performance, 21, 40-46.
27. Hogarth, R. M. (1989). On combining diagnostic 'forecasts': Thoughts and some evidence. International Journal of Forecasting, 5, 593-597.
28. Johnson, T. R., Budescu, D. V., & Wallsten, T. S. (2001). Averaging probability judgments: Monte Carlo analyses of asymptotic diagnostic value. Journal of Behavioral Decision Making, 14, 123-140.
29. Johnson-Laird, P. N. (1994). Mental models and probabilistic thinking. Cognition, 50, 189-209.
30. Juslin, P., Olsson, H., & Bjorkman, M. (1997). Brunswikian and Thurstonian origins of bias in probability assessment: On the interpretation of stochastic components of judgment. Journal of Behavioral Decision Making, 10, 189-210.
31. Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80, 237-251.
32. Karelaia, N. (2006). Thirst for confirmation in multi-attribute choice: Does search for consistency impair decision performance? Organizational Behavior and Human Decision Performance, 100, 128-143.
33. Keren, G., & Teigen, K. H. (2001). Why is p = 0.90 better than p = 0.70? Preferences for definitve predictions by lay consumers of probability judgments. Psychonomic Bulletin and Review, 21, 191-202.
34. Kuhn, K. M., & Sniezek, J. A. (1996). Confidence and uncertainty in judgmental forecasting: Differential effects of scenario presentation. Journal of Behavioral Decision Making, 9, 231-247.
35. Larrick, R. P., & Soil, J. B. (2006). Intuitions about combining opinions: Misappreciation of the averaging principle. Management Science, 52, 111-127.
36. Maines, L. (1996). An experimental examination of subjective forecast combination. International Journal of Forecasting, 12, 223-233.
37. Massey, C., & Wu, G. (2005). Detecting regime shifts: The causes of under- and over-reactions. Management Science, 51, 932-947.
38. Mitchell Gibbons, A. A. (2003). Alternative forms of advice in natural decision settings. Unpublished MA Thesis. University of Illinois at Urbana-Champaign.
39. Morris, P. A. (1986). Comment on Genest and Zideck's "Combining probability distributions: A critique and annotated bibliography". Statistical Science, 1, 141-144.
40. Payne, J., Bettman, J., & Johnson, E. (1993). The adaptive decision maker. Cambridge University Press.
41. Price, P. C., & Stone, E. R. (2004). Intuitive evaluation of likelihood judgment producers: Evidence for a confidence heuristic. Journal of Behavioral Decision Making, 17, 39-57.
42. Salvadori, L., Van Swol, L. M., & Sniezek, J. A. (2001). Information sampling and confidence within groups and judge advisor systems. Communication Research, 28, 737-771.
43. Slovic, P. (1972). From Shakespeare to Simon. Speculation-and some evidence-about man's ability to process information. Oregon Research Institute Bulletin, 12(3).
44. Shanteau, J. (2001). What does it mean when experts disagree? In E. Salas , & G. Klein (Eds.), Linking expertise and naturalistic decision making. Mahwa, NJ: Erlbaum.
45. Sniezek, J. A. (1992). Groups under uncertainty: An examination of confidence in group decision making. Organizational Behavior and Human Decision Processes, 52, 124-155.
46. Sniezek, J. A., & Buckley, T. (1991). Confidence depends on level of aggregation. Journal of Behavioral Decision Making, 4, 263-272.
47. Sniezek, J. A., & Buckley, T. (1995). Cueing and cognitive conflict in judge-advisor decision making. Organizational Behavior and Human Decision Processes, 62, 159-174.
48. Sniezek, J. A., & Henry, R. A. (1989). Accuracy and confidence in group judgment. Organizational Behavior and Human Decision Processes, 43, 1-28.
49. Sniezek, J. A., & Henry, R. A. (1990). Revision, weighting, and commitment in consensus group judgment. Organizational Behavior and Human Decision Processes, 45, 66-84.
50. Soll, J. B. (1999). Intuitive theories of information: Beliefs about the value of redundancy. Cognitive Psychology, 38, 311-346.
51. Soil, J. B., & Larrick, R. (2000). The 80/20 rule and the revision of judgment in light another's opinion: Why do we believe ourseleves so much? Paper presented at the annual meeting of the Society for Judgment and Decision Making (SJDM), Los Angeles, CA.
52. Trafimow, D., & Sniezek, J. A. (1994). Perceived expertise and its effect on confidence. Organizational Behavior and Human Decision Processes, 57, 290-302.
53. Van Swol, L. M., & Sniezek, J. A. (2005). Factors affecting the acceptance of expert advice. British Journal of Social Psychology, 44, 443-461.
54. Wallsten, T. S., Budescu, D. V., & Tsao, C. (1997). Combining linguistic probabilities. In R. W. Scholz, & A. C. Zimmer (Eds.), Qualitative aspects of decision making (pp. 27-55). Lengerich: Pabst Science Publishers.
55. Wallsten, T. S., & Diederich, A. (2001). Understanding pooled subjective probability estimates. Mathematical Social Sciences, 18, 1-18.
56. Winkler, R. L. (1971). Probabilistic prediction: Some experimental results. Journal of the American Statistical Association, 66, 675-685.
57. Winkler, R. L., & Poses, R. M. (1993). Evaluating and combining physician's probabilities of survival in an intensive care unit. Management Science, 39, 1526-1543.
58. Yaniv, I., & Kleinberger, E. (2000). Advice taking in decision-making: Egocentric discounting and reputation formation. Organizational Behavior and Human Decision Processes, 83, 260-281.
59. Yates, J. F. (1982). External correspondence: Decomposition of the mean probability score. Organizational Behavior and Human Performance, 30, 132-156.
60. Yates, J. F., Lee, J., Shinotsuka, H., Palatano, A. L., & Sieck, W. R. (1998). Cross-cultural variations in probability judgment accuracy: Beyond general knowledge overconfidence? Organizational Behavior and Human Decision Processes, 74, 89-117.
61. Yates, J. F., Price, P. C., Lee, J., & Ramirez, J. (1996). Good probabilistic forecasters: The "consumer's" perspective. International Journal of Forecasting, 12, 41-56.