The importance of the task in analyzing expert judgment

Organizational Behavior and Human Decision Processes

Volumn 69, Issue 3, 1997, Pages 205-219

  Stewart, Thomas R ^b   Roebber, Paul J ^a   Bosart, Lance F ^c  

^a UNIVERSITY OF WISCONSIN MILWAUKEE   (United States)

^b STATE UNIVERSITY OF NEW YORK   (United States)

^c NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0031097632     PISSN: 07495978     EISSN: None     Source Type: Journal    
DOI: 10.1006/obhd.1997.2682     Document Type: Article

References (95)

1. Anderson N. H., Shanteau J. C. Weak inference with linear models. Psychological Bulletin. 84:1977;1155-1170.
2. Birnbaum M. H. The devil rides again: Correlation as an index of fit. Psychological Bulletin. 79:1973;239-242.
3. Blattberg R. C., Hoch S. J. Database models and managerial intuition: 50% model + 50% manager. Management Science. 36:1990;887-899.
4. Bolger F., G Wright. Assessing the quality of expert judgment: Issues and analysis. Decision Support Systems. 11:1994;1-24.
5. Bosart L. F. SUNYA experimental results in forecasting daily temperature and precipitation. Monthly Weather Review. 103:1975;1013-1020.
6. Bosart L. F. An update on trends in skill of daily forecasts of temperature and precipitation at the State University of New York at Albany. Bulletin of the American Meteorological Society. 64:1983;346-354.
7. Brehmer A., Brehmer B. What have we learned about human judgment from thirty years of policy capturing? Human judgment: The social judgment theory view. 1988;North-Holland, Amsterdam. p. 75-114.
8. Brehmer B. Note on clinical judgment and the formal characteristics of clinical tasks. Psychological Bulletin. 83:1976;778-782.
9. Brehmer B. Response consistency in probabilistic inference tasks. Organizational Behavior and Human Performance. 22:1978;103-115.
10. Brehmer B. The development of social judgment theory. Brehmer B., Joyce C. R. B. Human judgment: The social judgment theory view. 1988;13-40 North-Holland, Amsterdam.
11. Brehmer B. The psychology of linear judgment models. Acta-Psychologica. 87:1994;137-154.
12. Brier G. W. Verification of weather forecasts expressed in terms of probability. Monthly Weather Review. 78:1950;1-3.
13. Brunswik E. The conceptual framework of psychology. 1952;University of Chicago Press, Chicago.
14. Brunswik E. Perception and the representative design of psychological experiments. 1956;University of California Press, Berkeley.
15. Bunn D., Wright G. Interaction of judgemental and statistical forecasting methods: issues and analysis. Management Science. 31:1991;501-518.
16. Camerer C. General conditions for the success of bootstrapping models. Organizational Behavior and Human Performance. 27:1981;411-422.
17. Carter G. M., Dallavalle J. P., Glahn H. R. Statistical forecasts based on the National Meteorological Center's numerical weather prediction system. Weather and Forecasting. 4:1989;401-412.
18. Castellan N. J. Jr. Comments on the "lens model" equation and the analysis of multiple-cue judgment tasks. Psychometrika. 38:1973;87-100.
19. Chi M.T.H., Glaser R., Farr M. J. The nature of expertise. 1988;Erlbaum, Hillsdale.
20. Clemen R. T. Combining forecasts: A review and annotated bibliography. International Journal of Forecasting. 5:1989;559-583.
21. Clemen R. T., Murphy A. H. Objective and subjective precipitation probability forecasts: statistical analysis of some interrelationships. Weather and Forecasting. 1:1986;56-65.
22. Clemen R. T., Winkler R. L. Limits for the precision and value of information from dependent sources. Operations Research. 33:1985;427-442.
23. Clemen R. T., Winkler R. L. Calibrating and combining precipitation probability forecasts. Viertl R. Probability and Bayesian statistics. 1987;97-110 Plenum, New York.
24. Cooksey R. Judgment analysis: Theory, methods, and applications. 1996;Academic Press, New York.
25. Cooksey R. W., Freebody P. Generalized multivariate lens model analysis for complex human inference tasks. Organizational Behavior and Human Decision Processes. 35:1985;46-72.
26. R. M. Dawes, 1988, Rational choice in an uncertain world, Harcourt, Brace, Jovanivich, New York.
27. Dawes R. M., Corrigan B. Linear models in decision making. Psychological Bulletin. 81:1974;95-106.
28. Dawes R. M., Faust D., Meehl P. E. Clinical versus actuarial judgment. Science. 243:1989;1668-1674.
29. DiMego G. J., Mitchell K. E., Petersen R. A., Hoke J. E., Gerrity J. P., Tuccilllo J. J., Wobus R. L., Juang H.-M. H. Changes to NMCs regional analysis and forecast system. Weather and Forecasting. 7:1992;185-198.
30. Doherty M. E., Kurz E. Social judgement theory. Thinking and Reasoning. 2:1996;109-140.
31. Ebbesen E. B., Konecni V. J. On the external validity of decision-making research: What do we know about decisions in the real world? Wallsten T. Cognitive processes in choice and decision behavior. 1980;21-45 Erlbaum, Hillsdale.
32. Einhorn H. J. Use of nonlinear, noncompensatory models as a function of task and amount of information. Organizational Behavior and Human Performance. 6:1971;1-27.
33. Einhorn H. J. Cue definition and residual judgment. Organizational Behavior and Human Decision Processes. 12:1974;30-49.
34. Ericsson K. A., Simon H. A. Protocol analysis: Verbal reports as data. 1984;Massachusetts Institute of Technology Press, Cambridge.
35. Faust D. Research on human judgment and its application to clinical practice. Professional Psychology. 17:1986;420-430.
36. Gaeth G. J., Shanteau J. Reducing the influence of irrelevant information on experienced decision makers. Organizational Behavior and Human Performance. 33:1984;263-282.
37. Glahn H. R., Lowry D. A. The use of model output statistics (MOS) in objective weather forecasting. Journal of Applied Meteorology. 11:1972;1203-1211.
38. Goldberg L. R. Diagnosticians vs. diagnostic signs. The diagnosis of psychosis vs. neurosis from the MMPI. Psychological Monographs. 79 (Whole No. 602):1965.
39. Goldberg L. R. Human mind versus regression equation: Five contrasts. Cicchetti D., Grove W. M. Thinking clearly about psychology: Vol. 1. Matters of public interest. 1991;173-184 University of Minnesota Press, Minneapolis.
40. Guilford J. P. Psychometric methods. 1954;McGraw-Hill, New York.
41. Hammond K. R. Probabilistic functioning and the clinical method. Psychological Review. 62:1955;255-262.
42. K. R. Hammond, 1980, The integration of research in judgment and decision theory, University of Colorado, Boulder, CO.
43. K. R. Hammond, 1981, Principles of organization in intuitive and analytical cognition, University of Colorado, Boulder, CO.
44. Hammond K. R. Naturalistic decision making from a Brunswikian viewpoint: Its past, present, future. Klein G., Orasanu J., Calderwood R., Zsambok C. E. Decision making in action: Models and methods. 1993;205-227 Ablex, Norwood, NJ.
45. Hammond K. R. Human judgment and social policy: Irreducible uncertainty, inevitable error, unavoidable injustice. 1996;Oxford University Press, New York.
46. Hammond K. R., Hamm R. M., Grassia J. Generalizing over conditions by combining the multitrait-multimethod matrix and the representative design of experiments. Psychological Bulletin. 100:1986;257-269.
47. Hammond K. R., Stewart T. R., Brehmer B., Steinmann D. O. Social judgment theory. Kaplan M. F., Schwartz S. Human judgment and decision processes. 1975;271-312 Academic Press, New York.
48. Harvey N. Why are judgments less consistent in less predictable task situations? Organizational Behavior and Human Decision Processes. 63:1995;247-263.
49. Heideman K. F., Stewart T. R., Moninger W. R., Reagan-Cirincione. The weather information and skill experiment (WISE): The effect of varying levels of information on forecast skill. Weather and Forecasting. 8:1993;25-36.
50. Hogarth R. Judgement and choice. 1987;Wiley, New York.
51. Hursch C. J., Hammond K. R., Hursch J. L. Some methodological considerations in multiple-cue probability studies. Psychological Review. 71:1964;42-60.
52. Johnson E. J. Expertise and decision under uncertainty: Performance and process. Chi M., Glaser R., Farr M. The nature of expertise. 1988;209-228 Erlbaum, Hillsdale, NJ.
53. Kahneman D., Tversky A. Prospect theory: An analysis of decision under risk. Econometrica. 47:1979;263-291.
54. Kahneman D., Slovic P., Tversky S. Judgment under uncertainty: Heuristics and biases. 1982;Cambridge University Press, Cambridge.
55. Klein G. A. Twenty questions - Suggestions for research in naturalistic decision making. Klein G. A., Orasanu J., Calderwood R., Zsambok C. E. Decision making in action: Models and methods. 1993;389-403 Ablex, Norwood.
56. Klein G. A., Orasanu J., Calderwood R., Zsambok C. E. Decision making in action: Models and methods. 1993;Ablex, Norwood.
57. Klein W. H., Lewis B. M., Enger I. Objective prediction of five-day mean temperatures during winter. Journal of Meteorology. 16:1959;672-682.
58. Lee J.-W., Yates J. F. How quantity judgment changes as the number of cues increases: An analytical framework and review. Psychological Bulletin. 112:1992;363-377.
59. Lusk C. M., Stewart T. R., Hammond K. R., Potts R. J. Judgment and decision making in dynamic tasks: The case of forecasting the microburst. Weather and Forecasting. 5:1990;627-639.
60. Makridakis S., Winkler R. L. Averages of forecasts: Some empirical results. Management Science. 29:1983;987-996.
61. McClelland G. H., Judd C. M. Statistical difficulties of detecting interactions and moderator effects. Psychological Bulletin, 114:1993;376-390.
62. Meehl P. E. Clinical vs. statistical prediction. 1954;University of Minneapolis Press, Minneapolis, MN.
63. Monahan J., Steadman H. J. Violent storms and violent people: How meteorology can inform risk communication in mental health law. American Psychologist. 51(9):1996;931-938.
64. Morrison D. G., Schmittlein D. C. How many forecasters do you really have? Mahanobis provides the intuition for the surprising Clemens and Winkler result. Operations Research. 39:1991;519-523.
65. Murphy A. H. Skill scores based on the mean square error and their relationships to the correlation coefficient. Monthly Weather Review. 116:1988;2417-2424.
66. Murphy A. H., Brown B. G. A comparative evaluation of objective and subjective weather forecasts in the United States. Journal of Forecasting. 3:1984;369-393.
67. Murphy A. H., Daan H. Impacts of feedback and experience on the quality of subjective probability forecasts: Comparison of results from the first and second years of the Zerikzee experiment. Monthly Weather Review. 112:1984;413-423.
68. Murphy A. H., Winkler R. L. Probability forecasts: A survey of national weather service forecasters. Bulletin of the American Meteorological Society. 55:1974;1449-1453.
69. Murphy A. H., Chen Y., Clemen R. T. Statistical analysis of interrelationships between objective and subjective temperature forecasts. Monthly Weather Review. 116:1988;2121-2131.
70. Murphy A. H., Lichtenstein S., Fischhoff B., Winkler R. L. Misinterpretations of precipitation probability forecasts. Bulletin of the American Meteorological Society. 61:1980;695-701.
71. Newell A., Simon H. A. Human problem solving. 1972;Prentice-Hall, Englewood Cliffs.
72. Oskamp S. Overconfidence in case study judgments. Journal of Consulting Psychology. 29:1965;261-265.
73. Roebber P. J., Bosart L. F. The complex relationship between forecast skill and forecast value: A real world analysis. Weather and Forecasting. 11:1996a;544-559.
74. Roebber P. J., Bosart L. F. The contributions of education and experience to forecast skill. Weather and Forecasting. 11:1996b;21-40.
75. Roebber P. J., Bosart L. F., Forbes G. J. Does distance from the forecast site affect skill? Weather and Forecasting. 11:1996;582-589.
76. Sanders F. F. Trends in skill of Boston forecasts made at MIT, 1966-1984. Bulletin of the American Meteorological Society. 67:1986;170-176.
77. Shanteau J. Competence in experts: The role of task characteristics. Organizational Behavior and Human Decision Processes. 53:1992;252-266.
78. J. Shanteau, 1995, Expert judgment and financial decision making, Kansas State University, Manhattan, Kansas.
79. Slovic P., Lichtenstein S. Comparison of Bayesian and regression approaches to the study of information processing in judgment. Organizational Behavior and Human Performance. 6:1971;649-744.
80. Slovic P., Fischhoff B., Lichtenstein S. Behavioral decision theory. Annual Review of Psychology. 28:1977;1-39.
81. Snellman L. W. Operational Forecasting Using Automated Guidance. Bulletin of the American Meteorological Society. 58:1977;1036-1044.
82. Stewart T. R. Components of correlations and extensions of the lens model equation. Psychometrika. 41:1976;101-120.
83. Stewart T. R. A decomposition of the correlation coefficient and its use in analyzing forecasting skill. Weather and Forecasting. 5:1990;661-666.
84. Stewart T. R. Meta-analysis of the relation between task predictability and accuracy of expert judgment. 1997;Center for Policy Research, University at Albany, State University of New York, Albany.
85. Stewart T. R., Lusk C. M. Seven components of judgmental forecasting skill: Implications for research and the improvement of forecasts. Journal of Forecasting. 13:1994;579-599.
86. Stewart T. R., Moninger W. R., Grassia J., Brady R. H., Merrem F. H. Analysis of expert judgment in a hail forecasting experiment. Weather and Forecasting. 4:1989;24-34.
87. Stewart T. R., Moninger W. R., Heideman K. F., Reagan-Cirincione P. Effects of improved information on the components of skill in weather forecasting. Organizational Behavior and Human Decision Processes. 53:1992;107-134.
88. Swets J. A. Measuring the accuracy of diagnostic systems. Science. 240:1988;1285-1293.
89. Tennekes H. The outlook: Scattered showers. Bulletin of the American Meteorological Society. 69:1988;368-372.
90. Trenberth K. E. On the interpretation of the diagnostic quasi-geostrophic omega equation. Monthly Weather Review. 106:1978;131-137.
91. Tucker L. R. A suggested alternative formulation in the developments by Hursch, Hammond, and Hursch, and by Hammond, Hursch, and Todd. Psychological Review. 71:1964;528-530.
92. Tversky A., Kahneman D. Belief in the law of small numbers. Psychological Bulletin. 76:1971;105-110.
93. Tversky A., Kahneman D. Judgment under uncertainty: Heuristics and biases. Science. 185:1974;1124-1131.
94. von Winterfeldt D., Edwards W. Decision analysis and behavioral research. 1986;Cambridge University Press, Cambridge.
95. Wigton R. S. Use of linear models to analyze physicians decisions. Medical Decision Making. 8:1988;241-252.