Expertise in Judgment and Decision Making: A Case for Training Intuitive Decision Skills

Blackwell Handbook of Judgment and Decision Making

Volumn , Issue , 2008, Pages 297-315

  Phillips, Jennifer K ^a   Klein, Gary ^a   Sieck, Winston R ^a  

^a Klein Associates   (United States)

Author keywords

Domains; Experience; Expertise; Heuristics; Judgment

Indexed keywords

EID: 84889298472     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9780470752937.ch15     Document Type: Chapter

References (98)

1. Anderson, J. R. (1983) The Architecture of Cognition. Cambridge, MA: Harvard University Press.
2. Anzai, Y. (1991) Learning and use of representation in physics expertise. In K. A. Ericsson and J. Smith (eds.), Toward a General Theory of Expertise: Prospects and Limits. Cambridge: Cambridge University Press.
3. Arkes, H. R., Wortmann, R. L., Saville, P. D., & Harkness, A. R. (1981) Hindsight bias among physicians weighing the likelihood of diagnoses, Journal of Applied Psychology, 66, 252-5.
4. Asare, S. K. & Wright, A. (1995) Normative and substantive expertise in multiple hypothesis evaluation, Organizational Behavior and Human Decision Processes, 64, 171-84.
5. Balzer, W. K., Doherty, M. E., & O'Connor, R. O. (1989) The effects of cognitive feedback on performance, Psychological Bulletin, 106, 410-33.
6. Bazerman, M., Loewenstein, G., & Moore, D. (2002) Why good accountants do bad audits, The Harvard Business Review, 80 (11), 97-102.
7. Bendor, J. (2003) Herbert A. Simon: Political scientist, Annual Review of Political Science, 6, 433-71.
8. Bornstein, B. H., Emler, A. C., & Chapman, G. B. (1999) Rationality in medical treatment decisions: Is there a sunk-cost effect? Social Sciences & Medicine, 49, 215-22.
9. Brehmer, B. (1980) In one word: Not from experience, Acta Psychologica, 45, 223-41.
10. Camerer, C. F. & Johnson, E. J. (1991) The process-performance paradox in expert judgment: How can experts know so much and predict so badly? In K. A. Ericsson and J. Smith (eds.), Toward a General Theory of Expertise: Prospects and Limits (pp. 195-217). Cambridge, MA: Cambridge University Press.
11. Cannon-Bowers, J. A. & Salas, E. (2001) Reflections on shared cognition, Journal of Organizational Behavior, 22, 195-202.
12. Chase, W. G. (1983) Spatial representations of taxi drivers. In D. R. Rogers and J. H. Sloboda (eds.), Acquisition of Symbolic Skills (pp. 391-405). New York: Plenum.
13. Chase, W. G. & Simon, H. A. (1973). The mind's eye in chess. In W. G. Chase (ed.), Visual Information Processing. New York: Academic Press.
14. Chi, M. T. H. (1978) Knowledge structures and memory development. In R. S. Siegler (ed.), Children's Thinking: What Develops? (pp. 73-96). Hillsdale, NJ: Lawrence Erlbaum Associates.
15. Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1980) Representation of Physics Knowledge by Experts and Novices (Technical report no. 2). Pittsburgh, PA: University of Pittsburgh: Learning Research and Development Center.
16. Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981) Categorization and representation of physics problems by experts and novices, Cognitive Science, 5, 121-52.
17. Chi, M. T. H., Glaser, R. & Farr, M. J. (eds.) (1988) The Nature of Expertise. Mahwah, NJ: Lawrence Erlbaum Associates.
18. Christensen-Szalanski, J. J. J. & Bushyhead, J. B. (1981) Physicians' use of probabilistic information in a real clinical setting, Journal of Experimental Psychology, 7, 928-35.
19. Cohen, M. S. (1993) The naturalistic basis of decision biases. In G. A. Klein, J. Orasanu, R. Calderwood, and C. E. Zsambok (eds.), Decision Making in Action: Models and Methods (pp. 51-99). Norwood, NJ: Ablex.
20. DeGroot, A. D. (1965) Thought and Choice in Chess. The Hague: Mouton.
21. Dreyfus, H. L. (1997) Intuitive, deliberative, and calculative models of expert performance. In C. Zsambok and G. Klein (eds.), Naturalistic Decision Making (pp. 17-28). Mahwah, NJ: Lawrence Erlbaum.
22. Einhorn, H. J. & Hogarth, R. M. (1981) Behavioral decision theory: Processes of judgment and choice, Annual Review of Psychology, 32, 53-88.
23. Ericsson, K. A. & Charness, N. (1997) Cognitive and developmental factors in expert performance. In P. J. Feltovich, K. M. Ford, and R. R. Hoffman (eds.), Expertise in Context: Human and Machine (pp. 3-41). Cambridge, MA: MIT Press.
24. Ericsson, K. A. & Smith, J. (1991) Prospects and limits of the empirical study of expertise: An introduction. In K. A. Ericsson and J. Smith (eds.), Toward a General Theory of Expertise: Prospects and Limits. Cambridge, MA: Cambridge University Press.
25. Feltovich, P. J., Ford, K. M., & Hoffman, R. R. (1997) Expertise in Context. Menlo Park, CA: AAAI Press/The MIT Press.
26. Feltovich, P. J., Johnson, P. E., Moller, J. H., & Swanson, D. B. (1984) LCS: The role and development of medical knowledge in diagnostic expertise. In W. J. Clancey and E. H. Shortliffe (eds.), Readings in Medical Artificial Intelligence: The First decade (pp. 275-319). Reading, MA: Addison-Wesley.
27. Feltovich, P. J., Spiro, R. J., & Coulson, R. L. (1997). Issues of expert flexibility in contexts characterized by complexity and change. In P. J. Feltovich, K. M. Ford, and R. R. Hoffman (eds.), Expertise in Context (pp. 125-46). Menlo Park, CA: AAAI/MIT Press.
28. Fischhoff, B. (1982) Debiasing. In D. Kahneman, P. Slovic, and A. Tverskey (eds.), Judgment Under Uncertainty: Heuristics and Biases. Cambridge, MA: Cambridge University Press.
29. Fischhoff, B. (1996) The real world: What good is it? Organizational Behavior and Human Decision Processes, 65, 232-48.
30. Flin, R., Salas, E., Strub, M., & Martin, L. (eds.) (1997) Decision Making Under Stress: Emerging Themes and Application. Aldershot, England: Ashgate Publishing.
31. Flin, R., Slaven, G., & Stewart, K. (1996) Emergency decision making in the offshore oil and gas industry, Human Factors, 38 (2), 262-77.
32. Gentner, D. R. (1988) Expertise in typewriting. In M. T. H. Chi, R. Glaser, and M. J. Farr (eds.), The Nature of Expertise. Mahwah, NJ: Lawrence Erlbaum Associates.
33. Gilovich, T. & Griffin, D. (2002) Introduction-Heuristics and biases: Then and now. In T. Gilovich, D. Griffin, and D. Kahneman (eds.), Heuristics and Biases: The Psychology of Intuitive Judgment (pp. 1-18). Cambridge, MA: Cambridge University Press.
34. Glaser, R. (1996) Changing the agency for learning: Acquiring expert performance. In K. A. Ericsson (ed.), The Road to Excellence (pp. 303-11). Mahwah, NJ: Erlbaum.
35. Glaser, R. & Chi, M. T. H. (1988) Overview. In M. T. H. Chi, R. Glaser, and M. J. Farr (eds.), The Nature of Expertise. Mahwah, NJ: Lawrence Erlbaum Associates.
36. Hammond, J. S., Keeney, R. L., & Raiffa, H. (1999) Smart Choices: A Practical Guide to Making Better Decisions. Boston, MA: Harvard Business School Press.
37. Hammond, K. R., McClelland, G. H., & Mumpower, J. (1980) Human Judgment and Decision Making: Theories, Methods, and Procedures. New York: Praeger.
38. Hammond, K. R., Summers, D. A., & Deane, D. H. (1973) Negative effects of outcomefeedback in multiple-cue probability learning, Organizational Behavior and Human Decision Processes, 9, 30-4.
39. Hayes, J. R. (1985) Three problems of teaching general skills. In J. W. Segal, S. F. Chipman, and R. Glaser (eds.), Thinking and Learning Skills: Relating Instruction to Research (vol. 1, pp. 391-405). Hillsdale, NJ: Lawrence Erlbaum Associates.
40. Hoffman, R. R. (1994) Constructivism versus realism or constructivism and realism? Journal of Experimental and Theoretical Artificial Intelligence, 6, 431-5.
41. Hoffman, R. R., Crandall, B. W., & Shadbolt, N. R. (1998) Use of the critical decision method to elicit expert knowledge: A case study in cognitive task analysis methodology, Human Factors, 40 (2), 254-76.
42. Hogarth, R. (2001) Educating Intuition. Chicago: University of Chicago Press.
43. Hutton, R. J. B., Miller, T. E., & Thordsen, M. L. (2003) Decision-centered design: Leveraging cognitive task analysis in design. In E. Holnagel (ed.), Handbook of Cognitive Task Design (pp. 383-416). Mahwah, NJ: Lawrence Erlbaum Associates.
44. Johnson, J. G. & Raab, M. (2003) Take the first: Option generation and resulting choices, Organizational Behavior and Human Decision Processes, 91 (2), 215-29.
45. Kahneman, D. & Tversky, A. (1972) Subjective probability: A judgment of representativeness, Cognitive Psychology, 3, 430-54.
46. Kahneman, D. & Tversky, A. (1982) On the study of statistical intuitions, Cognition, 11, 123-41.
47. Keren, G. (1987) Facing uncertainty in the game of bridge: A calibration study, Organizational Behavior and Human Decision Processes, 39 (1), 98-114.
48. Klein, G. (1998) Sources of Power: How People Make Decisions. Cambridge, MA: MIT Press.
49. Klein, G. & Militello, L. (in press). The knowledge audit as a method for cognitive task analysis. In H. Montgomery, R. Lipshitz, and B. Brehmer (eds.), How Professionals Make Decisions. Mahwah, NJ: Lawrence Erlbaum Associates.
50. Klein, G. & Wolf, S. (1998) The role of leverage points in option generation, IEEE Transactions on Systems, Man and Cybernetics: Applications and Reviews, 28 (1), 157-60.
51. Klein, G., Wolf, S., Militello, L., & Zsambok, C. (1995) Characteristics of skilled option generation in chess, Organizational Behavior and Human Decision Processes, 62 (1), 63-9.
52. Klein, G. A. & Brezovic, C. P. (1986) Design engineers and the design process: Decision strategies and human factors literature, Proceedings of the Human Factors and Ergonomics Society 30th Annual Meeting, 2, 771-5.
53. Klein, G. A., Calderwood, R., & Clinton-Cirocco, A. (1986) Rapid decision making on the fireground, Proceedings of the Human Factors and Ergonomics Society 30th Annual Meeting, 1, 576-80.
54. Klein, G. A. & Crandall, B. W. (1995) The role of mental simulation in naturalistic decision making. In P. Hancock, J. Flach, J. Caird, and K. Vicente (eds.), Local Applications of the Ecological Approach to Human-Machine Systems (vol. 2, pp. 324-58). Mahwah, NJ: Lawrence Erlbaum Associates.
55. Klein, G. A. & Hoffman, R. (1993) Seeing the invisible: Perceptual/cognitive aspects of expertise. In M. Rabinowitz (ed.), Cognitive Science Foundations of Instruction (pp. 203-26). Mahwah, NJ: Lawrence Erlbaum Associates.
56. Klein, G. A., Orasanu, J., Calderwood, R., & Zsambok, C. E. (eds.) (1993) Decision Making in Action: Models and Methods. Norwood, NJ: Ablex.
57. Klinger, D. W. & Militello, L. G. (2001). Designing for performance: A cognitive systems engineering and cognitive task analysis approach to the modification of the AWACS weapons director interface. Manuscript submitted 10/2001.
58. Kobus, D. A., Proctor, S., Bank, T. E., & Holste, S. T. (2000) Decision-making in a Dynamic Environment: The Effects of Experience and Information Uncertainty. San Diego, CA: Space and Naval Warfare Systems Center.
59. Koehler, D. J., Brenner, L., & Griffin, D. (2002) The calibration of expert judgment: Heuristics and biases beyond the laboratory. In T. Gilovich, D. Griffin, and D. Kahneman (eds.), Heuristics and Biases: The Psychology of Intuitive Judgment (pp. 686-715). Cambridge, MA: Cambridge University Press.
60. Larkin, J., McDermott, J., Simon, D. P., & Simon, H. A. (1980) Expert and novice performance in solving physics problems, Science, 20, 1335-42.
61. Larkin, J. H. (1983) The role of problem representation in physics. In D. Gentner and A. L. Stevens (eds.), Mental Models. Mahwah, NJ: Lawrence Erlbaum Associates.
62. Lee, J. W. & Yates, J. F. (1992) How quantity judgment changes as the number of cues increase: An analytical framework and review, Psychological Bulletin, 112, 363-77.
63. Lipshitz, R. & Cohen, M. S. (submitted) Warrants for prescription: Analytically and empirically based approaches to improving decision making. Manuscript submitted for publication-HFES.
64. Lipshitz, R., Klein, G., Orasanu, J., & Salas, E. (2001) Focus article: Taking stock of naturalistic decision making, Journal of Behavioral Decision Making, 14, 331-52.
65. Lipshitz, R. & Strauss, O. (1997) Coping with uncertainty: A naturalistic decision making analysis, Organizational Behavior and Human Decision Processes, 66, 149-63.
66. Loftus, E. F. & Wagenaar, W. A. (1988) Lawyer's prediction of success, Jurimetrics Journal, 437-53.
67. McNeil, B. J., Pauker, S. G., Sox, H. C., Jr., & Tversky, A. (1982) On the elicitation of preferences for alternative therapies, New England Journal of Medicine, 306, 1259-62.
68. Mosier, K. L. (1991) Expert decision making strategies. In P. Jersen (ed.), Proceedings of the Sixth International Symposium on Aviation Psychology (pp. 266-71). Columbus, OH.
69. Newell, A. & Simon, H. A. (1972) Human Problem Solving. Englewood Cliffs, NY: Prentice Hall.
70. Pascual, R. & Henderson, S. (1997) Evidence of naturalistic decision making in military command and control. In C. E. Zsambok and G. Klein (eds.), Naturalistic Decision Making. Mahwah, NJ: Lawrence Erlbaum Associates.
71. Phillips, J. K. & Battaglia, D. A. (2003) Instructional methods for training sensemaking skills, Proceedings of the Interservice/Industry Training, Simulation, and Education Conference 2003, CD-ROM proceedings.
72. Pliske, R. M., Klinger, D., Hutton, R., Crandall, B., Knight, B., & Klein, G. (1997) Understanding Skilled Weather Forecasting: Implications for Training and the Design of Forecasting Tools (Technical Report No. AL/HR-CR-1997-0003 for the Air Force Material Command, Armstrong Laboratory, Human Resources Directorate Brooks AFB, TX). Fairborn, OH: Klein Associates Inc.
73. Pliske, R. M., McCloskey, M. J., & Klein, G. (2001) Decision skills training: Facilitating learning from experience. In E. Salas & G. Klein (eds.), Linking Expertise and Naturalistic Decision Making (pp. 37-53). Mahwah, NJ: Lawrence Erlbaum Associates.
74. Polanyi, M. (1966) The Tacit Dimension. Garden City, NY: Doubleday.
75. Randel, J. M., Pugh, H. L., & Reed, S. K. (1996) Methods for analyzing cognitive skills for a technical task, International Journal of Human-Computer Studies, 45, 579-97.
76. Ross, K. G., Battaglia, D. A., Phillips, J. K., Domeshek, E. A., & Lussier, J. W. (2003) Mental models underlying tactical thinking skills, Proceedings of the Interservice/Industry Training, Simulation, and Education Conference 2003, CD-ROM proceedings.
77. Rouse, W. B. & Morris, N. M. (1986) On looking into the black box: Prospects and limits on the search for mental models, Psychological Bulletin, 100 (3), 349-63.
78. Russo, J. E., Schoemaker, P. J. H., & Hittleman, M. (2001) Winning Decisions: Getting it Right the First Time. New York: Doubleday.
79. Salas, E. & Klein, G. (2001) Linking Expertise and Naturalistic Decision Making. Mahwah, NJ: Lawrence Erlbaum Associates.
80. Salas, E., Wilson, K. A., Burke, C. S., & Bowers, C. A. (2002) Myths about crew resource management training, Ergonomics in Design, Fall, 20-4.
81. Schmitt, J. F. & Klein, G. (1996) Fighting in the fog: Dealing with battlefield uncertainty, Marine Corps Gazette, 80, 62-9.
82. Schneider, W. (1985) Training high-performance skills: Fallacies and guidelines, Human Factors, 27, 285-300.
83. Schunn, C. D. & Anderson, J. R. (1999) The generality/specificity of expertise in scientific reasoning, Cognitive Science, 23, 337-70.
84. Serfaty, D., MacMillan, J., Entin, E. E., & Entin, E. B. (1997) The decision-making expertise of battle commanders. In C. Zsambok and G. Klein (eds.), Naturalistic Decision Making (pp. 233-46). Mahwah, NJ: Lawrence Erlbaum Associates.
85. Shanteau, J. (1988) Psychological characteristics and strategies of expert decision makers, Acta Psychologica, 68, 203-15.
86. Shanteau, J. (1989) Cognitive heuristics and biases in behavioral auditing: Review, comments, and observations, Accounting Organizations and Society, 14(1-2), 165-77.
87. Shanteau, J. (1992) Competence in experts: The role of task characteristics, Organizational Behavior and Human Decision Processes, 53, 252-66.
88. Sieck, W. R. (2003) Effects of choice and relative frequency elicitation on overconfidence: Further tests of an exemplar-retrieval model, Journal of Behavioral Decision Making, 16, 127-45.
89. Simon, H. A. (1957) Models of Man: Social and Rational. New York: John Wiley & Sons.
90. Simon, H. A. (1975) The functional equivalence of problem solving skills, Psychology, 7, 268-88.
91. Smith, E. E. & Osherson, D. N. (1989) Similarity and decision making. In S. Vosniadou and A. Ortony (eds.), Similarity and Analogical Reasoning (pp. 60-76). Cambridge, MA: Cambridge University Press.
92. Smith, J. F. & Kida, T. (1991) Heuristics and biases: Expertise and task realism in auditing, Psychological Bulletin, 109 (3), 472-89.
93. Stewart, T. R., Roebber, P. J., & Bosart, L. F. (1997) The importance of the task in analyzing expert judgment, Organizational Behavior and Human Decision Processes, 69, 205-19.
94. US Army Research Institute (2001) Think like a Commander (Computer CD-ROM and Materials). Ft Leavenworth, KS: US Army Research Institute for the Behavioral and Social Sciences.
95. Wright, G. & Bolger, F. (eds.) (1993) Expertise and Decision Support: The Language of Science. New York, NY: Plenum.
96. Wright, G. & Goodwin, P. (eds.) (1998) Forecasting with Judgment. New York: John Wiley & Sons.
97. Yates, J. F., Veinott, E. S., & Patalano, A. L. (2003) Hard decisions, bad decisions: On decision quality and decision aiding. In S. L. Schneider and J. Shanteau (eds.), Emerging Perspectives on Judgment and Decision Research. New York: Cambridge University Press.
98. Zsambok, C. E. & Klein, G. (eds.) (1997) Naturalistic Decision Making. Mahwah, NJ: Lawrence Erlbaum Associates.