Performance measurement in simulation-based training: A review and best practices

Simulation and Gaming

Volumn 40, Issue 3, 2009, Pages 328-376

  Salas, Eduardo ^a   Rosen, Michael A ^a   Held, Janet D ^b   Weissmuller, Johnny J ^c  

^a UNIVERSITY OF CENTRAL FLORIDA   (United States)

^b and Technology (NPRST)   (United States)

^c HQ AFPC/DPSF ^*  (United States)

Author keywords

Best practices; Causes; Corrective feedback; Design; Diagnosis; Diagnostic measurment; Effectiveness; Feedback; Methdologies; Performance; Performance diagnosis; Performance measurement; Remediatiopn; SBT; Simulation based training; Team performance; Teams; Theoretical foundations

Indexed keywords

EID: 66149117460     PISSN: 10468781     EISSN: 1552826X     Source Type: Journal    
DOI: 10.1177/1046878108326734     Document Type: Conference Paper

References (160)

1. Arvey, R.D., & Murphy, K.R. (1998). Performance evaluation in work settings. Annual Review of Psychology, 49, 141-168.
2. Austin, J.R. (2003). Transactive memory in organizational groups: The effects of content, consensus, specialization, and accuracy on group performance. Journal of Applied Psychology, 88(5), 866-878.
3. Baddeley, A.D., & Hitch, G. (1993). The recency effect: Implicit learning with explicit retrieval? Memory and Cognition, 21, 146-155.
4. Bainbridge, L., & Sanderson, P. (2005). Verbal protocol analysis. In J. R. Wilson & E. M. Corlett (Eds.), Evaluation of human work: A practical ergonomics methodology (3rd ed., pp. 159-184). Boca Raton, FL: Taylor & Francis.
5. Baker, D.P., & Salas, E. (1992). Principles for measuring teamwork skills. Human Factors, 34(4), 469-475.
6. Bandow, D. (2001). Time to create sound teamwork. The Journal for Quality and Participation, 24, 41-47.
7. Banks, D., Bader, P., Fleming, P., Zaccaro, S., & Barber, H. (2001, April). Leader adaptability: The role of work experiences and leader attributes. Paper presented at the 16th Annual Conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
8. Bedard, J., & Chi, M.T.H. (1992). Expertise. Current Directions in Psychological Science, 1(4), 135-139.
9. Berger, H. (1999, November-December). Realtime feedback in truck driver training. Paper presented at the Interservice/Industry Simulation, Training, and Education Conference, Orlando, FL.
10. Bloom, B. (1956). Taxonomy of educational objectives: The cognitive domain. New York: Donald McKay.
11. Bordage, G., & Zacks, R. (1984). The structure of medical knowledge in the memories of medical students and general practitioners: Categories and prototypes. Medical Education, 18, 406-416.
12. Bowers, C.A., Jentsch, F., Salas, E., & Braun, C.C. (1998). Analyzing communication sequences for team training needs assessment. Human Factors, 40(4), 672-679.
13. Brannick, M.T., & Prince, C. (1997). Overview of team performance measurement. In M. T. Brannick, E. Salas, & C. Prince (Eds.), Team performance assessment and measurement: Theory, methods, and applications (pp. 3-16). Mahwah, NJ: Lawrence Erlbaum.
14. Broadbent, D. (1972). Decision and stress. New York: Academic Press.
15. Burke, C.S., Fiore, S.M., & Salas, E. (2004). The role of shared cognition in enabling shared leadership and team adaptability. In C. L. Pearce & J. A. Conger (Eds.), Shared leadership: Reframing the hows and whys of leadership (pp. 103-121). Thousand Oaks, CA: Sage.
16. Burke, C.S., Stagl, K.C., Salas, E., Pierce, L., & Kendall, D. (2006). Understanding team adaptation: A conceptual analysis & model. Journal of Applied Psychology, 91(6), 1189-1207.
17. Cacciabue, P.C., & Hollnagel, E. (1995). Simulation of cognition: Applications. In J. M. Hoc, P. C. Cacciabue, & E. Hollnagel (Eds.), Expertise and technology: Cognition and human-computer cooperation (pp. 55-73). Mahwah, NJ: Lawrence Erlbaum.
18. Campbell, J.P. (1990). Modeling the performance prediction problem in industrial and organizational psychology. In M. D. Dunette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (pp. 687-732). Palo Alto, CA: Consulting Psychologists Press.
19. Campbell, J.P., McCloy, R.A., Oppler, S.H., & Sager, E. (1993). A theory of performance. In N. Schmitt & W. Borman (Eds.), Personnel selection in organizations (pp. 35-70). San Francisco: Jossey-Bass.
20. Cannon-Bowers, J.A., & Salas, E. (1997). A framework for developing team performance measures in training. In M. T. Brannick, E. Salas, & C. Prince (Eds.), Team performance and measurement: Theory, methods, and applications (pp. 45-62). Mahwah, NJ: Lawrence Erlbaum.
21. Cannon-Bowers, J.A., Prince, C., Salas, E., Owens, J.M., Morgan, B.B., & Gonos, G.H. (1989). Determining aircrew coordination training effectiveness. Paper presented at the 11th annual meeting of the Interservice/Industry Training Systems Conference, Ft. Worth, TX.
22. Cannon-Bowers, J.A., Salas, E., & Converse, S. (1993). Shared mental models in expert team decision making. In N. J. J. Castellan (Ed.), Individual and group decision making (pp. 221-246). Hillsdale, NJ: Lawrence Erlbaum.
23. Card, S.K., Moran, T.P., & Newell, A. (1983). The psychology of human-computer interaction. Hillsdale, NJ: Lawrence Erlbaum.
24. Carley, K.M. (1997). Extracting team mental models through textual analysis. Journal of Organizational Behavior, 18, 533-558.
25. Carolan, T., MacMillan, J., Entin, E.B., Morley, R.M., Schreiber, B.T., Portrey, A., et al. (2003, December). Integrated performance measurement and assessment in distributed mission operations environments: Relating measures to competencies. Paper presented at the Interservice/Industry Training, Simulation, and Education Conference, Orlando, FL.
26. Chandler, T.N., & Owens, N.E. (2003, December). Using performance based coaching techniques for intelligent tutoring systems. Paper presented at the Interservice/Industry Training, Simulation, and Education Conference, Orlando, FL.
27. Chi, M.T.H. (2006). Two approaches to the study of experts' characteristics. In K. A. Ericsson, N. Charness, P. J. Fletovich, & R. R. Hoffman (Eds.), The Cambridge handbook of expertise and expert performance (pp. 21-30). New York: Cambridge University Press.
28. Chi, M.T.H., Feltovich, P.J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121-152.
29. Cohen, M.S., Freeman, J.T., & Wolf, S. (1996). Metarecognition in time-stressed decision making: Recognizing, critiquing and correcting. Human Factors, 38(2), 206-219.
30. Cooke, N.J., Salas, E., Cannon-Bowers, J.A., & Stout, R.J. (2000). Measuring team knowledge. Human Factors, 42, 151-173.
31. Cooke, N.J., Salas, E., Kiekel, P.A., & Bell, B. (2004). Advances in measuring team cognition. In E. Salas & S. M. Fiore (Eds.), Team cognition: Understanding the factors that drive process and performance (pp. 83-106). Washington, DC: American Psychiatric Association.
32. Cooke, N.J., Stout, R., & Salas, E. (1997, September). Broadening the measurement of situational awareness through cognitive engineering methods. Paper presented at the 41st Annual Meeting of the Human Factors and Ergonomics Society, Albuquerque, New Mexico.
33. Cooke, N.J., Stout, R.J., & Salas, E. (2001). A knowledge elicitation approach to the measurement of team situation awareness. In M. McNeese, E. Salas, & M. R. Endsley (Eds.), New trends in cooperative activities (pp. 114-139). Santa Monica, CA: Human Factors and Ergonomics Society.
34. Cooper, M., Viney, L., & McDermott, T. (2003, December). Electronic warfare rangeless embedded training: A cost effective training approach. Paper presented at the Interservice/Industry Training, Simulation, and Education Conference, Orlando, FL.
35. Cooper, R., & Sawaf, A. (1996). Executive EQ: Emotional intelligence in leadership and organizations. New York: Grosset/Putnam.
36. Drewes, P., & Gonzalez, A. (1997, March). Automated performance monitoring. Paper presented at the Interservice/Industry Simulation, Training and Education Conference, Orlando, FL.
37. Driskell, J.E., & Salas, E. (1991). Group decision making under stress. Journal of Applied Psychology, 76(3), 473-478.
38. Driskell, J.E., & Salas, E. (1992). Collective behavior and team performance. Human Factors, 34, 277 - 288. Dyer, J.L. (1984). Team research and team training: A state of the art review. In F. A. Muckler (Ed.), Human factors review (pp. 285 - 323). Santa Monica, CA: Human Factors Society.
39. Endsley, M.R. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors, 37(1), 32-64.
40. Endsley, M.R. (1997). The role of situation awareness in naturalistic decision making. In C. E. Zsambok & G. Klein (Eds.), Naturalistic decision making (pp. 269-284). Hillsdale, NJ: Lawrence Erlbaum.
41. Ericsson, K.A. (2006). An introduction to Cambridge handbook of expertise and expert performance: Its development, organization, and content. In K. A. Ericsson, N. Charness, P. J. Feltovich, & R. R. Hoffman (Eds.), The Cambridge handbook of expertise and expert performance (pp. 3-19). New York: Cambridge University Press.
42. Ericsson, K.A., & Kintsch, W. (1995). Long-term working memory. Psychological Review, 102(2), 211-245.
43. Ericsson, K.A., & Lehmann, A.C. (1996). Expert and exceptional performance: Evidence of maximal adaptation to task constraints. Annual Review of Psychology, 47, 273-305.
44. Ericsson, K.A., & Simon, H.A. (1993). Protocol analysis: Verbal reports as data. Cambridge, MA: The MIT Press.
45. Ericsson, K.A., & Smith, J. (1991). Prospects and limits of the empirical study of expertise: An introduction. In K. A. Ericsson & J. Smith (Eds.), Toward a general theory of expertise: Prospects and limits (pp. 1-38). Cambridge, UK: Cambridge University Press.
46. Ericsson, K.A., & Ward, P. (2007). Capturing the naturally occurring superior performance of experts in the laboratory: Toward a science of expert and exceptional performance. Current Directions in Psychological Science, 16(6), 346-350.
47. Faraj, S.A., & Sproull, L. (2000). Coordinating expertise in software development teams. Management Science, 46, 1554-1568.
48. Feltovich, P.J., Prietula, M.J., & Ericsson, K.A. (2006). Studies of expertise from psychological perspectives. In K. A. Ericsson, N. Charness, P. J. Feltovich, & R. R. Hoffman (Eds.), Cambridge handbook of expertise and expert performance (pp. 41-67). New York: Cambridge University Press.
49. Fivar, G. (1980). The critical incident technique: A bibliography (2nd ed.). Palo Alto, CA: American Institute for Research.
50. Flavell, J.H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-development inquiry. American Psychologist, 34(10), 906-911.
51. Flanagan, J.C. (1954). The critical incident technique. Psychological Bulletin, 51, 327-358.
52. Fleishman, E.A., Mumford, M.D., Zaccaro, S.J., Levin, K.Y., Korotkin, A.L., & Hein, M.B. (1991). Taxonomic efforts in the description of leader behavior: A synthesis and functional interpretation. Leadership Quarterly, 4, 245-287.
53. Fleming, B., Barto, J., & Johnson, C. (1998, November-December). Joint collective performance measures: Performance validation through measurement. Paper presented at the Interservice/Industry Training, Simulation and Education Conference, Orlando, FL.
54. Foushee, H.C., & Manos, K.L. (1981). Information transfer within the cockpit: Problems in intra cockpit communications (No. NASA TP-1875). Moffett Field, CA: NASA-Ames Research Center.
55. Fowlkes, J.E., Dwyer, D.J., Milham, L.M., Burns, J.J., & Pierce, L.G. (1999, November-December). Team skills assessment: A test and evaluation component for emerging weapon systems. Paper presented at the Interservice/Industry Training, Simulation, and Education Conference, Orlando, FL.
56. Fowlkes, J.E., Dwyer, D.J., Oser, R.L., & Salas, E. (1998). Event-based approach to training (EBAT). The International Journal of Aviation Psychology, 8(3), 209-221.
57. Fowlkes, J.E., Lane, N.E., Salas, E., Franz, T., & Oser, R. (1994). Improving the measurement of team performance: The TARGETs methodology. Military Psychology, 6, 47-61.
58. Frank, G., Meeds, H., Perkins, K., Whiteford, B., Arnold, P., Hubal, R., et al. (2004, December). Performance assessment for distributed learning using after action review reports generated by simulations. Paper presented at the Interservice/Industry Training, Simulation and Education Conference, Orlando, FL.
59. Gagnè, R.M. (1984). Learning outcomes and their effects: Useful categories of human performance. American Psychologist, 39, 377-385.
60. Gersick, C.J. (1988). Time and transition in work teams: Toward a new model of group development. Academy of Management Journal, 31(1), 9-41.
61. Glaser, R. (1987). Thoughts on expertise. In C. Schooler & K. W. Schaie (Eds.), Cognitive functioning and social structure over the life course (pp. 81-94). Norwood, NJ: Ablex.
62. Glaser, R., & Chi, M.T.H. (1988). Overview. In M. T. H. Chi, R. Glaser, & M. Farr (Eds.), The nature of expertise (pp. xv-xxviii). Hillsdale, NJ: Lawrence Erlbaum.
63. Graham, J., Schneider, M., Bauer, A., Bessiere, K., & Gonzalez, C. (2004, September). Shared mental models in military command and control organizations: Effect of social network distance. Paper presented at the Human Factors and Ergonomics Society 48th Annual Meeting, New Orleans, LA.
64. Gray, B., Bougon, M.G., & Donnellon, A. (1985). Organizations as constructions and destructions of meaning. Journal of Management, 11, 83-98.
65. Gully, S.M., Devine, D.J., & Whiney, D.J. (1995). A meta-analysis of the cohesion and performance: Effects of level of analysis and task interdependence. Small Group Research, 26, 497-520.
66. Guzzo, R.A., & Dickson, M.W. (1996). Teams in Organizations: Recent Research on Performance and Effectiveness. Annual Review of Psychology, 47, 307-338.
67. Hackman, J.R. (1983). A normative model of work team effectiveness (No. 2). New Haven, CT: Yale University Press.
68. Hackman, J.R. (2002). Leading teams: Setting the stage for great performances. Boston: Harvard Business School Press.
69. Hinsz, V.B., Tindale, R.S., & Vollrath, D.A. (1997). The Emerging conceptualization of Groups as Information Processors. Psychological Bulletin, 121(1), 43-64.
70. Hoffman, R.R., Crandall, B., & Shadbolt, N. (1998). Use of the critical decision method to elicit expert knowledge: A case study in the methodology of cognitive task analysis. Human Factors, 40(2), 254-276.
71. Hoffman, R.R., Feltovich, P.J., & Ford, K.M. (1997). A general conceptual framework for conceiving of expertise and expert systems. In P. J. Feltovich, K. M. Ford, & R. R. Hoffman (Eds.), Expertise in context (pp. 543-580). Menlo Park, CA: American Association for Artificial Intelligence Press/MIT Press.
72. Jentsch, F., Sellin-Wolters, S., Bowers, C.A., & Salas, E. (1995, October). Crew coordination behaviors as predictors of problem detection and decision making times. Paper presented at the Human Factors and Ergonomics Society 30th Annual Meeting, Santa Monica, CA.
73. Johnson-Laird, P.N. (1983). Mental models: Towards a cognitive science of language, inference, and consciousness. Cambridge, MA: Harvard University Press.
74. Jones, G., & George, J. (1998). The experience and evolution of trust: Implications for cooperation and teamwork. Academy of Management Review, 23, 531-546.
75. Kanki, B.G., Lozito, S., & Foushee, H.C. (1989). Communication indices of crew coordination. Aviation, Space, & Environmental Medicine, 60, 56-60.
76. Keikel, P.A., Cooke, N.J., Foltz, P.W., & Shope, S.M. (2001). Automating measurement of team cognition through analysis of communication data. In M. J. Smith, G. Salvendy, D. Harris, & R. J. Koubek (Eds.), Usability evaluation and interface design (pp. 1382-1386). Mahwah, NJ: Lawrence Erlbaum.
77. Kelly, M. (1999, November-December). After action review in synthetic environment based training systems: A training system not a technology. Paper presented at the Interservice/Industry Simulation, Training, and Education Conference, Orlando, FL.
78. Kendall, D.L., & Salas, E. (2004). Measuring team performance: Review of current methods and consideration of future needs. In J. W. Ness, V. Tepe, & D. Ritzer (Eds.), The science and simulation of human performance (pp. 307-326). Boston: Elsevier.
79. Klein, G. (1993). A recognition primed decision (RPD) model of rapid decision making. In G. Klein, J. Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in action (pp. 138-147). Norwood, NJ: Ablex.
80. Klein, G.A., Calderwood, R., & MacGregor, D. (1989). Critical decision method for eliciting knowledge. IEEE Transactions on Systems, Man, and Cybernetics, 19(3), 462-472.
81. Klein, G., Orasanu, J., Calderwood, R., & Zsambok, C.E. (1993). Decision making in action. Norwood, NJ: Ablex.
82. Klein, G., Ross, K.G., Moon, B.M., Klein, D.E., Hoffman, R.R., & Hollnagel, E. (2003). Macrocognition. IEEE Intelligent Systems, 18(3), 81-85.
83. Klein, K., & Kozlowski, S.W.J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent processes. In K. J. Klein (Ed.), Multilevel theory, research, and methods in organizations: Foundations, extensions, and new directions (pp. 3-90). San Francisco: Jossey-Bass.
84. Kleinman, D.L., & Serfaty, D. (1989, June). Team performance assessment in distributed decision-making. Paper presented at the Symposium on Interactive Network Simulation for Training, Orlando, FL.
85. Klimoski, R., & Mohammed, S. (1994). Team mental model: Construct or metaphor? Journal of Management, 20(2), 403-437.
86. Kluwe, R.H. (1987). Executive decisions and regulation of problem-solving behavior. In F. E. Weinert & R. H. Kluwe (Eds.), Metacognition, motivation, and understanding (pp. 31-64). Hillsdale, NJ: Lawrence Erlbaum.
87. Kozlowski, S.W.J. (1998). Training and developing adaptive teams: Theory, principles, and research. In J. A. Cannon-Bowers & E. Salas (Eds.), Making decisions under stress: Implications for individual and team training (pp. 115-153). Washington, DC: American Psychiatric Association.
88. Kozlowski, S.W.J., & Bell, B.S. (2003). Work groups and teams in organizations. In W. C. Borman, D. R. Ilgen, & R. J. Klimoski (Eds.), Handbook of psychology: Industrial and organizational psychology (Vol. 12, pp. 333-375). London: Wiley.
89. Kozlowski, S.W.J., Gully, S.M., Nason, E.R., & Smith, E.M. (1999). Developing adaptive teams: A theory of compilation and performance across levels and time. In D. R. Ilgen & E. D. Pulakos (Eds.), The changing nature of work and performance: Implications for staffing, personnel actions, and development (pp. 240 - 292). San Francisco: Jossey-Bass.
90. Kozlowski, S.W., Gully, S.M., Smith, E.M., Nason, E.R., & Brown, K.G. (1995). Sequenced mastery training and advanced organizers: Effects on learning, self-efficacy, performance, and generalization. Paper presented at the Symposium conducted at the Tenth Annual Conference of the Society for Industrial and Organizational Psychology, Orlando, FL.
91. Kozlowski, S.W.J., & Klein, K.J. (2000). A Multilevel Approach to Theory and Research in Organizations: Contextual, Temporal, and Emergent Processes. In K. J. Klein & S. W. J. Kozlowski (Eds.), Multilelvel Theory, Research, and Methods in Organizations: Foundations, Extensions, and New Directions (pp. 3 - 90). San Francisco, CA: Jossey-Bass.
92. Kraiger, K., Ford, J.K., & Salas, E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology, 78(2), 311-328.
93. Leddo, J., Zhang, Z., & Pokorny, R. (1998, November-December). Automated performance assessment tools. Paper presented at the Interservice/Industry Training, Simulation, and Education Conference, Orlando, FL.
94. Lesgold, A.M., Rubinson, H., Feltovich, P.J., Glaser, R., Klopfer, D., & Wang, Y. (1988). Expertise in a complex skill: Diagnosing x-ray pictures. In M. T. H. Chi, R. Glaser, & M. Farr (Eds.), The nature of expertise (pp. 311-342). Hillsdale, NJ: Lawrence Erlbaum.
95. Lewis, K. (2000, August). Transactive memory and performance of management consulting teams: Examining construct and predictive validity of a new scale. Paper presented at the Annual Meeting of the Academy of Management, Toronto, Ontario, Canada.
96. Lord, R.G., & Emrich, C.G. (2001). Thinking outside the box by looking inside the box: Extending the cognitive revolution in leadership research. Leadership Quarterly, 11, 551-579.
97. MacBryde, J., & Mendibil, K. (2003). Designing performance measurement systems for teams: Theory and practice. Management Decision, 41(8), 722-733.
98. Marks, M.A., Mathieu, J.E., & Zaccaro, S.J. (2001). A temporally based framework and taxonomy of team processes. Academy of Management Review, 26, 356-376.
99. Marks, M.A., Zaccaro, S.J., & Mathieu, J.E. (2000). Performance implications of leader briefings and team interaction training for team adaptation to novel environments. Journal of Applied Psychology, 85, 971-986.
100. Marsh, S.M., Kiechel, L., Boyce, L.A., & Zaccaro, S.J. (2001, April). Leadership and team adaptation: Examining the attributes and training strategies that promote effective performance in dynamic environments. Paper presented at the 13th Annual Conference of Society for Industrial and Organizational Psychology, San Diego, CA.
101. Marshall-Mies, J.C., Fleishman, E.A., Martin, J.A., Zaccaro, S.J., Baughman, W.A., & McGee, M.L. (2000). Development and evaluation of cognitive and metacognitive measures for predicting leadership potential. Leadership Quarterly, 11, 135-154.
102. Mathieu, J.E., Heffner, T.S., Goodwin, G.F., Salas, E., & Cannon-Bowers, J. (2000). The influence of shared mental models on team process and performance. Journal of Applied Psychology, 85(2), 273-283.
103. Mazzoni, G., & Nelson, T. O. (Eds.). (1998). Metacognition and cognitive neuropsychology: Monitoring and control processes. Mahwah, NJ: Lawrence Erlbaum.
104. McCloy, R.A., Campbell, J.P., & Cudeck, R. (1994). A confirmatory test of a model of performance determinants. Journal of Applied Psychology, 79(4), 493-505.
105. McIntyre, R.M., & Salas, E. (1995). Measuring and managing for team performance: Emerging principles from complex environments. In R. A. Guzzo & E. Salas (Eds.), Team Effectiveness and Decision Making in Organizations (pp. 9-45). San Francisco, CA: Jossey Bass.
106. Meliza, L.L., & Brown, B. (1996, December). Increasing speed and flexibility of feedback systems for DIS exercises. Paper presented at the Interservice/Industry Training, Simulation and Education Conference, Orlando, FL.
107. Meliza, L.L., & Paz, B. (1996, December). Integrating exercise control and feedback systems in DIS. Paper presented at the Interservice/ Industry Training, Simulation, and Education Conference, Orlando, FL.
108. Mohammed, S., & Dumville, B.C. (2001). Team mental models in a team knowledge framework: Expanding theory and measure across disciplinary boundaries. Journal of Organizational Behavior, 22(2), 89-103.
109. Mohammed, S., Klimoski, R., & Rentsch, J.R. (2000). The measurement of team mental models: We have no shared schema. Organizational Research Methods, 3, 123-165.
110. Moors, A., & De Houwer, J. (2006). Automaticity: A theoretical and conceptual analysis. Psychological Bulletin, 132(2), 297-326.
111. Moreland, R.L. (1999). Transactive memory: Learning who knows what in work groups and organizations. In L. L. Thompson & J. M. Levine (Eds.), Shared cognition in organizations: The management of knowledge (pp. 3-31). Mahwah, NJ: Lawrence Erlbaum.
112. Moreland, R.L., Argote, L., & Krishnan, R. (1996). Socially shared cognition at work: Transactive memory and group performance. In J. L. Nye & A. M. Brower (Eds.), What's social about social cognition? Research on socially shared cognition in small groups (pp. 57-84). Thousand Oaks, CA: Sage.
113. Morgan, B.B., Jr., Lassiter, D.L., & Salas, E. (1989). Networked simulation application for team training and performance research. Paper presented at the meeting for Interactive Networked Simulation in Training, Orlando, FL.
114. Morgan, B.B., Jr., Salas, E., & Glickman, A.S. (2001). An analysis of team evolution and maturation. The Journal of General Psychology, 120(3), 277-291.
115. Motowildo, S.J. (2003). Job performance. In W. C. Borman, D. R. Ilgen, & R. J. Klimoski (Eds.), Comprehensive handbook of psychology (Vol. 12: Industrial and Organizational Psychology, pp. 39-53). New York: Wiley.
116. Murphy, K.R., & Constans, J.L. (1987). Behavioral anchors as a source of bias in rating. Journal of Applied Psychology, 72, 573-577.
117. Murphy, K.R., & Pardaffy, V.A. (1989). Bias in behaviorally anchored rating scales: Global or scale-specific? Journal of Applied Psychology, 74, 343-346.
118. Orasanu, J. (1990). Shared mental models and crew performance. Princeton, NJ: Laboratory of Cognitive Science, Princeton University.
119. Orasanu, J., & Fischer, U. (1991, April). Information transfer and shared mental models of decision making. Paper presented at the Sixth International Symposium on Aviation Psychology, Columbus, OH.
120. Oser, R.L., Cannon-Bowers, J.A., Salas, E., & Dwyer, D.J. (1999). Enhancing human performance in technology-rich environments: Guidelines for scenario-based training. In E. Salas (Ed.), Human/technology interaction in complex systems (Vol. 9, pp. 175-202). Greenwich, CT: JAI Press.
121. Pearce, C.L., & Sims, H.P. (2002). Vertical versus shared leadership as predictors of the effectiveness of change management teams: An examination of aversive, directive, transactional, transformational and empowering leader behaviors. Group Dynamics: Theory, Research, and Practice, 6(2), 172-197.
122. Porter, C.O., Hollenbeck, J.R., Ilgen, D.R., Ellis, A.P., West, B.J., & Moon, H. (2003). Backing up behaviors in teams: The role of personality and legitimacy of need. Journal of Applied Psychology, 88(3), 391-403.
123. Posner, M.I. (1978). Chronometric explorations of the mind. Hillsdale, NJ: Lawrence Erlbaum.
124. Prince, C., & Salas, E. (1997). Situation assessment for routine and flight decision making. International Journal of Cognitive Ergonomics, 1, 315-204.
125. Rosen, M.A., Salas, E., Silvestri, S., Wu, T., & Lazzara, E.H. (2008). A measurement tool for simulation-based training in emergency medicine: The Simulation Module for Assessment of Resident Targeted Event Responses (SMARTER) approach. Simulation in Healthcare, 3(3), 170-179.
126. Rosen, M.A., Salas, E., Wu, T.S., Silvestri, S., Lazzara, E.H., Lyons, R., et al. (in press). Promoting teamwork: An event-based approach to simulation-based teamwork training for emergency medicine residents. Academic Emergency Medicine.
127. Salas, E., Burke, C.S., Fowlkes, J.E., & Preist, H.A. (2003). On measuring teamwork skills. In J. C. Thomas (Ed.), Comprehensive Handbook of Psychological Assessment (Vol. 4, Industrial and organizational assessment, pp. 427-442). Hoboken, NJ: John Wiley & Sons.
128. Salas, E., Burke, C.S., & Stagl, K.C. (2004). Developing teams and team leaders: Strategies and principles. In D. Day, S. J. W. Zaccaro, & S. M. Halpin (Eds.), Leader development for transforming organizations: Growing leaders for tomorrow (pp. 325-355). Mahwah, NJ: Lawrence Erlbaum.
129. Salas, E., Dickinson, T., Converse, S., & Tannenbaum, S. (1992). Toward and understanding of team performance and training. In R. Swezey & E. Salas (Eds.), Teams: Their training and performance (pp. 3-29). Norwood, NJ: Ablex.
130. Salas, E., & Fiore, S.M. (2004). Why team cognition? An overview. In E. Salas & S. M. Fiore (Eds.), Team cognition (pp. 3-8). Washington, DC: American Psychological Association.
131. Salas, E., Priest, H.A., & Burke, C.S. (2005). Teamwork and team performance measurement. In J. R. Wilson & N. Corlett (Eds.), Evaluation of human work (3rd ed., pp. 793-808). Boca Raton, FL: Taylor & Francis.
132. Salas, E., Prince, C., Baker, D.P., & Shrestha, L. (1995). Situation awareness in team performance: Implications for measurement and training. Human Factors, 37(1), 123-136.
133. Salas, E., & Rosen, M.A. (in press). Experts at work: Principles developing expertise in organizations. In S. W. J. Kozlowski and E. Salas (Eds.), Learning, training, and development in organizations. Mahwah, NJ: Lawrence Erlbaum.
134. Salas, E., Rosen, M.A., Burke, C.S., Nicholson, D., & Howse, W.R. (2007). Markers for enhancing team cognition in complex environments: The power of team performance diagnosis. Aviation, Space, and Environmental Medicine Special Supplement on Operational Applications of Cognitive Performance Enhancement Technologies, 78(5), B77-B85.
135. Salas, E., Sims, D.E., & Burke, C.S. (2005). Is there a big five in teamwork ? Small Group Research, 36(5), 555-599.
136. Salas, E., Stagl, K.C., & Burke, C.S. (2004). 25 years of team effectiveness in organizations: Research themes and emerging needs. International Review of Industrial and Organizational Psychology, 19, 47-91.
137. Sanderson, P.M., & Fisher, C. (1994). Exploratory sequential data analysis: Foundations. Human-Computer Interaction, 9, 251-317.
138. Schreiber, B.T., Watz, E.A., & Winston, B. (2003, December). Objective human performance measurement in a distributed environment: Tomorrow's needs. Paper presented at the Interservice/Industry Training, Simulation and Education Conference, Orlando, FL.
139. Schvaneveldt, R.W. (1990). Proximities, networks, and schemata. In R. W. Schvaneveldt (Ed.), Pathfinder associative networks: Studies in knowledge organization (pp. 135-148). Norwood, NJ: Ablex.
140. Shadbolt, N. (2005). Eliciting expertise. In J. R. Wilson & N. Corlett (Eds.), Evaluation of human work (pp. 185-218). Boca Raton, FL: Taylor & Francis.
141. Simon, H.A., & Chase, W.G. (1973). Skill in chess. American Scientist, 61, 393-403.
142. Sonnentag, S., Niessen, C., & Volmer, J. (2006). Expertise in software design. In K. A. Ericsson, N. Charness, P. J. Feltovich, & R. R. Hoffman (Eds.), The Cambridge handbook of expertise and expert performance (pp. 373-387). Cambridge, UK: Cambridge University Press.
143. Smith, P.C., & Kendall, L.M. (1963). Retranslations or expectations: An approach to the construction of unambiguous anchors for rating scales. Journal of Applied Scales, 47, 149-155.
144. Smith-Jentsch, K.A., Johnston, J.H., & Payne, S.C. (1998). Measuring team-related expertise in complex environments. In J. A. Cannon-Bowers & E. Salas (Eds.), Decision Making Under Stress: Implications for Individual and Team Training (pp. 39-60). Washington, DC: APA.
145. Stacy, W., Freeman, J., Lackey, S., & Merket, D. (2004, December). Enhancing simulation-based training with performance measurement objects. Paper presented at the Interservice/Industry Simulation, Training, and Education Conference, Orlando, FL.
146. Sternberg, R.J. (1997). Cognitive conceptions of expertise. In P. J. Fletovich, K. M. Ford, & R. R. Hoffman (Eds.), Expertise in context (pp. 149-162). Menlo Park, CA: American Association for Artificial Intelligence Press/MIT Press.
147. Sternberg, R.J. (1998). Metacognition, abilities, and developing expertise: What makes an expert student? Instructional Science, 26, 127-140.
148. Sternberg, S. (1969). The discovery of processing stages: Extensions of Donder's method. Acta Psychologica, 30, 276-315.
149. Stout, R.J., Cannon-Bowers, J.A., & Salas, E. (1996). The role of shared mental models in developing team situational awareness: Implications for training. Training Research Journal, 2, 85-116.
150. Tagger, S., & Seijts, G.H. (2003). Leader and staff role efficacy as antecedents of collective-efficacy and team performance. Human Performance, 16, 131-156.
151. Tannenbaum, S.I., Beard, R.L., & Salas, E. (1992). Team building and its influence on team effectiveness: An examination of conceptual and empirical developments. In K. Kelley (Ed.), Issue, theory, and research in industrial/organizational psychology (pp. 117-153). Amsterdam: Elsevier.
152. Tubré, T., Arthur, W., Jr., & Bennett, W., Jr. (2006). General models of job performance: Theory and practice. In W. Bennett, C. E. Lance, W. Bennett, Jr., & D. J. Woehr (Eds.), Performance measurement: Current perspectives and future challenges (pp. 175-203). Mahwah, NJ: Lawrence Erlbaum.
153. Webber, S.S. (2002). Leadership and trust facilitating cross-functional team success. Journal of Management Development, 21, 201-214.
154. Wegner, D.M. (1986). Transactive memory: A contemporary analysis of the group mind. In B. Mullen & G. R. Goethals (Eds.), Theories of group behavior (pp. 185-208). New York: Springer-Verlag.
155. Weick, K.E., & Bougon, M.G. (1986). Organizations as cognitive maps: Charting ways to success and failure. In H. P. Sims & D. A. Gioia (Eds.), The thinking organization: Dynamics of organizational social cognition (pp. 102-135). San Francisco: Jossey Bass.
156. Wickens, C.D., & Carswell, C.M. (1997). Information processing. In G. Salvendy (Ed.), Handbook of human factors and ergonomics (pp. 89-129). New York: John Wiley.
157. Yoo, Y., & Kanawattanachai, P. (2001). Developments of transactive memory systems and collective mind in virtual teams. International Journal of Organizational Analysis, 9(2), 187-208.
158. Zaccaro, S.J., Rittman, A.L., & Marks, M.A. (2001). Team leadership. Leadership Quarterly, 12, 451-483.
159. Zachary, W., Bilazarian, P., Burns, J., & Cannon-Bowers, J.A. (1997,
160. Zeitz, C.M. (1997). Some concrete advantages of abstraction: How experts' representations facilitate reasoning. In P. J. Feltovich, K. M. Ford, & R. R. Hoffman (Eds.), Expertise in context (pp. 43-65). Menlo Park, CA: American Association for Artificial Intelligence Press/MIT Press.