Conceptualization and measurement of team workload: A critical need

Human Factors

Volumn 54, Issue 1, 2012, Pages 36-51

  Funke, Gregory J ^a   Knott, Benjamin A ^a   Salas, Eduardo ^b   Pavlas, Davin ^b   Strang, Adam J ^c  

^a AIR FORCE RESEARCH LABORATORY   (United States)

^b UNIVERSITY OF CENTRAL FLORIDA   (United States)

^c Consortium Research Fellows Program ^*  (United States)

Author keywords

measurement; performance measures; physiological measures; strategy shift measures; subjective measures; team workload

Indexed keywords

PERFORMANCE MEASURE; PHYSIOLOGICAL MEASURES; STRATEGY SHIFT MEASURES; SUBJECTIVE MEASURES; TEAM WORKLOAD;

MEASUREMENTS; PHYSIOLOGY;

RESEARCH;

COGNITION; CONCEPT FORMATION; HUMAN; NONBIOLOGICAL MODEL; PHYSIOLOGY; PSYCHOLOGICAL ASPECT; PSYCHOMOTOR PERFORMANCE; PUBLIC RELATIONS; QUESTIONNAIRE; REVIEW; TASK PERFORMANCE; WORK; WORKLOAD;

COGNITION; CONCEPT FORMATION; HUMANS; INTERPROFESSIONAL RELATIONS; MODELS, ORGANIZATIONAL; PSYCHOMOTOR PERFORMANCE; QUESTIONNAIRES; TASK PERFORMANCE AND ANALYSIS; WORK; WORKLOAD;

EID: 84856424781     PISSN: 00187208     EISSN: 15478181     Source Type: Journal    
DOI: 10.1177/0018720811427901     Document Type: Review

References (89)

1. Abarbanel H. D. I. Analysis of observed chaotic data. New York, NY: Springer-Verlag ; 1996 :
2. Bailey L. L., Willems B. F. The moderator effects of taskload on the interplay between en route intra-sector team communications, situation awareness, and mental workload. Washington, DC: National Technical Information Service ; 2002 :
3. Beith B. H.. Subjective workload under individual and team performance conditions. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 1987 ; 31: 67-71
4. Bell B. S., Kozlowski S. W. J.. A typology of virtual teams: Implications for effective leadership. Group & Organization Management. 2002 ; 27: 14-49 (Pubitemid 36649934)
5. Belyavin A. Foundation of augmented cognition. Schmorrow D., ed. Mahwah, NJ: Lawrence Erlbaum ; 2005: 430-437.
6. Bolia R. S., Nelson W. T.. Characterizing team performance in network-centric operations: Philosophical and methodological issues. Aviation, Space, and Environmental Medicine. 2007 ; 78: B71 - B76
7. Bolstad C. A., Endsley M. R. The effects of task load and shared displays on team situation awareness. Washington, DC: Defense Technical Information Center ; 1999 :
8. Bonito J. A.. The analysis of participation in small groups: Methodological and conceptual issues related to interdependence. Small Group Research. 2002 ; 33: 412-438 (Pubitemid 36668685)
9. Bonner M. A., Wilson G. F.. Heart rate measures of flight test and evaluation. International Journal of Aviation Psychology. 2002 ; 12: 63-77 (Pubitemid 37110130)
10. Bowers C., Braun C., Morgan B. Team performance assessment and measurement: Theory, methods, and applications. Brannick M. T. Salas E. Prince C., ed. Mahwah, NJ: Lawrence Erlbaum ; 1997: 85-108.
11. Bowers C. A., Jentsch F. Handbook of human factors and ergonomic methods. Stanton N. Hedge A. Brookhuis K. Salas E. Hendrick H., ed. Boca Raton, FL: CRC Press ; 2005: 57-1-57-3.
12. Bowers C. A., Urban J. M., Morgan B. B. The study of crew coordination and performance in hierarchical team decision making. Orlando: University of Central Florida, Team Performance Laboratory ; 1992 :
13. Brannick M. T., Prince A., Prince C., Salas E.. The measurement of team process. Human Factors. 1995 ; 37: 641-651 (Pubitemid 26012126)
14. Cannon-Bowers J. A., Salas E., Blickensderfer E., Bowers C. A.. The impact of cross-training and workload on team functioning: A replication and extension of initial findings. Human Factors. 1998 ; 40: 92-101 (Pubitemid 28194525)
15. Cooke N. J., Stout R. J., Salas E. New trends in cooperative activities: System dynamics in complex settings. McNeese M. Endsley M. Salas E., ed. Santa Monica, CA: Human Factors and Ergonomics Society ; 2001: 114-139.
16. Dale R., Spivey M. J.. Unraveling the dyad: Using recurrence analysis to explore patterns of syntactic coordination between children and caregivers in conversation. Language Learning. 2006 ; 56: 391-430 (Pubitemid 44181339)
17. Dickinson T. L., McIntyre R. M. Team performance assessment and measurement: Theory, methods, and applications. Brannick M. T. Salas E. Prince C., ed. Mahwah, NJ: Lawrence Erlbaum ; 1997: 19-44.
18. Elkins A. N., Muth E. R., Hoover A. W., Walker A. D., Carpenter T. L., Switzer F. S.. Physiological compliance and team performance. Applied Ergonomics. 2009 ; 40: 997-1003
19. Entin E. E., Entin E. B.. Measures for evaluation of team processes and performance in experiments and exercises. Paper presented at the International Command and Control Research and Technology Symposium ; Annapolis, MD ;..
20. Entin E. E., MacMillan J., Serfaty D.. Analysis of organizational processes in adaptive command and control architectures. Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics ; 1998 ; 1998. 3664.
21. Entin E. E., Serfaty D.. Adaptive team coordination. Human Factors. 1999 ; 41: 312-325 (Pubitemid 29330654)
22. Entin E. E., Serfaty D., Kerrigan C.. Choice and performance under three command and control architectures. Paper presented at the 1998 Command and Control Research and Technology Symposium ; Monterey, CA ;..
23. Finomore V. S., Knott B. A., Nelson W. T., Galster S. M., Bolia R. S.. The effects of multimodal collaboration technology on subjective workload profiles of tactical air battle management teams. Proceedings of the International Symposium on Aviation Psychology ; 2007 ; 2007. 190.
24. Fishel S. R., Muth E. R., Hoover A. W.. Establishing appropriate physiological baseline procedures for real-time physiological measurement. Journal of Cognitive Engineering and Decision Making. 2007 ; 1: 286-308
25. Funke G. J., Bennett A. M., Nelson W. T., Galster S. M.. Collaboration technologies and the supervisory control of UCAVs in tactical C2: Effects on performance and workload. Proceedings of the International Symposium on Aviation Psychology ; 2007 ; 2007. 216.
26. Funke G. J., Galster S. M.. The effects of cognitive processing load and collaboration technology on team performance in a simulated command and control environment. International Journal of Industrial Ergonomics. 2009 ; 39: 541-547
27. Funke G. J., Knott B. A., Galster S. M., Brown R. D.. Team workload: A construct worth pursuing?. Proceedings of the International Symposium on Aviation Psychology ; 2009 ; 2009. 378.
28. Funke G. J., Russell S. M., Knott B. A., Miller B. T.. What can a multidimensional measure of stress tell us about team collaborative tools?. Proceedings of the International Symposium on Aviation Psychology ; 2009 ; 2009. 480.
29. GalsterS. M.KnottB. A. (2007). The Team Workload Assessment Scale (TWAS). Unpublished manuscript.
30. George J. A., James L. R.. Personality, affect, and behavior in groups revisited: Comment on aggregation, levels of analysis, and a recent application of within and between analysis. Journal of Applied Psychology. 1993 ; 78: 798-804
31. Glick W. H.. Conceptualizing and measuring organizational and psychological climate: Pitfalls in multilevel research. Academy of Management Review. 1985 ; 10: 601-616
32. Glickman A. S., Zimmer S., Montero R. C., Guerette P. J., Campbell W. J., Morgan B. B., Salas E. The evolution of teamwork skills: An empirical assessment with implications for training. Orlando, FL: Naval Training Systems Center ; 1987 :
33. Gopher D., Donchin E. Handbook of perception and human performance: Vol. 2. Cognitive processes and performance. Boff K. R. Kaufmann L. Thomas J. P., ed. New York, NY: John Wiley ; 1986: 41-1-41-49.
34. Hancock P. A., Mihaly T., Rahimi M., Meshkati N. Human mental workload. Hancock P. A. Meshkati N., ed. New York, NY: North-Holland ; 1988: 329-382.
35. Hart S. G., Staveland L. E. Human mental workload. Hancock P. A. Meshkati N., ed. Amsterdam, Netherlands: North-Holland ; 1988: 139-183.
36. Henning R. A., Armstead A. G., Ferris J. K.. Social psychophysiological compliance in a four-person research team. Applied Ergonomics. 2009 ; 40: 1004-1010
37. Henning R. A., Boucsein W., Gil M. C.. Social- physiological compliance as a determinant of team performance. International Journal of Psychophysiology. 2001 ; 40: 221-232 (Pubitemid 32157072)
38. Hildebrand G. A., Pharmer J. A., Weaver M. D.. Performance and usability evaluation of an automated task management concept prototype design. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 2003 ; 47: 1913-1917
39. Hollan J., Hutchins E., Kirsch D.. Distributed cognition: Toward a new foundation for human-computer interaction research. ACM Transactions on Computer-Human Interaction. 2000 ; 7: 174-196
40. Kirkman B. L., Tesluk P. E., Rosen B.. Assessing the incremental validity of team consensus ratings over aggregation of individual-level data in predicting team effectiveness. Personnel Psychology. 2001 ; 54: 645-667 (Pubitemid 33406135)
41. Knott B. A., Bolia R. S., Nelson W. T., Galster S. M.. The impact of instant messaging on team performance, subjective workload, and situation awareness in tactical command and control. Paper presented at the 11th International Command and Control Research and Technology Symposium ; 2006, September Cambridge, UK ;..
42. Knott C. C., Nelson W. T., McCroskey M. K., Miller B. T. Analysis of eye movements in change detection with teams using a simulated tactical situation display. Washington, DC: Defense Technical Information Center ; 2008 :
43. Kozlowski S. W. J., Ilgen D. R.. Enhancing the effectiveness of work groups and teams. Psychological Science in the Public Interest. 2006 ; 7: 77-124
44. Kozlowski S. W. J., Klein K. J. Multilevel theory, research, and methods in organizations. Klein K. J. Kozlowski S. W. J., ed. San Francisco, CA: Jossey-Bass ; 2000: 3-90.
45. Lin C. J., Hsieh T. L., Tsai P. J., Yang C. W., Yenn T. C.. Development of a team workload assessment technique for the main control room of advanced nuclear power plants. Human Factors and Ergonomics in Manufacturing & Service Industries. 2011 ; 21: 397-411
46. MacMillan J., Entin E. E., Serfaty D. Team cognition: Understanding the factors that drive process and performance. Salas E. Fiore S. M., ed. Washington, DC: American Psychological Association ; 2004: 61-82.
47. MacMillan J., Paley M. J., Entin E. B., Entin E. E. Handbook of human factors and ergonomics methods. Stanton N. Hedge A. Brookhuis K. Salas E. Hendrick H., ed. Boca Raton, FL: CRC Press ; 2005: 51-1-51-9.
48. Marks M. A., Mathieu J. E., Zaccaro S. J.. A temporally based framework and taxonomy of team processes. Academy of Management Review. 2001 ; 26: 356-376
49. Mathieu J. E., Heffner T. S., Goodwin G. F., Salas E., Cannon-Bowers J. A.. The influence of shared mental models on team process and performance. Journal of Applied Psychology. 2000 ; 85: 273-283
50. McGrath J. E. Groups: Interaction and performance. Englewood Cliffs, NJ: Prentice Hall ; 1984 :
51. Moray N. Mental workload: Its theory and measurement. Moray N., ed. New York, NY: Plenum ; 1979: 13-22.
52. Morgan B. B., Salas E., Glickman A. S.. An analysis of team evolution and maturation. Journal of General Psychology. 1994 ; 120: 277-291
53. Murai K., Okazaki K., Hayashi Y.. Measurement for mental workload of bridge team on leaving/entering port. Paper presented at the 2004 IEEE Position, Location, and Navigation Symposium ; Monterey, CA ;..
54. Murray D. M., Varnell S. P., Blitstein J. L.. Design and analysis of group-randomized trials: A review of recent methodological developments. American Journal of Public Health. 2004 ; 94: 423-432 (Pubitemid 38269495)
55. Norman D. A., Bobrow D. G.. On data-limited and resource-limited processes. Cognitive Psychology. 1975 ; 7: 44-64
56. Parasuraman R., Sheridan T. B., Wickens C. D.. Situation awareness, mental workload, and trust in automation: Viable, empirically supported cognitive engineering constructs. Journal of Cognitive Engineering and Decision Making. 2008 ; 2: 140-160
57. Pharmer J., Cropper K., McKneely J., Williams E. Tactical Tomahawk weapon control system V6 land attack combat system prototype human-computer interface: Test report for FY 03 fleet operability test. San Diego, CA: Naval Air Systems Command, Space and Naval Warfare Systems Center ; 2004 :
58. Reid G. B., Nygren T. E. Human mental workload. Hancock P. A. Meshkati N., ed. New York, NY: North-Holland ; 1988: 185-218.
59. Richman J. S., Moorman J. R.. Physiological time series analysis using approximate entropy and sample entropy. American Journal of Physiology: Heart and Circulatory Physiology. 2000 ; 278: H2039 - H2049
60. RileyM. A.Van OrdenG. C. (Eds.). (2005). Tutorials in contemporary nonlinear methods for the behavioral sciences. Retrieved from http://www.nsf.gov/sbe/bcs/pac/nmbs/nmbs.jsp
61. Rouse W. B., Cannon-Bowers J. A., Salas E.. The role of mental models in team performance in complex systems. IEEE Transactions on Systems, Man, and Cybernetics. 1992 ; 22: 1296-1308 (Pubitemid 23625913)
62. Team effectiveness in complex organizations: Cross-disciplinary perspectives and approaches. Salas E. Burke S. Goodwin J., ed. San Francisco, CA: Jossey-Bass ; 2009 :
63. Team cognition. Salas E. Fiore S. M., ed. Washington, DC: American Psychological Association ; 2004 :
64. SchwartzD. (2008). Exploring team dynamics: The evolution of coordination in a complex command and control environment (Unpublished doctoral dissertation). Wright State University, Dayton, OH.
65. Sebok A.. Team performance in process control: Influences of interface design and staffing levels. Ergonomics. 2000 ; 43: 1210-1236 (Pubitemid 30657657)
66. Serfaty D., Entin E. E., Deckert J. C. Team adaptation to stress in decision making and coordination with implications for CIC team training. Burlington, MA: ALPHATECH ; 1993 :
67. Shi J., Luh P., Kleinman D. L., Serfaty D.. Information and command strategies in hierarchical team decision making. Paper presented at the American Control Conference ; Boston, MA ;..
68. Shingledecker C. A. The practical assessment of pilot workload. Roscoe A., ed. Neuilly sur Seine, France: North Atlantic Treaty Organization, Advisory Group for Aerospace Research and Development ; 1987: 11-14.
69. ShockleyK. (2005). Cross recurrence quantification of interpersonal postural activity. In RileyM. A.Van OrdenG. C. (Eds.), Tutorials in contemporary nonlinear methods for the behavioral sciences (pp. 142-177). Retrieved from http://www.nsf.gov/sbe/bcs/pac/nmbs/nmbs.jsp
70. Singleton W. T. The mind at work: Psychological ergonomics. Cambridge, UK: Cambridge University Press ; 1989 :
71. Steiner I. D. Group processes and productivity. New York, NY: Academic Press ; 1972 :
72. Stern R. M., Ray W. J., Quigley K. S. Psychophysiological recording. New York, NY: Oxford University Press ; 2001 :
73. Stevens R. H., Galloway T., Berka C., Sprang M. Foundations of augmented cognition: Neuroergonomics and operational neuroscience. Schmorrow D. D. Estabrooke I. V. Grootjen M., ed. Berlin, Germany: Springer-Verlag ; 2009: 658-667.
74. Strang A., Funke G. J., Knott B. A., Warm J. S.. Physio-behavioral synchronicity as an index of processes supporting team performance. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 2011 ; 55: 1447-1451
75. Strang A. J., Knott B. A., Funke G. J., Russell S. M., Miller B. T., Dukes A. W., Courtice A. M., Lyons J., Brown R. D., Hyson J., Bolia R. S.. Collaboration technologies improve performance and communication in air battle management. Military Psychology. 2011 ; 23: 390-409
76. Thornton C., Braun C., Bowers C., Morgan B.. Automation effects in the cockpit: A low-fidelity investigation. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 1992 ; 36: 30-34 (Pubitemid 23583657)
77. Throne M. H., Deatz R. C., Holden W. T., Campbell C. H., Sterling B. S., Lickteig C. W. Prototype staff training and evaluation methods for future forces. Washington, DC: Defense Technical Information Center ; 1999 :
78. Tognoli E., Lagarde J., DeGuzman G. C., Kelso J. A. S.. The phi complex as a neuromarker of human social coordination. Proceedings of the National Academy of Science. 2007 ; 104: 8190-8195 (Pubitemid 47185897)
79. Tsang P., Vidulich M. Handbook of human factors and ergonomics. 2006 ). Workload and situation awareness. In SalvendyG. (Ed.), Handbook of human factors and ergonomics (3rd ed., pp. 243-268). New York, NY: John Wiley. Salvendy G., ed. New York, NY: John Wiley ; 2006: 243-268.
80. Urban J. M., Weaver J. L., Bowers C. A., Rhodenizer L.. Effects of workload and structure on team processes and performance: Implications for complex team decision making. Human Factors. 1996 ; 38: 300-310 (Pubitemid 26362371)
81. Van de Ven A. H., Delbecq A. L., Koenig R.. Determinants of coordination modes within organizations. American Sociological Review. 1976 ; 41: 322-328
82. Van Orden K. F. Monitoring moment-to-moment operator workload using task load and system-state information. San Diego, CA: SPAWAR Systems Center ; 2001 :
83. Vidulich M. A. Principles and practice of aviation psychology. Tsang P. S. Vidulich M. A., ed. Mahwah, NJ: Lawrence Erlbaum ; 2003: 115-146.
84. WebberC. L.ZbilutJ. P. (2005). Recurrence quantification analysis of nonlinear dynamical systems. In RileyM. A.Van OrdenG. C. (Eds.), Tutorials in contemporary nonlinear methods for the behavioral sciences. Retrieved from http://www.nsf.gov/sbe/bcs/pac/nmbs/chap3.pdf
85. Wickens C. D., Hollands J. G. Engineering psychology and human performance. 2000 ). Engineering psychology and human performance (3rd ed.). New York, NY: HarperCollins. New York, NY: HarperCollins ; 2000 :
86. Wierwille W. W., Eggemeier F. T.. Recommendations for mental workload measurement in a test and evaluation environment. Human Factors. 1993 ; 35: 263-281 (Pubitemid 23204670)
87. Yang C. W., Yenn T. C., Lin C. J.. Assessing team workload under automation based on a subjective performance measure. Safety Science. 2010 ; 48: 914-920
88. Young M. S., Stanton N. A.. Attention and automation: New perspectives on mental underload and performance. Theoretical Issues in Ergonomic Science. 2002a ; 3: 178-194
89. Young M. S., Stanton N. A.. Malleable attentional resources theory: A new explanation for the effects of mental underload on performance. Human Factors. 2002b ; 44: 365-375 (Pubitemid 35448639)