Managing workload in human-robot interaction: A review of empirical studies

Computers in Human Behavior

Volumn 26, Issue 5, 2010, Pages 840-856

  Prewett, Matthew S ^a   Johnson, Ryan C ^a   Saboe, Kristin N ^a   Elliott, Linda R ^b   Coovert, Michael D ^a  

^a UNIVERSITY OF SOUTH FLORIDA   (United States)

^b U S ARMY RESEARCH LABORATORY   (United States)

Author keywords

Automation; Display; Perception; Performance; Teleoperation

Indexed keywords

AUTOMATION; DISPLAY DEVICES; MAN MACHINE SYSTEMS; REMOTE CONTROL; SENSORY PERCEPTION;

COMPLEX ENVIRONMENTS; DECREASING OPERATOR; EMPIRICAL LITERATURE; EMPIRICAL STUDIES; MULTI-MODAL DISPLAYS; MULTIPLE RESOURCES; PERFORMANCE; SOCIOTECHNICAL SYSTEMS;

HUMAN ROBOT INTERACTION;

EID: 77955269395     PISSN: 07475632     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.chb.2010.03.010     Document Type: Review

References (100)

1. J.J. Adams Multiple robot/single human interaction: Effects on perceived workload Behavior Information and Technology 28 2 2009 183 198
2. Adelstein, B. D., Thomas, G. L., & Ellis, S. R. (2003). Head tracking latency in virtual environments: Psychophysics and a model. In Proceedings of human factors and ergonomics society 47th annual meeting (pp. 2083-2087).
3. Allison, R. S., Zacher, J. E., Wang, D., & Shu, J. (2004). Effects of network delay on a collaborative motor task with telehaptic and televisual feedback. In Proceedings of 2004 ACM SIGGRAPH international conference on virtual reality continuum and its applications in industry (pp. 375-381).
4. S. Beer Decision and control: The meaning of operational research and management cybernetics 1966 John Wiley Chichester
5. D.J. Bruemmer, R.L. Boring, D.A. Few, J.L. Marble, and M.C. Walton "I call shotgun!": An evaluation of mixed-initiative control for novice users of a search and rescue robot IEEE Transactions on Systems, Man, and Cybernetics 34 2004 2847 2852
6. J.L. Burke, M.S. Prewett, A. Gray, L. Yang, F.R.B. Stilson, and E. Redden Comparing the effects of visual-auditory and visual-tactile feedback on user performance. A meta-analysis Proceedings of the Eighth International Conference on Multimodal Interfaces 108 2006 117
7. Calhoun, G., Draper, M., Nelson, J., Lefebvre, A., & Ruff, H. (2006). Simulation assessment of synthetic vision concepts for UAV operations. (Technical Report). Air Force Research Laboratory, Human Effectiveness Directorate Warfighter Interface Division at Wright-Patterson AFB, OH.
8. Cassidy, A. M. (2009). Mental models, trust, and reliance: Exploring the effect of human perceptions on automation use (Master's Thesis, Naval Post-Graduate School).
9. Chadwick, R. A. (2006). Operating multiple semi-autonomous robots: Monitoring, responding, detecting. In Proceedings of the human factors and ergonomics society's 50th annual meeting. San Francisco, CA. HFES '06 (pp. 329-333).
10. Chadwick, R. A. (2005). The impacts of multiple robots and display views: An urban search and rescue simulation. In Proceedings of the human factors and ergonomics society's 49th annual meeting. Orlando, FL. HFES '05 (pp. 387-391).
11. Chen, J. Y. C. (2009). Concurrent performance of military tasks and robotics tasks: Effects of automation unreliability and individual differences. In Proceedings of the fourth human robot interaction conference 2009, La Jolla, CA. HRI '09 (pp. 181-188).
12. J.Y.C. Chen, and C. Joyner Concurrent performance of gunner's and robotics operator's tasks in a multi-tasking environment Military Psychology 21 2009 98 113
13. J.Y.C. Chen, and J.E. Thropp Review of low frame rate effects on human performance IEEE Transactions on Systems, Man and Cybernetics-Part A: Systems and Humans 37 6 2007 1063 1076
14. J.Y.C. Chen, P.J. Durlach, J.A. Sloan, and L.D. Bowers Human-robot interaction in the context of simulated route reconnaissance missions Military Psychology 20 2008 135 149
15. J.Y.C. Chen, E. Haas, and M. Barnes Human performance issues and user interface design for teleoperated robots IEEE Transactions on Systems, Man, and Cybernetics 37 6 2007 1231 1245
16. Coovert, M. D., & Elliott, L. R. (2009). UAV Operator Specifications from ONET. In Proceedings of the 15th international symposium on aviation psychology. Dayton, OH (pp. 552-557).
17. M.D. Coovert, K. Ramakrishna, and E. Salas Preferences for power in expert systems by novice users AI & Society 3 1989 59 61
18. M.D. Coovert, A. Walvoord, L. Elliott, and E. Redden A tool for the accumulation and evaluation of multimodal research IEEE Transactions on Systems, Man, and Cybernetics 24 5 2008 1884 1906
19. Cosenzo, K. A., Parasuraman, R., Novak, A., & Barnes, M. (2006). Implementation of automation for control of robotic systems. (Technical Report). Army Research Laboratory (ARL) at Aberdeen Proving Grounds, MD.
20. Crandall, J. W., & Cummings, M. L. (2007). Developing performance metrics for the supervisory control of multiple robots. In Proceedings of the second ACM/IEEE international conference on human-robot interaction. Arlington, VA. HRI '07 (pp. 33-40).
21. R.P. Darken, and H. Cervik Map usage in virtual environments: Orientation issues Proceedings of IEEE Conference on Virtual Reality 133 1999 140
22. Darken, R. P., Kempster, K., & Peterson, B. (2003). Effects of streaming video quality of service on spatial comprehension in a reconnaissance task. In Proceedings of IAITSEC 2001, Orlando, FL.
23. S.R. Dixon, and C.D. Wickens Automation reliability in unmanned aerial vehicle control: A reliance-compliance model of automation dependence in high workload Human Factors 48 2006 474 486
24. Dixon, S. R., & Wickens, C. D. (2003). Control of multiple-UAVs: A workload analysis. In Proceedings of the 12th international symposium on aviation psychology. Dayton, OH (pp. 1-5).
25. Draper, M., Calhoun, G., & Nelson, J (2006). Evaluation of synthetic vision overlay concepts for UAV sensor operations: Landmark cues and picture-in-picture. (Technical Report). Human Effectiveness Directorate, Warfighter Interface Division at Wright-Patterson AFB, OH.
26. Draper, M., Calhoun, G., Ruff, H., Williamson, D., & Barry, T. (2003). Manual versus speech input for unmanned aerial vehicle control stations. In Proceedings of the human factors and ergonomics society's 47th annual meeting. Denver, CO. HFES '03 (pp. 387-391).
27. Draper, J. V., Handel, S., Hood, C. C., & Kring, C. T. (1991). Three experiments with stereoscopic television: when it works and why. In IEEE transactions on systems, man and cybernetics (pp. 1047-1052).
28. D. Drascic, and J. Grodski Defense teleoperation and stereoscopic video Proceedings of SPIE: Stereoscopic Displays and Applications IV 1915 1993 1 12
29. Drury, J. L., Keyes, B., & Yanco, H. A. (2007). LASSOing HRI: Analyzing situation awareness in map-centric and video-centric interfaces. In Proceedings of the second ACM/IEEE international conference on human-robot interaction. Arlington, VA. HRI '07 (pp. 279-286).
30. Elliott, L. R., Prewett, M. S., Coovert, M. D., Walvoord, A. G., Saboe, K. N., & Johnson, R. J. (2009). A meta-analysis of vibrotactile and visual information displays for improving task performance. Unpublished manuscript, Army Research Laboratory at Fort Benning, GA.
31. Ellis, S. R., Mania, K., Adelstein, B. D., & Hill, M. (2004). Generalizeability of latency detection in a variety of virtual environments. In Proceedings of the human factors and ergonomics society's 48th annual meeting. New Orleans, LA, HFES '04 (p. 2632).
32. M.R. Endsley, and D.B. Kaber Level of automation effects on performance, situation awareness and workload in a dynamic control task Ergonomics 42 1999 462 492
33. Fisher, A., McDermott, P. L., & Fagan, S. (2009). Bandwidth allocation in a military teleoperation task. In Proceedings of the human factors and ergonomics society's 53rd annual meeting. La Jolla, CA. HFES '09 (pp. 287-288).
34. Folds, D. J., & Gerth, J. (1994). Auditory monitoring of up to eight simultaneous sources. In Proceedings of the human factors and ergonomics society's 38th annual meeting. Santa Monica, CA, HFES '94 (pp. 505-509).
35. Galster, S. M., Knott, B. A., & Brown, R. D. (2006). Managing multiple UAVs: Are we asking the right questions? In Proceedings of the human factors and ergonomics society's 50th annual meeting. San Francisco, CA. HFES '06 (pp. 545-549).
36. Goodrich, M. A., McLain, T. W., Anderson, J. D., Sun, J., & Crandall, J. W. (2007). Managing autonomy in robot teams: Observations from four experiments. In Proceedings of the second human robot interaction conference. Arlington, VA. HRI '07 (pp. 25-32).
37. Hardin, B., & Goodrich, M. (2009). On using mixed-initiative control: A perspective for managing large-scale robot teams. In Proceedings of the fourth ACM/IEEE international conference on human-robot interaction. La Jolla, CA. HRI '09 (pp. 165-172).
38. Heath-Pastore, T. (1994). Improved operator awareness of teleoperated land vehicle attitude. (Technical Report). Naval Command, Control and Ocean Surveillance Center, CA.
39. K. Hendy, J. Lao, and P. Milgram Combining time and intensity effects in assessing operator information-processing load Human Factors 30 1997 30 47
40. Hill, S. G., & Bodt, B. (2007). A field experiment of autonomous mobility: Operator workload for one or two robots. In Proceedings of the second ACM/IEEE international conference on human-robot interaction. Arlington, VA. HRI '07 (pp. 169-176).
41. S. Hughes, and M. Lewis Task-driven camera operations for robotic exploration IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans 35 4 2005 513 522
42. Humphrey, C. M., Henck, C., Sewell, G., Williamson, B. W., & Adams, J. A. (2007). Assessing the scalability of a multi-robot interface. In Proceedings of the second ACM/IEEE international conference on human-robot interaction. Arlington, VA. HRI '07 (pp. 239-246).
43. D.B. Kaber, and M.R. Endsley The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task Theoretical Issues in Ergonomics Science, preview 2003 1 40
44. D.B. Kaber, E. Onal, and M. Endsley Design of automation for telerobots and the effect on performance, operator situation awareness and subjective workload Human Factors and Ergonomics in Manufacturing 10 2000 409 430
45. Klein, G., Pliske, R. M., Crandall, B., & Woods, D. (1999). Features of problem detection. In Proceedings of the human factors and ergonomics society's 43rd annual meeting. Houston, TX. HFES '99 (pp. 133-137).
46. Krotkov, E., Simmons, R., Cozman, F., & Koenig, S. (1996). Safeguarded teleoperation for lunar rovers: From human factors to field trials. In Proceedings of the 26th international conference on environmental systems. Santa Monica, CA.
47. Lane, J. C., Carignan, C. R., Sullivan, B. R., Akin, D. L., Hunt, T., & Cohen, R. (2002). Effects of time delay on telerobotic control of neutral buoyancy vehicles. In Proceedings of IEEE international conference on robotics and automation. Washington, DC (pp. 2874-2879).
48. Levinthal, B. R., & Wickens, C. D. (2006). Management of multiple UAVs with imperfect automation. In Proceedings of the human factors and ergonomics society's 50th annual meeting. San Francisco, CA. HFES '06 (pp. 1941-1944).
49. M. Lewis, J. Wang, S. Hughes, and X. Liu Experiments with attitude: Attitude displays for teleoperation IEEE Transactions on Systems Man and Cybernetics 33 2 2003 1345 1349
50. Lif, P., Jander, H., & Borgwall, J. (2007). Tactical evaluation of unmanned ground vehicle during a MOUT exercise. In Proceedings of the 7th international conference of engineering psychology and cognitive ergonomics. Beijing, China. HCI '07 (pp. 731-740).
51. Lion, D. M. (1993). Three dimensional manual tracking using a head-tracked stereoscopic display. (Technical Report). Human Interface Technology Lab, WA.
52. E.A. Locke, and G.P. Latham A theory of goal setting and task performance 1990 Prentice-Hall Englewood Cliffs, NJ
53. Luck, J. P., McDermott, P. L., Allender, L., & Russell, D. C. (2006, March). An investigation of real world control of robotic assets under communication latency. In Proceedings of the first human robot interaction conference 2006, Salt Lake City, UT. HRI '06 (pp. 202-209).
54. M.J. Massimino, and T.B. Sheridan Teleoperator performance with varying force and visual feedback Human Factors 36 1 1994 145 157
55. J. Meyer, L. Feinshreiber, and Y. Parmet Levels of automation in a simulated failure detection task IEEE Transactions on Systems, Man and Cybernetics 33 3 2003 2101 2106
56. Miller, C.A., Parasuraman, R. (2003). Beyond levels of automation: An architecture for more flexible human-automation collaboration. In Proceedings of the 47th human factors and ergonomics society (pp. 82-85).
57. K.L. Mosier, N. Sethi, S. McCauley, L. Khoo, and J.M. Orasanu What you don't know can hurt you: Factors impacting diagnosis in the automated cockpit Human Factors 49 2 2007 300 310
58. Murray, S. A. (1995). Human-machine interaction with multiple autonomous sensors. In Proceedings of the 6th IFAC/IFIP/IFORS/IEA symposium on analysis, design and evaluation of man-machine systems.
59. Muthard, E. K. & Wickens, C. D. (2003). Factors that mediate flight plan monitoring and errors in plan revision: Planning under automated and high workload conditions. In Proceedings of the 12th international symposium on aviation psychology, Dayton, OH.
60. Nielson, C. W., & Goodrich, M. A. (2006). Comparing the usefulness of video and map information in navigation tasks. In Proceedings of the first human robot interaction conference. Salt Lake City, UT. HRI '06 (pp. 95-101).
61. O. Olmos, C.D. Wickens, and A. Chudy Tactical displays for combat awareness: An examination of dimensionality and frame of reference concepts and the application of cognitive engineering The International Journal of Aviation Psychology 10 3 2000 247 271
62. R. Parasuraman, S. Galster, and C. Miller Human control of multiple robots in the RoboFlag simulation environment IEEE Transactions on Systems, Man and Cybernetics 33 4 2003 3233 3237
63. R. Parasuraman, S.M. Gaslter, P. Squire, H. Furukawa, and C. Miller Flexible delegation-type interface enhances system performance in human supervision of multiple robots: Empirical studies with RoboFlag IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans 35 4 2005 481 493
64. S.H. Park, and J.C. Woldstad Multiple two-dimensional displays as an alternative to three-dimensional displays in telerobotic tasks Human Factors 42 4 2000 592 603
65. Pazuchanics, S. L. (2006). The effects of camera perspective and field of view on performance in teleoperated navigation. In Proceedings of the human factors and ergonomics society's 50th annual meeting. San Francisco, CA. HFES '06 (pp. 1528-1532).
66. M.S. Prewett, A. Gray, J.L. Burke, L. Yang, F.R.B. Stilson, and L.R. Elliott The benefits of multimodal information: A meta-analysis comparing visual and visual-tactile feedback Proceedings of the Eighth International Conference on Multimodal Interfaces 2006 333 338 (Pubitemid 47113466)
67. Reddy, M. (1997). The effects of low frame rate on a measure of user performance in virtual environments. (Technical Report). Dept of Computer Science, University of Edinburgh.
68. R. Revans The origins and growth of action learning 1982 Chartwell Bratt Bromley
69. P. Richard, G. Birebent, P. Coiffet, G. Burdea, D. Gomez, and N. Lagrana Effect of frame rate and force feedback on virtual object manipulation Presence 5 1 1996 95 108
70. E. Rovira, K. McGarry, and R. Parasuraman Effects of imperfect automation on decision making in a simulated command and control task Human Factors 49 2007 76 87
71. H.A. Ruff, S. Narayan, and M.H. Draper Human interaction with levels of automation and decision-aid fidelity in the supervisory control of multiple simulated unmanned air vehicles Presence. Teleoperators and Virtual Environments 11 4 2002 335 351
72. Ruff, H. A., Calhoun, G., Draper, M., Fontejon, J., & Abbott, J. (2004). Exploring automation issues in supervisory control of multiple UAVs. In Proceedings of the 5th conference of human performance, situation awareness, and automation technology (pp. 218-222).
73. Schermerhorn, P., & Schultz, M. (2009). Dynamic robot autonomy: Investigating the effects of robot decision-making in a human-robot team task. In Proceedings of the 2009 international conference on multimodal interfaces. Cambridge, MA. ICMI-MLMI '09 (pp. 63-70).
74. Schipani, S. P. (2003). An evaluation of operator workload during partially autonomous vehicle operations. (Technical Report). Army Research Laboratory at Aberdeen Proving Grounds, MD.
75. Scribner, D. R., & Gombash, J. W. (1998). The effect of stereoscopic and wide field of view conditions on teleoperator performance. (Technical Report). Army Research Laboratory at Aberdeen Proving Grounds, MD.
76. Sellner, B. P., Hiatt, L. M., Simmons, R., & Singh, S. (2006). Attaining situational awareness for sliding autonomy. In Proceedings of the 1st ACM SIGCHI/SIGART conference on human-robot interaction. Salt Lake City, UT. HRI '06.
77. M.A. Shreik-Nainar, D.B. Kaber, and M.-Y. Chow Control gain adaptation in virtual reality mediated human-telerobot interaction Human Factors and Ergonomics in Manufacturing 15 3 2003 259 274
78. Smyth, C. C. (2002). Modeling indirect vision driving with fixed flat panel displays: Task performance and mental workload. (Technical Report). Army Research Laboratory at Aberdeen Proving Grounds, MD.
79. Smyth, C. C., Gombash, J. W., & Burcham, P. M. (2001). Indirect vision driving with fixed flat panel displays for near-unity, wide, and extended fields of camera view. (Technical Report). Army Research Laboratory at Aberdeen Proving Grounds, MD.
80. Squire, P., Trafton, G., & Parasuraman, R. (2006). Human control of multiple unmanned vehicles: effects of interface type on execution and task switching times. In Proceedings of the 1st ACM SIGCHI/SIGART conference on human-robot interaction. Salt Lake City, UT. HRI '06 (pp. 26-32).
81. Thomas, L. C., & Wickens, C. D. (2000). Effects of display frames of reference on spatial judgments and change direction. (Technical Report). Army Research at Aberdeen Proving Grounds, MD.
82. Trouvain, B., Schlick, C., & Mervert, M. (2005). Comparison of a map- vs. camera-based user interface in a multi-robot navigation task. In Proceedings of IEEE interantional conference on systems, man and cybernetics. Washington DC, USA. SMC '03 (pp. 3224-3231).
83. Trouvain, B., & Wolf, C. (2003). Design and evaluation of a multi-robot interface. In The role of humans in intelligent and automated systems. Symposium presented at the 22nd RTO HFM. Warsaw, Poland, (pp. 1-12).
84. K.J. Vicente Cognitive work analysis: Toward safe, productive, and healthy computer-based work 1999 LEA Mahwah, NJ
85. Wang, H., Lewis, M., Velagapudi, P., Scerri, P., & Sycara, K. (2009a). How search and its subscale tasks in N robots. Proceedings of the fourth ACM/IEEE international conference on human-robot interaction. La Jolla, CA. HRI '09 (pp. 141-147).
86. Wang, H., Chien, S.Y., Lewis, M. Velagapudi, P., Scerri, P., & Sycara, K. (2009b). Human teams for large scale multirobot control. Proceedings of the IEEE international conference on systems, man, & cybernetics. San Antonio, TX, (pp. 1269-1274).
87. Wang, J., Wang, H., & Lewis, M. (2008). Assessing cooperation in human control of heterogeneous robots. In Proceedings of the third ACM/IEEE international conference on human-robot interaction. Amsterdam, Netherlands. HRI '08 (pp. 9-15).
88. Wang, J., & Lewis, M. (2007). Human control for cooperating robot teams. In Proceedings of the second human-robot interaction conference. Arlington, VA. HRI '07 (pp. 9-16).
89. L. Wang, G.A. Jamieson, and J.G. Hollands Trust and reliance on an automated combat identification system Human Factors 51 2009 281 291
90. Wang, W., & Milgram, P. (2003). Effects of viewpoint displacement on navigational performance in virtual environments. In Proceedings of human factors and ergonomics society 47th annual meeting. Denver, CO. HFES '03 (pp. 139-143).
91. B. Watson, N. Walker, W. Ribarsky, and V. Spaulding Effects of variation in system responsiveness on user performance in virtual environments Human Factors 40 3 1998 403 414
92. B. Watson, N. Walker, P. Woytiuk, and W. Ribarsky Maintaining usability during 3D placement despite delay Proceedings of IEEE Conference on Virtual Reality 133 2003 140
93. C. Wickens Multiple resources and performance prediction Theoretical Issues in Ergonomics Science 3 2 2002 159 177
94. C. Wickens Multiple resources and mental workload Human Factors 50 3 2008 449 454
95. Wickens, C. D., Dixon, S. R., & Chang, D. (2003). Using interference models to predict performance in a multiple-task UAV environment- 2 UAVs. (Technical Report). University of Illinois at Urbana-Champaign.
96. Wickens, C. D., Dixon, S. R., Goh, J., & Hammer, B. (2005). Pilot dependence on imperfect diagnostic automation in simulated UAV flights: An attentional visual scanning analysis. (Technical Report). University of Illinois at Urbana-Champaign, Institute of Aviation.
97. D. Wier Sequences of failure in complex socio-technical systems: Some implications of decision and control Kybernetes 33 2004 522 537
98. B.G. Witmer, and P.B. Kline Judging perceived and traversed distance in virtual environments Presence 7 2 1998 144 167
99. M. Yeh, and C.D. Wickens Display signaling in augmented reality: Effects of cue reliability and image realism on attention allocation and trust calibration Human Factors 43 3 2001 355 365
100. H. Yi, B. Song, D. Ji, and T. Yu Experimental research on situation awareness of the operators for unmanned aerial vehicle IEEE Transactions on Systems, Man and Cybernetics 36 6 2006 1225 1228