Supporting interruption management and multimodal interface design: Three meta-analyses of task performance as a function of interrupting task modality

Human Factors

Volumn 55, Issue 4, 2013, Pages 697-724

  Lu, Sara A ^a   Wickens, Christopher D ^b   Prinet, Julie C ^a   Hutchins, Shaun D ^b   Sarter, Nadine ^a   Sebok, Angelia ^b  

^a UNIVERSITY OF MICHIGAN   (United States)

^b Alion Science Corporation   (United States)

Author keywords

auditory; interface design guidelines; interruption management; meta analysis; multimodal interfaces; multiple resources; redundancy; tactile; time sharing

Indexed keywords

AUDITORY; INTERFACE DESIGNS; INTERRUPTION MANAGEMENT; META-ANALYSIS; MULTI-MODAL INTERFACES; MULTIPLE RESOURCES; TACTILE; TIME-SHARING;

DATA PROCESSING; DESIGN; KNOWLEDGE BASED SYSTEMS; MODERATORS; REDUNDANCY;

INTERACTIVE COMPUTER SYSTEMS;

ARTICLE; ATTENTION; HEARING; HUMAN; MENTAL FUNCTION; META ANALYSIS; PHYSIOLOGY; REACTION TIME; TASK PERFORMANCE; TOUCH; VISION;

ATTENTION; AUDITORY PERCEPTION; HUMANS; MENTAL PROCESSES; REACTION TIME; TASK PERFORMANCE AND ANALYSIS; TOUCH PERCEPTION; VISUAL PERCEPTION;

EID: 84880309082     PISSN: 00187208     EISSN: 15478181     Source Type: Journal    
DOI: 10.1177/0018720813476298     Document Type: Article

References (103)

1. Almen L. Comparing audio and tactile inputs as driver attention control. Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2002: 1777-1781.
2. Banbury S. P., Macken W. J., Tremblay S., Jones D. M.. Auditory distraction and short-term memory: Phenomena and practical implications. Human Factors. 2001 ; 43: 12-29
3. Begault D. R., Pittman M. T.. Three-dimensional audio versus head-down traffic alert and collision avoidance system displays. International Journal of Aviation Psychology. 1996 ; 6: 79-93
4. Belz S. M., Robinson G. S., Casali J.. A new class of auditory warning signals for complex systems: Auditory icons. Human Factors. 1999 ; 41: 608-618
5. Bliss J., Liebman R., Brill J. Alert modality and behavioral compliance during virtual combat. Proceedings of the Human Factors and Ergonomics Society 54th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2010: 2373-2377.
6. Bouis D., Voss M., Geiser G., Haller R. Proceedings of the Human Factors and Ergonomics Society 23rd Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 1979: 35-39.
7. Bronkhorst A. W., Veltman J. A., van Breda L.. Application of a three-dimensional auditory display in a flight task. Human Factors. 1996 ; 38: 23-33
8. Burke J. L., Prewett M. S., Gray A. A., Yang L., Stilson F., Coovert M. D., Elliot L. R., Redden E. Comparing the effects of visual-auditory and visual-tactile feedback on user performance: A meta-analysis. Proceedings of the 8th International Conference on Multimodal Interfaces. New York, NY: ACM Press ; 2006: 108-117.
9. Calhoun G. L., Draper M. H., Guilfoos B. J., Ruff H. A. Tactile and aural alerts in high auditory load UAV control environments. Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2005: 145-149.
10. Calhoun G., Fontejon J., Draper M., Ruff H., Guilfoos B. Proceedings of the Human Factors and Ergonomics Society 48th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2004: 137-141.
11. Chang A., O'Modhrain S., Jacob R., Gunther E., Ishii H. ComTouch: Design of a vibrotactile communication device. Proceedings of the Conference on Designing Interactive System. New York, NY: ACM Press ; 2002: 312-320.
12. Chen W. H., Zeng J. J., Kao K. C. Effect of auditory intersection collision avoidance warnings on driving behaviors in different distracted driving conditions. Paper presented at the 19th International Technical Conference on the Enhanced Safety of Vehicles ; 2005 Washington, DC.
13. Colcombe A., Wickens C. D. Cockpit display of traffic information automated conflict alerting: Parameters to maximize effectiveness and minimize disruption in multi-task environments. Moffett Field, CA: NASA Ames Research Center ; 2006:
14. Dingus T. A., McGehee D. V., Hankey J. M.. Human factors field evaluation of automotive headway maintenance/collision warning devices. Human Factors. 1997 ; 39: 216-229
15. Dixon S. R., Wickens C. D., Chang D.. Mission control of multiple unmanned aerial vehicles: A workload analysis. Human Factors. 2005 ; 47: 479-487
16. Dowell J., Shmueli Y.. Blending speech output and visual text in the multimodal interface. Human Factors. 2008 ; 50: 782-788
17. Elliott L., Coovert M., Redden E.. Overview of meta-analyses investigating vibrotactile versus visual display options. Proceedings of 13th International Conference of Human-Computer Interaction ; 2009 ; 2009. 435.
18. Ferris T., Penfold R., Hameed S., Sarter N. The implications of crossmodal links in attention for the design of multimodal interfaces: A driving simulation study. Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2006: 406-409.
19. Ferris T., Sarter N.. Cross-modal links among vision, audition, and touch in complex environments. Human Factors. 2008 ; 50: 17-26
20. Ferris T., Sarter N.. When content matters: The roles of processing code in tactile display design. IEEE Transactions on Haptics. 2010 ; 3: 199-210
21. Finomore V., Popik D., Castle C., Dallman R. Effects of a network-centric multi-modal communication tool on a communication monitoring task. Proceedings of the Human Factors and Ergonomics Society 54th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2010: 2125-2129.
22. Fitch G., Kiefer R., Hankey J., Kleiner B.. Toward developing an approach for alerting drivers to the direction of a crash threat. Human Factors. 2007 ; 49: 710-720
23. Gish K. W., Staplin L., Stewart J., Perel M.. Sensory and cognitive factors affecting automotive head-up display effectiveness. Transportation Research Record. 1999 ; 1694: 10-19
24. Glumm M., Kehring K., White T. Effects of tactile, visual, and auditory cues about threat location on target acquisition and attention to visual and auditory communications. Adelphi, MD: U.S. Army Research Laboratory ; 2006:
25. Haas E., Hill S., Stachowiak C., Fields M. Proceedings of the Human Factors and Ergonomics Society 53rd Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2009: 259-263.
26. Hale K., Stanney K.. Deriving haptic design guidelines from human physiological, psychophysical, and neurological foundations. IEEE Computer Graphics and Applications. 2004 ; 24 (2). 33-39
27. Hameed S., Ferris T., Jayaraman S., Sarter N.. Using informative peripheral visual and tactile cues to support task and interruption management. Human Factors. 2009 ; 51: 126-135
28. Helleberg J., Wickens C. D.. Effects of data-link modality and display redundancy on pilot performance: An attentional perspective. International Journal of Aviation Psychology. 2003 ; 13 (3). 9-21
29. Ho C., Hong T., Spence C. Warning signals go multisensory. Proceedings of the 11th International Conference on Human-Computer Interaction: Vol. 9. Advances in virtual environments technology: Musings on design, evaluation, and applications. Mahwah, NJ: Lawrence Erlbaum ; 2005: 22-27.
30. Ho C., Hong T., Spence C.. The differential effect of vibrotactile and auditory cues on visual spatial attention. Ergonomics. 2006 ; 49: 724-738
31. Ho C., Reed N., Spence C.. Multisensory in-car warning signals for collision avoidance. Human Factors. 2007 ; 49: 1107-1114
32. Ho C., Spence C.. Using peripersonal warning signals to orient a driver's gaze. Human Factors. 2009 ; 51: 539-556
33. Ho C. Y., Nikolic M., Waters M., Sarter N.. Not now! Supporting interruption management by indicating the modality and urgency of pending tasks. Human Factors. 2004 ; 46: 399-409
34. Horrey W. J., Wickens C. D.. Driving and side task performance: The effects of display clutter, separation, and modality. Human Factors. 2004 ; 46: 611-624
35. Hurwitz J. B., Wheatley D. J. Using driver performance measures to estimate workload. Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2002: 1804-1808.
36. Iani C., Wickens C. D.. Factors affecting task management in aviation. Human Factors. 2007 ; 49: 16-24
37. Kaber D. B., Wright M. C., Sheik-Nainer M. A.. Investigation of multi-modal interface features for adaptive automation of a human-robot system. International Journal of Human-Computer Studies. 2006 ; 64: 527-540
38. Kramer A. F., Cassavaugh N., Horrey W. J., Becic E., Mayhugh J. L.. Influence of age and proximity warning devices on collision avoidance in simulated driving. Human Factors. 2007 ; 49: 935-949
39. Kramer G. Auditory display: Soniïcation, audiïcation, and auditory interfaces. Kramer G., ed. Reading, MA: Addison-Wesley ; 1994: 1-77.
40. Krausman A., Elliott L., Pettitt R. Effects of visual, auditory, and tactile alerts on platoon leader performance and decision making. Adelphi, MD: U.S. Army Research Laboratory ; 2005:
41. Lancaster J. A., Casali J. G.. Investigating pilot performance using mixed-modality simulated data link. Human Factors. 2008 ; 50: 183-193
42. Larkin R. J. A comparison of audio, visual, and tactile warning devices in a simulated flight environment. Monterey, CA: Naval Postgraduate School ; 1983:
43. LatorellaK. A. (1996). Investigating interruptions: Implications for flight deck performance (Unpublished doctoral dissertation). State University of New York at Buffalo.
44. Latorella K. A. Effects of modality on interrupted flight deck performance: Implications for data link. Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 1998: 87-91.
45. Latorella K. A. Investigating interruptions: Implications for flightdeck performance. Hampton, VA: NASA Langley Research Center ; 1999:
46. Lee J., McGehee D., Brown T., Marshall D.. Effects of adaptive cruise control and alert modality on driver performance. Transportation Research Record. 2006 ; 1980: 49-56
47. Lee J., Spence C.. Assessing the benefits of multimodal feedback on dual-task performance under demanding conditions. Proceedings of the 22nd British HCI Group Annual Conference: People and Computers XXII. Culture, Creativity, Interaction ; 2008 ; 2008. 185.
48. Lewandowski L., Hursh S., Kobus D. A. Multimodal versus unimodal information processing of words. Groton, CT: Naval Submarine Medical Research Laboratory ; 1985:
49. Liu Y. C.. Comparative study of the effects of auditory, visual and multimodality displays on drivers performance in advanced traveler information systems. Ergonomics. 2001 ; 44: 425-442
50. Lu S. A., Wickens C. D., Sarter N. B., Sebok A. Informing the design of multimodal displays: A meta-analysis of empirical studies comparing auditory and tactile interruptions. Proceedings of the Human Factors and Ergonomics Society 55th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2011: 1170-1174.
51. Maltz M., Shinar D.. Imperfect in-vehicle collision avoidance warning systems can aid drivers. Human Factors. 2004 ; 46: 357-366
52. Merat N., Jamson A.. The effect of stimulus modality on signal detection: Implications for assessing the safety of in-vehicle technology. Human Factors. 2008 ; 50: 145-158
53. Mohebbi R., Gray R., Tan H.. Driver reaction time to tactile and auditory rear-end collision warnings while talking on a cell phone. Human Factors. 2009 ; 51: 102-110
54. Mollenhauer M. A., Lee J., Cho K., Hulse M. C., Dingus T. A. The effects of sensory modality and information priority on in-vehicle signing and information systems. Proceedings of the Human Factors and Ergonomics Society 38th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 1994: 1072-1076.
55. Mortimer C. Affects of task difficulty on target guidance using auditory and tactile cues. Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2006: 2109-2113.
56. Navarro J., Mars F., Hoc J.-M.. Lateral control assistance for car drivers: A comparison of motor priming and warning systems. Human Factors. 2007 ; 49: 950-960
57. Navon D., Gopher D.. On the economy of the human processing systems. Psychological Review. 1979 ; 86: 254-255
58. Oskarsson P.-A, Eriksson L., Carlander O.. Enhanced perception and performance by multimodal threat cueing in simulated combat vehicle. Human Factors. 2012 ; 54: 122-137
59. Oskarsson P.-A., Eriksson L., Lif P., Lindahl B., Hedström J. Multimodal threat cueing in simulated combat vehicle. Proceedings of the Human Factors and Ergonomics Society 52nd Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2008: 1287-1291.
60. Oviatt S. Multimodal interfaces. The human-computer interaction handbook: Fundamentals, evolving technologies and emerging applications. Jacko J. Sears A., ed. Mahwah, NJ: Lawrence Erlbaum ; 2003: 286-304.
61. Perry N. C., Stevens C. J., Wiggins M. W., Howell C. E.. Cough once for danger: Icons versus abstract warnings as informative alerts in civil aviation. Human Factors. 2007 ; 49: 1061-1071
62. Ratwani R. M., Andrews A. E., Sousk J. D., Trafton J. G. The effect of interruption modality on primary task resumption. Proceedings of the Human Factors and Ergonomics Society 52nd Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2008: 393-397.
63. Rosenthal R.. The "file drawer problem" and tolerance for null results. Psychological Bulletin. 1979 ; 86: 638-641
64. Rosenthal R., DiMatteo M. R.. Meta-analysis: Recent developments in quantitative methods for literature review. Annual Review of Psychology. 2001 ; 52: 59-82
65. Santamaria A., Quesada S. Technical status report: Task No. HMI-08-02-001. Experimentation support and data analysis. Adelphi, MD: Robotics Collaborative Technical Alliance ; 2007:
66. Sarter N. B. Advances in human performance and cognitive engineering research. Salas E., ed. New York, NY: JAI Press ; 2002: 13-36.
67. Sarter N. B.. Coping with complexity through adaptive interface design. Human-Computer Interaction. 2007 ; 4552: 493-498
68. Savick D., Elliott L., Zubal O., Stachowiak C. The effect of audio and tactile cues on soldier decision making and navigation in complex simulation scenarios. Adelphi, MD: U.S. Army Research Laboratory ; 2006:
69. Scerbo M. W. Automation and human performance: Theory and applications (Human Factors in Transportation). Parasuruman R. Mouloua M., ed. Mahwah, NJ: Lawrence Erlbaum ; 1996: 37-63.
70. Scott J., Gray R.. A comparison of tactile, visual, and auditory warnings for rear-end collision prevention in simulated driving. Human Factors. 2008 ; 50: 264-275
71. Seagull F., Wickens C., Loeb R. When less is more? Attention and workload in auditory, visual, and redundant patient monitoring conditions. Proceedings of the Human Factors and Ergonomics Society 45th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2001: 1395-1399.
72. Seppelt B., Wickens C. D. In-vehicle tasks: Effects of modality, driving relevance, and redundancy. Savoy: University of Illinois, Aviation Human Factors Division ; 2003:
73. Simmons J. P., Nelson L. D., Simonsohn U.. False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science. 2011 ; 22: 1359-1366
74. Sklar A. E., Sarter N. B.. Good vibrations: Tactile feedback in support of attention allocation and human-automation coordination in event-driven domains. Human Factors. 1999 ; 41: 543-552
75. Smith C., Clegg B., Heggestad E., Hopp-Levine P.. Interruption management: A comparison of auditory and tactile cues for both alerting and orienting. International Journal of Human-Computer Studies. 2009 ; 67: 777-786
76. Srinivasan R.. Effect of selected in-vehicle route guidance systems on driver reaction times. Human Factors. 1997 ; 39: 200-215
77. Srinivasan R., Yang C.-Z., Jovanis P. P., Kitamura R., Anwar M.. Simulation study of driving performance with selected route guidance systems. Transportation Research. 1994 ; 2 (2). 73-90
78. Stanley L. Haptic and auditory cues for lane departures warnings. Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2006: 2405-2408.
79. Steelman-Allen K., McCarley J., Wickens C. D.. Modeling the control of attention in visual work spaces. Human Factors. 2011 ; 53: 142-153
80. Steelman-Allen K., Talleur D., Carbonari R., Yamani Y., McCarley J., Nunes A... Paper presented at the Human Factors and Ergonomics Society 54th Annual Meeting ; 2010 San Francisco, CA ; 2010..
81. Stone R., Bisantz A., Llinas J., Paquet V.. Augmented multisensory interface design (amid): A human-centric approach to unisensory and multisensory augmented reality design. Journal of Cognitive Engineering and Decision Making. 2009 ; 3: 362-388
82. Straughn S., Gray R., Tan H.. To go or not to go: Stimulus-response compatibility for tactile and auditory pedestrian collision warnings. IEEE Transactions on Haptics. 2009 ; 2: 111-117
83. Tilak R., Xholi I., Schowalter D., Ferris T., Hameed S., Sarter N. Crossmodal links in attention in the driving environment: The roles of cueing modality, signal timing, and workload. Proceedings of the Human Factors and Ergonomics Society 52th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2008: 1815-1819.
84. Trafton G., Monk C.. Task interruptions. Reviews of Human Factors and Ergonomics. 2007 ; 3: 111-126
85. Trumbly J. E., Arnett K. P., Johnson P. C.. Productivity gains via an adaptive user interfaces. Journal of Human-Computer Studies. 1994 ; 40: 63-81
86. Van Erp J., van Veen H. Vibro-tactile information presentation in automobiles. Proceedings of Eurohaptics. Paris, France: Eurohaptics Society ; 2001: 99-104.
87. Van Erp J., van Veen H.. Vibrotactile in-vehicle navigation system. Transportation Research Part F: Traffic Psychology and Behaviour. 2004 ; 7: 247-256
88. Walker J., Alicandri E., Sedney C., Roberts K.. In-vehicle navigation devices: Effects on the safety of driver performance. Proceedings of Vehicle Navigation and Information Systems Conference ; 1991 New York, NY: IEEE ; 1991. 499.
89. Whitmire J., Morgan J. F., Oron-Gilad T., Hancock P. A. The effect of in-vehicle warning systems on speed compliance in work zones. Proceedings of Human Factors and Ergonomics Society 54th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2010: 2023-2027.
90. Wickens C. D. The structure of attentional resources. Attention and performance VIII. Nickerson R. S., ed. Hillsdale, NJ: Lawrence Erlbaum ; 1980: 239-258.
91. Wickens C. D.. Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science. 2002 ; 3: 159-177
92. Wickens C. D.. Multiple resources and mental workload. Human Factors. 2008 ; 50: 449-455
93. Wickens C. D., Dixon S., Seppelt B. In-vehicle displays and control task interferences: The effects of display location and modality. Savoy: University of Illinois, Aviation Research Lab ; 2002:
94. Wickens C. D., Dixon S. R., Seppelt B. Auditory preemption versus multiple resources: Who wins in interruption management? Proceedings of the Human Factors and Ergonomics Society 49th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2005: 463-466.
95. Wickens C., Goh J., Helleberg J., Horrey W., Talleur D.. Attentional models of multitask pilot performance using advanced display technology. Human Factors. 2003 ; 45: 360-380
96. Wickens C. D., Gosney J. Redundancy, modality, and priority in dual task interference. Proceedings of the Human Factors and Ergonomics Society 47th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2003: 1590-1594.
97. Wickens C. D., Hollands J. G., Banbury S., Parasuraman R. (2012 ). Engineering psychology and human performance (4th ed.). New York, NY: Prentice Hall.
98. Wickens C. D., Hutchins S., Carolan T., Cummings J.. Effectiveness of part task training and increasing difficulty training strategies: A meta-analysis approach. Human Factors. 2012 ;:
99. Wickens C. D., Liu Y.. Codes and modalities in multiple resources: A success and a qualification. Human Factors. 1988 ; 30: 599-616
100. Wickens C. D., Prinet J., Hutchins S., Sarter N. B., Sebok A. Auditory-visual redundancy in vehicle control interruptions: Two meta-analyses. Proceedings of Human Factors and Ergonomics Society 55th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society ; 2011: 1155-1159.
101. Wickens C. D., Sandry D. L., Vidulich M.. Compatibility and resource competition between modalities of input, central processing, and output. Human Factors. 1983 ; 25: 227-248
102. Wickens C., Small R., Andre T., Bagnall T., Brenaman C.. Multisensory enhancement of command displays for unusual attitude recovery. International Journal of Aviation Psychology. 2008 ; 18: 255-267
103. Wogalter M., Conzola V., Smith-Jackson T.. Research-based guidelines for warning design and evaluation. Applied Ergonomics. 2002 ; 33: 219-23