Using multimodal displays to signify critical handovers of control to distracted autonomous car drivers

International Journal of Mobile Human Computer Interaction

Volumn 9, Issue 3, 2017, Pages 1-16

  Politis, Ioannis ^a   Brewster, Stephen ^a   Pollick, Frank ^a  

^a UNIVERSITY OF GLASGOW   (United Kingdom)

Author keywords

Audio; Autonomous cars; Games; Handover; Multimodal feedback; Speech; Tactile; Tactons; Urgency; Visual; Warnings

Indexed keywords

SPEECH; USER INTERFACES;

AUDIO; AUTONOMOUS CAR; GAMES; HANDOVER; MULTIMODAL FEEDBACK; TACTILE; TACTONS; URGENCY; VISUAL; WARNINGS;

HUMAN COMPUTER INTERACTION;

EID: 85019257283     PISSN: 1942390X     EISSN: 19423918     Source Type: Journal    
DOI: 10.4018/ijmhci.2017070101     Document Type: Article

References (37)

1. Apple. (2015). CarPlay. Retrieved from http://www.apple.com/uk/ios/carplay/
2. Baldwin, C. L. (2011). Verbal collision avoidance messages during simulated driving: Perceived urgency, alerting effectiveness and annoyance. Ergonomics, 54(4), 328-337. doi:10.1080/00140139.2011.558634 PMID:21491275
3. Edworthy, J., Hellier, E., Walters, K., Clift-Mathews, W., & Crowther, M. (2003). Acoustic, semantic and phonetic influences in spoken warning signal words. Applied Cognitive Psychology, 17(8), 915-933. doi:10.1002/acp.927
4. Edworthy, J., Loxley, S., & Dennis, I. (1991). Improving auditory warning design: Relationship between warning sound parameters and perceived urgency. Human Factors: The Journal of the Human Factors and Ergonomics Society, 33(2), 205-231. doi:10.1177/001872089103300206 PMID:1860703
5. Gold, C., & Bengler, K. (2014). Taking Over Control from Highly Automated Vehicles. In Advances in Human Aspects of Transportation: Part II (pp. 64-69).
6. Gold, C., Damböck, D., Lorenz, L., & Bengler, K. (2013). "Take over!" How long does it take to get the driver back into the loop? In HFES Annual Meeting (Vol. 57, pp. 1938-1942). SAGE Publications. http://doi.org/doi:〈ALIGNMENT.qj〉〈/ALIGNMENT〉10.1177/1541931213571433
7. Google. (2015a). Android Auto. Retrieved from https://www.android.com/auto/
8. Google. (2015b). Google Self-Driving Car Project - Monthly reports. Retrieved from http://www.google.com/selfdrivingcar/reports/
9. Ho, C., Reed, N., & Spence, C. (2007). Multisensory In-Car Warning Signals for Collision Avoidance. Human Factors: The Journal of the Human Factors and Ergonomics Society, 49(6), 1107-1114. doi:10.1518/001872007X249965 PMID:18074709
10. Ho, C., Tan, H. Z., & Spence, C. (2005). Using spatial vibrotactile cues to direct visual attention in driving scenes. Transportation Research Part F: Traffic Psychology and Behaviour, 8(6), 397-412. doi:10.1016/j.trf.2005.05.002
11. Koo, J., Kwac, J., Ju, W., Steinert, M., Leifer, L., & Nass, C. (2014). Why did my car just do that? Explaining semiautonomous driving actions to improve driver understanding, trust, and performance. International Journal on Interactive Design and Manufacturing. Doi:〈ALIGNMENT.qj〉〈/ALIGNMENT〉10.1007/s12008-014-0227-2
12. Krome, S., Goddard, W., Greuter, S., Walz, S. P., & Gerlicher, A. (2015). A Context-Based Design Process for Future Use Cases of Autonomous Driving: Prototyping AutoGym. Proceedings of AutomotiveUI 2015 (pp. 265-272). ACM Press. doi:10.1145/2799250.2799257
13. Kyriakidis, M., Happee, R., & De Winter, J.C.F. (2014). Public opinion on automated driving : Results of an international questionnaire among 5,000 respondents.
14. Lee, J. D., Bricker, D. L., & Hoffman, J. D. (2008). A Dynamic Programming Algorithm for Scheduling In-Vehicle Messages. IEEE Transactions on Intelligent Transportation Systems, 9(2), 226-234. doi:10.1109/TITS.2008.922876
15. Lindgren, A., Angelelli, A., Mendoza, P. A., & Chen, F. (2009). Driver behaviour when using an integrated advisory warning display for advanced driver assistance systems. IET Intelligent Transport Systems, 3(4), 390-399. doi:10.1049/iet-its.2009.0015
16. Liu, Y. C. (2001). Comparative study of the effects of auditory, visual and multimodality displays on drivers' performance in advanced traveller information systems. Ergonomics, 44(4), 425-442. doi:10.1080/00140130010011369 PMID:11291824
17. Merat, N., Jamson, A. H., Lai, F. F. C., Daly, M., & Carsten, O. M. (2014). Transition to manual: Driver behaviour when resuming control from a highly automated vehicle. Transportation Research Part F: Traffic Psychology and Behaviour, 27, 274-282. doi:10.1016/j.trf.2014.09.005
18. Miller, D., Sun, A., Johns, M., Ive, H., Sirkin, D., Aich, S., & Ju, W. (2015). Distraction Becomes Engagement in Automated Driving. Proceedings of the Human Factors and Ergonomics Society Annual Meeting (pp. 1676-1680). SAGE.
19. Mok, B. K., Johns, M., Lee, K. J., Ive, P. H., Miller, D., & Ju, W. (2015). Timing of Unstructured Transitions of Control in Automated Driving. Proceedings of the 2015 IEEE Intelligent Vehicles Symposium (IV) (pp. 1167-1172). IEEE. doi:10.1109/IVS.2015.7225841
20. National Highway Traffic Safety Administration. (2013). Preliminary Statement of Policy Concerning Automated Vehicles. Washington, DC.
21. Naujoks, F., Mai, C., & Neukum, A. (2014). The effect of urgency of take-over requests during highly automated driving under distraction conditions. Proceedings of the 5th International Conference on Applied Human Factors and Ergonomics AHFE '14 (pp. 431-438).
22. Neubauer, C., Matthews, G., & Saxby, D. (2014). Fatigue in the Automated Vehicle: Do Games and Conversation Distract or Energize the Driver? Proceedings of the Human Factors and Ergonomics Society Annual Meeting (pp. 2053-2057). SAGE. doi:10.1177/1541931214581432
23. Pfromm, M., Khan, R., Oppelt, S., Abendroth, B., & Bruder, R. (2015). Investigation of Take-Over Performance of Driving Tasks by the Driver due to System Failure of Semi-Automated and Assisted Driving. Proceedings of the 19th Triennial Congress of the IEA (pp. 1-8).
24. Politis, I., Brewster, S., & Pollick, F. (2013). Evaluating Multimodal Driver Displays of Varying Urgency. Proceedings of the AuomotiveUI '13 (pp. 92-99). ACM Press; doi:10.1145/2516540.2516543
25. Politis, I., Brewster, S., & Pollick, F. (2014a). Evaluating Multimodal Driver Displays under Varying Situational Urgency. Proceedings of CHI '14 (pp. 4067-4076). ACM Press. doi:10.1145/2556288.2556988
26. Politis, I., Brewster, S., & Pollick, F. (2014b). Speech Tactons Improve Speech Warnings for Drivers. Proceedings of the AutomotiveUI '14 (pp. 1-8). ACM Press. doi:10.1145/2667317.2667318
27. Politis, I., Brewster, S., & Pollick, F. (2015a). Language-Based Multimodal Displays for the Handover of Control in Autonomous Cars. Proceedings of the Automotive UI '15 (pp. 3-10). ACM Press. doi:10.1145/2799250.2799262
28. Politis, I., Brewster, S., & Pollick, F. (2015b). To Beep or Not to Beep? Comparing Abstract versus Language-Based Multimodal Driver Displays. Proceedings of CHI '15 (pp. 3971-3980). ACM Press.
29. SAE. (2002). Its In-Vehicle Message Priority Standard J2395.
30. SAE J3016, & J3016, S. (2014). Taxonomy and Definitions for Terms Related to On-Road Motor Vehicle Automated Driving Systems.
31. Serrano, J., Di Stasi, L. L., Megías, A., & Catena, A. (2011). Effect of directional speech warnings on road hazard detection. Traffic Injury Prevention, 12(6), 630-635. doi:10.1080/15389588.2011.620661 PMID:22133340
32. Spence, C. (2010). Crossmodal spatial attention. Annals of the New York Academy of Sciences, 1191(1), 182-200. doi:10.1111/j.1749-6632.2010.05440.x PMID:20392281
33. Telpaz, A., Rhindress, B., Zelman, I., & Tsimhoni, O. (2015). Haptic Seat for Automated Driving : Preparing the Driver to Take Control Effectively. Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (pp. 23-30). ACM Press. doi:10.1145/2799250.2799267
34. Terken, Z., Haex, R., Beursgens, L., & Szostak, D. (2013). Unwinding After Work: An In-Car Mood Induction System for Semi-Autonomous Driving. AutomotiveUI, 246-249, 246-249. doi:10.1145/2516540.2516571
35. Walch, M., Lange, K., Baumann, M., & Weber, M. (2015). Autonomous Driving: Investigating the Feasibility of Car-Driver Handover Assistanc. Proceedings of Automotive UI '15 (pp. 11-18). ACM Press. doi:〈ALIGNMENT.qj〉〈/ALIGNMENT〉10.1145/2799250.2799268
36. Warnock, D., McGee-Lennon, M., & Brewster, S. (2011). The Role of Modality in Notification Performance. In Interact (pp. 572-588). Springer. doi:10.1007/978-3-642-23771-3-43
37. Zhao, S., Brumby, D. P., Chignell, M., Salvucci, D., & Goyal, S. (2013). Shared Input Multimodal Mobile Interfaces: Interaction Modality Effects on Menu Selection in Single-Task and Dual-Task Environments. Interacting with Computers, 25(5), 386-403. doi:10.1093/iwc/iws021