Analyzing crowdsourced ratings of speech-based take-over requests for automated driving

Applied Ergonomics

Volumn 64, Issue , 2017, Pages 56-64

  Bazilinskyy, P ^a   de Winter, J C F ^a  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

Auditory displays; Autonomous driving; Human automation interaction; Speech perception

Indexed keywords

AUTOMATION; CROWDSOURCING; ERGONOMICS;

AUDITORY DISPLAY; AUTOMATED DRIVING; AUTONOMOUS DRIVING; BACKGROUND NOISE; HUMAN FACTORS AND ERGONOMICS; HUMAN-AUTOMATION INTERACTIONS; SPEECH PERCEPTION; TRAFFIC SITUATIONS;

SPEECH;

ADULT; ARTICLE; AUTOMATED DRIVING; AUTOMATIC SPEECH RECOGNITION; AUTOMATION; CAR DRIVING; CROWDSOURCING; DECISION MAKING; ERGONOMICS; FEMALE; HUMAN; HUMAN EXPERIMENT; INDIAN; MALE; NOISE; SATISFACTION; SPEECH RATE; VOICE; CAR; HEARING; INTERNATIONAL COOPERATION; MAN MACHINE INTERACTION; SPEECH; SPEECH INTELLIGIBILITY; TRAFFIC NOISE; YOUNG ADULT;

ADULT; AUDITORY PERCEPTION; AUTOMATION; AUTOMOBILE DRIVING; AUTOMOBILES; CROWDSOURCING; FEMALE; HUMANS; INTERNATIONALITY; MALE; MAN-MACHINE SYSTEMS; NOISE, TRANSPORTATION; SPEECH ACOUSTICS; SPEECH INTELLIGIBILITY; YOUNG ADULT;

EID: 85019569213     PISSN: 00036870     EISSN: 18729126     Source Type: Journal    
DOI: 10.1016/j.apergo.2017.05.001     Document Type: Article

References (75)

1. Arik, S.Ö., Chrzanowski, M., Coates, A., Diamos, G., Gibiansky, A., Kang, Y., et al. Deep voice: Real-time neural text-to-speech. 2017 Retrieved from https://arxiv.org/pdf/1702.07825.pdf.
2. Arrabito, G.R., Effects of talker sex and voice style of verbal cockpit warnings on performance. Hum. Factors J. Hum. Factors Ergonomics Soc. 51 (2009), 3–20.
3. Arrowsmith, J., Trial watch: phase II failures: 2008–2010. Nat. Rev. Drug Discov. 10 (2011), 328–329.
4. Asendorpf, J.B., Conner, M., De Fruyt, F., De Houwer, J., Denissen, J.J., Fiedler, K., Perugini, M., Recommendations for increasing replicability in psychology. Eur. J. Personality 27 (2013), 108–119.
5. Baldwin, C.L., Verbal collision avoidance messages during simulated driving: perceived urgency, alerting effectiveness and annoyance. Ergonomics 54 (2011), 328–337.
6. Bazilinskyy, P., De Winter, J.C.F., Auditory interfaces in automated driving: an international survey. PeerJ Comput. Sci., 1, 2015, e13.
7. Begley, C.G., Ellis, L.M., Drug development: raise standards for preclinical cancer research. Nature 483 (2012), 531–533.
8. Behrend, T.S., Sharek, D.J., Meade, A.W., Wiebe, E.N., The viability of crowdsourcing for survey research. Behav. Res. Methods 43 (2011), 800–813.
9. Bent, T., Bradlow, A.R., The interlanguage speech intelligibility benefit. J. Acoust. Soc. Am. 114 (2003), 1600–1610.
10. Buhrmester, M., Kwang, T., Gosling, S.D., Amazon's Mechanical Turk a new source of inexpensive, yet high-quality, data?. Perspect. Psychol. Sci. 6 (2011), 3–5.
11. Button, K.S., Ioannidis, J.P., Mokrysz, C., Nosek, B.A., Flint, J., Robinson, E.S., Munafò, M.R., Power failure: why small sample size undermines the reliability of neuroscience. Nat. Rev. Neurosci. 14 (2013), 365–376.
12. Chandler, J., Mueller, P., Paolacci, G., Nonnaïveté among Amazon Mechanical Turk workers: consequences and solutions for behavioral researchers. Behav. Res. Methods 46 (2014), 112–130.
13. Cooke, M., Mayo, C., Valentini-Botinhao, C., Stylianou, Y., Sauert, B., Tang, Y., Evaluating the intelligibility benefit of speech modifications in known noise conditions. Speech Commun. 55 (2013), 572–585.
14. Crump, M.J., McDonnell, J.V., Gureckis, T.M., Evaluating Amazon's Mechanical Turk as a tool for experimental behavioral research. Plos One, 8, 2013, e57410.
15. Damböck, D., Weißgerber, T., Kienle, M., Bengler, K., 2013. Requirements for cooperative vehicle guidance. Proceedings of the 16th International IEEE Annual Conference on Intelligent Transportation Systems, The Hague, the Netherlands, 1656–1661.
16. De Winter, J.C.F., Dodou, D., Five-point Likert items: t test versus Mann-Whitney-Wilcoxon. Pract. Assess. Res. Eval., 15, 2010, 11.
17. De Winter, J.C.F., Dodou, D., National correlates of self-reported traffic violations across 41 countries. Personality Individ. Differ. 98 (2016), 145–152.
18. De Winter, J. C. F., Kyriakidis, M., Dodou, D., Happee, R., 2015. Using CrowdFlower to study the relationship between self-reported violations and traffic accidents. Proceedings of the 6th International Conference on Applied Human Factors and Ergonomics (AHFE), Las Vegas, NV, 2518–2525.
19. Edworthy, J., Hellier, E., Walters, K., Clift-Mathews, W., Crowther, M., Acoustic, semantic and phonetic influences in spoken warning signal words. Appl. Cogn. Psychol. 17 (2003), 915–933.
20. Edworthy, J., Hellier, E., Rivers, J., The use of male or female voices in warnings systems: a question of acoustics. Noise Health 6 (2003), 39–50.
21. Eichelberger, A.H., McCartt, A.T., Volvo drivers’ experiences with advanced crash avoidance and related technologies. Traffic Inj. Prev. 15 (2014), 187–195.
22. Eriksson, A., Stanton, N.A., Takeover time in highly automated vehicles: noncritical transitions to and from manual control. Hum. Factors, 2017, 10.1177/0018720816685832.
23. Fastl, H., 2006. Psychoacoustic basis of sound quality evaluation and sound engineering. Proceedings of the Thirteenth International Congress on Sound and Vibration, Vienna, Austria.
24. Freedman, L.P., Cockburn, I.M., Simcoe, T.S., The economics of reproducibility in preclinical research. PLOS Biol., 13, 2015, e1002165.
25. Gadbury, G.L., Allison, D.B., Inappropriate fiddling with statistical analyses to obtain a desirable p-value: tests to detect its presence in published literature. Plos One, 7, 2012, e46363.
26. Gold, C., Damböck, D., Lorenz, L., Bengler, K., 2013. “Take over!” How long does it take to get the driver back into the loop? Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 57, 1938–1942.
27. Gold, C., Berisha, I., Bengler, K., 2015. Utilization of drivetime – Performing non-driving related tasks while driving highly automated. Proceedings of the Human Factors and Ergonomics Society 59th Annual Meeting, 59, 1666–1670.
28. Greenwald, A.G., Within-subject designs: to use or not to use?. Psychol. Bull. 83 (1976), 314–320.
29. Gururaj, G., Road traffic deaths, injuries and disabilities in India: current scenario. Natl. Med. J. India 21 (2008), 14–20.
30. Hellier, E., Edworthy, J., Weedon, B., Walters, K., Adams, A., The perceived urgency of speech warnings: semantics versus acoustics. Hum. Factors J. Hum. Factors Ergonomics Soc. 44 (2002), 1–17.
31. Henrich, J., Heine, S.J., Norenzayan, A., The weirdest people in the world?. Behav. Brain Sci. 33 (2010), 61–135.
32. Hergeth, S., Lorenz, L., Krems, J. F., Toenert, L., 2015. Effects of take-over requests and cultural background on automation trust in highly automated driving. Proceedings of the Eighth International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design, Salt Lake City, UT, 331–337.
33. Hollander, T. D., Wogalter, M. S., 2000. Connoted hazard of voiced warning signal words: An examination of auditory components. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 44, 702–705.
34. Ioannidis, J.P., Why most published research findings are false. PLOS Med., 2, 2005, e124.
35. Ioannidis, J., Doucouliagos, C., What's to know about the credibility of empirical economics?. J. Econ. Surv. 27 (2013), 997–1004.
36. Jang, P.S., Designing acoustic and non-acoustic parameters of synthesized speech warnings to control perceived urgency. Int. J. Industrial Ergonomics 37 (2007), 213–223.
37. Keren, G., Between-or within-subjects design: a methodological dilemma. Keren, G., Lewis, C., (eds.) A Handbook for Data Analysis in the Behaviorial Sciences: Methodological Issues, 1993, Erlbaum, Hillsdale, NJ, 257–272.
38. Klein, R.A., Ratliff, K.A., Vianello, M., Adams, R.B. Jr., Bahník, Š., Bernstein, M.J., Cemalcilar, Z., Investigating variation in replicability: a “many labs” replication project. Soc. Psychol. 45 (2014), 142–152.
39. Kuchar, J.K., Yang, L.C., A review of conflict detection and resolution modeling methods. IEEE Trans. Intelligent Transp. Syst. 1 (2000), 179–189.
40. Kyriakidis, M., Happee, R., De Winter, J.C.F., Public opinion on automated driving: results of an international questionnaire among 5,000 respondents. Transp. Res. Part F Traffic Psychol. Behav. 32 (2015), 127–140.
41. Langlois, S., Suied, C., Lageat, T., Charbonneau, A., 2008. Cross cultural study of auditory warnings. Proceedings of the 14th International Conference on Auditory Display (ICAD2008), Paris, France.
42. Large, D.R., Burnett, G.E., Drivers’ preferences and emotional responses to satellite navigation voices. Int. J. Veh. Noise Vib. 9 (2013), 28–46.
43. Lerner, N., Singer, J., Kellman, D., Traube, E., 2015. In-vehicle noise alters the perceived meaning of auditory signals. Proceedings of the Eighth International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design, Salt Lake City, UT, 401–407.
44. Ljungberg, J.K., Parmentier, F.B., Hughes, R.W., Macken, W.J., Jones, D.M., Listen out! Behavioural and subjective responses to verbal warnings. Appl. Cogn. Psychol. 26 (2012), 451–461.
45. Ljungberg, J.K., Parmentier, F., The impact of intonation and valence on objective and subjective attention capture by auditory alarms. Hum. Factors J. Hum. Factors Ergonomics Soc. 54 (2012), 826–837.
46. Machado, S., Duarte, E., Teles, J., Reis, L., Rebelo, F., Selection of a voice for a speech signal for personalized warnings: the effect of speaker's gender and voice pitch. Work 41 (2012), 3592–3598.
47. Maxwell, S.E., Lau, M.Y., Howard, G.S., Is psychology suffering from a replication crisis? What does “failure to replicate” really mean?. Am. Psychol. 70 (2015), 487–498.
48. Merat, N., Jamson, A. H., 2009. How do drivers behave in a highly automated car. Proceedings of the 5th International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design, Big Sky, MT, 514–521.
49. Mok, B., Johns, M., Lee, K. J., Miller, D., Sirkin, D., Ive, P., Ju, W., 2015. Emergency, automation off: Unstructured transition timing for distracted drivers of automated vehicles. Proceedings of the 2015 IEEE 18th International Conference on Intelligent Transportation Systems, Canary Islands, Spain, 2458–2464.
50. Munro, M.J., Derwing, T.M., Foreign accent, comprehensibility, and intelligibility in the speech of second language learners. Lang. Learn. 45 (1995), 73–97.
51. Munro, M.J., Derwing, T.M., Morton, S.L., The mutual intelligibility of L2 speech. Stud. Second Lang. Acquis. 28 (2006), 111–131.
52. Munro, M.J., Foreign accent and speech intelligibility. Hansen Edwards, J.G., Zampini, M.L., (eds.) Phonology and Second Language Acquisition, 2008, John Benjamins, Philadelphia, 193–218.
53. Naujoks, F., Mai, C., Neukum, A., 2014. The effect of urgency of take-over requests during highly automated driving under distraction conditions. In T. Ahram, W. Karwowski, & T. Marek (Eds.), Proceedings of the 5th International Conference on Applied Human Factors and Ergonomics (AHFE), Kraków, Poland.
54. Nilsson, J., Strand, N., Falcone, P., Vinter, J., 2013. Driver performance in the presence of adaptive cruise control related failures: Implications for safety analysis and fault tolerance. Proceedings of the 43rd Annual IEEE/IFIP Conference on Dependable Systems and Networks Workshop, Budapest, Hungary.
55. Nixon, C.W., Morris, L.J., McCavitt, A.R., McKinley, R.L., Anderson, T.R., McDaniel, M.P., Yeager, D.G., Female voice communications in high levels of aircraft cockpit noises–part I: spectra, levels, and microphones. Aviat. Space, Environ. Med. 69 (1998), 675–683.
56. Noyes, J.M., Hellier, E., Edworthy, J., Speech warnings: a review. Theor. Issues Ergonomics Sci. 7 (2006), 551–571.
57. Oedegaarde, Tesla Autopilot Test (EU Version) - on Country Road, Highway and in Town. 2015 https://www.youtube.com/watch?v=7YVNW7-lFx8.
58. Open Science Collaboration. Estimating the reproducibility of psychological science. Science, 349, 2015, aac4716.
59. Özkan, T., Lajunen, T., The role of personality, culture, and economy in unintentional fatalities: an aggregated level analysis. Personality Individ. Differ. 43 (2007), 519–530.
60. Parasuraman, R., Riley, V., Humans and automation: use, misuse, disuse, abuse. Hum. Factors 39 (1997), 230–253.
61. Park, K.S., Jang, P.S., Effects of synthesized voice warning parameters on perceived urgency. Int. J. Occup. Saf. Ergonomics 5 (1999), 73–95.
62. Petermeijer, S., De Winter, J.C.F., Bengler, K., Vibrotactile displays: a survey with a view on highly automated driving. IEEE Trans. Intelligent Transp. Syst. 17 (2016), 897–907.
63. Pfromm, M., Khan, R., Oppelt, S., Abendroth, B., Brudera, R., 2015. Investigation of take-over performance of driving tasks by the driver due to system failure of semi-automated and assisted driving. Proceedings 19th Triennial Congress of the IEA, Melbourne, Australia.
64. Podlipský, V.J., Šimáčková, Š., Petráž, D., Is there an interlanguage speech credibility benefit?. Top. Linguist. 17 (2016), 30–44.
65. Politis, I., Brewster, S., Pollick, F., 2015. Language-based multimodal displays for the handover of control in autonomous cars. Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Application, Nottingham, UK.
66. Rand, D.G., The promise of Mechanical Turk: how online labor markets can help theorists run behavioral experiments. J. Theor. Biol. 299 (2012), 172–179.
67. Smith, L.E., Rafiqzad, K., English for cross-cultural communication: the question of intelligibility. TESOL Q. 13 (1979), 371–380.
68. SAE International, Taxonomy and Definitions for Terms Related To Driving Automation Systems for On-Road Motor Vehicles (Standard No. J3016). 2016.
69. Stewart, N., Ungemach, C., Harris, A.J., Bartels, D.M., Newell, B.R., Paolacci, G., Chandler, J., The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers. Judgm. Decis. Mak. 10 (2015), 479–491.
70. Stroebe, W., Are most published social psychological findings false?. J. Exp. Soc. Psychol. 66 (2016), 134–144.
71. Wagenmakers, E.J., Verhagen, J., Ly, A., Bakker, M., Lee, M.D., Matzke, D., Morey, R.D., A power fallacy. Behav. Res. Methods 47 (2015), 913–917.
72. Wogalter, M.S., Silver, N.C., Warning signal words: connoted strength and understandability by children, elders, and non-native English speakers. Ergonomics 38 (1995), 2188–2206.
73. Wogalter, M.S., Conzola, V.C., Smith-Jackson, T.L., Research-based guidelines for warning design and evaluation. Appl. Ergon. 33 (2002), 219–230.
74. World Health Organization, WHO Global Status Report on Road Safety 2015., 2015, World Health Organization, Geneva, Switzerland.
75. Zeeb, K., Buchner, A., Schrauf, M., What determines the take-over time? An integrated model approach of driver take-over after automated driving. Accid. Analysis Prev. 78 (2015), 212–221.