Self-report measures for the assessment of human–machine interfaces in automated driving

Cognition, Technology and Work

Volumn 22, Issue 4, 2020, Pages 703-720

  Forster, Yannick ^a,b   Hergeth, Sebastian ^a   Naujoks, Frederik ^a   Krems, Josef F ^b   Keinath, Andreas ^a  

^a BMW GROUP   (Germany)

^b CHEMNITZ UNIVERSITY OF TECHNOLOGY   (Germany)

Author keywords

Automated driving; Human computer interaction; Human machine interfaces; Psychometrics; Tools and methods

Indexed keywords

AUTOMATION; DECELERATION; RELIABILITY ANALYSIS;

AUTOMATED DRIVING; COMPUTER INTERACTION; MACHINE INTERFACES; PSYCHOMETRICS; TOOLS AND METHODS;

HUMAN COMPUTER INTERACTION;

EID: 85073817504     PISSN: 14355558     EISSN: 14355566     Source Type: Journal    
DOI: 10.1007/s10111-019-00599-8     Document Type: Article

References (90)

1. AdaptIVe Consortium (2017) Final functional human factors recommendations (Deliverable D3.3)
2. Adell E (2010) Acceptance of driver support systems. Proc Eur Conf Hum Centered Design Intell Transp Syst 2:475–486
3. Adell E, Nilsson L, Várhelyi A (2014) How is acceptance measured? Overview of measurement issues, methods and tools. In: Horberry T, Regan MA, Stevens A (eds) Driver acceptance of new technology theory measurement and optimisation. CRC Press, London, UK, pp 73–89
4. Anthoine E, Moret L, Regnault A, Sébille V, Hardouin J-B (2014) Sample size used to validate a scale: a review of publications on newly-developed patient reported outcomes measures. Health Qual Life Outcomes 12:176. 10.1186/s12955-014-0176-2
5. Auricht M, Stark R, Blume C (2014) Integrating user experience validation into a new engineering development process for advanced driver assistance systems. In: Boyle LN (ed) The 6th international conference of automotive user interfaces and interactive vehicular applications. Seattle, WA, USA
6. Bangor A, Kortum P, Miller J (2009) Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Stud 4(3):114–123
7. Bazilinskyy P, Eriksson A, Petermeijer B, de Winter J (2017) Usefulness and satisfaction of take-over requests for highly automated driving. In: Road safety and simulation international conference (RSS 2017), The Hague, Netherlands
8. Beggiato M, Pereira M, Petzoldt T, Krems JF (2015) Learning and development of trust, acceptance and the mental model of ACC. A longitudinal on-road study. Transp Res Part F Traffic Psychol Behav 35:75–84. 10.1016/j.trf.2015.10.005
9. Brooke J (1996) SUS—a quick and dirty usability scale. Usability Eval Ind 194(189):4–7
10. Bubb H, Bengler K, Grünen RE, Vollrath M (2015) Automobilergonomie. Springer, Berlin
11. Bühner M (2011) Einführung in die Test-und Fragebogenkonstruktion. Pearson Deutschland GmbH, München
12. Campbell DT, Fiske DW (1959) Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol Bull 56(2):81
13. Chien S-Y, Semnani-Azad Z, Lewis M, Sycara K (2014) Towards the development of an inter-cultural scale to measure trust in automation. In: International conference on cross-cultural design
14. Crash Avoidance Metrics Partnership (2016) Automated vehicles research for enhanced safety. NHTSA, Department of Transportation, Washington, DC
15. Cronbach LJ (1951) Coefficient alpha and the internal structure of tests. Psychometrika 16(3):297–334. 10.1007/BF02310555
16. Cronbach LJ, Meehl PE (1955) Construct validity in psychological tests. Psychol Bull 52(4):281–302. 10.1037/h0040957
17. Davis FD (1985) A technology acceptance model for empirically testing new end-user information systems: theory and results. Massachusetts Institute of Technology, Boston
18. Davis FD (1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quart 13(3):319. 10.2307/249008
19. Eichinger A (2011) Untersuchungskonzepte für die Evaluation von Systemen zur Erkennung des Fahrerzustands: BASt-Forschungsbericht: FE 82.369/2009. Berichte der Bundesanstalt für Straßenwesen 80:45–94
20. Fornell C, Larcker DF (1981) Evaluating structural equation models with unobservable variables and measurement error. J Market Res 18:39–50
21. Forster Y, Naujoks F, Neukum A (2016) Your turn or my turn? Design of a human–machine interface for conditional automation. In: Green P (ed) Proceedings of the 8th international conference on automotive user interfaces and interactive vehicular applications. Ann Arbor, MI, USA
22. Forster Y, Naujoks F, Neukum A (2017) Increasing anthropomorphism and trust in automated driving functions by adding speech output. Intelligent Vehicles Symposium (IV), 2017 IEEE, Redondo Beach, California, USA
23. Forster Y, Kraus J, Feinauer S, Baumann M (2018a) Calibration of trust expectancies in conditionally automated driving by brand, reliability information and introductionary videos: an online study. In: Donmez B, Walker BN, Fröhlich K (Chairs) Proceedings of the 10th international conference on automotive user interfaces and interactive vehicular applications, Toronto, CN
24. Forster Y, Hergeth S, Naujoks F, Krems, JF (2018b) How usability can save the day: methodological considerations for making automated driving a success story. In: Donmez B, Walker BN, Fröhlich K (Chairs) Proceedings of the 10th international conference on automotive user interfaces and interactive vehicular applications, Toronto, CN
25. Forster Y, Hergeth S, Naujoks F, Beggiato M, Krems JF, Keinath A (2019a) Learning and development of mental models in interaction with driving automation: a simulator study. Driving Assessment Conference, Santa Fe, NM, USA
26. Forster Y, Hergeth S, Naujoks F, Beggiato M, Krems JF, Keinath A (2019b) Learning to use automation: behavioral changes in interaction with automated driving systems. Transp Res Part F Traffic Psychol Behav 62:599–614
27. Forster Y, Hergeth S, Naujoks F, Krems JF, Keinath A (2019) Empirical validation of a checklist for heuristic evaluation of automated vehicle HMIs. In: 10th international conference on applied human factors and ergonomics, Washington D.C., USA
28. François M, Osiurak F, Fort A, Crave P, Navarro J (2016) Automotive HMI design and participatory user involvement: review and perspectives. Ergonomics 60(4):541–552
29. Frison A-K, Wintersberger P, Riener A, Schartmüller C (2017) Driving hotzenplotz: a hybrid interface for vehicle control aiming to maximize pleasure in highway driving. In: Boll (ed) Proceedings of the 9th international conference on automotive user interfaces and interactive vehicular applications, Oldenburg, Germany
30. Frison A-K, Wintersberger P, Riener A, Schartmüller C, Boyle LN, Miller E, Weigl K (2019) UX We Trust: investigation of aesthetics and usability of driver-vehicle interfaces and their impact on the perception of automated driving. The 2019 CHI Conference, Glasgow, UK
31. Ghazizadeh M, Lee JD, Boyle LN (2012) Extending the technology acceptance model to assess automation. Cognit Technol Work 14(1):39–49
32. Gold C, Körber M, Hohenberger C, Lechner D, Bengler K (2015) Trust in automation—before and after the experience of take-over scenarios in a highly automated vehicle. Procedia Manufact 3:3025–3032
33. Gold C, Naujoks F, Radlmayr J, Bellem H, Jarosch O (2017) Testing scenarios for human factors research in level 3 automated vehicles. In: International conference on applied human factors and ergonomics, Los Angeles, CA, USA
34. Green P, Levison W, Paelke G, Serafin C (1994) Suggested human factors design guidelines for driver information systems. UMTRI, Michigan
35. Guttman L (1945) A basis for analyzing test–restest reliability. Psychometrika 10:255–282
36. Hassenzahl M (2001) The effect of perceived hedonic quality on product appealingness. Int J Hum Comput Interact 13(4):481–499. 10.1207/S15327590IJHC1304_07
37. Hassenzahl M, Burmester M, Koller F (2003) AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität [AttracDiff: A questionnaire to measure perceived hedonic and pragmatic quality]. In: Ziegler J, Szwillus G (eds) Mensch & Computer 2003. Interaktion in Bewegung. B. G. Teubner, Stuttgart, Leipzig, pp 187–196
38. Häuslschmid R, von Bülow M, Pfleging B, Butz A (2017) SupportingTrust in autonomous driving. In: Papadopoulos GA, Kuflik T, Chen F, Duarte C, Fu W-T (Chairs) The 22nd international conference on intelligent user interfaces, Limassol, Cyprus
39. Henrich J, Heine SJ, Norenzayan A (2010) The weirdest people in the world? Behav Brain Sci 33(2–3):61–83. 10.1017/S0140525X0999152X (discussion 83–135)
40. Hergeth S (2016) Automation trust in conditional automated driving systems: approaches to operationalization and design. PhD Thesis. Chemnitz University of Technology. Chemnitz, Germany
41. Hergeth S, Lorenz L, Krems JF (2017) Prior familiarization with takeover requests affects drivers’ takeover performance and automation trust. Hum Factors 59(3):457–470
42. Hoff K, Bashir M (2016) Trust in automation: integrating empirical evidence on factors that influence trust. Hum Factors 57(3):407–434
43. Hornbæk K (2006) Current practice in measuring usability: challenges to usability studies and research. Int J Hum Comput Stud 64(2):79–102. 10.1016/j.ijhcs.2005.06.002
44. Hornbæk K, Law EL-C (2007) Meta-analysis of correlations among usability measures. The 2007 CHI conference, San Jose, California, USA. 10.1145/1240624.1240722
45. International Organization for Standardization (2018) Ergonomics of Human–System Interaction—Part 11: Usability: Definitions and Concepts. Geneva, Switzerland: ISO 9241-11
46. ISO (2012) Road vehicles—ergonomic aspects of transport information and control systems—calibration tasks for methods which assess driver demand due to the use of in-vehicle systems. (ISO, 14198). Geneva, Switzerland
47. Jarosch O, Kuhnt M, Paradies S, Bengler K (2017) It’s out of our hands now! Effects of non-driving related tasks during highly automated driving on drivers’ fatigue. In: Driving Assessment Conference, Manchester Village, Vermont, USA
48. Jian J-Y, Bisantz AM, Drury CG (2000) Foundations for an empirically determined scale of trust in automated systems. Int J Cognit Ergon 4(1):53–71
49. Jones PS, Lee JW, Phillips LR, Zhang XE, Jaceldo KB (2001) An adaptation of Brislin’s translation model for cross-cultural research. Nursing Res 50(5):300–304
50. Kline TJB (1999) The team player inventory: reliability and validity of a measure of predisposition toward organizational team-working environments. J Specialists Group Work 24(1):102–112
51. König M, Neumayr L (2017) Users’ resistance towards radical innovations: the case of the self-driving car. Transp Res Part F Traffic Psychol Behav 44:42–52
52. Kyriakidis M, Happee R, de Winter JCF (2015) Public opinion on automated driving: results of an international questionnaire among 5000 respondents. Transp Res Part F Traffic Psychol Behav 32:127–140
53. Laugwitz B, Held T, Schrepp M (2008) Construction and evaluation of a user experience questionnaire. Symp Aust HCI Usability Eng Group. 10.1002/9783527617272.ch1
54. Lee JD, See KA (2004) Trust in automation: designing for appropriate reliance. Hum Factors 46(1):50–80
55. Lewis JR (2002) Psychometric evaluation of the PSSUQ using data from 5 years of usability studies. Int J Hum Comput Interact 14(3–4):463–488. 10.1515/9783110887242.3
56. Lewis JR, Sauro J (2009) The factor structure of the system usability scale. International conference on human centered design
57. Lüdtke O, Robitzsch A, Trautwein U, Köller O (2007) Umgang mit fehlenden Werten in der psychologischen Forschung. Psychologische Rundschau 58(2):103–117. 10.1026/0033-3042.58.2.103
58. Minge M, Thüring M, Wagner I, Kuhr CV (2016) The meCUE questionnaire: a modular tool for measuring user experience. In: Advances in ergonomics modeling 2016, pp 115–128
59. Naujoks F, Purucker C, Neukum A, Wolter S, Steiger R (2015) Controllability of partially automated driving functions—does it matter whether drivers are allowed to take their hands off the steering wheel? Transp Res Part F Traffic Psychol Behav 35:185–198
60. Naujoks F, Forster Y, Wiedemann K, Neukum A (2016) Speech improves human-automation cooperation in automated driving. In: Workshopband Mensch und Computer 2016. Aachen, Germany
61. Naujoks F, Forster Y, Wiedemann K, Neukum A (2017) Improving usefulness of automated driving by lowering primary task interference through HMI design. J Adv Transp. 10.1155/2017/6105087
62. Naujoks F, Hergeth S, Keinath A, Wiedemann K, Schömig N (2018) Use cases for assessing, testing, and validating the human–machine interface of automated driving systems. Human Factors and Ergonomics Society Annual Meeting, Philadelphia
63. Naujoks F, Hergeth S, Wiedemann K, Schömig N, Forster Y, Keinath A (2019a) Test procedure for evaluating the human–machine-interface of vehicles with automated driving. Traffic Injury Prevent 20:146–151
64. Naujoks F, Wiedemann K, Schömig N, Hergeth S, Keinath A (2019b) Towards guidelines and verification methods for automated vehicle HMIs. Transp Res Part F Traffic Psychol Behav 60:121–136
65. Nees M (2016) Acceptance of self-driving cars: an examination of idealized versus realistic portrayals with a self-driving cars acceptance scale. In: Human Factors and Ergonomics Society 60th Annual Meeting, Washington, D.C
66. Nielsen J, Levy J (1994) Measuring usability: preference vs. performance. Commun ACM 37(4):66–75
67. Nordhoff S, van Arem B, Happee R (2016) Conceptual model to explain, predict, and improve user acceptance of driverless podlike vehicles. Transp Res Record J Transp Res Board 2602:60–67. 10.3141/2602-08
68. Norman D, Miller J, Henderson A (1995) What you see, some of what’s in the future, and how we go about doing it: HI at apple computer. In: Conference companion on human factors in computing systems
69. Nunes A, Reimer B, Coughlin JF (2018) People must retain control of autonomous vehicles. Nature 556(7700):169–171. 10.1038/d41586-018-04158-5
70. Nunnally JC (1978) Psychometric theory. McGraw-Hill, New York
71. O’connor BP (2000) SPSS and SAS programs for determining the number of components using parallel analysis and Velicer’s MAP test. Behav Res Methods Instrum Comput 32(3):396–402. 10.3758/BF03200807
72. Osswald S, Wurhofer D, Trösterer S, Beck E, Tscheligi M (2012) Predicting information technology usage in the car: towards a car technology acceptance model. In: Kun AL, Boyle LN, Reimer B, Riener A (Chairs) Proceedings of the 4th international conference on automotive user interfaces and interactive vehicular applications. symposium conducted at the meeting of ACM, Portsmouth, New Hampshire, US
73. Payre W, Cestac J, Delhomme P (2014) Intention to use a fully automated car: attitudes and a priori acceptability. Transp Res Part F Traffic Psychol Behav 27:252–263. 10.1016/j.trf.2014.04.009
74. Rahman MM, Lesch MF, Horrey WJ, Strawderman L (2017) Assessing the utility of TAM, TPB, and UTAUT for advanced driver assistance systems. Accid Anal Prevent 108:361–373. 10.1016/j.aap.2017.09.011
75. Revelle W, Zinbarg RE (2009) Coefficients alpha, beta, omega, and the glb: comments on Sijtsma. Psychometrika 74(1):145
76. Society of Automotive Engineers International J3016 (2018) Taxonomy and definitions for terms related to on-road motor vehicle automated driving systems. SAE International, Warrendale, PA
77. Sijtsma K (2009) On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika 74(1):107–120. 10.1007/s11336-008-9101-0
78. Spain RD, Bustamante EA, Bliss JP (2008) Towards an empirically Developed Scale for System Trust: take Two. Proc Hum Factors Ergon Soc Ann Meeting 52(19):1335–1339. 10.1177/154193120805201907
79. Stevens A, Quimby A, Board A, Kersloot T, Burns P (2002) Design guidelines for safety in-vehicle information systems. TRL Limited, Crowthorne
80. Tabachnick BG, Fidell LS (2007) Using multivariate statistics. Allyn & Bacon/Pearson Education, Boston
81. Tullis TS, Boynton TL, Hersh H (1995) Readability of fonts in the windows environment. Conference companion on Human factors in computing systems
82. Van der Laan JD, Heino A, de Waard D (1997) A simple procedure for the assessment of acceptance of advanced transport telematics. Transp Res Part C Emerg Technol 5(1):1–10
83. Vanderhaegen F, Carsten O (2017) Can dissonance engineering improve risk analysis of human–machine systems? Cogn Technol Work 19(1):1–12. 10.1007/s10111-017-0405-7
84. Velicer WF (1976) Determining the number of components from the matrix of partial correlations. Psychometrika 41(3):321–327. 10.1007/BF02293557
85. Venkatesh V, Morris MG, Davis GB, Davis FD (2003) User acceptance of information technology: toward a unified view. MIS Quart (27:3):425–478
86. Verberne FMF, Ham J, Midden CJH (2012) Trust in smart systems: sharing driving goals and giving information to increase trustworthiness and acceptability of smart systems in cars. Hum Factors 54(5):799–810
87. Walch M, Baumann M, Jaschke L, Weber M, Hock P (2017) Touch screen maneuver approval mechanisms for highly automated vehicles: a first evaluation. Adjunct proceedings of the 9th international ACM conference on automotive user interfaces and interactive vehicular applications, Oldenburg, Germany
88. Waytz A, Heafner J, Epley N (2014) The mind in the machine: anthropomorphism increases trust in an autonomous vehicle. J Exp Soc Psychol 52:113–117. 10.1016/j.jesp.2014.01.005
89. Wickens CD, Hollands JG, Banbury S, Parasuraman R (2015) Engineering psychology and human performance. Psychology Press, Boca Raton
90. Zoellick JC, Kuhlmey A, Schenk L, Schindel D, Blüher S (2019) Assessing acceptance of electric automated vehicles after exposure in a realistic traffic environment. PLOS One. 10.1371/journal.pone.0215969