Controversy in human factors constructs and the explosive use of the NASA-TLX: A measurement perspective

Cognition, Technology and Work

Volumn 16, Issue 3, 2014, Pages 289-297

  de Winter, J C F ^a  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

Human factors constructs; Measurement theory; Operationalism; Representationalism; Situation awareness; Workload

Indexed keywords

EXPLOSIVES; ITERATIVE METHODS; MEASUREMENT THEORY; NASA; SURVEYS;

COOPER-HARPER RATING; HUMAN FACTORS RESEARCH; MEASUREMENT PROCEDURES; OPERATIONALISM; REPRESENTATIONALISM; SITUATION AWARENESS; SUBJECTIVE WORKLOAD; WORKLOAD;

HUMAN ENGINEERING;

EID: 84904541762     PISSN: 14355558     EISSN: 14355566     Source Type: Journal    
DOI: 10.1007/s10111-014-0275-1     Document Type: Article

References (52)

1. Adams S (1998) Practical considerations for measuring Situational Awareness. In: Proceedings for the third annual symposium and exhibition on situational awareness in the tactical air environment, Piney Point, MD, pp 157-164.
2. Boring EG, Stevens SS (1936) The nature of tonal brightness. Proc Natl Acad Sci 22: 514-521.
3. Bridgman PW (1927) The logic of modern physics. MacMillan, New York.
4. Byers JC, Bittner AC, Hill SG (1989) Traditional and raw task load index (TLX) correlations: are paired comparisons necessary? In: Mital A (ed) Advances in industrial ergonomics and safety 1. Taylor and Francis, London, pp 481-485.
5. Campbell NR (1920) Physics: the elements. Cambridge University Press, Cambridge.
6. Campbell NR, Jeffreys H (1938) Symposium: measurement and its importance for philosophy. Proc Aristot Soc Suppl Vol 17: 121-151.
7. Dawes RM, Smith TL (1985) Attitude and opinion measurement. In: Lindzey G, Aronson E (eds) The handbook of social psychology. Random House, New York, pp 509-566.
8. Dekker SWA (2013) On the epistemology and ethics of communicating a Cartesian consciousness. Saf Sci 56: 96-99.
9. Dekker S, Hollnagel E (2004) Human factors and folk models. Cogn Technol Work 6: 79-86.
10. Dekker SW, Woods DD (2002) Maba-maba or abracadabra? progress on human-automation co-ordination. Cogn Technol Work 4: 240-244.
11. Dekker S, Nyce JM, van Winsen R, Henriqson E (2010) Epistemological self-confidence in human factors research. J Cogn Eng Decis Mak 4: 27-38.
12. Dickinson J, Byblow WD, Ryan LA (1993) Order effects and the weighting process in workload assessment. Appl Ergon 24: 357-361.
13. Ferguson A, Myers CS, Bartlett RJ, Banister H, Bartlett FC, Brown W et al (1938) Quantitative estimates of sensory events: interim report. Br Assoc Adv Sci 108: 277-334.
14. Ferguson A, Myers CS, Bartlett RJ, Banister H, Bartlett FC, Brown W et al (1940) Quantitative estimates of sensory events. Adv Sci 1: 331-349.
15. Flach JM (1995) Situation awareness: proceed with caution. Hum Factors 37: 149-157.
16. Hancock PA (1996) Effects of control order, augmented feedback, input device and practice on tracking performance and perceived workload. Ergonomics 39: 1146-1162.
17. Hancock PA, Szalma JL (2004) On the relevance of qualitative methods for ergonomics. Theor Issues Ergon Sci 5: 499-506.
18. Hand DJ (1996) Statistics and the theory of measurement. J R Stat Soc A 159: 445-492.
19. Hand DJ (2004) Measurement: theory and practice. Arnold, London.
20. Hand DJ (2005) Size matters-how measurement defines our world. Significance 2: 81-83.
21. Hand DJ, Keynes M (1993) Letter to the Editor: Velleman PF, Wilkinson L (1993) Nominal, ordinal, interval, and ratio typologies are misleading. Am Stat 47, 65-72: Comments by Huberty and Hand; rejoinder by Velleman and Wilkinson. Am Stat 47: 314-315.
22. Hart SG, Staveland LE (1988) Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In: Hancock PA, Meshkati N (eds) Human mental workload, North Holland Press, Amsterdam, pp 139-183. http://humanfactors. arc. nasa. gov/groups/TLX/downloads/NASA-TLXChapter. pdf.
23. Hendy KC, Hamilton KM, Landry LM (1993) Measuring subjective workload: when is one scale better than many? Hum Factors 35: 579-601.
24. Hill SG, Iavecchia HP, Byers JC, Bittner AC Jr, Zaklade AL, Christ RE (1992) Comparison of four subjective workload rating scales. Hum Factors 34: 429-439.
25. Ioannidis JP (2012) Why science is not necessarily self-correcting. Perspect Psychol Sci 7: 645-654.
26. Kantowitz BH (1992) Selecting measures for human factors research. Hum Factors 34: 387-398.
27. Khurshid A, Sahai H (1993) Scales of measurements: an introduction and a selected bibliography. Qual Quant 27: 303-324.
28. Liu YL, Wickens CD (1994) Mental workload and cognitive task automaticity: an evaluation of subjective and time-estimation metrics. Ergonomics 37: 1843-1854.
29. Lord FM (1953) On the statistical treatment of football numbers. Am Psychol 8: 750-751.
30. Luce RD, Suppes P (2002) Representational measurement theory. In: Pashler H, Wixted J (eds) Stevens' handbook of experimental psychology, vol 4. Methodology in experimental psychology, 3rd edn. Wiley, New York, pp 1-41.
31. Marcus-Roberts HM, Roberts FS (1987) Meaningless statistics. J Educ Behav Stat 12: 383-394.
32. Marks LE (2006) S. S. Stevens: a brief scientific biography. In: Fechner Day 2006: Proceedings of the twenty-second annual meeting of the international society for psychophysics, St. Albans, England.
33. Mather M, Cacioppo JT, Kanwisher N (2013) Introduction to the Special Section: 20 Years of fMRI-what has it done for understanding cognition? Persp Psychol Sci 8: 41-43.
34. Merton RK (1968) The Matthew effect in science. Science 159: 56-63.
35. Michel J-B, Shen YK, Aiden AP et al (2010) Quantitative analysis of culture using millions of digitized books. Science 331: 176-182.
36. Miller GA (1974) Stanley Smith Stevens: 1906-1973. Am J Psychol 87: 279-288.
37. Moroney WF, Biers DW, Eggemeier FT, Mitchell JA (1992) A comparison of two scoring procedures with the NASA Task Load Index in a simulated flight task. In: Proceedings of the IEEE national aerospace and electronics conference 2, Dayton, OH, pp 734-740.
38. NASA (1986) Task Load Index (NASA-TLX). v. 1. 0. Paper and pencil package (instruction manual). NASA Ames Research Center, Moffett Field, CA. http://humanfactors. arc. nasa. gov/groups/TLX/downloads/TLX. pdf.
39. NASA (2014) Task Load Index paper and pencil version. NASA Ames Research Center, Moffett Field, CA. http://humansystems. arc. nasa. gov/groups/TLX/downloads/TLXScale. pdf.
40. Nygren TE (1991) Psychometric properties of subjective workload measurement techniques: implications for their use in the assessment of perceived mental workload. Hum Factors 33: 17-33.
41. Parasuraman R, Sheridan TB, Wickens CD (2008) Situation awareness, mental workload, and trust in automation: viable, empirically supported cognitive engineering constructs. J Cogn Eng Decis Mak 2: 140-160.
42. Reid GB, Shingledecker CA, Eggemeier FT (1981) Application of conjoint measurement to workload scale development. Proc Hum Factors Ergon Soc Annu Meet 25: 522-526.
43. Rubio S, Díaz E, Martín J, Puente JM (2004) Evaluation of subjective mental workload: a comparison of SWAT, NASA-TLX, and Workload Profile methods. Appl Psychol 53: 61-86.
44. Siegel S (1956) Nonparametric statistics for the behavioral sciences. McGraw-Hill, New York.
45. Stevens SS (1946) On the theory of scales of measurement. Science 103: 677-688.
46. Stevens SS (1959) Measurement, psychophysics and utility. In: Churchman CW, Ratoosh P (eds) Measurement: definitions and theories. Wiley, New York, pp 18-63.
47. Stevens SS (1961) To honor Fechner and repeal his law. Science 133: 80-86.
48. Teghtsoonian R (2001) S. S. Stevens. In: Smelser NJ, Baltes PB (eds) International encyclopedia of the social and behavioral sciences 22: 15104-15108. Pergamon, Oxford.
49. Townsend JT, Ashby FG (1984) Measurement scales and statistics: the misconception misconceived. Psychol Bull 96: 394-401.
50. Underwood BJ (1957) Psychological research. Prentice-Hall, Englewood Cliffs, NJ.
51. Uttal WR (2008) Time, space, and number in physics and psychology. Sloan Publishing. http://j. b5z. net/i/u/2084689/f/Online_Time_Space_Number. pdf.
52. Velleman PF, Wilkinson L (1993) Nominal, ordinal, interval, and ratio typologies are misleading. Am Stat 47: 65-72.