Using near infrared spectroscopy and heart rate variability to detect mental overload

Behavioural Brain Research

Volumn 259, Issue , 2014, Pages 16-23

  Durantin, G ^b   Gagnon, J F ^a   Tremblay, S ^a   Dehais, F ^b  

^a INSTITUT SUPÉRIEUR DE L'AÉRONAUTIQUE ET DE L'ESPACE   (France)

^b UNIVERSITÉ LAVAL   (Canada)

Author keywords

HRV; Human factors; Mental overload; NIRS; Remotely operated vehicle

Indexed keywords

ADULT; ARTICLE; AUTONOMIC NERVOUS SYSTEM; BRAIN REGION; CENTRAL NERVOUS SYSTEM; FEMALE; HEART RATE VARIABILITY; HUMAN; HUMAN EXPERIMENT; MALE; MENTAL CAPACITY; MENTAL LOAD; MENTAL OVERLOAD; MENTAL PERFORMANCE; NEAR INFRARED SPECTROSCOPY; OXYGEN CONSUMPTION; PREFRONTAL CORTEX; PRIORITY JOURNAL; PSYCHOMOTOR PERFORMANCE; TASK PERFORMANCE; WORKING MEMORY;

HRV; HUMAN FACTORS; MENTAL OVERLOAD; NIRS; REMOTELY OPERATED VEHICLE;

ADULT; ANALYSIS OF VARIANCE; ELECTROCARDIOGRAPHY; FEMALE; FOURIER ANALYSIS; HEART RATE; HEMOGLOBINS; HUMANS; MALE; MENTAL PROCESSES; PSYCHOMETRICS; SPECTROSCOPY, NEAR-INFRARED; TASK PERFORMANCE AND ANALYSIS; WORKLOAD; YOUNG ADULT;

EID: 84887531883     PISSN: 01664328     EISSN: 18727549     Source Type: Journal    
DOI: 10.1016/j.bbr.2013.10.042     Document Type: Article

References (72)

1. Cooke N.J. Human factors of remotely operated vehicles. Proceedings of the Human Factors and Ergonomics Society Annual Meeting (vol. 50, pp. 166-169) 2006, SAGE Publications. 10.1177/154193120605000135.
2. Hancock P.A., Verwey W.B. Fatigue, workload and adaptive driver systems. Accident Analysis and Prevention 1997, 29(4):495-506.
3. Parasuraman R., Riley V. Humans and automation: use, misuse, disuse, abuse. Human Factors 1997, 39(2):230-253.
4. Williams K.W. A Summary of unmanned aircraft accident/incident data: human factors implications. Security 2004, 18.
5. Johnson C.W. Failure in safety-critical systems: a handbook of accident and incident reporting 2003, University of Glasgow Press, Glasgow, Scotland.
6. Reason J. The contribution of latent human failures to the breakdown of complex systems. Philosophical Transactions of the Royal Society of London. B, Biological Sciences 1990, 327(1241):475-484.
7. Shappel S.A., Wiegmann D.A. The human factors analysis and classification system - HFACS (No. DOT/FAA/AM-00/7) 2000, US Federal Aviation Administration, Office of Aviation Medicine.
8. Chen J.Y.C., Haas E.C., Barnes M.J. Human performance issues and user interface design for teleoperated robots. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 2007, 37(6):1231-1245. IEEE.
9. Cummings M., Bruni S., Mercier S., Mitchell P.J. Automation architecture for single operator, multiple UAV command and control. Security 2007, 1(2):1-24.
10. Ruff H.A., Narayanan S., Draper M.H. Human interaction with levels of automation and decision-aid fidelity in the supervisory control of multiple simulated. Technology 2002, 11(4):335-351.
11. Kahneman D. Attention and effort 1973, Prentice-Hall, Englewood Cliffs, NJ.
12. Koechlin E., Ody C., Kouneiher F. The architecture of cognitive control in the human prefrontal cortex. Science 2003, 302(5648):1181-1185.
13. Dehais F., Causse M., Vachon F., Tremblay S. Cognitive conflict in human-automation interactions: a psychophysiological study. Applied Ergonomics 2012, 43:588-595.
14. Thomas L.C., Wickens C.D. Visual displays and cognitive tunneling: frames of reference effects on spatial judgments and change detection. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2001, 45(4):336-340. 10.1177/154193120104500415.
15. Hockey G.R.J. Operator functional state: the prediction of breakdown in human performance. Speed, control and age: in honor of Patrick rabbit 2005, 373-394. Oxford University Press, Oxford.
16. Hancock P.A., Szalma J.L. The future of neuroergonomics. Theoretical Issues in Ergonomics Science 2003, 4(1/2):238-249.
17. Parasuraman R., Rizzo M. Neuroergonomics: the brain at work 2007, Oxford University Press, New York.
18. Sarter N., Sarter M. Neuroergonomics: opportunities and challenges of merging cognitive neuroscience with cognitive ergonomics. Theoretical Issues in Ergonomics Science 2003, 4(1/2):142-150.
19. Freeman F.G., Mikulka P.J., Prinzel L.J., Scerbo M.W. Evaluation of an adaptive automation system using three EEG indices with a visual tracking task. Biological Psychology 1999, 50(1):61-76.
20. Sheridan T. Adaptive automation, level of automation, allocation authority, supervisory control, and adaptive control: distinctions and modes of adaptation. Systems, Man and Cybernetics, Part A: Systems 2011, 41(4):662-667.
21. Kaber D.B., Perry C.M., Segall N., Sheik-Nainar M.A. Workload state classification with automation during simulated air traffic control. The International Journal of Aviation Psychology 2007, 17(4):371-390.
22. Dehais F., Causse M., Tremblay S. Mitigation of conflicts with automation: use of cognitive countermeasures. Human Factors: The Journal of the Human Factors and Ergonomics Society 2011, 53(5):448-460. 10.1177/0018720811418635.
23. Visser E.D., Parasuraman R. Adaptive aiding of human-robot teaming: effects of imperfect automation on performance, trust, and workload. Journal of Cognitive Engineering and Decision Making 2011, 5(2):209-231. 10.1177/1555343411410160.
24. Tsai Y.F., Viirre E., Strychacz C., Chase B., Jung T.P. Task performance and eye activity: predicting behavior relating to cognitive workload. Aviation, Space, and Environmental Medicine 2007, 78(suppl. 1):B176-B185.
25. Haapalainen E., Kim S., Forlizzi J.F., Dey A.K. Psycho-physiological measures for assessing cognitive load. Proceedings of the 12th ACM international conference on Ubiquitous computing 2010 September, 301-310.
26. Kamath M.V., Fallen E.L. Power spectral analysis of heart rate variability: a noninvasive signature of cardiac autonomic function. Critical Reviews in Biomedical Engineering 1993, 21(3):245.
27. Hjortskov N., Rissen D., Blangsted A.K., Fallentin N., Lundberg U., Søgaard K. The effect of mental stress on heart rate variability and blood pressure during computer work. European Journal of Applied Physiology 2004, 92(1-2):84-89.
28. Murai K., Hayashi Y., Okazaki T., Stone L.C., Mitomo N. Evaluation of ship navigator's mental workload using nasal temperature and heart rate variability. International Conference on Systems, Man and Cybernetics, SMC 2008, IEEE 2008 October, 1528-1533.
29. Miyake S. Multivariate workload evaluation combining physiological and subjective measures. International Journal of Psychophysiology 2001, 40(3):233-238.
30. Parasuraman R., Caggiano D. Neural and genetic assays of human mental workload. Quantifying Human Information Processing 2005, 123-149.
31. Cui X., Bray S., Bryant D.M., Glover G.H., Reiss A.L. A quantitative comparison of NIRS and fMRI across multiple cognitive tasks. NeuroImage 2011, 54(4):2808-2821.
32. Ayaz H., Shewokis P.A., Bunce S., Izzetoglu K., Willems B., Onaral B. Optical brain monitoring for operator training and mental workload assessment. NeuroImage 2011, 59(1):36-47. Elsevier Inc. 10.1016/j.neuroimage.2011.06.023.
33. Takeuchi Y. Change in blood volume in the brain during a simulated aircraft landing task. Journal of Occupational Health 2000, 42(2):60-65.
34. Ayaz H., Cakir M.P., Izzetoglu K., Curtin A., Shewokis P.A., Bunce S.C., et al. Monitoring expertise development during simulated UAV piloting tasks using optical brain imaging. Aerospace Conference, 2012 IEEE 2012 March, 1-11. IEEE.
35. Tsujii T., Yamamoto E., Ohira T., Takahashi T., Watanabe S. Antihistamine effects on prefrontal cortex activity during working memory process in preschool children: a near-infrared spectroscopy (NIRS) study. Neuroscience Research 2010, 67(1):80-85.
36. Yang H., Zhou Z., Liu Y., Ruan Z., Gong H., Luo Q., et al. Gender difference in hemodynamic responses of prefrontal area to emotional stress by near-infrared spectroscopy. Behavioural Brain Research 2007, 178(1):172-176.
37. Herrmann M.J., Plichta M.M., Ehlis A.C., Fallgatter A.J. Optical topography during a Go-NoGo task assessed with multi-channel near-infrared spectroscopy. Behavioural Brain Research 2005, 160(1):135-140.
38. Boecker M., Buecheler M.M., Schroeter M.L., Gauggel S. Prefrontal brain activation during stop-signal response inhibition: an event-related functional near-infrared spectroscopy study. Behavioural Brain Research 2007, 176(2):259-266.
39. Goldberg T.E., Berman K.F., Fleming K., Ostrem J., Van Horn J.D., Esposito G., et al. Uncoupling cognitive workload and prefrontal cortical physiology: a PET rCBF study. NeuroImage 1998, 7(4):296303.
40. Yerkes R.M., Dodson J.D. The relation of strength of stimulus to rapidity of habit-formation. Journal of Comparative Neurology and Psychology 1908, 18(5):459-482.
41. Jaeggi S.M., Studer-Luethi B., Buschkuehl M., Su Y., Jonides J., Perrig W.J. The relationship between n-back performance and matrix reasoning - implications for training and transfer. Intelligence 2010, 38(6):625-635. 10.1016/j.intell.2010.09.001.
42. Causse M., Dehais F., Arexis M., Pastor J. Cognitive aging and flight performances in general aviation pilots. Aging, Neuropsychology, and Cognition 2011, 18(5):544-561.
43. Morrow D.G., Menard W.E., Stine-Morrow E.A., Teller T., Bryant D. The influence of expertise and task factors on age differences in pilot communication. Psychology and Aging 2001, 16(1):31-46.
44. Taylor J.L., O'Hara R., Mumenthaler M.S., Rosen A.C., Yesavage J.A. Cognitive ability, expertise, and age differences in following air-traffic control instructions. Psychology and Aging 2005, 20(1):117-133.
45. Brehmer B., Dörner D. Experiments with computer-simulated microworlds: escaping both the narrow straits of the laboratory and the deep blue sea of the field study. Computers in Human Behavior 1993, 9(2-3):171-184. 10.1016/0747-5632(93)90005-D.
46. Oron-Gilad T., Chen J., Hancock P.A. Remotely operated vehicles (ROVs) from the top-down and the bottom-up. Advances in Human Performance and Cognitive Engineering Research 2006, 7:37-47. 10.1016/S1479-3601(05)07003-7.
47. Smith E.E., Jonides J. Working memory: a view from neuroimaging. Cognitive Psychology 1997, 33(1):5-42.
48. Owen A.M., McMillan K.M., Laird A.R., Bullmore E. N-back working memory paradigm: a meta-analysis of normative functional neuroimaging studies. Human Brain Mapping 2005, 25(1):46-59.
49. Schreppel T., Egetemeir J., Schecklmann M., Plichta M.M., Pauli P., Ellgring H., et al. Activation of the prefrontal cortex in working memory and interference resolution processes assessed with near-infrared spectroscopy. Neuropsychobiology 2008, 57(4):188-193.
50. Menda J., Hing J.T., Ayaz H., Shewokis P.A., Izzetoglu K., Onaral B., et al. Optical brain imaging to enhance UAV operator training, evaluation, and interface development. Journal of Intelligent and Robotic Systems 2010, 61(1-4):423-443. 10.1007/s10846-010-9507-7.
51. Baddeley A. Working memory: looking back and looking forward. Nature Reviews. Neuroscience 2003, 4:829-839.
52. Qin S., Hermans E.J., van Marle H.J., Luo J., Fernández G. Acute psychological stress reduces working memory-related activity in the dorsolateral prefrontal cortex. Biological Psychiatry 2009, 66(1):25-32.
53. Simpson J.R., Drevets W.C., Snyder A.Z., Gusnard D.A., Raichle M.E. Emotion-induced changes in human medial prefrontal cortex: II. During anticipatory anxiety. Proceedings of the National Academy of Sciences 2001, 98(2):688-693.
54. Dolcos F., Kragel P., Wang L., McCarthy G. Role of the inferior frontal cortex in coping with distracting emotions. Neuroreport 2006, 17(15):1591-1594.
55. Causse M., Dehais F., Péran P., Sabatini U., Pastor J. The effects of emotion on pilot decision-making: a neuroergonomic approach to aviation safety. Transportation Research Part C: Emerging Technologies 2013 August, 33:272-281.
56. Izzetoglu M., Bunce S.C., Izzetoglu K., Onaral B., Pourrezaei K. Functional brain imaging using near-infrared technology. IEEE Engineering in Medicine and Biology Magazine 2007, 26(4):38.
57. Ayaz H., Izzetoglu M., Bunce S., Heiman-Patterson T., Onaral B. Detecting cognitive activity related hemodynamic signal for brain computer interface using functional near infrared spectroscopy. 3rd International IEEE/EMBS Conference on Neural Engineering, 2007. CNE'07 2007 May, 342-345.
58. Kramer A.F., Wickens C.D., Donchin E. Processing of stimulus properties: evidence for dual-task integrality. Journal of Experimental Psychology: Human Perception and Performance 1985, 11(4):393.
59. Wickens C., Kramer A., Vanasse L., Donchin E. Performance of concurrent tasks: a psychophysiological analysis of the reciprocity of information-processing resources. Science 1983.
60. Cohen J.D., Perlstein W.M., Braver T.S., Nystrom L.E., Noll D.C., Jonides J., et al. Temporal dynamics of brain activation during a working memory task. Nature 1997, 386:604-608.
61. Barch D.M., Braver T.S., Nystrom L.E., Forman S.D., Noll D.C., Cohen J.D. Dissociating working memory from task difficulty in human prefrontal cortex. Neuropsychologia 1997, 35(10):1373-1380.
62. Rowe J.B., Passingham R.E. Working memory for location and time: activity in prefrontal area 46 relates to selection rather than maintenance in memory. Neuroimage 2001, 14(1):77-86.
63. Causse M., Sénard J.M., Démonet J.F., Pastor J. Monitoring cognitive and emotional processes through pupil and cardiac response during dynamic versus logical task. Applied Psychophysiology and Biofeedback 2010, 35(2):115-123.
64. Granholm E., Asarnow R.F., Sarkin A.J., Dykes K.L. Pupillary responses index cognitive resource limitations. Psychophysiology 1996, 33(4):457-461.
65. Cabestrero R., Crespo A., Quirós P. Pupillary dilation as an index of task demands 1, 2. Perceptual and Motor Skills 2009, 109(3):664-678.
66. Gianaros P.J., Van der Veen F.M., Jennings J.R. Regional cerebral blood flow correlates with heart period and high-frequency heart period variability during working-memory tasks: implications for the cortical and subcortical regulation of cardiac autonomic activity. Psychophysiology 2004, 41(4):521-530.
67. Napadow V., Dhond R., Conti G., Makris N., Brown E.N., Barbieri R. Brain correlates of autonomic modulation: combining heart rate variability with fMRI. Neuroimage 2008, 42(1):169.
68. Backs R.W., Lenneman J.K., Sicard J.L. The use of autonomic components to improve cardiovascular assessment of mental workload in flight. The International Journal of Aviation Psychology 1999, 9(1):33-47.
69. Engineering psychophysiology: issues and applications. 2000, Lawrence Erlbaum, Mahwah, NJ. R.W. Backs, W. Boucsein (Eds.).
70. Wilson G.F., Russell C.A. Performance enhancement in an uninhabited air vehicle task using psychophysiologically determined adaptive aiding. Human Factors: The Journal of the Human Factors and Ergonomics Society 2007, 49(6):1005-1018.
71. St.John M., Kobus D.A., Morrison J.G., Schmorrow D. Overview of the DARPA augmented cognition technical integration experiment. International Journal of Human-Computer Interaction 2004, 17(2):131-149.
72. Hancock P.A., Chignell M.H. Mental workload dynamics in adaptive interface design. IEEE Transactions on Systems, Man and Cybernetics 1988, 18(4):647-658.