Workload estimation in physical human-robot interaction using physiological measurements

Interacting with Computers

Volumn 27, Issue 6, 2015, Pages 616-629

  Novak, Domen ^a   Beyeler, Benjamin ^a   Omlin, Ximena ^a   Riener, Robert ^a  

^a ETH ZURICH   (Switzerland)

Author keywords

assistive technologies; human robot interaction; novel interaction paradigms; tangible interaction; virtual reality

Indexed keywords

ELECTROENCEPHALOGRAPHY; ELECTROPHYSIOLOGY; EYE TRACKING; MAN MACHINE SYSTEMS; NASA; PHYSIOLOGICAL MODELS; VIRTUAL REALITY;

ASSISTIVE TECHNOLOGY; AUTONOMIC NERVOUS SYSTEM; INFORMATION SOURCES; INTERACTION PARADIGM; NON-LINEAR ESTIMATION; PHYSICAL HUMAN-ROBOT INTERACTIONS; PHYSIOLOGICAL MEASUREMENT; TANGIBLE INTERACTION;

HUMAN ROBOT INTERACTION;

EID: 84983349064     PISSN: 09535438     EISSN: None     Source Type: Journal    
DOI: 10.1093/iwc/iwu021     Document Type: Article

References (42)

1. Badesa, F., Morales, R., Garcia-Aracil, N., Sabater, J., Perez-Vidal, C. and Fernandez, E. (2012) Multimodal interfaces to improve therapeutic outcomes in robot-assisted rehabilitation. IEEE Trans. Syst. Man Cybern. - Part C : Appl. Rev., 42, 1152-1158.
2. Berka, C., Levendowski, D.J., Lumicao, M.N., Yau, A., Davis, G., Zivkovic, V.T., Olmstead, R.E., Tremoulet, P.D. and Craven, P.L. (2007) EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks.Aviat. Space Environ. Med., 78, B231-B2344.
3. Boiten, F. (1993) Component analysis of task-related respiratory patterns. Int. J. Psychophysiol., 15, 91-104.
4. Breiman, L. (2001) Random forests. Mach. Learn., 45, 5-32.
5. Brookings, J.B., Wilson, G.F. and Swain, C.R. (1996) Psychophysiological responses to changes in workload during simulated air traffic control. Biol. Psychol., 42, 361-377.
6. Byrne, E.A. and Parasuraman, R. (1996) Psychophysiology and adaptive automation. Biol. Psychol., 42, 249-268.
7. Cameirão, M.S., Badia, S.B.I., Oller, E.D. and Verschure, P.F.M.J. (2010) Neurorehabilitation using the virtual reality based Rehabilitation Gaming System : Methodology, design, psychometrics, usability and validation. J. NeuroEng. Rehabil., 7, 1-14.
8. Chanel, G., Rebetez, C., Bétrancourt, M. and Pun, T. (2011) Emotion assessment from physiological signals for adaptation of game difficulty. IEEE Trans. Syst. Man Cybern. - Part A : Syst. Hum., 41, 1052-1063.
9. DeRivecourt, M.,Kuperus, M.N., Post,W.J. and Mulder, L.J.M. (2008) Cardiovascular and eye activity measures as indices for momentary changes in mental effort during simulated flight. Ergonomics, 51, 1295-1319.
10. Di Stasi, L.L., Catena, A., Cañas, J.J., Macknik, S.L. and Martinez- Conde, S. (2013) Saccadic velocity as an arousal index in naturalistic tasks. Neurosci. Biobehav. Rev., 37, 968-975.
11. Fairclough, S.H. (2009) Fundamentals of physiological computing. Interact. Comput., 21, 133-145.
12. Fairclough, S.H. and Venables, L. (2006) Prediction of subjective states from psychophysiology : A multivariate approach. Biol. Psychol., 71, 100-110.
13. George, L., Marchal, M., Glondu, L. and Lecuyer, A. (2012) Combining brain-computer interfaces and haptics? : Detecting mental workload to adapt haptic assistance. In : Proceedings of EuroHaptics 2012, pp. 124-135. Tampere Finland.
14. Gonzalez, J., Soma, H., Sekine, M. and Yu, W. (2012) Psychophysiological assessment of a prosthetic hand sensory feedback system based on an auditory display : A preliminary study. J. NeuroEng. Rehabil., 9, 1-14.
15. Guerrero, C.R., Fraile Marinero, J.C., Turiel, J.P. and Muñoz,V. (2013) Using 'human state aware'robots to enhance physical human-robot interaction in a cooperative scenario. Comput. Methods Programs Bomed., 112, 250-259.
16. Hart, S. (2006) NASA-Task Load Index (NASA-TLX); 20 years later. In Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting, San Francisco, USA, pp. 904-908.
17. Hart, S. and Staveland, L.E. (1988) Development of NASA-TLX (Task Load Index) : Results of empirical and theoretical research. In : Hancock, P.A. & Meshkati, N. (eds), Human MentalWorkload. Amsterdam : North Holland Press.
18. He, P., Wilson, G. & Russell, C. (2004) Removal of ocular artifacts from electro-encephalogram by adaptive filtering. Med. Biol. Eng. Comput., 42, 407-12.
19. Hussain, M.S., Calvo R.A. and Chen, F. (2014) Automatic cognitive load detection from face, physiology, task performance and fusion during affective interference. Interact. Comput., 26, 256-268.
20. Klimesch, W. (1999) EEG alpha and theta oscillations reflect cognitive and memory performance : A review and analysis. Brain Res. Rev., 29, 169-195.
21. Koenig, A., Novak, D., Omlin, X., Pulfer, M., Perreault, E., Zimmerli, L., Mihelj, M. and Riener, R. (2011) Real-time closed-loop control of cognitive load in neurological patients during robotassisted gait training. IEEE Trans. Neural Syst. Rehabil. Eng., 19, 453-464.
22. Kreibig, S.D. (2010) Autonomic nervous system activity in emotion : A review. Biol. Psychol., 84, 394-421.
23. Liu, C., Conn, K., Sarkar, N. and Stone, W. (2008) Online affect detection and robot behavior adaptation for intervention of children with autism. IEEE Trans. Robotics, 24, 883-896.
24. Liu, C.,Agrawal, P., Sarkar, N. and Chen, S. (2009) Dynamic difficulty adjustment in computer games through real-time anxiety-based affective feedback. Int. J. Hum.-Comput. Interact., 25, 506-529.
25. Mandryk, R.L. and Atkins, M.S. (2007) A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. Int. J. Hum.-Comput. Stud., 65, 329-347.
26. Nef, T., Guidali, M. and Riener, R. (2009). ARMin III-arm therapy exoskeleton with an ergonomic shoulder actuation. Appl. Bionics Biomech., 6, 127-142.
27. Novak, D., Mihelj, M., Ziherl, J., Olenšek, A. and Munih, M. (2011a). Psychophysiological measurements in a biocooperative feedback loop for upper extremity rehabilitation. IEEE Trans. Neural Syst. Rehabil. Eng., 19, 400-410.
28. Novak, D., Mihelj, M. and Munih, M. (2011b) Psychophysiological responses to different levels of cognitive and physical workload in haptic interaction. Robotica, 29, 367-374.
29. Novak, D., Mihelj, M. and Munih, M. (2012) A survey of methods for data fusion and system adaptation using autonomic nervous system responses in physiological computing. Interact. Comput., 24, 154-172.
30. Pedrotti, M., Mirzaei, M. A., Tedesco, A., Chardonnet, J.-R., Mérienne, F., Benedetto, S. and Baccino,T. (2014)Automatic stress classification with pupil diameter analysis. Int. J. Hum.-Comput. Interact., 30, 220-236.
31. Picard, R.W., Vyzas, E. and Healey, J. (2001) Toward machine emotional intelligence : Analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell., 23, 1175-1191.
32. Proakis, J.G. and Manolakis, D.K. (2007) Digital Signal Processing (4th edn). Prentice Hall, Upper Saddle River, NJ.
33. Rani, P., Sims, J., Brackin, R. and Sarkar, N. (2002). Online stress detection using psychophysiological signals for implicit human-robot cooperation. Robotica, 20, 673-685.
34. Rani, P., Sarkar, N., Smith, C.A. and Kirby, L.D. (2004) Anxiety detecting robotic system-towards implicit human-robot collaboration. Robotica, 22, 85-95.
35. Shirzad, N. and Van der Loos, H.F.M. (2013) Adaptation of task difficulty in rehabilitation exercises based on the user's motor performance and physiological responses. In Proceedings of the 2013 IEEE International Conference on Rehabilitation Robotics, Seattle, USA.
36. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. (1996) Heart rate variability : Standards of measurement, physiological interpretation, and clinical use. Eur. Heart J., 17, 354-381.
37. Ting, C.-H., Mahfouf, M., Nassef, A., Linkens, D.A., Panoutsos, G., Nickel, P., Roberts, A.C. and Hockey, G. (2010) Real-time adaptive automation system based on identification of operator functional state in simulated process control operations. IEEE Trans. Syst. Man Cybernet. - Part A : Syst. Hum., 40, 251-262.
38. Van Orden, K.F., Limbert, W., Makeig, S. and Jung, T.-P. (2001) Eye activity correlates of workload during a visuospatial memory task. Hum. Factors, 43, 111-121.
39. Veltman, J.A. and Gaillard, A.W.K. (1998) Physiological workload reactions to increasing levels of task difficulty. Ergonomics, 41, 656-659.
40. Vidulich, M.A. and Tsang, P.S. (2012) Mental workload and situation awareness. In Salvendy, G. (ed.), Handbook of Human Factors and Ergonomics (4th edn). JohnWiley & Sons, Inc., Hoboken, NJ.
41. Whitham, E.M. et al. (2007) Scalp electrical recording during paralysis : Quantitative evidence that EEG frequencies above 20 Hz are contaminated by EMG. Clin. Neurophysiol., 118, 1877-1888.
42. Wilson, G.F. and Russell, C.a. (2007). Performance enhancement in an uninhabited air vehicle task using psychophysiologically determined adaptive aiding. Hum. Factors, 49, 1005-1018.