Affective man-machine interface: Unveiling human emotions through biosignals

Communications in Computer and Information Science

Volumn 52, Issue , 2010, Pages 21-47

  Van Den Broek, Egon L ^a   Lisý, Viliam ^b   Janssen, Joris H ^c,d   Westerink, Joyce H D M ^c   Schut, Marleen H ^c   Tuinenbreijer, Kees ^c  

^a UNIVERSITY OF TWENTE   (Netherlands)

^b CZECH TECHNICAL UNIVERSITY IN PRAGUE   (Czech Republic)

^c PHILIPS RESEARCH LABORATORIES   (Netherlands)

^d EINDHOVEN UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

BIOSIGNALS; ELECTRICAL PROFILES; ELECTRODERMAL ACTIVITY; EMOTION CLASSIFICATION; EMOTION DETECTION; FACIAL EMG SIGNALS; GENERIC FRAMEWORKS; HUMAN EMOTION; MAN-MACHINE INTERFACE; PARALLEL PROCESSING; PERSONAL PROFILE; STATE-OF-THE-ART REVIEWS;

BIOMEDICAL ENGINEERING; CONSUMER PRODUCTS;

INDUSTRIAL RESEARCH;

EID: 77950354983     PISSN: 18650929     EISSN: None     Source Type: Book Series    
DOI: 10.1007/978-3-642-11721-3_2     Document Type: Article

References (76)

1. Picard, R.W.: Affective Computing. MIT Press, Boston (1997)
2. van Tulder, M., Malmivaara, A., Koes, B.: Repetitive strain injury. The Lancet 369(9575), 1815-1822 (2007)
3. Schuler, J.L.H., O'Brien, W.H.: Cardiovascular recovery from stress and hypertension factors: A meta-analytic view. Psychophysiology 34(6), 649-659 (1997)
4. Frederickson, B.L.,Manusco, R.A., Branigan, C., Tugade, M.M.: The undoing effect of positive emotions. Motivation and Emotion 24(4), 237-257 (2000)
5. Ader, R., Cohen, N., Felten, D.: Psychoneuroimmunology: Interactions between the nervous system and the immune system. The Lancet 345(8942), 99-103 (1995)
6. Solomon, G.F., Amkraut, A.A., Kasper, P.: Immunity, emotions, and stress with special reference to the mechanisms of stress effects on the immune system. Psychotherapy and Psychosomatics 23(1-6), 209-217 (1974)
7. Fairclough, S.H.: Fundamentals of physiological computing. Interacting with Computers 21(1-2), 133-145 (2009)
8. Mauss, I.B., Robinson, M.D.:Measures of emotion: A review. Cognition and Emotion 23(2), 209-237 (2009)
9. Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: Analysis of affective physiological state. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(10), 1175-1191 (2001)
10. van den Broek, E.L., Janssen, J.H., Westerink, J.H.D.M., Healey, J.A.: Prerequisits for Affective Signal Processing (ASP). In: Encarnação, P., Veloso, A. (eds.) Biosignals 2009: Proceedings of the International Conference on Bio-Inspired Systems and Signal Processing, Porto - Portugal, pp. 426-433 (2009)
11. Critchley, H.D., Elliott, R., Mathias, C.J., Dolan, R.J.: Neural activity relating to generation and representation of galvanic skin conductance responses: A functional magnetic resonance imaging study. The Journal of Neuroscience 20(8), 3033-3040 (2000)
12. Zeng, Z., Pantic, M., Roisman, G.I., Huang, T.S.: A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(1), 39-58 (2009)
13. James,W.: Review: La pathologie des emotions by Ch. F́eŕe. The Philosophical Review 2(3), 333-336 (1893)
14. Marwitz, M., Stemmler, G.: On the status of individual response specificity. Psychophysiology 35(1), 1-15 (1998)
15. Gunes, H., Piccardi, M.: Automatic temporal segment detection and affect recognition from face and body display. IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics 39(1), 64-84 (2009)
16. Whitehill, J., Littlewort, G., Fasel, I., Bartlett, M., Movellan, J.: Towards practical smile detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(11), 2106-2111 (2009)
17. Daly, A.: Movement analysis: Piecing together the puzzle. TDR - The Drama Review: A Journal of Performance Studies 32(4), 40-52 (1988)
18. Ververidis, D., Kotropoulos, C.: Emotional speech recognition: Resources, features, and methods. Speech Communication 48(9), 1162-1181 (2006)
19. Van den Broek, E.L.: Emotional Prosody Measurement (EPM): A voice-based evaluation method for psychological therapy effectiveness. Studies in Health Technology and Informatics (Medical and Care Compunetics) 103, 118-125 (2004)
20. van den Broek, E.L., Schut,M.H.,Westerink, J.H.D.M., Tuinenbreijer, K.: Unobtrusive Sensing of Emotions (USE). Journal of Ambient Intelligence and Smart Environments 1(3), 287-299 (2009)
21. Gamboa, H., Silva, F., Silva, H., Falcão, R.: PLUX - Biosignals Acquisition and Processing (2010), http://www.plux.info (Last accessed January 30, 2010)
22. van den Broek, E.L., Westerink, J.H.D.M.: Considerations for emotion-aware consumer products. Applied Ergonomics 40(6), 1055-1064 (2009)
23. Berntson, G.G., Bigger, J.T., Eckberg, D.L., Grossman, P., Kaufmann, P.G., Malik, M., Nagaraja, H.N., Porges, S.W., Saul, J.P., Stone, P.H., van der Molen, M.W.: Heart rate variability: Origins, methods, and interpretive caveats. Psychophysiology 34(6), 623-648 (1997)
24. Boucsein, W.: Electrodermal activity. Plenum Press, New York (1992)
25. Grossman, P., Taylor, E.W.: Toward understanding respiratory sinus arrhythmia: Relations to cardiac vagal tone, evolution and biobehavioral functions. Biological Psychology 74(2), 263-285 (2007)
26. Fridlund, A.J., Cacioppo, J.T.: Guidelines for human electromyographic research. Psychophysiology 23(5), 567-589 (1986)
27. Reaz, M.B.I., Hussain, M.S., Mohd-Yasin, F.: Techniques of EMG signal analysis: detection, processing, classification and applications. Biological Procedures Online 8(1), 11-35 (2006)
28. Grandjean, D., Scherer, K.R.: Unpacking the cognitive architecture of emotion processes. Emotion 8(3), 341-351 (2008)
29. Lotte, F., Congedo, M., Ĺecuyer, A., Lamarche, F., Arnaldi, B.: A review of classification algorithms for EEG-based brain-computer interfaces. Journal of Neural Engineering 4(2), R1-R13 (2007)
30. Bimber, O.: Brain-Computer Interfaces. IEEE Computer 41(10) (2008); [special issue]
31. Minsky, M.: The Emotion Machine: Commonsense Thinking, Artificial Intelligence, and the Future of the Human Mind. Simon & Schuster, New York (2006)
32. Aarts, E.: Ambient intelligence: Vision of our future. IEEE Multimedia 11(1), 12-19 (2004)
33. Kim, J., Andŕe, E.: Emotion recognition based on physiological changes in music listening. IEEE Transactions on Pattern Analysis and Machine Intelligence 30(12), 2067-2083 (2008)
34. Liu, C., Rani, P., Sarkar, N.: Human-robot interaction using affective cues. In: Proceedings of the 15th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN 2006), Hatfield, UK, pp. 285-290. IEEE Computer Society, Los Alamitos (2006)
35. Rani, P., Sims, J., Brackin, R., Sarkar, N.: Online stress detection using psychophysiological signals for implicit human-robot cooperation. Robotica 20(6), 673-685 (2002)
36. Cacioppo, J.T., Tassinary, L.G., Berntson, G.: Handbook of Psychophysiology, 3rd edn. Cambridge University Press, New York (2007)
37. Sinha, R., Parsons, O.A.: Multivariate response patterning of fear. Cognition and Emotion 10(2), 173-198 (1996)
38. Scheirer, J., Fernandez, R., Klein, J., Picard, R.W.: Frustrating the user on purpose: A step toward building an affective computer. Interacting with Computers 14(2), 93-118 (2002)
39. Nasoz, F., Alvarez, K., Lisetti, C.L., Finkelstein, N.: Emotion recognition from physiological signals for presence technologies. International Journal of Cognition, Technology and Work 6(1), 4-14 (2003)
40. Takahashi, K.: Remarks on emotion recognition from bio-potential signals. In: Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, Palmerston North, New Zealand, October 5-8, vol.2, pp. 1655-1659 (2003)
41. Haag, A., Goronzy, S., Schaich, P.,Williams, J.: Emotion recognition using bio-sensors: First steps towards an automatic system. In: Andŕe, E., Dybkjær, L., Minker, W., Heisterkamp, P. (eds.) ADS 2004. LNCS (LNAI), vol.3068, pp. 36-48. Springer, Heidelberg (2004)
42. Kim, K.H., Bang, S.W., Kim, S.R.: Emotion recognition system using short-term monitoring of physiological signals. Medical & Biological Engineering & Computing 42(3), 419-427 (2004)
43. Lisetti, C.L., Nasoz, F.: Using noninvasive wearable computers to recognize human emotions from physiological signals. EURASIP Journal on Applied Signal Processing 2004(11), 1672-1687 (2004)
44. Wagner, J., Kim, J., Andŕe, E.: From physiological signals to emotions: Implementing and comparing selected methods for feature extraction and classification. In: Proceedings of the IEEE International Conference on Multimedia and Expo. (ICME), Amsterdam, The Netherlands, July 6-8, pp. 940-943 (2005)
45. Yoo, S.K., Lee, C.K., Park, J.Y., Kim, N.H., Lee, B.C., Jeong, K.S.: Neural network based emotion estimation using heart rate variability and skin resistance. In: Wang, L., Chen, K., S. Ong, Y. (eds.) ICNC 2005. LNCS, vol.3610, pp. 818-824. Springer, Heidelberg (2005)
46. Choi, A., Woo, W.: Physiological sensing and feature extraction for emotion recognition by exploiting acupuncture spots. In: Tao, J., Tan, T., Picard, R.W. (eds.) ACII 2005. LNCS, vol.3784, pp. 590-597. Springer, Heidelberg (2005)
47. Healey, J.A., Picard, R.W.: Detecting stress during real-world driving tasks using physiological sensors. IEEE Transactions on Intelligent Transportation Systems 6(2), 156-166 (2005)
48. Rani, P., Liu, C., Sarkar, N., Vanman, E.: An empirical study of machine learning techniques for affect recognition in human-robot interaction. Pattern Analysis & Applications 9(1), 58-69 (2006)
49. Zhai, J., Barreto, A.: Stress detection in computer users through noninvasive monitoring of physiological signals. Biomedical Science Instrumentation 42, 495-500 (2006)
50. Jones, C.M., Troen, T.: Biometric valence and arousal recognition. In: Thomas, B.H. (ed.) Proceedings of the Australasian Computer-Human Interaction Conference (OzCHI), Adelaide, Australia, pp. 191-194 (2007)
51. Leon, E., Clarke, G., Callaghan, V., Sepulveda, F.: A user-independent real-time emotion recognition system for software agents in domestic environments. Engineering Applications of Artificial Intelligence 20(3), 337-345 (2007)
52. Liu, C., Conn, K., Sarkar, N., Stone, W.: Physiology-based affect recognition for computerassisted intervention of children with Autism Spectrum Disorder. International Journal of Human-Computer Studies 66(9), 662-677 (2008)
53. Katsis, C.D., Katertsidis, N., Ganiatsas, G., Fotiadis, D.I.: Toward emotion recognition in car-racing drivers: A biosignal processing approach. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans 38(3), 502-512 (2008)
54. Yannakakis, G.N., Hallam, J.: Entertainment modeling through physiology in physical play. International Journal of Human-Computer Studies 66(10), 741-755 (2008)
55. Task Force: Heart rate variability: Standards of measurement, physiological interpretation, and clinical use. European Heart Journal 17(3), 354-381 (1996)
56. Ravenswaaij-Arts, C.M.A.V., Kollee, L.A.A., Hopman, J.C.W., Stoelinga, G.B.A., Geijn, H.P.: Heart rate variability. Annals of Internal Medicine 118(6), 436-447 (1993)
57. Butler, E.A.,Wilhelm, F.H., Gross, J.J.: Respiratory sinus arrhythmia, emotion, and emotion regulation during social interaction. Psychophysiology 43(6), 612-622 (2006)
58. van den Broek, E.L., Schut, M.H., Westerink, J.H.D.M., van Herk, J., Tuinenbreijer, K.: Computing emotion awareness through facial electromyography. In: Huang, T.S., Sebe, N., Lew, M., Pavlovíc, V., Kölsch, M., Galata, A., Kisa?canin, B. (eds.) ECCV 2006 Workshop on HCI. LNCS, vol.3979, pp. 52-63. Springer, Heidelberg (2006)
59. Westerink, J.H.D.M., van den Broek, E.L., Schut, M.H., van Herk, J., Tuinenbreijer, K.: 14. In: Computing emotion awareness through galvanic skin response and facial electromyography. Philips Research Book Series, vol.8, pp. 137-150. Springer, Dordrecht (2008)
60. Cacioppo, J., Tassinary, L.: Inferring psychological significance from physiological signals. American Psychologist 45(1), 16-28 (1990)
61. Mitchell, T.M.: Machine Learning. The McGraw-Hill Companies, Inc., Columbus (1997)
62. Bishop, C.M.: Pattern Recognition and Machine Learning. Information Science and Statistics. Springer Science+Business Media, LLC, New York (2006)
63. Schölkopf, B., Smola, A.J.: Learning with kernels: Support Vector Machines, Regularization, Optimization, and Beyond. In: Adaptive Computation and Machine Learning. The MIT Press, Cambridge (2002)
64. Rencher, A.C.: Methods of Multivariate Analysis, 2nd edn. Wiley Series in Probability and Statistics. John Wiley & Sons, Inc., New York (2002)
65. Rottenberg, J., Ray, R.R., Gross, J.J.: 1. In: Emotion elicitation using films, pp. 9-28. Oxford University Press, New York (2007)
66. Kreibig, S.D., Wilhelm, F.H., Roth, W.T., Gross, J.J.: Cardiovascular, electrodermal, and respiratory response patterns to fear- and sadness-inducing films. Psychophysiology 44(5), 787-806 (2007)
67. Kring, A.M., Gordon, A.H.: Sex differences in emotion: Expression, experience, and physiology. Journal of Personality and Social Psychology 74(3), 686-703 (1998)
68. Carrera, P., Oceja, L.: Drawing mixed emotions: Sequential or simultaneous experiences? Cognition & Emotion 21(2), 422-441 (2007)
69. Russell, J.A.: A circumplex model of affect. Journal of Personality and Social Psychology 39(6), 1161-1178 (1980)
70. Cover, T.M., van Campenhout, J.M.: On the possible orderings in the measurement selection problem. IEEE Transactions on Systems, Man, and Cybernetics SMC-7(9), 657-661 (1977)
71. Lawrence, S., Giles, C.L., Tsoi, A.: What size neural network gives optimal generalization? Convergence properties of backpropagation. Technical Report UMIACS-TR-96-22 and CSTR- 3617 (April 1996)
72. Barrett, L.F.: Valence as a basic building block of emotional life. Journal of Research in Personality 40, 35-55 (2006)
73. Russel, J.A., Barrett, L.F.: Core affect, prototypical emotional episodes, and other things called emotion: Dissecting the elephant. Journal of Personality and Social Psychology 26(5), 805-819 (1999)
74. Gendolla, G.H.E.: On the impact of mood on behavior: An integrative theory and a review. Review of General Psychology 4(4), 378-408 (2000)
75. Cooper, C.L., Pervin, L.A.: Personality: Critical concepts in psychology, 1st edn. Critical concepts in psychology. Routledge, New York (1998)
76. Lukowicz, P.:Wearable computing and artificial intelligence for healthcare applications. Artificial Intelligence in Medicine 42(2), 95-98 (2008)