Using noninvasive wearable computers to recognize human emotions from physiological signals

Eurasip Journal on Applied Signal Processing

Volumn 2004, Issue 11, 2004, Pages 1672-1687

  Lisetti, Christine Lætitia ^a   Nasoz, Fatma ^b  

^a EURECOM   (France)

^b University of Central Florida   (United States)

Author keywords

Emotion recognition; Multimodal affective user interfaces; Multimodal human computer interaction

Indexed keywords

COGNITIVE SYSTEMS; DECISION MAKING; HUMAN COMPUTER INTERACTION; NEUROLOGY; PSYCHOPHYSIOLOGY; REAL TIME SYSTEMS; USER INTERFACES;

AUTONOMIC NERVOUS SYSTEMS (ANS); EMOTION RECOGNITION; MULTIMODAL AFFECTIVE USER INTERFACES; MULTIMODAL HUMAN-COMPUTER INTERACTIONS;

WEARABLE COMPUTERS;

EID: 10244240638     PISSN: 11108657     EISSN: None     Source Type: Journal    
DOI: 10.1155/S1110865704406192     Document Type: Review

References (54)

1. G. Bower, "Mood and memory," American Psychologist, vol. 36, no. 2, pp. 129-148, 1981.
2. S. Kalyuga, P. Chandler, and J. Sweller, "Levels of expertise and user-adapted formats of instructional presentations: a cognitive load approach," in Proceedings of the 6th International Conference on User Modeling (UM '97), A. Jameson, C. Paris, and C. Tasso, Eds., pp. 261-272, New York, NY, USA, 1997.
3. D. Derryberry and D. Tucker, "Neural mechanisms of emotion," Journal of Consulting and Clinical Psychology, vol. 60, no. 3, pp. 329-337, 1992.
4. R. Zajonc, "On the primacy of affect," American Psychologist, vol. 39, no. 2, pp. 117-123, 1984.
5. G. Linden, S. Hanks, and N. Lesh, "Interactive assessment of user preference models: the automated travel assistant," in Proceedings of the 6th International Conference on User Modeling (UM '97), A. Jameson, C. Paris, and C. Tasso, Eds., pp. 67-78, New York, NY, USA, 1997.
6. A. Damasio, Descartes' Error, Avon Books, New York, NY, USA, 1994.
7. J. Ledoux, "Emotion and the amygdala," in The Amygdala: Neurobiological Aspects of Emotion, Memory, and Mental Dysfunction, J. P. Aggleton, Ed., pp. 339-351, Wiley-Liss, New York, NY, USA, 1992.
8. P. Paranagama, F. Burstein, and D. Arnott, "Modeling the personality of decision makers for active decision support," in Proceedings of the 6th International Conference on User Modeling (UM '97), pp. 79-81, Sardinia, Italy, June 1997.
9. S. Bull, "See yourself write: a simple student model to make students think," in Proceedings of the 6th International Conference User Modeling (UM '97), pp. 315-326, New York, NY, USA, 1997.
10. N. H. Frijda, The Emotions, Cambridge University Press, New York, NY, USA, 1986.
11. R. Birdwhistle, Kinesics and Context: Essays on Body Motion and Communication, University of Pennsylvania Press, Philadelphia, Pa, USA, 1970.
12. P. Ekman and W. V. Friesen, Unmasking the Face: A Guide to Recognizing Emotions from Facial Expressions, Prentice-Hall, Englewood Cliffs, NJ, USA, 1975.
13. N. Chovil, "Discourse-oriented facial displays in conversation," Research on Language and Social Interaction, vol. 25, pp. 163-194, 1991.
14. D. Goleman, Emotional Intelligence, Bantam Books, New York, NY, USA, 1995.
15. H. Leventhal and K. Sherer, "The relationship of emotion to cognition: a functional approach to a semantic controversy," Cognition and Emotion, vol. 1, no. 1, pp. 3-28, 1987.
16. C. L. Lisetti and F. Nasoz, "MAUI: a multimodal affective user interface," in Proceedings of the ACM Multimedia International Conference, Juan les Pins, France, December 2002.
17. N. Bianchi-Berthouze and C. L. Lisetti, "Modeling multimodal expression of user's affective subjective experience," User Modeling and User-Adapted Interaction, vol 12, no. 1, pp. 49-84, 2002.
18. G. E. Schwartz, P. L. Fair, P. S. Greenberg, Friedman M. J., and G. L. Klerman, "Facial EMG in the assessment of emotion," Psychophysiology, vol. 11, no. 2, pp. 237, 1974.
19. P. Ekman, R. W. Levenson, and W. V. Friesen, "Autonomic nervous system activity distinguishes among emotions," Science, vol. 221, no. 4616, pp. 1208-1210, 1983.
20. J. T. Lanzetta and S. P. Orr, "Excitatory strength of expressive faces: effects of happy and fear expressions and context on the extinction of a conditioned fear response," Journal of Personality and Social Psychology, vol. 50, no. 1, pp. 190-194, 1986.
21. S. R. Vrana, B. N. Cuthbert, and P. J. Lang, "Fear imagery and text processing," Psychophysiology, vol. 23, no. 3, pp, 247-253, 1986.
22. R. A. Wright, R. J. Contrada, and M. J. Patane, "Task difficulty, cardiovascular response and the magnitude of goal valence," Journal of Personality and Social Psychology, vol. 51, no. 4, pp. 837-843, 1986.
23. C. A. Smith, "Dimensions of appraisal and physiological response in emotion," Journal of Personality and Social Psychology, vol. 56, no. 3, pp. 339-353, 1989.
24. G. Stemmler, "The autonomic differentiation of emotions revisited: convergent and discriminant validation," Psychophysiology, vol. 26, no. 6, pp. 617-632, 1989.
25. R. W. Levenson, P. Ekman, and W. V. Friesen, "Voluntary facial action generates emotion-specific autonomic nervous system activity," Psychophysiology, vol. 27, pp. 363-384, 1990.
26. R. W. Levenson, P. Ekman, K. Heider, and W. V. Friesen, "Emotion and autonomic nervous system activity in the Minangkabau of west Sumatra," Journal of Personality and Social Psychology, vol. 62, no. 6, pp. 972-988, 1992.
27. S. Vrana, "The psychophysiology of disgust: differentiating negative emotional contexts with facial EMG," Psychophysiology, vol. 30, no. 3, pp. 279-286, 1993.
28. R. A. Wright and J. C. Dill, "Blood pressure responses and incentive appraisals as a function of perceived ability and objective task demand," Psychophysiology, vol. 30, no. 2, pp. 152-160, 1993.
29. A. Pecchinenda and C. Smith, "Affective significance of skin conductance activity during difficult problem-solving task," Cognition and Emotion, vol. 10, no. 5, pp. 481-503, 1996.
30. R. Sinha and O. Parsons, "Multivariate response patterning of fear and anger," Cognition and Emotion, vol. 10, no. 2, pp. 173-198, 1996.
31. C. Collet, E. Vernet-Maury, G. Delhomme, and A. Dittmar, "Autonomic nervous system response patterns specificity to basic emotions," J. Auton. Nerv. Syst, vol. 62, no. 1-2, pp. 45-57, 1997.
32. J. J. Gross and R. W. Levenson, "Hiding feelings: the acute effects of inhibiting negative and positive emotion," Journal of Abnormal Psychology, vol. 106, no. 1, pp. 95-103, 1997.
33. W. Ark, D. C. Dryer, and D. J. Lu, "The emotion mouse," in Human-Computer Interaction: Ergonomics and User Interfaces, H. J. Bullinger and J. Ziegler, Eds., pp. 818-823, Lawrence Erlbaum, London, UK, 1999.
34. D. Palomba, M. Sarlo, A. Angrilli, and A. Mini, "Cardiac responses associated with affective processing of unpleasant film stimuli," International Journal of Psychophysiology, vol. 36, no. 1, pp. 45-57, 2000.
35. M. P. Tarvainen, A. S. Koistinen, M. Valkonen-Korhonen, J. Partanen, and P. A. Karjalainen, "Analysis of galvanic skin responses with principal components and clustering techniques," IEEE Transactions on Biomedical Engineering, vol. 48, no. 10, pp. 1071-1079, 2001.
36. M. E. Crosby, B. Auernheimer, C. Aschwanden, and C. Ikehara, "Physiological data feedback for application in distance education," in Proceedings of Workshop on Perceptive User Interfaces (PUI '01), Orlando, Fl, USA, November 2001.
37. R. W. Picard, E. Vyzas, and J. Healey, "Toward machine emotional intelligence: analysis of affective physiological state," IEEE Transactions Pattern Analysis and Machine Intelligence, vol. 23, no. 10, pp. 1175-1191, 2001.
38. J. Scheirer, R. Fernandez, J. Klein, and R. W. Picard, "Frustrating the user on purpose: a step toward building an affective computer," Interacting With Computers, vol. 14, no. 2, pp. 93-118, 2002.
39. J. Healey, Wearable and automotive systems for affect recognition from physiology, Ph.D. thesis, Massachusetts Institute of Technology, Mass, USA, May 2000.
40. D. M. Clynes, Sentics: The Touch of Emotions, Anchor Press, New York, NY, USA, 1977.
41. J. J. Gross and R. W. Levenson, "Emotion elicitation using films," Cognition and Emotion, vol. 9, no. 1, pp. 87-108, 1995.
42. F. Nasoz, K. Alvarez, C. L. Lisetti, and N. Finkelstein, "Emotion recognition from physiological signals for presence technologies," Cognition, Technology, and Work, vol. 6, no. 1, pp. 4-14, 2004.
43. T. M. Mitchell, Machine Learning, McGraw-Hill, New York, NY, USA, 1997.
44. A. A. Nicol and P. M. Pexman, Presenting Your Findings: A Practical Guide for Creating Tables, American Psychological Association, Wash, DC, USA, 1999.
45. M. T. Hagan and M. B. Menhaj, "Training feedforward networks with the Marquardt algorithm," IEEE Transactions on Neural Networks, vol. 5, no. 6, pp. 989-993, 1994.
46. L. Feldman Barrett, J. J. Gross, T. Conner Christensen, and M. Benvenuto, "Knowing what you're feeling and knowing what to do about it: mapping the relation between emotion differentiation and emotion regulation," Cognition and Emotion, vol. 15, no. 6, pp. 713-724, 2001.
47. L. James, Road Rage and Aggressive Driving: Steering Clear of Highway Warfare, Prometheus Books, Amherst, NY, USA, 2000.
48. J. Larson and C. Rodriguez, Road Rage to Road-Wise, Tom Doherty Associates, New York, NY, USA, 1999.
49. V. E. Lewis and R. N. Williams, "Mood-congruent vs. mood-state-dependent learning: implications for a view of emotion," The Journal of Social Behavior and Personality, vol. 4, no. 2, pp. 157-171, 1989, Special Issue on Mood and Memory: Theory, Research, and Applications.
50. J. J. Martocchio, "Effects of conceptions of ability on anxiety, self-efficacy, and learning in training," Journal of Applied Psychology, vol. 79, no. 6, pp. 819-825, 1994.
51. P. Warr and D. Bunce, "Trainee characteristics and the outcomes of open learning," Personnel Psychology, vol. 48, no. 2, pp. 347-375, 1995.
52. C. L. Lisetti, F. Nasoz, C. LeRouge, O. Ozyer, and K. Alvarez, "Developing multimodal intelligent affective interfaces for tele-home health care," International Journal of Human-Computer Studies, vol. 59, no. 1-2, pp. 245-255, 2003, Special Issue on Applications of Affective Computing in Human-Computer Interaction.
53. C. L. Lisetti and D. J. Schiano, "Facial expression recognition: where human-computer interaction, artificial intelligence and cognitive science intersect," Pragmatics and Cognition, vol. 8, no. 1, pp. 185-235, 2000.
54. C. L. Lisetti and D. Rumelhart, "Facial expression recognition using a neural network," in Proceedings of the 11th International Florida Artificial Intelligence Research Society Conference (FLAIRS '98), pp. 328-332, AAAI Press, Menlo Park, Calif, USA, 1998.