Real-time classification of evoked emotions using facial feature tracking and physiological responses

International Journal of Human Computer Studies

Volumn 66, Issue 5, 2008, Pages 303-317

  Bailenson, Jeremy N ^a   Pontikakis, Emmanuel D ^b   Mauss, Iris B ^c   Gross, James J ^a   Jabon, Maria E ^b   Hutcherson, Cendri A C ^a   Nass, Clifford ^a   John, Oliver ^d  

^a STANFORD UNIVERSITY   (United States)

^b Stanford University   (United States)

^c UNIVERSITY OF DENVER   (United States)

^d UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Affective computing; Computer vision; Emotion; Facial tracking

Indexed keywords

FACE RECOGNITION; FEATURE EXTRACTION; LEARNING ALGORITHMS; LEARNING SYSTEMS; MATHEMATICAL MODELS; OBJECT RECOGNITION;

AFFECTIVE COMPUTING; EMOTION; FACIAL TRACKING; PERSON SPECIFIC MODELS;

REAL TIME SYSTEMS;

EID: 41449104132     PISSN: 10715819     EISSN: 10959300     Source Type: Journal    
DOI: 10.1016/j.ijhcs.2007.10.011     Document Type: Article

References (42)

1. Bailenson, J., Yee, N., Merget, D., Schroeder, R., 2006. The Effect of Behavioral Realism and Form Realism of Real-Time Avatar Faces on Verbal Disclosure, Non-/verbal Disclosure, Emotion Recognition, and Copresence in Dyadic Interaction, Presence, vol. 15, No. 4, August 2006, pp. 359-372.
2. Bartlett, M., Littlewort, G., Frank, M., Lainscsek, C., Fasel, I., Movellan, J., 2005. Recognizing Facial Expression: Machine Learning and Application to Spontaneous Behavior, IEEE CVPR 2005.
3. Bonanno A.G., and Keltner D. The coherence of emotion systems: comparing "on-line" measures of appraisal and facial expressions, and self-report. Cognition and Emotion 18 (2004) 431-444
4. Bradley M. Emotion and motivation. In: Cacioppo J., Tassinary L., and Brenston G. (Eds). Handbook of Psychophysiology (2000), Cambridge University Press, New York
5. Curhan J.R., and Pentland A. Thin slices of negotiation: predicting outcomes from conversational dynamics within the first 5 mins. Journal of Applied Psychology 92 (2007) 802-811
6. Deng, Z., Bailenson, J.N., Lewis, J.P., Neumann, U., 2006. Perceiving visual emotions with speech. In: Proceedings of the 6th International Conference on Intelligent Virtual Agents.
7. Ehrlich S.M., Schiano D.J., and Sheridan K. Communicating facial affect: it's not the realism, it's the motion. CHI '00 (2000), ACM Press, New York, NY pp. 251-252
8. Ekman P. Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage (2001), Norton, New York
9. Ekman P., and Friesan W. The Facial Action Coding System: A Technique for the Measurement of Facial Movement (1978), Consulting Psychologists Press
10. Ekman P., Davidson R.J., and Friesen W.V. The Duchenne smile: emotional expression and brain physiology II. Journal of Personality and Social Psychology 58 (1990) 342-353
11. el Kaliouby, R., Robinson, P., 2005. Real-time inference of complex mental states from facial expressions and head gestures. In: Real-time Vision for HCI. Springer, pp. 181-200. ISBN 0-387-27697-1.
12. el Kaliouby, R., Robinson, P., Keates, S., 2003. Temporal context and the recognition of emotion from facial expression. In: Proceedings of HCI International, vol. 2, Crete, June 2003, pp. 631-635. ISBN 0-8058-4931-9.
13. Essa I., and Pentland A. Coding, analysis, interpretation, and recognition of facial expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence 19 7 (1997) 757-763
14. Freund, Y., Schapire, R., 1996. Experiments with a New boosting algorithm. In: Machine Learning-Proceedings of the Thirteenth International Conference. Morgan Kaufmann, San Francisco, pp. 148-156.
15. Friedman J., Hastie T., and Tibshirani R. Additive logistic regression: a statistical view of boosting. Annals of Statistics 28 (2000) 337-407
16. Gross J.J., and Levenson R.W. Emotion elicitation using films. Cognition and Emotion 9 (1995) 87-108
17. Gross J.J., and Thompson R. Emotion regulation: conceptual foundations. In: Gross J.J. (Ed). Handbook of Emotion Regulation (2007), Guilford Press, New York 3-24
18. Hall J.A. Nonverbal Sex Differences: Communication Accuracy and Expressive Style (1984), The Johns Hopkins University Press, Baltimore, MD
19. Kimura, S., Yachida, M., 1997. Facial expression recognition and its degree estimation. In: Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 295-300.
20. Kring A.M. Gender and anger. In: Fischer A.H. (Ed). Gender and Emotion: Social Psychological Perspectives (2000), Cambridge University Press, New York 211-231
21. Li, S.Z., Jain, A.K., 2005. Handbook of Face Recognition. Springer, New York. ISBN# 0-387-40595 (Chapter 11).
22. Lien, J.-J., Kanade, T., Cohn, J., Li, C., 1998. Subtly different facial expression recognition and expression intensity estimation. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 853-859.
23. Lyons, M.J., 2004. Facial gesture interfaces for expression and communication. In: IEEE International Conference on Systems, Man, and Cybernetics, vol. 1, pp. 598-603.
24. Lyons, M.J., Bartneck, C., 2006. HCI and the face. In: CHI '06 Extended Abstracts on Human Factors in Computing Systems. ACM, pp. 1671-1674.
25. Mauss I.B., Levenson R.W., McCarter L., Wilhelm F.H., and Gross J.J. The tie that binds?. Coherence among Emotion Experience, Behavior, and Physiology 5 2 (2005) 175-190
26. Mauss I.B., Evers C., Wilhelm F.H., and Gross J.J. How to bite your tongue without blowing your top: Implicit evaluation of emotion regulation predicts affective responding to anger provocation. Personality and Social Psychology Bulletin 32 (2006) 389-602
27. Michel, P., el Kaliouby, R., 2003. Real time facial expression recognition in video using support vector machines. In: Proceedings of the 5th International Conference on Multimodal Interfaces, pp. 258-264. ISBN: 1-58113-621-8
28. Nass, C., Brave, S., 2005. Wired for Speech: How Voice Activates and Advances the Human-Computer Relationship. The MIT Press. ISBN: 0262140926.
29. Pantic, M., Patras, I., 2006. Dynamics of Facial Expression: Recognition of Facial Actions and Their Temporal Segments from Face Profile Image Sequences. IEEE Transactions on Systems, Man and Cybernetics-Part B, vol. 36, no. 2, pp. 433-449, April 2006.
30. Picard R.W. Affective Computing (1997), MIT Press, 1997, Cambridge, MA
31. Picard, R.W., Daily, S.B., 2005. Evaluating affective interactions: alternatives to asking what users feel. In: CHI Workshop on Evaluating Affective Interfaces: Innovative Approaches, April 2005, Portland, OR.
32. Philippot P. Inducing and assessing differentiated emotion-feeling states in the laboratory. Cognition and Emotion 7 2 (1993) 171-193
33. Rosenberg E.L., and Ekman P. Emotion: methods of study. In: Kazdin A.E. (Ed). Encyclopedia of Psychology vol. 3 (2000), American Psychological Association 171-175
34. Sayette M., Cohn J., Wertz J., Perrott M., and Parrott D. A psychometric evaluation of the facial action coding system for assessing spontaneous expression. Journal of Nonverbal Behavior 25 (2001) 167-186
35. Schiano, D.J., Ehrlich, S.M., Sheridan, K., 2004. Categorical Imperative NOT: Facial Affect is Perceived Continuously. CHI, pp. 49-56.
36. Sebe, N., Lew, M.S., Cohen, I., Garg, A., Huang, T.S., 2002. Emotion recognition using a cauchy naive bayes classifier. In: Proceedings of ICPR, vol. 1, pp. 17-20.
37. Tian, Y.-L., Kanade, T., Cohn, J., 2000. Eye-state action unit detection by Gabor wavelets. In: Proceedings of International Conference on Multi-modal Interfaces (ICMI), pp. 143-150.
38. Timmers M., Fischer A.H., and Manstead A.S.R. Gender differences in motives for regulating emotions. Personality and Social Psychology Bulletin 24 (1998) 974-985
39. Watson D., and Tellegen A. Toward a consensual structure of mood. Psychological Bulletin 98 2 (1985) 219-235
40. Witten H.I., and Fank E. Data Mining: Practical Machine Learning Tools and Techniques. second ed. (2005), Morgan Kaufmann, San Francisco
41. Zhang, Z., Lyons, M., Schuster, M., Akamatsu, S., 1998. Comparison between geometry-based and gabor-wavelets-based facial expression recognition using multi-layer perceptron. In: 3rd International Conference on Face and Gesture Recognition, p. 454.
42. Zlochower, A., Cohn, J., Lien, J., Kanade, T., 1998. A computer vision based method of facial expression analysis in parent-infant interaction. In: International Conference on Infant Studies.