Modeling naturalistic affective states via facial, vocal, and bodily expressions recognition

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 4451 LNAI, Issue , 2007, Pages 91-112

  Karpouzis, Kostas ^a   Caridakis, George ^a   Kessous, Loic ^b   Amir, Noam ^b   Raouzaiou, Amaryllis ^a   Malatesta, Lori ^a   Kollias, Stefanos ^a  

^a NATIONAL TECHNICAL UNIVERSITY OF ATHENS   (Greece)

^b AFEKA TEL AVIV ACADEMIC COLLEGE OF ENGINEERING   (Israel)

Author keywords

Affective interaction; Facial expressions; Hand gestures; Multimodal analysis; Neural networks; Prosody

Indexed keywords

ARTIFICIAL INTELLIGENCE; FEEDBACK; IMAGING TECHNIQUES; PHOTOGRAPHY; SPEECH RECOGNITION; STATE FEEDBACK; VIDEO RECORDING;

AFFECTIVE INTERACTION; AFFECTIVE STATES; AUDIO-VISUAL MATERIAL; DIMENSIONAL REPRESENTATION; DYNAMIC APPROACH; DYNAMIC EVENTS; EMOTIONAL STATES; FACIAL EXPRESSIONS; HAND GESTURES; HUMAN COMPUTING; HUMAN-CENTERED COMPUTING; HUMAN-MACHINE INTERACTION; HYDERABAD , INDIA; INTERNATIONAL WORKSHOPS; JOINT CONFERENCE; MULTIMODAL ANALYSIS; MULTIMODAL INPUTS; NEURAL NETWORKS; PROSODY; PROSODY INFORMATION; REAL-WORLD SITUATIONS; SIMPLE RECURRENT NETWORK; VARIATIONS OF; VIDEO SEQUENCING;

HUMAN COMPUTER INTERACTION;

EID: 49949087271     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-540-72348-6_5     Document Type: Conference Paper

References (54)

1. A. Jaimes, Human-Centered Multimedia: Culture, Deployment, and Access, IEEE Multimedia Magazine, Vol. 13, No.1, 2006.
2. A. Mehrabian, Communication without Words, Psychology Today, vol. 2, no. 4, pp. 53-56, 1968.
3. A. Nogueiras, A. Moreno, A. Bonafonte and J.B. Marino, Speech emotion recognition using hidden markov models. Proceedings of Eurospeech, Aalborg, Denmark, 2001.
4. A. Pentland, Socially Aware Computation and Communication, Computer, vol. 38, no. 3, pp. 33-40, 2005.
5. A. Raouzaiou, N. Tsapatsoulis, K. Karpouzis and S. Kollias, Parameterized facial expression synthesis based on MPEG-4, EURASIP Journal on Applied Signal Processing, Vol. 2002, No 10, 2002, pp. 1021-1038.
6. A.J. Fridlund, Human Facial Expression: An Evolutionary Perspective, Academic Press, 1994.
7. B. Hartmann, M. Mancini, C. Pelachaud, Formational parameters and adaptive prototype instantiation for MPEG-4 compliant gesture synthesis, In Computer Animation'02, Geneva, Switzerland. IEEE Computer Society Press, 2002.
8. C. Tomasi and T. Kanade, Detection and Tracking of Point Features, Carnegie Mellon University Technical Report CMU-CS-91-132, April 1991.
9. F. Freitag, E. Monte, Acoustic-phonetic decoding based on Elman predictive neural networks, Proceedings of ICSLP 96, Fourth International Conference on, Page(s): 522-525, vol.1.
10. FP5 IST ERMIS, Emotionally Rich Man-machine Intelligent System IST-2000-29319, http://www.image.ntua.gr/ermis
11. FP6 1ST HUMAINE, Human-Machine Interaction Network on Emotion, 2004-2007, http://www.emotion-research.net
12. G. Caridakis, A. Raouzaiou, K. Karpouzis, S. Kollias. Synthesizing Gesture Expressivity Based on Real Sequences. Workshop on multimodal corpora: from multimodal behaviour theories to usable models, LREC 2006 Conference, Genoa, Italy, 24-26 May.
13. G. W. Williams, Comparing the joint agreement of several raters with another rater", Biometrics, vol32, pp. 619-627, 1976.
14. H. G. Zimmermann, R. Grothmann, A. M. Schaefer, and Ch. Tietz. Identification and forecasting of large dynamical systems by dynamical consistent neural networks. In S. Haykin, T. Sejnowski J. Principe, and J. McWhirter, editors, New Directions in Statistical Signal Processing: From Systems to Brain. MIT Press, 2006.
15. I. Cohen, A. Garg, and T. S. Huang, Emotion Recognition using Multilevel-HMM, NIPS Workshop on Affective Computing, Colorado, Dec 2000.
16. I. Cohen, N. Sebe, F. Cozman, M. Cirelo, and T.S. Huang. Learning Bayesian network classifiers for facial expression recognition using both labeled and unlabeled data. In Proc. Conf. on Computer Vision and Pattern Recognition, volume 1, pages 595-601, 2003.
17. J. Lien, Automatic recognition of facial expressions using hidden markov models and estimation of expression intensity, Ph.D. dissertation, Carnegie Mellon University, Pittsburg, PA, 1998.
18. J. W. Young, Head and Face Anthropometry of Adult U.S. Civilians, FAA Civil Aeromedical Institute, 1963-1993 (final report 1993)
19. J. Weizenbaum, ELIZA - A Computer Program For the Study of Natural Language Communication Between Man and Machine, Communications of the ACM, Volume 9, Number 1, 1966, pp. 36-35.
20. J.L. Elman, Finding structure in time, Cognitive Science, vol. 14, 1990, pp. 179-211.
21. K. Karpouzis, A. Raouzaiou, A. Drosopoulos, S. Ioannou, T. Balomenos, N. Tsapatsoulis and S. Kollias, Facial expression and gesture analysis for emotionally-rich man-machine interaction, in N. Sarris, M. Strintzis, (eds.), 3D Modeling and Animation: Synthesis and Analysis Techniques, pp. 175-200, Idea Group Publ., 2004.
22. K. Karpouzis, A. Raouzaiou, A. Drosopoulos, S. Ioannou, T. Balomenos, N. Tsapatsoulis and S. Kollias, Facial expression and gesture analysis for emotionally-rich man-machine interaction, N. Sarris, M. Strintzis, (eds.), 3D Modeling and Animation: Synthesis and Analysis Techniques, pp. 175-200, Idea Group Publ., 2004.
23. K.M. Lam, H. Yan, Locating and Extracting the Eye in Human Face Images, Pattern Recognition, Vol.29, No.5, 1996, pp. 771-779.
24. K.R. Scherer, Adding the affective dimension: A new look in speech analysis and synthesis, In Proc. International Conf. on Spoken Language Processing, pp. 1808-1811, 1996.
25. L. Vincent, Morphological Grayscale Reconstruction in Image Analysis: Applications and Efficient Algorithms, IEEE Trans. Image Processing, vol. 2, no. 2, 1993, pp. 176-201.
26. L.C. De Silva and P.C Ng, Bimodal emotion recognition, In Proc. Automatic Face and Gesture Recognition, pp. 332-335, 2000.
27. L.M. Wang, X.H. Shi, G.J. Chen, H.W. Ge, H.P. Lee, Y .C Liang, Applications of PSO Algorithm and OIF Elman Neural Network to Assessment and Forecasting for Atmospheric Quality, ICANNGA 2005, 2005
28. L.S. Chen and T.S. Huang, Emotional expressions in audiovisual human computer interaction, In Proc. International Conference on Multimedia and Expo, pp. 423-426, 2000.
29. L.S. Chen, Joint processing of audio-visual information for the recognition of emotional expressions in human-computer interaction, PhD thesis, University of Illinois at Urbana-Champaign, Dept. of Electrical Engineering, 2000.
30. M. H. Yang, D. Kriegman, N. Ahuja, Detecting Faces in Images: A Survey, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.24(1), 2002, pp. 34-58.
31. M. Pantic and L.J.M. Rothkrantz, Automatic analysis of facial expressions: The state of the art. IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(12):1424-1445, 2000.
32. M. Pantic and L.J.M. Rothkrantz, Towards an Affect-sensitive Multimodal Human-Computer Interaction, Proceedings of the IEEE, vol. 91, no. 9, pp. 1370-1390, 2003.
33. M. Pantic, Face for Interface, in The Encyclopedia of Multimedia Technology and Networking, M. Pagani, Ed., Idea Group Reference, vol. 1, pp. 308-314, 2005.
34. M. Pantic, N. Sebe, J. Cohn, T. Huang, Affective Multimodal Human-Computer Interaction, Proceedings of the 13th annual ACM international conference on Multimedia, pp. 669 - 676, 2005.
35. Mathworks, Manual of Neural Network Toolbox for MATLAB
36. Murat Tekalp, Joern Ostermann, Face and 2-D mesh animation in MPEG-4, Signal Processing: Image Communication 15, Elsevier, pp. 387-421, 2000.
37. N. Sebe, I. Cohen, T.S. Huang, Handbook of Pattern Recognition and Computer Vision, World Scientific, January 2005
38. P. Ekman and W. Friesen, Pictures of Facial Affect, Palo Alto, CA: Consulting Psychologists Press, 1978.
39. P. Mertens, The Prosogram: Semi-Automatic Transcription of Prosody based on a Tonal Perception Model, in B. Bel & I. Marlien (eds.), Proc. of Speech Prosody, Japan, 2004.
40. P. Oudeyer, The production and recognition of emotions in speech: features and algorithms. International Journal of Human Computer Interaction, 59(1-2):157-183, 2003.
41. R. Cowie and E. Douglas-Cowie, Automatic statistical analysis of the signal and prosodic signs of emotion in speech. In Proc. International Conf on Spoken Language Processing, pp. 1989-1992, 1996.
42. R. Cowie, E. Douglas-Cowie, N.Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz and J. G. Taylor, Emotion Recognition in Human-Computer Interaction, IEEE Signal Processign Magazine, pp 33- 80, January 2001.
43. R. Cowie, E. Douglas-Cowie, S. Savvidou, E. McMahon, M. Sawey and M. Schröder, FEELTRACE: An instrument for recording perceived emotion in real time, ISCA Workshop on Speech and Emotion, Northern Ireland, pp. 19-24, 2000.
44. R. Fransens, Jan De Prins, SVM-based Nonparametric Discriminant Analysis, An Application to Face Detection, Ninth IEEE International Conference on Computer Vision, Volume 2, October 13-16, 2003.
45. R. W. Picard, Affective Computing, MIT Press, 1997.
46. R. W. Picard, Towards computers that recognize and respond to user emotion, IBM Syst. Journal, 39 (3-4), 705-719, 2000.
47. R.L. Hsu, M. Abdel-Mottaleb, Anil K. Jain, Face Detection in Color Images, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.24, No.5, May 2002
48. R.W. Picard, E. Vyzas, and J. Healey, Toward machine emotional intelligence: Analysis of affective physiological state, IEEE Trans. on Pattern Analysis and Machine Intelligence, 23(10): 1175-1191, 2001.
49. S. Haykin. Neural Networks: A. Comprehensive Foundation. Macmlllan, New York, 1994.
50. S. Ioannou, A. Raouzaiou, V. Tzouvaras, T. Mailis, K. Karpouzis, S. Kollias, Emotion recognition through facial expression analysis based on a neurofuzzy network, Neural Networks, Elsevier, Vol. 18, Issue 4, May 2005, pp. 423-435
51. T. Balomenos, A. Raouzaiou, S. Ioannou, A. Drosopoulos, K. Karpouzis, S. Kollias, Emotion Analysis in Man-Machine Interaction Systems, Samy Bengio, Hervé Bourlard (Eds.), Machine Learning for Multimodal Interaction, Lecture Notes in Computer Science, Vol. 3361, 2004, pp. 318 - 328, Springer-Verlag.
52. U. Williams, K. N. Stevens, Emotions and Speech: some acoustical correlates, JASA 52, pp. 1238-1250, 1972.
53. Z. Zeng, J. Tu, M. Liu, T.S. Huang, Multi-stream Confidence Analysis for Audio-Visual Affect Recognition, ACII 2005, pp. 964-971.
54. Z. Zeng, Y. Hu, G. Roisman, Y. Fu, T. Huang, "Audio-visual, Emotion Recognition in Adult Attachment Interview, this volume.