Modeling naturalistic affective states via facial and vocal expressions recognition

ICMI'06: 8th International Conference on Multimodal Interfaces, Conference Proceeding

Volumn , Issue , 2006, Pages 146-154

  Caridakis, George ^a   Malatesta, Lori ^a   Kessous, Loic ^b   Amir, Noam ^b   Raouzaiou, Amaryllis ^a   Karpouzis, Kostas ^a  

^a NATIONAL TECHNICAL UNIVERSITY OF ATHENS   (Greece)

^b AFEKA TEL AVIV ACADEMIC COLLEGE OF ENGINEERING   (Israel)

Author keywords

Affective interaction; Facial expression recognition; Image processing; Multimodal analysis; Naturalistic data; Prosodic feature extraction; User modeling

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION LANGUAGES; FEEDBACK; IMAGE RECOGNITION; SPEECH RECOGNITION; USER INTERFACES;

AFFECTIVE STATES; AUDIOVISUAL MATERIAL; FACIAL EXPRESSIONS RECOGNITION; VOCAL EXPRESSIONS RECOGNITION;

HUMAN COMPUTER INTERACTION;

EID: 34547153664     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1180995.1181029     Document Type: Conference Paper

References (47)

1. R. W. Picard, Affective Computing, MIT Press, 1997.
2. A. Jaimes, Human-Centered Multimedia: Culture, Deployment, and Access, IEEE Multimedia Magazine, Vol. 13, No. 1, 2006.
3. A. Pentland, A., Socially Aware Computation and Communication, Computer, vol. 38, no. 3, pp. 33-40, 2005.
4. R. W. Picard, Towards computers that recognize and respond to user emotion, IBM Syst. Journal, 39 (3-4), 705-719, 2000.
5. A. Mehrabian, Communication without Words, Psychology Today, vol. 2, no. 4, pp. 53-56, 1968.
6. A.J. Fridlund, Human Facial Expression: An Evolutionary Perspective, Academic Press, 1994.
7. M, Pantic, Face for Interface, in The Encyclopedia of Multimedia Technology and Networking, M. Pagani, Ed., Idea Group Reference, vol. 1, pp. 308-314, 2005.
8. A. Nogueiras, A. Moreno, A. Bonafonte and J.B. Marino, Speech emotion recognition using hidden markov models. Proceedings of Eurospeech, Aalborg, Denmark, 2001.
9. J. Lien, Automatic recognition of facial expressions using hidden markov models and estimation of expression intensity, Ph.D. dissertation, Carnegie Mellon University, Pittsburg, PA, 1998.
10. J.L. Elman, Finding structure in time, Cognitive Science, vol. 14, 1990, pp. 179-211.
11. P. Mertens, The Prosogram: Semi-Automatic Transcription of Prosody based on a Tonal Perception Model, in B. Bel & I. Marlien (eds.), Proc. of Speech Prosody, Japan, 2004.
12. K. Karpouzis, A. Raouzaiou, A. Drosopoulos, S. Ioannou, T. Balomenos, N. Tsapatsoulis and S. Kollias, Facial expression and gesture analysis for emotionally-rich man-machine interaction, in N. Sarris, M. Strintzis, (eds.), 3D Modeling and Animation: Synthesis and Analysis Techniques, pp. 175-200, Idea Group Publ., 2004.
13. S. Ioannou, A. Raouzaiou, V. Tzouvaras, T. Mailis, K. Karpouzis, S. Kollias, Emotion recognition through facial expression analysis based on a neurofuzzy network, Neural Networks, Elsevier, Vol. 18, Issue 4, May 2005, pp. 423-435
14. R. Cowie and E. Douglas-Cowie, Automatic statistical analysis of the signal and prosodic signs of emotion in speech. In Proc. International Conf. on Spoken Language Processing, pp. 1989-1992, 1996.
15. K.R. Scherer, Adding the affective dimension: A new look in speech analysis and synthesis, In Proc. International Conf. on Spoken Language Processing, pp. 1808-1811, 1996.
16. R.W. Picard, E. Vyzas, and J. Healey, Toward machine emotional intelligence: Analysis of affective physiological state, IEEE Trans, on Pattern Analysis and Machine Intelligence, 23(10):1175-1191, 2001.
17. L.S. Chen, Joint processing of audio-visual information for the recognition of emotional expressions in human-computer interaction, PhD thesis, University of Illinois at UrbanaChampaign, Dept. of Electrical Engineering, 2000.
18. M. Pantic and L.J.M. Rothkrantz, Automatic analysis of facial expressions: The state of the art. IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(12):1424-1445, 2000.
19. L.C. De Silva and P.C Ng, Bimodal emotion recognition, In Proc. Automatic Face and Gesture Recognition, pp. 332-335, 2000.
20. L.S. Chen and T.S. Huang, Emotional expressions in audio-visual human computer interaction, In Proc. International Conference on Multimedia and Expo, pp. 423-426, 2000.
21. L.M. Wang, X.H. Shi, G.J. Chen, H.W. Ge, H.P. Lee, Y.C. Liang, Applications of PSO Algorithm and OIF Elman Neural Network to Assessment and Forecasting for Atmospheric Quality, ICANNGA 2005, 2005
22. N. Sebe, I. Cohen, T.S. Huang, Handbook of Pattern Recognition and Computer Vision, World Scientific, January 2005
23. R. Cowie, E. Douglas-Cowie, S. Savvidou, E. McMahon, M. Sawey and M. Schröder, FEELTRACE: An instrument for recording perceived emotion in real time, ISCA Workshop on Speech and Emotion, Northern Ireland, pp. 19-24, 2000.
24. AP Engelbrecht, I. Cloete, A Sensitivity Analysis Algorithm for Pruning Feedforward Neural Networks, IEEE International Conference in Neural Networks, Washington, Vol 2, 1996, pp 1274-1277.
25. A. Murat Tekalp, Joern Ostermann, Face and 2-D mesh animation in MPEG-4, Signal Processing: Image Communication 15, Elsevier, pp. 387-421, 2000.
26. Rein-Lien Hsu, Mohamed Abdel-Mottaleb, Anil K. Jain, Face Detection in Color Images, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.24, No.5, May 2002
27. ERMIS, Emotionally Rich Man-machine Intelligent System IST-2000-29319, http://www.image.ntua.gr/ermis
28. HUMAINE IST, Human-Machine Interaction Network on Emotion, 2004-2007, http://www.emotion-research.net
29. J. W. Young, Head and Face Anthropometry of Adult U.S. Civilians, FAA Civil Aeromedical Institute, 1963-1993 (final report 1993)
30. R. Fransens, Jan De Prins, SVM-based Nonparametric Discriminant Analysis, An Application to Face Detection, Ninth IEEE International Conference on Computer Vision, Volume 2, October 13-16, 2003.
31. Kin-Man Lam and Hong Yan, Locating And Extracting the Eye in Human Face Images, Pattern Recognition, Vol.29, No.5,1996, pp. 771-779.
32. G Tomasi and T. Kanadie, Detection and Tracking of Point Features, Carnegie Mellon University Technical Report CMU-CS-91-132, April 1991.
33. L. Vincent, Morphological Grayscale Reconstruction in Image Analysis: Applications and Efficient Algorithms, IEEE Trans. Image Processing, vol. 2, no. 2, 1993, pp. 176-201.
34. A. Raouzaiou, N. Tsapatsoulis, K. Karpouzis and S. Kollias, Parameterized facial expression synthesis based on MPEG-4, EURASIP Journal on Applied Signal Processing, Vol. 2002, No 10, 2002, pp. 1021-1038.
35. M. H. Yang, D. Kriegman, N. Ahuja, Detecting Faces in Images: A Survey, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol.24(1), 2002, pp. 34-58.
36. P. Ekman and W. Friesen, Pictures of Facial Affect, Palo Alto, CA: Consulting Psychologists Press, 1978.
37. U. Williams, K. N. Stevens, Emotions and Speech: some acoustical correlates, JASA 52, pp. 1238-1250, 1972.
38. Mathworks, Manual of Neural Network Toolbox for MAT-LAB
39. S. Haykin. Neural Networks: A Comprehensive Foundation. Macmillan, New York, 1994.
40. H. G. Zimmermann, R. Grothmann, A. M. Schaefer, and Ch. Tietz. Identification and forecasting of large dynamical systems by dynamical consistent neural networks. In S. Haykin, T. Sejnowski J. Principe, and J. McWhirter, editors, New Directions in Statistical Signal Processing: From Systems to Brain. MIT Press, 2006.
41. M. Pantic, N. Sebe, J. Cohn, T. Huang, Affective Multimodal Human-Computer Interaction, Proceedings of the 13th annual ACMinternational conference on Multimedia, pp. 669-676, 2005.
42. R. Cowie, E. Douglas-Cowie, N.Tsapatsouulis, G. Votsis, S. Kollias, W. Fellenz and J. G. Taylor, Emotion Recognition in Human-Computer Interaction, IEEE Signal Processign Magazine, pp 33- 80, January 2001.
43. M. Pantic and L.J.M. Rothkrantz, Towards an Affectsensitive Multimodal Human-Computer Interaction, Proceedings of the IEEE, vol. 91, no. 9, pp. 1370-1390, 2003.
44. P. Oudeyer, The production and recognition of emotions in speech: features and algorithms. International Journal of Human Computer Interaction, 59(1-2):157-183, 2003.
45. I. Cohen, A. Garg, and T. S. Huang, Emotion Recognition using Multilevel-HMM, NIPS Workshop on Affective Computing, Colorado, Dec 2000.
46. F. Freitag, E. Monte, Acoustic-phonetic decoding based on Elman predictive neural networks, Proceedings of ICSLP 96, Fourth International Conference on, Page(s): 522-525, vol.1.
47. Z. Zeng, J. Tu, M. Liu, T.S. Huang, Multi-stream Confidence Analysis for Audio-Visual Affect Recognition, ACII 2005, pp. 964-971.