Context-sensitive multimodal emotion recognition from speech and facial expression using bidirectional LSTM modeling

Proceedings of the 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010

Volumn , Issue , 2010, Pages 2362-2365

  Wöllmer, Martin ^a   Metallinou, Angeliki ^b   Eyben, Florian ^a   Schuller, Björn ^a   Narayanan, Shrikanth ^b  

^a TECHNICAL UNIVERSITY OF MUNICH   (Germany)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

Context modeling; Emotion recognition; Hidden Markov models; Long short term memory; Multimodality

Indexed keywords

BRAIN; CLASSIFICATION (OF INFORMATION); FACE RECOGNITION; HIDDEN MARKOV MODELS; LONG SHORT-TERM MEMORY; SPEECH COMMUNICATION; SUPPORT VECTOR MACHINES; VECTOR SPACES; SPEECH RECOGNITION;

AUDIO-VISUAL DATABASE; CLASSIFICATION TECHNIQUE; CONTEXT MODELING; CONTEXTUAL INFORMATION; EMOTION RECOGNITION; EMOTIONAL EXPRESSIONS; MULTI-MODALITY; MULTIMODAL EMOTION RECOGNITION; CONTEXT MODELS; CONTEXT-SENSITIVE; EMOTION RECOGNITION FROM SPEECH; FACIAL EXPRESSIONS; HIDDEN-MARKOV MODELS; MEMORY NETWORK; SENSITIVE TECHNIQUES;

SPEECH RECOGNITION; EMOTION RECOGNITION;

EID: 79958734716     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (14)

1. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J. G. Taylor, "Emotion recognition in humancomputer interaction," IEEE Signal Processing Magazine, vol. 18, no. 1, pp. 32-80, 2001.
2. A. Mehrabian, "Communication without words," Psychology today, vol. 2, pp. 53-56, 1968.
3. A. Metallinou, S. Lee, and S. Narayanan, "Decision level combination of multiple modalities for recognition and analysis of emotional expression," in Proc. of ICASSP, Dallas, Texas, 2010, pp. 2462-2465.
4. Y. Zhang and Q. Ji, "Active and dynamic information fusion for facial expression understanding from image sequences," IEEE Transactions On Pattern Analysis And Machine Intelligence, vol. 27, no. 5, pp. 1-16, May 2005.
5. M. Wöllmer, F. Eyben, B. Schuller, E. Douglas-Cowie, and R. Cowie, "Data-driven clustering in emotional space for affect recognition using discriminatively trained LSTM networks," in Proc. of Interspeech, Brighton, UK, 2009, pp. 1595-1598.
6. C.-C. Lee, C. Busso, S. Lee, and S. Narayanan, "Modeling mutual influence of interlocutor emotion states in dyadic spoken interactions," in In Proceedings of Interspeech, UK, 2009.
7. S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural Computation, vol. 9(8), pp. 1735-1780, 1997. (Pubitemid 127462305)
8. M. Wöllmer, B. Schuller, F. Eyben, and G. Rigoll, "Combining long short-term memory and dynamic bayesian networks for incremental emotion-sensitive artificial listening," IEEE Journal of Selected Topics in Signal Processing, Special Issue on Speech Processing for Natural Interaction with Intelligent Environments, 2010.
9. C. Busso, M. Bulut, C.-C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. Chang, S. Lee, and S. Narayanan, "IEMOCAP: interactive emotional dyadic motion capture database," Language Resources and Evaluation, vol. 42, pp. 335-359, 2008.
10. I. Cohen, Q. T. Xiang, S. Zhou, X. Sean, Z. Thomas, and T. S. Huang, "Feature selection using principal feature analysis," 2002.
11. A. Metallinou, C. Busso, S. Lee, and S. Narayanan, "Visual emotion recognition using compact facial representations and viseme information," in Proc. of ICASSP, Dallas, Texas, 2010, pp. 2474-2477.
12. F. Eyben, M. Wöllmer, and B. Schuller, "openEAR - introducing the Munich open-source emotion and affect recognition toolkit," in Proc. of ACII, Amsterdam, The Netherlands, 2009, pp. 576-581.
13. A. Graves, "Supervised sequence labelling with recurrent neural networks," Ph.D. dissertation, Technische Universität München, 2008.
14. S. Young, G. Evermann, M. Gales, T. Hain, D. Kershaw, X. Liu, G. Moore, J. Odell, D. Ollason, D. Povey, V. Valtchev, and P. Woodland, The HTK Book. Entropic Cambridge Research Laboratory, Cambridge, England, 2006.