A robust multimodal approach for emotion recognition

Neurocomputing

Volumn 71, Issue 10-12, 2008, Pages 1913-1920

  Song, Mingli ^a   You, Mingyu ^a   Li, Na ^a   Chen, Chun ^a  

^a ZHEJIANG UNIVERSITY   (China)

Author keywords

Emotion recognition; Lipschitz embedding; Multimodal; Tripled hidden Markov model

Indexed keywords

CORRELATION METHODS; ESTIMATION; FEATURE EXTRACTION; GEODESY; HIDDEN MARKOV MODELS;

EMOTION RECOGNITION; INTELLIGENT HUMAN/COMPUTER COMMUNICATION; LIPSCHITZ EMBEDDING; ROBUST MULTIMODAL APPROACH;

IMAGE RECOGNITION;

ACCURACY; ACOUSTICS; ARTICLE; AUDIO RECORDING; AUDITORY STIMULATION; CORRELATION ANALYSIS; EMOTIONALITY; FACIAL EXPRESSION; FACIES; HIDDEN MARKOV MODEL; INTERMETHOD COMPARISON; MATHEMATICAL COMPUTING; PATTERN RECOGNITION; PRIORITY JOURNAL; SPEECH; VIDEORECORDING; VISUAL INFORMATION;

EID: 44649157612     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2007.07.041     Document Type: Article

References (29)

1. N. Amir, S. Ron, Towards an automatic classification of emotion in speech, in: Proceedings of the International Conference on Spoken Language Processing, Sydney, Australia, December 1998, pp. 225-228.
2. Bengio S. An asynchronous hidden Markov model for audio-visual speech recognition. In: Becker S., Thrun S., and Obermayer K. (Eds). Advances in Neural Information Processing System, NIPS 15 (2003), MIT Press, Cambridge, MA 1237-1244
3. J. Bu, M. Song, Q. Wu, C. Chen, Sketch based facial expression recognition using graphics hardware, in: The First International Conference on Affective Computing & Intelligent Interaction 2005, Lecture Notes in Computer Science, vol. 3784, Beijing, China, October 2005, pp. 72-80.
4. L. Chen, T.S. Huang, Emotional expressions in audiovisual human computer interaction, in: Proceedings of the International Conference on Multimedia & Expo, New York, NY, June 2000, pp. 423-426.
5. E.S. Chuang, H. DeDeshpande, C. Bregler, Expression space learning, in: Proceedings of the Pacific Conference on Computer Graphics and Applications, Beijing, China, 2002, pp. 68-77.
6. Cohen I., Sebe N., Chen L., Garg A., and Huang T.S. Facial expression recognition from video sequences temporal and static modeling. Comput. Vision Image Understanding 91 1-2 (2003) 160-187 (Special issue on face recognition & facial expression recognition from video sequences temporal and static modeling)
7. I. Cohen, N. Sebe, F.G. Cozman, M.C. Cirelo, T.S. Huang, Learning Bayesian network classifiers for facial expression recognition with both labeled and unlabeled data, in: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, Toronto, Canada, 2003, pp. 595-601.
8. Cootes T.F., Cooper D., Taylor C.J., and Graham J. Active shape models-their training and application. Comput. Vision Image Understanding 61 1 (1995) 38-59
9. M. Covell, C. Bregler, Eigen-points: control-point location using principle component analysis, in: Proceedings of the International Conference on Automatic Face & Gesture Recognition, Killington, Vermont, October 1996, pp. 122-127.
10. R. Cowie, E. Douglas-Cowie, S. Savvidou, E. McMahon, M. Sawey, M. Schöder, Feedtrace: an instrument for recording perceived emotion in real time, in: Proceedings of the ISCA Workshop on Speech and Emotion, 2000, pp. 19-24.
11. L.C. De Silva, P.C. Ng, Bimodal emotion recognition, in: Proceedings of the International Conference on Automatic Face & Gesture Recognition, Grenoble, France, 2000, pp. 332-335.
12. F. Dellaert, T. Polzin, A. Waibel, Recognizing emotion in speech, in: Proceedings of the International Conference Spoken Language Processing, Philadelphia, PA, October 1996, pp. 1970-1973.
13. Ekman P. Facial expression and emotion. Am. Psychol. 48 4 (1993) 384-392
14. Ekman P. Strong evidence for universals in facial expressions: a reply to Russell's mistaken critique. Psychol. Bull. 115 2 (1994) 268-287
15. Ekman P., and Friesen W.V. Facial Action Coding System: Investigator's Guide (1978), Consulting Psychologists Press
16. Fragopanagos F., and Taylor J.G. Emotion recognition in human-computer interaction. Neural Networks 18 (2005) 389-405
17. International Organization for Standardization. Facial animation parameters. Text for ISO/IEC FDIS 14496-2 Visual, ISO/IEC JTC1/SC29/WG11 N2502, 1998.
18. Lavagetto F., and Pockaj R. The facial animation engine: towards a high-level interface for the design of MPEG-4 compliant animated faces. IEEE Trans. Circuits Systems Video Technol 9 2 (1999) 277-289
19. J.J. Lien, T. Kanade, J.F. Cohn, C.-C. Li, Subtly different facial expression recognition and expression intensity estimation, in: Proceedings of the IEEE International Conference Computer Vision and Pattern Recognition, Santa Barbara, California, 1998, pp. 853-859.
20. Mase K. Recognition of facial expression from optical flow. IEICE Trans. E74 (1991) 3474-3483
21. A.V. Nefian, L.L. Hong, X.X. Liu, X. Pi, C. Mao, K. Murphy, A coupled HMM for audio-visual speech recognition, in: Proceedings of the IEEE ICASSP'2002, vol. 2, Orlando, Florida, May 2002, pp. 2013-2016.
22. A. Nogueiras, A. Moreno, A. Bonafonte, J.B. Mariño, Speech emotion recognition using hidden Markov models, in: Proceedings of the EUROSPEECH'01, Aalborg, Denmark, September 2001, pp. 2679-2682.
23. V. Petrushin, Emotion recognition in speech signals: experimental study, development, and application, in: Proceedings of the International Conference on Spoken Language Processing, Beijing, China, June 2000, pp. 98-112.
24. M. Song, J. Bu, C. Chen, N. Li, Audio-visual based emotion recognition-a new approach, in: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, Washington, DC, June 2004, pp. 1020-1025.
25. Tao D., Li X., Hu W., Maybank S.J., and Wu X. Supervised tensor learning. Knowledge Inform. Syst. 13 1 (2007) 1-42
26. D. Tao, X. Li, X. Wu, S.J. Maybank, General tensor discriminant analysis and Gabor features for gait recognition, IEEE Trans. Pattern Anal. Machine Intell. 29 (10) (2007) 1,700-1,715.
27. M. You, C. Chen, J. Bu, J. Liu, J. Tao, Emotional speech analysis on nonlinear manifold, in: Proceedings of the 18th International Conference on Pattern Recognition, Hong Kong, China, August 2006, pp. 91-94.
28. Z. Zeng, Y. Hu, G.I. Roisman, Z. Wen, Y. Fu, T.S. Huang, Audio-visual emotion recognition in adult attachment interview, in: Proceedings of the ACM International Conference on Machine Intelligence, Banff, Canada, November 2006, pp. 139-145.
29. Z. Zeng, J. Tu, P. Pianfetti, M. Liui, T. Zhang, et al., Audio-visual affect recognition through multi-stream fused HMM for HCI, in: Proceedings of the International Conference on Computer Vision and Pattern Recognition, San Diego, CA, June 2005, pp. 967-972.