Semantic audio-visual data fusion for automatic emotion recognition

EUROMEDIA 2008 - 14th Annual Scientific Conference on Web Technology, New Media Communications and Telematics Theory Methods, Tools and Applications Medical Imaging and D-TV

Volumn , Issue , 2008, Pages 58-65

  Datcu, Dragos ^a   Rothkrantz, Leon J M ^a  

^a DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

Active Appearance Models; Automatic emotion recognition; Data fusion; Face detection; Facial characteristic point extraction; Facial feature extraction; Speech analysis; Support Vector Machines

Indexed keywords

DATA FUSION; FACE RECOGNITION; FEATURE EXTRACTION; GESTURE RECOGNITION; MEDICAL IMAGING; SEMANTICS; SPEECH ANALYSIS; SUPPORT VECTOR MACHINES; TOOLS; WIRELESS TELECOMMUNICATION SYSTEMS;

ACTIVE APPEARANCE MODELS; AUDIO-VISUAL DATA FUSION; AUTOMATIC EMOTION RECOGNITION; CHARACTERISTIC POINT; EMOTION RECOGNITION FROM SPEECH; FACIAL EXPRESSION RECOGNITION; FACIAL FEATURE EXTRACTION; RECOGNITION OF EMOTION;

PSYCHOLOGY COMPUTING;

EID: 84898877448     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (31)

1. Aleksic, P. S., A. K. Katsaggelos, "Automatic Facial Expression Recognition using Facial Animation Parameters and Multi-Stream HMMs", ISSN: 1556-6013, in IEEE Transactions on Information Forensics and Security, 2006.
2. Boersma, P., Weenink, D., "Praat: doing phonetics by computer (Version 4.3.14) " [Computer program], 2005.
3. Burkhardt, F., A. Paeschke, M. Rolfes, W. Sendlmeier, B. Weiss, "A Database of German Emotional Speech", Proceedings Interspeech, Lissabon, Portugal 2005.
4. Busso, C, Z. Deng, S. Yildirim, M. Bulut, C. M. Lee, A. Kazemzadeh, S. Lee, U. Neumann, S. Narayanan, "Analysis of Emotion Recognition using Facial Expressions, Speech and Multimodal Information", Proceedings of ACM 6th International Conference on Mutlmodal Interfaces (ICMI 2004), ISBN: 1-58113-890-3, State College, PA, 2004.
5. Chang, C. C, C. J. Lin, "LIBSVM: a library for support vector machines", http://www.csie.ntu.edu.tw/~cjlin/libsvm, 2001.
6. Cootes, T. F., G. J. Edwards, C. J. Taylor, "Active appearance models", Lecture Notes in Computer Science, vol. 1407, pp. 484-98, 1998.
7. Datcu, D., L. J. M. Rothkrantz, "Multimodal workbench for human emotion recognition", Software Demo at IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR'07, Minneapolis, Minnesota, USA, 2007.
8. Datcu, D., L. J. M. Rothkrantz, "The recognition of emotions from speech using GentleBoost Classifier", CompSysTech'06, June 2006.
9. Datcu, D., L. J. M. Rothkrantz, "Facial Expression Recognition in still pictures and videos using Active Appearance Models. A comparison approach.", CompSysTech'07, ISBN 978-954-9641-50-9, pp. VI.13-1-VI.13-6, Rousse, Bugaria, June 2007.
10. Datcu, D., L. J. M. Rothkrantz, "The use of Active Appearance Model for facial expression recognition in crisis environments", Proceedings ISCRAM2007, ISBN 9789054874171, pp. 515-524, 2007.
11. Datcu, D., L. J. M. Rothkrantz, "Machine learning techniques for face analysis", Euromedia 2005, ISBN 90-77381-17-1, pp. 105-109, 2005.
12. Ekman, P., W. Friesen, "Facial Action Coding System", Consulting Psychologists Press, Inc., Palo Alto California, USA, 1978.
13. Fellenz, W. A., J. G. Taylor, R. Cowie, E. Douglas-Cowie, F. Piat, S. Kollias, C. Orovas, B. Apolloni, "On emotion recognition of faces and of speech using neuralnetworks, fuzzy logic and the ASSESS system", Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks, IJCNN 2000, ISBN: 0-7695-0619-4, pp. 93-98, 2000.
14. Go, H. J., K. C. Kwak, D. J. Lee, "Emotion recognition from the facial image and speech signal", SICE Annual Conference, Japan, pp. 2890-2895, 2003.
15. Hoch, S., F. Althoff, G. McGlaun, G. Rigoll, "Bimodal fusion of emotional data in an automotive environment", IEEE International Conference on Acoustics, Speech, and Signal Processing ICASSP'05, ISBN: 0-7803-8874-7, Vol. 2, pp. 1085-1088, 2005.
16. th International Symposium on Advanced Intelligent Systems ISIS'07, 2007.
17. Kanade, T., J. F. Cohn, Y. Tian, "Comprehensive database for facial expression analysis, ". Proc. of the 4th IEEE Int. Con. On Automatic Face and Gestures Reco., France, 2000.
18. Martin, O., I. Kotsia, B. Macq, I. Pitas, "The eNTERFACE'05 Audio-Visual Emotion Database", Proceedings of the 22nd International Conference on Data Engineering Workshops (ICDEW06), ISBN:0-7695-2571-7, 2006.
19. Meng, H., J. Pittermann, A. Pittermann, W. Minker, "Combined speech-emotion recognition for spoken human-computer interfaces", IEEE International Conference on Signal Processing and Communications, Dubai (United Emirates), 2007.
20. OpenCV: Open Source Computer Vision Library; http://www.intel.com/ technology/computing/opencv/.
21. Pantic, M., L. J. M. Rothkrantz, "Toward an affect-sensitive multimodal human-computer interaction", Proceedings of the IEEE, Special Issue on human-computer multimodal interface, 91(9):1370-1390, 2003.
22. Paleari, M., C. L. Lisetti, "Toward Multimodal Fusion of Affective Cues", HCM'06, Santa Barbara, California, USA, pp. 99-108, October 27, 2006.
23. Sebe N., I. Cohen, T. Gevers, T. S. Huang, "Emotion Recognition Based on Joint Visual and Audio Cues", Proceedings of the 18th International Conference on Pattern Recognition (ICPR'06), 2006.
24. Song, M., J. J. Bu, C. Chen, N. Li, "Audio-visual based emotion recognition-a new approach", Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004 - CVPR 2004, ISBN: 0-7695-2158-4, pp. 1020-1025, 2004.
25. Snack Sound Toolkit; http://www.speech.kth.se/snack/.
26. Stegmann, M. B., "The AAM-API: An Open Source Active Appearance Model Implementation", Medical Image Computing and Computer-Assisted Intervention - MICCAI 2003, 6th Int. Conference, Montréal, Canada, Springer, pp. 951-952, 2003.
27. Vapnik, V., "The Nature of Statistical Learning Theory", Springer-Verlag, New York, 1995.
28. Vapnik, V., "Statistical Learning Theory", John Wiley and Sons, Inc., New York, 1998.
29. Viola, P., M. Jones, "Robust Real-time Object Detection. " Second International Workshop on Statistical and Computational Theories of Vision-Modeling, Learning, Computing, and Sampling, 2001.
30. Zeng, Z., Y. Hu, G. I. Roisman, Z. Wen, Y. Fu, T. S. Huang, "Audio-Visual Spontaneous Emotion Recognition", Artificial Intelligence for Human Computing, ISBN: 978-3-540-72346-2, Springer Berlin/Heidelberg, pp. 72-90, 2007.
31. Wimmer, M., B. Schuller, D. Arsic, G. Rigoll, B. Radig, "LOW-LEVEL FUSION OF AUDIO AND VIDEO FEATURE FOR MULTI-MODAL EMOTION RECOGNITION", In 3rd International Conference on Computer Vision Theory and Applications (VISAPP), Madeira, Portugal, 2008.