Audiovisual three-level fusion for continuous estimation of Russell's emotion circumplex

AVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge

Volumn , Issue , 2013, Pages 31-39

  Sánchez Lozano, Enrique ^a   Lopez Otero, Paula ^a   Docio Fernandez, Laura ^a   Argones Rúa, Enrique ^b   Alba Castro, José Luis ^a  

^a UNIVERSITY OF VIGO   (Spain)

^b Multimodal Information Area Gradiant ^*  (Spain)

Author keywords

Affective computing; Audiovisual features; Eigen models; Multimodal fusion; Support vector regression

Indexed keywords

AFFECTIVE COMPUTING; AUDIO-VISUAL FEATURES; EIGEN-MODELS; MULTI-MODAL FUSION; SUPPORT VECTOR REGRESSION (SVR);

ITERATIVE METHODS;

EID: 84887432798     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2512530.2512534     Document Type: Conference Paper

References (41)

1. T. Ahonen, A. Hadid, and M. Pietikäinen. Face recognition with local binary patterns. In Computer Vision-ECCV 2004, pages 469-481. Springer, 2004.
2. T. Almaev and M. Valstar. Local gabor binary patterns from three orthogonal planes for automatic facial expression recognition. In Proc. Affective Computing and Interaction (to appear) (ACII'13), 2013.
3. D. Bolme, B. Draper, and J. Beveridge. Average of synthetic exact filters. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR'09), 2009.
4. C.-C. Chang and C.-J. Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1-27:27, 2011. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
5. A. C. Cruz, B. Bhanu, and S. Yang. A psychologically-inspired match-score fusion model for video-based facial expression recognition. In ACII (2), pages 341-350, 2011.
6. P. Dago-Casas, D. González-Jiménez, L. L. Yu, and J. L. Alba-Castro. Single-and cross-database benchmarks for gender classification under unconstrained settings. In Computer Vision Workshops (ICCV Workshops), 2011 IEEE International Conference on, pages 2152-2159. IEEE, 2011.
7. M. El Ayadi, M. S. Kamel, and F. Karray. Survey on speech emotion recognition: Features, classification schemes, and databases. Pattern Recogn., 44(3):572-587, Mar. 2011.
8. D. Gabor. Theory of communication. Journal of Institute for Electrical Engnineering, 53, 1946.
9. D. González-Jiménez and J. L. Alba-Castro. Toward pose-invariant 2-d face recognition through point distribution models and facial symmetry. Information Forensics and Security, IEEE Transactions on, 2(3):413-429, 2007.
10. M. A. Hall. Correlation-based Feature Subset Selection for Machine Learning. PhD thesis, University of Waikato, Hamilton, New Zealand, 1998.
11. D. R. Hardoon, S. Szedmak, O. Szedmak, and J. Shawe-taylor. Canonical correlation analysis; an overview with application to learning methods. Technical report, 2007.
12. B. Jiang, M. F. Valstar, and M. Pantic. Action unit detection using sparse appearance descriptors in space-time video volumes. In Automatic Face & Gesture Recognition and Workshops (FG 2011), 2011 IEEE International Conference on, pages 314-321. IEEE, 2011.
13. P. Kenny, G. Boulianne, and P. Dumouchel. Eigenvoice modeling with sparse training data. IEEE Transactions on Speech and Audio Processing, 13(3), 2005.
14. R. Kuhn, P. Nguyen, J. C. Junqua, and L. Goldwasser. Eigenfaces and eigenvoices: dimensionality reduction for specialized pattern recognition. In Multimedia Signal Processing, 1998 IEEE Second Workshop on, pages 71-76, 1998.
15. Y. Linde, A. Buzo, and R. Gray. An algorithm for vector quantizer design. IEEE Transactions on Communications, 28(84), 1980.
16. G. Littlewort, J. Whitehill, T. Wu, I. Fasel, M. Frank, J. Movellan, and M. Bartlett. The computer expression recognition toolbox (cert). In Automatic Face & Gesture Recognition and Workshops (FG 2011), 2011 IEEE International Conference on, pages 298-305. IEEE, 2011.
17. H. Meng and N. Bianchi-Berthouze. Naturalistic affective expression classification by a multi-stage approach based on hidden markov models. In Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part II, ACII'11, pages 378-387, Berlin, Heidelberg, 2011. Springer-Verlag.
18. M. A. Nicolaou, H. Gunes, and M. Pantic. Continuos prediction of spontaneous affect from multiple cues and modalities in valence-arousal space. IEEE Trans. Affect. Comput., 2(2):92-105, July 2011.
19. J. Nicolle, V. Rapp, K. Bailly, L. Prevost, and M. Chetouani. Robust continuous prediction of human emotions using multiscale dynamic cues. In Proceedings of the 14th ACM international conference on Multimodal interaction, pages 501-508. ACM, 2012.
20. T. Ojala, M. Pietikainen, and T. Maenpaa. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 24(7):971-987, 2002.
21. R. Picard. Affective computing. MIT Press, 1997.
22. L. Rabiner and B.-H. Juang. Fundamentals of speech recognition. Prentice-Hall, Inc., Upper Saddle River, NJ, USA, 1993.
23. J. Russel. A circumspect model of affect. Journal of Psychology and Social Psychology, 39(6), 1980.
24. N. Sato and Y. Obuchi. Emotion recognition using mel-frequency cepstral coefficients. Information and Media Technologies, 2(3):835-848, 2007.
25. A. Savran, H. Cao, M. Shah, A. Nenkova, and R. Verma. Combining video, audio and lexical indicators of affect in spontaneous conversation via particle filtering. In Proceedings of the 14th ACM international conference on Multimodal interaction, ICMI '12, pages 485-492, New York, NY, USA, 2012. ACM.
26. A. Sayedelahl, P. Fewzee, M. S. Kamel, and F. Karray. Audio-based emotion recognition from natural conversations based on co-occurrence matrix and frequency domain energy distribution features. In Proceedings of the 4th international conference on Affective computing and intelligent interaction - Volume Part II, ACII'11, pages 407-414, Berlin, Heidelberg, 2011. Springer-Verlag.
27. B. Schuller, S. Steidl, and A. Batliner. The interspeech 2009 emotion challenge. In INTERSPEECH, pages 312-315, 2009.
28. B. Schuller, S. Steidl, A. Batliner, F. Burkhardt, L. Devillers, C. Muller, and S. S. Narayanan. The interspeech 2010 paralinguistic challenge. In In Proceedings of InterSpeech, Makuhari, Japan, Sept. 2010.
29. B. Schuller, S. Steidl, A. Batliner, E. Nöth, A. Vinciarelli, F. Burkhardt, R. van Son, F. Weninger, F. Eyben, T. Bocklet, G. Mohammadi, and B. Weiss. The INTERSPEECH 2012 Speaker Trait Challenge. In ISCA, editor, Proceedings of the 13th Annual Conference of the International Speech Communication Association (INTERSPEECH 2012), 2012.
30. B. Schuller, S. Steidl, A. Batliner, F. Schiel, and J. Krajewski. The interspeech 2011 speaker state challenge. In INTERSPEECH, pages 3201-3204, 2011.
31. B. Schuller, S. Steidl, A. Batliner, A. Vinciarelli, K. Scherer, F. Ringeval, M. Chetouani, F. Weninger, F. Eyben, E. Marchi, M. Mortillaro, H. Salamin, A. Polychroniou, F. Valente, and S. Kim. The interspeech 2013 computational paralinguistics challenge: Social signals, conflict, emotion, autism. 09/2013 2013.
32. B. Schuller, M. Valstar, F. Eyben, R. Cowie, and M. Pantic. Avec 2012: the continuous audio/visual emotion challenge. In Proceedings of the 14th ACM international conference on Multimodal interaction, pages 449-456. ACM, 2012.
33. B. Schuller, M. Valstar, F. Eyben, G. McKeown, R. Cowie, and M. Pantic. Avec 2011-the first international audio/visual emotion challenge. In Affective Computing and Intelligent Interaction, pages 415-424. Springer, 2011.
34. T. Senechal, V. Rapp, H. Salam, R. Seguier, K. Bailly, and L. Prevost. Facial action recognition combining heterogeneous features via multikernel learning. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 42(4):993-1005, 2012.
35. C. Shan, S. Gong, and P. W. McOwan. Facial expression recognition based on local binary patterns: A comprehensive study. Image and Vision Computing, 27(6):803-816, 2009.
36. M. Valstar, B. Jiang, M. Mehu, M. Pantic, and K. Scherer. The first facial expression recognition and analysis challenge. In Automatic Face Gesture Recognition and Workshops (FG 2011), 2011 IEEE International Conference on, pages 921-926, 2011.
37. M. Valstar, B. Schuller, K. Smith, F. Eyben, B. Jiang, S. Bilakhia, S. Schnieder, R. Cowie, and M. Pantic. AVEC 2013 - the continuous audio/visual emotion and depression recognition challange. In Proceedings of the 3rd International Audio/Visual Emotion Challange and Workshop (AVEC'13), 2013.
38. R. Westwood. Speaker Adaptation Using Eigenvoices. PhD thesis, Cambridge University, Cambridge, U.K., 1999.
39. L. Wolf, T. Hassner, Y. Taigman, et al. Descriptor based methods in the wild. In Workshop on Faces in'Real-Life'Images: Detection, Alignment, and Recognition, 2008.
40. M. Wollmer, M. Kaiser, F. Eyben, B. Schuller, and G. Rigoll. LSTM-modeling of continuous emotions in an audiovisual affect recognition framework. Image Vision Comput., 31(2):153-163, 2013.
41. Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang. A survey of affect recognition methods: Audio, visual, and spontaneous expressions, 2009.