Fusion of audio-visual features using hierarchical classifier systems for the recognition of affective states and the state of depression

ICPRAM 2014 - Proceedings of the 3rd International Conference on Pattern Recognition Applications and Methods

Volumn , Issue , 2014, Pages 671-678

  Kächele, Markus ^a   Glodek, Michael ^a   Zharkov, Dimitrij ^a   Meudt, Sascha ^a   Schwenker, Friedhelm ^a  

^a UNIVERSITY OF ULM   (Germany)

Author keywords

Affective computing; Emotion recognition; Information fusion; Multiple classifier systems

Indexed keywords

AUDIO SYSTEMS; HIERARCHICAL SYSTEMS; INFORMATION FUSION; KALMAN FILTERS; PATTERN RECOGNITION; TIME SERIES;

AFFECTIVE COMPUTING; AUDIO-VISUAL FEATURES; DISCRIMINATIVE POWER; EMOTION RECOGNITION; HIERARCHICAL CLASSIFIERS; MULTIPLE CLASSIFIER SYSTEMS; REAL-WORLD SCENARIO; TIME SERIES PREDICTORS;

CLASSIFICATION (OF INFORMATION);

EID: 84902358176     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.5220/0004828606710678     Document Type: Conference Paper

References (36)

1. Airas, M. and Alku, P. (2007). Comparison of multiple voice source parameters in different phonation types. In INTERSPEECH, pages 1410-1413.
2. Bishop, C. M. (2006). Pattern Recognition and Machine Learning. Springer.
3. Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W., and Weiss, B. (2005). A database of German emotional speech. In Proceedings of Interspeech 2005, pages 1517-1520.
4. Cohn, J., Kruez, T., Matthews, I., Yang, Y., Nguyen, M. H., Padilla, M., Zhou, F., and De la Torre, F. (2009). Detecting depression from facial actions and vocal prosody. In Affective Computing and Intelligent Interaction and Workshops (ACII 2009)., pages 1-7.
5. Drugman, T., Bozkurt, B., and Dutoit, T. (2011). Causal- Anticausal decomposition of speech using complex cepstrum for glottal source estimation. Speech Communication, 53(6):855-866.
6. Fragopanagos, N. and Taylor, J. (2005). Emotion recognition in human-computer interaction. Neural Networks, 18:389-405.
7. Glodek, M., Reuter, S., Schels, M., Dietmayer, K., and Schwenker, F. (2013). Kalman filter based classifier fusion for affective state recognition. In Proceedings of the International Workshop on Multiple Classifier Systems (MCS), volume 7872 of LNCS, pages 85-94. Springer.
8. Glodek, M., Schels, M., Palm, G., and Schwenker, F. (2012). Multi-modal fusion based on classification using rejection option and Markov fusion networks. In Proceedings of the International Conference on Pattern Recognition (ICPR), pages 1084-1087. IEEE.
9. Hinton, G. E., Osindero, S., and Teh, Y.-W. (2006). A fast learning algorithm for deep belief nets. Neural Comput., 18(7):1527-1554.
10. Jiang, B., Valstar, M. F., and Pantic, M. (2011). Action unit detection using sparse appearance descriptors in space-time video volumes. In Proceedings of IEEE International Conference on Automatic Face and Gesture Recognition, pages 314-321. IEEE.
11. Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Transactions of the ASME - Journal of Basic Engineering, 82(Series D):35-45.
12. Kanade, T., Cohn, J., and Tian, Y. (2000). Comprehensive database for facial expression analysis. In Automatic Face and Gesture Recognition, 2000., pages 46-53.
13. Kane, J. and Gobl, C. (2013). Wavelet maxima dispersion for breathy to tense voice discrimination. Audio, Speech, and Language Processing, IEEE Transactions on, 21(6):1170-1179.
14. Lee, C. M., Yildirim, S., Bulut, M., Kazemzadeh, A., Busso, C., Deng, Z., Lee, S., and Narayanan, S. S. (2004). Emotion recognition based on phoneme classes. In Proceedings of ICSLP 2004.
15. Luengo, I., Navas, E., and Hernáez, I. (2010). Feature analysis and evaluation for automatic emotion identification in speech. Multimedia, IEEE Transactions on, 12(6):490-501.
16. Lugger, M. and Yang, B. (2006). Classification of different speaking groups by means of voice quality parameters. ITG-Fachbericht-Sprachkommunikation 2006.
17. Lugger, M. and Yang, B. (2007). The relevance of voice quality features in speaker independent emotion recognition. In Acoustics, Speech and Signal Processing, 2007. ICASSP 2007. IEEE International Conference on, volume 4, pages IV-17. IEEE.
18. Meng, H., Huang, D., Wang, H., Yang, H., AI-Shuraifi, M., and Wang, Y. (2013). Depression recognition based on dynamic facial and vocal expression features using partial least square regression. In Proceedings of AVEC 2013, AVEC '13, pages 21-30. ACM.
19. Meudt, S., Zharkov, D., Kächele, M., and Schwenker, F. (2013). Multi classifier systems and forward backward feature selection algorithms to classify emotional coloured speech. In Proceedings of the International Conference onMultimodal Interaction (ICMI 2013).
20. Nwe, T. L., Foo, S. W., and De Silva, L. C. (2003). Speech emotion recognition using hidden markov models. Speech communication, 41(4):603-623.
21. Ojala, T., Pietikinen, M., and Harwood, D. (1996). A comparative study of texture measures with classification based on featured distributions. Pattern Recognition, 29(1):51 - 59.
22. Ojansivu, V. and Heikkil, J. (2008). Blur insensitive texture classification using local phase quantization. In Elmoataz, A., Lezoray, O., Nouboud, F., and Mammass, D., editors, Image and Signal Processing, volume 5099 of LNCS, pages 236-243. Springer Berlin Heidelberg.
23. Palm, G. and Schwenker, F. (2009). Sensor-fusion in neural networks. In Shahbazian, E., Rogova, G., and DeWeert, M. J., editors, Harbour Protection Through Data Fusion Technologies, pages 299-306. Springer.
24. Russell, J. A. and Mehrabian, A. (1977). Evidence for a three-factor theory of emotions. Journal of Research in Personality, 11(3):273 - 294.
25. Sánchez-Lozano, E., Lopez-Otero, P., Docio-Fernandez, L., Argones-Ruá, E., and Alba-Castro, J. L. (2013). Audiovisual three-level fusion for continuous estimation of russell's emotion circumplex. In Proceedings of AVEC 2013, AVEC '13, pages 31-40. ACM.
26. Saragih, J. M., Lucey, S., and Cohn, J. F. (2011). Deformable model fitting by regularized landmark meanshift. Int. J. Comput. Vision, 91(2):200-215.
27. Scherer, K. R., Johnstone, T., and Klasmeyer, G. (2003). Handbook of Affective Sciences - Vocal expression of emotion, chapter 23, pages 433-456. Affective Science. Oxford University Press.
28. Scherer, S., Kane, J., Gobl, C., and Schwenker, F. (2012). Investigating fuzzy-input fuzzy-output support vector machines for robust voice quality classification. Computer Speech and Language, 27(1):263-287.
29. Scherer, S., Schwenker, F., and Palm, G. (2008). Emotion recognition from speech using multi-classifier systems and rbf-ensembles. In Speech, Audio, Image and Biomedical Signal Processing using Neural Networks, pages 49-70. Springer Berlin Heidelberg.
30. Schwenker, F., Scherer, S., Schmidt, M., Schels, M., and Glodek, M. (2010). Multiple classifier systems for the recognition of human emotions. In Gayar, N. E., Kittler, J., and Roli, F., editors, Proceedings of the 9th International Workshop on Multiple Classifier Systems (MCS'10), LNCS 5997, pages 315-324. Springer.
31. Senechal, T., Rapp, V., Salam, H., Seguier, R., Bailly, K., and Prevost, L. (2012). Facial action recognition combining heterogeneous features via multikernel learning. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 42(4):993-1005.
32. Valstar, M., Schuller, B., Smith, K., Eyben, F., Jiang, B., Bilakhia, S., Schnieder, S., Cowie, R., and Pantic, M. (2013). Avec 2013: The continuous audio/visual emotion and depression recognition challenge. In Proceedings of AVEC 2013, AVEC '13, pages 3-10. ACM.
33. Viola, P. and Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, volume 1, pages I-511-I-518 vol.1.
34. Wöllmer, M., Kaiser, M., Eyben, F., Schuller, B., and Rigoll, G. (2013). LSTM-modeling of continuous emotions in an audiovisual affect recognition framework. Image and Vision Computing, 31(2):153 - 163. Affect Analysis In Continuous Input.
35. Yang, S. and Bhanu, B. (2011). Facial expression recognition using emotion avatar image. In Automatic Face Gesture Recognition and Workshops (FG 2011), 2011 IEEE International Conference on, pages 866-871.
36. Zeng, Z., Pantic, M., Roisman, G. I., and Huang, T. S. (2009). A survey of affect recognition methods: Audio, visual, and spontaneous expressions. pages 39- 58.