Face reading from speech - Predicting facial action units from audio cues

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn 2015-January, Issue , 2015, Pages 1977-1981

  Ringeval, Fabien ^a,d   Marchi, Erik ^a   Mehu, Marc ^b   Scherer, Klaus ^c   Schuller, Björn ^c,d,e  

^a TECHNICAL UNIVERSITY OF MUNICH   (Germany)

^b WEBSTER VIENNA PRIVATE UNIVERSITY   (Austria)

^c UNIVERSITY OF GENEVA   (Switzerland)

^d UNIVERSITY OF PASSAU   (Germany)

^e IMPERIAL COLLEGE LONDON   (United Kingdom)

Author keywords

Computational paralinguistics; FACS action units

Indexed keywords

BEHAVIORAL RESEARCH; COMPUTER KEYBOARDS; FORECASTING; SPEECH COMMUNICATION;

ACOUSTIC CUES; ACTION UNIT; AUTOMATIC DETECTION; AUTOMATIC RECOGNITION; FACIAL BEHAVIOURS; PARALINGUISTICS; REAL LIFE DATA; UNCONSTRAINED ENVIRONMENTS;

FACE RECOGNITION;

EID: 84959170695     PISSN: 2308457X     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (31)

1. F. Ringeval, F. Eyben, E. Kroupi, A. Yuce, J.-P. Thiran, T. Ebrahimi, D. Lalanne, and B. Schuller, "Prediction of asynchronous dimensional emotion ratings from audiovisual and phys-iological data, " Pattern Recognition Letters, November 2014, in press.
2. F. Schwenker, S. Scherer, and L.-P. Morency, Multimodal Pattern Recognition of Social Signals in Human-Computer-Interaction. Springer, 2015.
3. P. Ekman, W. V. Friesen, and J. Hager, Facial action coding system. Salt Lake City, UT: Research Nexus, 2002.
4. M. Valstar, B. Jiand, M. Mehu, M. Pantic, and K. Scherer, "The first facial expression recognition and analysis challenge, " in Proc. of the IEEE International Conference on Automatic Face & Gesture Recognition (FG), Santa Barbara (CA), USA, March 2011, pp. 921-926.
5. M. Valstar, G. McKeown, M. Mehu, L. Yin, M. Pantic, and J. Cohn, "FERA 2015-Second Facial Expression Recognition and Analysis Challenge, " in Proc. of the IEEE International Conference on Automatic Face & Gesture Recognition (FG), Ljublijana, Slovenia, May 2015.
6. D. L. Bolinger, Intonation and its parts. New York: London, Edward Arnold, 1985.
7. C. Cave, I. Guaitella, R. Bertrand, S. Santi, F. Harlay, and R. Espesser, "About the relationship between eyebrow movements and f0 variations, " in Proc. of the 4th International Conference on Spoken Language, Philadelphia (PA), USA, 1996, pp. 2175-2178.
8. G. Lejan, N. Souviraa-Labastie, and F. Bimbot, "Facial expression recognition from speech, " in Research Report, RR-8337, 2013.
9. F. Eyben, F. Weninger, F. Groß, and B. Schuller, "Recent developments in openSMILE, the Munich open-source multimedia feature extractor, " in Proc. of the 21st ACM International Conference on Multimedia (ACM MM), Barcelona, Spain, October 2013, pp. 835-838.
10. F. Eyben, "Real-time speech and music classification by large audio feature space extraction, " Ph. D. dissertation, Technische Universit ät München, 2014, submitted, to appear.
11. B. Schuller, S. Steidl, A. Batliner, E. Noth, A. Vinciarelli, F. Burkhardt, R. van Son, F. Weninger, F. Eyben, T. Bocklet, G. Mohammadi, and B. Weiss, "The INTERSPEECH 2012 Speaker Trait Challenge, " in Proc. of INTERSPEECH 2012, 13th Annual Conference of the International Speech Communication Association (ISCA), Portland (OR), USA, September 2012, pp. 254-257.
12. B. Schuller, S. Steidl, A. Batliner, A. Vinciarelli, K. Scherer, F. Ringeval, M. Chetouani, F. Weninger, F. Eyben, E. Marchi, M. Mortillaro, H. Salamin, A. Polychroniou, F. Valente, and S. Kim, "The INTERSPEECH 2013 Computational Paralinguistics Challenge: Social Signals, Conflict, Emotion, Autism, " in Proc. of INTERSPEECH 2013, 14th Annual Conference of the International Speech Communication Association (ISCA), Lyon, France, August 2013, pp. 148-152.
13. B. Schuller, S. Steidl, A. Batliner, J. Epps, F. Eyben, F. Ringeval, E. Marchi, and Y. Zhang, "The INTERSPEECH 2014 Computational Paralinguistics Challenge: Cognitive & Physical Load, " in Proc. of INTERSPEECH 2014, 15th Annual Conference of the International Speech Communication Association (ISCA), Singapore, Republic of Singapore, September 2014, pp. 427-431.
14. B. Schuller, S. Steidl, A. Batliner, S. Hantke, F. Hönig, J. R. Orozco-Arroyave, E. Nöth, Y. Zhang, and F. Weninger, "The INTERSPEECH 2015 Computational Paralinguistics Challenge: Nativeness, Parkinson's & Eating Condition, " in Proc. of INTERSPEECH 2015, 16th Annual Conference of the International Speech Communication Association (ISCA), Dresden, Germany, September 2015.
15. F. Weninger, F. Eyben, B. W. Schuller, M. Mortillaro, and K. R. Scherer, "On the acoustics of emotion in audio: What speech, music and sound have in common, " Frontiers in Psychology, Emotion Science, Special Issue on Expression of emotion in music and vocal communication, vol. 4, no. Article ID 292, pp. 1-12, May 2013.
16. D. Bone, C.-C. Lee, and S. S. Narayanan, "Robust Unsupervised Arousal Rating: A Rule-Based Framework with Knowledge-Inspired Vocal Features, " IEEE Transactions on Affective Computing (TAC), vol. 5, no. 2, pp. 201-213, April-June 2014.
17. F. Ringeval, S. Amiriparian, F. Eyben, K. Scherer, and B. Schuller, "Emotion recognition in the wild: Incorporating voice and lip activity in multimodal decision-level fusion, " in 2nd Emotion Recognition In The Wild Challenge and Workshop (EmotiW 2014), held in conjunction with the 16th International Conference on Multimodal Interaction (ICMI 2014), Istanbul, Turkey, September 2014, pp. 473-480.
18. F. Eyben, K. Scherer, B. Schuller, J. Sundberg, E. Andr, C. Busso, L. Devillers, J. Epps, P. Laukka, S. Narayanan, and K. Truong, "The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing, " IEEE Transactions on Affective Computing, 2015, to appear.
19. C. Cortes and V. Vapnik, "Support-Vector Networks, " Machine Learning, vol. 20, no. 3, September 1995.
20. S. Hochreiter and J. Schmidhuber, "Long short-term memory, " Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997.
21. A. Graves and J. Schmidhuber, "Framewise phoneme classification with bidirectional LSTM and other neural network architectures, " Neural Networks, Special Issue, 18th International Joint Conference on Neural Networks (IJCNN), vol. 18, no. 5-6, pp. 602-610, July-August 2005.
22. T. Bänziger, M. Mortillaro, and K. R. Scherer, "Introducing the Geneva Multimodal Expression Corpus for Experimental Research on Emotion Perception, " Emotion, vol. 12, no. 2, pp. 1161-1179, 2012.
23. M. Mortillaro, M. Mehu, and K. Scherer, "Subtly different positive emotions can be distinguished by their facial expressions, " Social Psychological and Personality Science, vol. 2, no. 3, pp. 262-271, May 2011.
24. M. Kipp, Gesture generation by imitation-from human behavior to computer character animation. Boca Raton, FL: Dissertation. com, 2004.
25. F. Ringeval, A. Sonderegger, J. Sauer, and D. Lalanne, "Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions, " in Proc. of Face & Gestures 2013, 2nd IEEE International Workshop on Emotion Representation, Analysis and Synthesis in Continuous Time and Space (EmoSPACE), Shanghai, China, April 2013.
26. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, "The WEKA data mining software: An update, " ACM SIGKDD Explorations Newsletter, vol. 11, no. 1, pp. 10-18, June 2009.
27. F. Weninger, J. Bergmann, and B. Schuller, "Introducing CURRENNT-the Munich Open-Source CUDA RecurREnt Neural Network Toolkit, " Journal of Machine Learning Research, vol. 15, 2014, in press.
28. B. Schuller, "The Computational Paralinguistics Challenge, " IEEE Signal Processing Magazine, vol. 29, no. 4, pp. 97-101, July 2012.
29. M. Wöllmer, F. Wenninger, F. Eyben, B. Schuller, and G. Rigoll, "Acoustic-Linguistic Recognition of Interest in Speech with Bottleneck-BLSTM Nets, " in Proc. of INTERSPEECH 2011, 12th Annual Conference of the International Speech Communication Association (ISCA), Florence, Italy, August 2011, pp. 77-80.
30. F. Ringeval and M. Chetouani, "Hilbert-Huang Transform for non-linear characterization of speech rhythm, " in ISCA Tutorial and Research Workshop on on Non-Linear Speech Processing (NOLISP 2009), Vic, Spain, June 2009.
31. F. Ringeval, M. Chetouani, and B. Schuller, "Novel metrics of speech rhythm for the assessment of emotion, " in Proc. of INTERSPEECH 2012, 13th Annual Conference of the International Speech Communication Association, Portland (OR), USA, September 2012, pp. 346-349.