A 3-D audio-visual corpus of affective communication

IEEE Transactions on Multimedia

Volumn 12, Issue 6, 2010, Pages 591-598

  Fanelli, Gabriele ^a   Gall, Juergen ^a   Romsdorfer, Harald ^b   Weise, Thibaut ^c   Van Gool, Luc ^a  

^a ETH ZURICH   (Switzerland)

^b GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

^c EPFL   (Switzerland)

Author keywords

3 D face modeling; Audio visual database; emotional speech; face tracking; visual speech modeling

Indexed keywords

3D FACES; AUDIO-VISUAL DATABASE; EMOTIONAL SPEECH; FACE TRACKING; VISUAL SPEECH;

FACE RECOGNITION; FREQUENCY ESTIMATION; GESTURE RECOGNITION; HUMAN COMPUTER INTERACTION; SCANNING; SOFTWARE AGENTS; SPEECH ANALYSIS; SPEECH RECOGNITION; SPEECH SYNTHESIS;

THREE DIMENSIONAL;

EID: 77956722066     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2010.2052239     Document Type: Article

References (37)

1. Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang, "A survey of affect recognition methods: Audio, visual, and spontaneous expressions," IEEE Trans. Pattern Anal. Mach. Intell., vol.31, no.1, pp. 39-58, Jan. 2009.
2. R. W. Picard, "Toward computers that recognize and respond to user emotion," IBM Syst. J., vol.39, no.3-4, pp. 705-719, 2000.
3. A. Mehrabian, "Communication without words," Psychol. Today, vol.2, no.9, pp. 52-55, 1968.
4. C. M. Whissell, The Dictionary of Affect in Language, R. Plutchik and H. Kellerman, Eds. Reading, MA: Addison-Wesley, 1972.
5. S. Frigo, "The relationship between acted and naturalistic emotional corpora," in Proc. Corpora for Research on Emotion and Affect (LREC), 2006.
6. M. Valstar, H. Gunes, and M. Pantic, "How to distinguish posed from spontaneous smiles using geometric features," in Proc. ACM Int. Conf. Multimodal Interfaces (ICMI'07), New York, Nov. 2007, pp. 38-45.
7. J. Gross and R. Levenson, "Emotion elicitation using films," Cognit. Emotion, vol.9, no.1, pp. 87-108, 1995.
8. M. Bradley, B. Cuthbert, and P. Lang, "Picture media and emotion: Effects of a sustained affective context," Psychophysiology, vol.33, no.6, pp. 662-670, 1996.
9. D. Clark, "On the induction of depressed mood in the laboratory: Evaluation and comparison of the velten and musical procedures," Adv. Behav. Res. Therapy, vol.5, no.1, pp. 27-49, 1983.
10. K. Scherer, T. Johnstone, and T. Bänziger, "Automatic verification of emotionally stressed speakers: The problem of individual differences," in Proc. Int. Workshop Speech and Computer, 1998.
11. R. Cowie and R. R. Cornelius, "Describing the emotional states that are expressed in speech," Speech Commun., vol.40, no.1-2, pp. 5-32, 2003.
12. R. Westrmann, K. Spies, G. Stahl, and F. W. Hesse, "Relative effectiveness and validity of mood induction procedures: A meta-analysis," Eur. J. Social Psychol., vol.26, no.4, pp. 557-580, 1996.
13. P. Ekman, "Constants across cultures in the face and emotion," J. Pers. Soc. Psychol., vol.17, no.2, pp. 124-129, 1971.
14. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J. G. Taylor, "Emotion recognition in human-computer interaction," IEEE Signal Process. Mag., vol.18, no.1, pp. 32-80, Jan. 2001.
15. R. Craggs and M. M.Wood, "A two dimensional annotation scheme for emotion in dialogue," in Proc. AAAI Spring Symp. Exploring Attitude and Affect in Text: Theories and Applications, 2004.
16. N. Sebe, M. S. Lew, I. Cohen, Y. Sun, T. Gevers, and T. S. Huang, "Authentic facial expression analysis," in Proc. AFGR, 2004.
17. E. Velten, "A laboratory task for induction of mood states," Behav. Res. Therapy, vol.6, pp. 473-482, 1968.
18. Y. Morlec, G. Bailly, and V. Aubergé, "Generating prosodic attitudes in French: Data, model and evaluation," Speech Commun., vol.33, no.4, pp. 357-371, 2001.
19. R. Cowie, E. Douglas-Cowie, and C. Cox, "Beyond emotion archetypes: Databases for emotion modelling using neural networks," Neural Netw., vol.18, no.4, pp. 371-388, 2005.
20. E. Douglas-Cowie, R. Cowie, I. Sneddon, C. Cox, O. Lowry, M. McRorie, J.-C. Martin, L. Devillers, S. Abrilian, A. Batliner, N. Amir, and K. Karpouzis, "The humaine database: Addressing the collection and annotation of naturalistic and induced emotional data," in Proc. 2nd Int. Conf. Affective Computing and Intelligent Interaction (ACII), Lisbon, Portugal, 2007, pp. 488-500.
21. M. Grimm, K. Kroschel, and S. Narayanan, "The vera am mittag German audio-visual emotional speech database," in 2008 IEEE Int. Conf. Multimedia and Expo., Apr. 23-26, 2008, pp. 865-868.
22. E. Douglas-Cowie, N. Campbell, R. Cowie, and P. Roach, "Emotional speech: Towards a newgeneration of databases," Speech Commun., vol.40, no.1-2, pp. 33-60, 2003.
23. J. C. Martin, G. Caridakis, L. Devillers, K. Karpouzis, and S. Abrilian, "Manual annotation and automatic image processing of multimodal emotional behaviors," Pers. Ubiq. Comput., vol.13, no.1, pp. 69-76, 2009.
24. U. Türk, The Technical Processing in Smartkom Data Collection: A Case Study, LMU Munich, Jul. 2001, Tech. Rep.
25. A. Zara, V. Maffiolo, J.-C. Martin, and L. Devillers, "Collection and annotation of a corpus of human-human multimodal interactions: Emotion and others anthropomorphic characteristics," in Proc. ACII, 2007, pp. 464-475.
26. T. Bänziger and K. R. Scherer, "Using actor portrayals to systematically study multimodal emotion expression: The gemep corpus," in Proc. ACII, 2007, pp. 476-487.
27. R. Banse and K. R. Scherer, "Acoustic profiles in vocal emotion expression," J. Pers. Soc. Psychol., vol.70, pp. 614-636, 1996.
28. L. S.-H. Chen, "Joint processing of audio-visual information for the recognition of emotional expressions in human-computer interaction," Ph.D. dissertation, Univ. Illinois at Urbana-Champaign, Champaign, IL, 2000.
29. P. Ekman and W. Friesen, Facial Action Coding System: A Technique for the Measurement of Facial Movement. Palo Alto, CA: Consulting Psychologists, 1978.
30. C. Busso, M. Bulut, C.-C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. Chang, S. Lee, and S. Narayanan, "Iemocap: Interactive emotional dyadic motion capture database," Lang. Resour. Eval., pp. 335-359, 2008.
31. Y. Cao, W. C. Tien, P. Faloutsos, and F. H. Pighin, "Expressive speech-driven facial animation," ACM Trans. Graph., vol.24, no.4, pp. 1283-1302, 2005.
32. K. Wampler, D. Sasaki, L. Zhang, and Z. Popovic, "Dynamic, expressive speech animation from a single mesh," in Proc. Symp. Computer Animation, 2007, pp. 53-62.
33. L. Yin, X. Chen, Y. Sun, T. Worm, and M. Reale, "A high-resolution 3D dynamic facial expression database," in Proc. FG, 2008.
34. T. Weise, B. Leibe, and L. V. Gool, "Fast 3D scanning with automatic motion compensation," in Proc. IEEE CVPR, 2007.
35. T. Weise, H. Li, L. V. Gool, and M. Pauly, "Face/off: Live facial puppetry," in Proc. SCA, 2009.
36. M. Schröder, "Expressive speech synthesis: Past, present, and possible futures," in Affective Information Processing. New York: Springer, 2008.
37. H. Romsdorfer, "Polyglot text-to-speech synthesis. Text analysis and prosody control," Ph.D. dissertation, Comput. Eng. Netw. Lab., ETH Zurich (TIK-Schriftenreihe Nr. 101), Zurich, Switzerland, Jan. 2009, No.18210.