Animated Lombard speech: Motion capture, facial animation and visual intelligibility of speech produced in adverse conditions

Computer Speech and Language

Volumn 28, Issue 2, 2014, Pages 607-618

  Alexanderson, Simon ^a   Beskow, Jonas ^a  

^a ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

Author keywords

Audio visual intelligibility; Facial animation; Lip reading; Lombard effect; Motion capture; Speech reading

Indexed keywords

AUDIO-VISUAL; FACIAL ANIMATION; LIP-READING; LOMBARD EFFECTS; MOTION CAPTURE; SPEECH-READING;

ANIMATION; AUDIO ACOUSTICS; DIGITAL STORAGE; SPEECH RECOGNITION; THREE DIMENSIONAL COMPUTER GRAPHICS;

SPEECH INTELLIGIBILITY;

EID: 84890567121     PISSN: 08852308     EISSN: 10958363     Source Type: Journal    
DOI: 10.1016/j.csl.2013.02.005     Document Type: Article

References (18)

1. S. Al Moubayed, J. Beskow, and B. Granström Auditory-visual prominence: from intelligibility to behavior Journal on Multimodal User Interfaces 3 4 2010 299 311
2. G. Bailly, M. Bérar, F. Elisei, and M. Odisio Audiovisual speech synthesis International Journal of Speech Technology 6 2003 331 346
3. R. Campbell, and T.J.E. Mohammed Speechreading for information gathering: a survey of scientific sources 2010 Deafness Cognition and Language (DCAL) Research Centre Division of Psychology and Language Sciences University College London
4. B. Choe, H. Lee, and H.-S. Ko Performance-driven muscle-based facial animation The Journal of Visualization and Computer Animation 12 2 2001 67 79
5. Z. Deng, and J. Noh Computer facial animation: a survey Z. Deng, U. Neumann, Data-driven 3D facial animation 2007 Springer London 1 28
6. O. Engwall, and P. Wik Can you tell if tongue movements are real or synthesized? Proceedings of Auditory-Visual Speech Processing AVSP 2009 Norwich, England 2009 pp. 96-101
7. M. Fitzpatrick, J. Kim, and C. Davis The effect of seeing the interlocutor on auditory and visual speech production in noise In AVSP-2011 2011 pp. 31-35
8. J. Kim, A. Sironic, and C. Davis Hearing speech in noise: seeing a loud talker is better Perception 40 7 2011 853 862
9. B. Lindblom Explaining phonetic variation: a sketch of the H & H theory W.J. Hardcastle, A. Marchal, Speech production and speech modelling 1990 Kluwer Academic Publishers Dordrecht 403 439
10. H. McGurk, and J. MacDonald Hearing lips and seeing voices Nature 264 1976 746
11. T. Öhman An audio-visual speech database and automatic measurements of visual speech TMH-QPSR, KTH 1-2 1998 61 76
12. F. Pighin, and J.P. Lewis Facial motion retargeting. ACM SIGGRAPH 2006 Courses 2006 ACM Boston
13. T. Raitio, A. Suni, J. Yamagishi, H. Pulakka, J. Nurminen, M. Vainio, and P. Alku HMM-based speech synthesis utilizing glottal inverse filtering IEEE Transactions on Audio, Speech, and Language Processing 19 1 2011 153 165
14. G. Salvi, J. Beskow, S. Al Moubayed, and B. Granström SynFace - speech-driven facial animation for virtual speech-reading support EURASIP Journal on Audio, Speech, and Music Processing 2009 I 2009 191940, http://dx.doi.org/10.1155/2009/191940
15. K. Sjölander An HMM-based system for automatic segmentation and alignment of speech Proceedings of Fonetik 2003 2003 pp. 93-96
16. Q. Summerfield Lipreading and audio-visual speech perception Philosophical Transactions: Biological Sciences 335 1273 1992 71 78
17. E. Vatikiotis-Bateson, A.V. Barbosa, C.Y. Chow, M. Oberg, J. Tan, and H.C. Yehia Audiovisual Lombard speech: reconciling production and perception J. Vroomen, M. Swerts, E. Krahmer, Proceedings of AVSP 2007: International Conference on Auditory Visual Speech Processing 2007 Kasteel Groenendaal Hilvarenbeek, The Netherlands 41 45 31 August-3 September
18. E. Vatikiotis-Bateson, V. Chung, K. Lutz, N. Mirante, J. Otten, and J. Tan Auditory, but perhaps not visual, processing of Lombard speech Journal of the Acoustical Society of America 119 4 2006 3444