Visual speech synthesis by modelling coarticulation dynamics using a non-parametric switching state-space model

International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction, ICMI-MLMI 2010

Volumn , Issue , 2010, Pages

  Deena, Salil ^a   Hou, Shaobo ^a   Galata, Aphrodite ^a  

^a UNIVERSITY OF MANCHESTER   (United Kingdom)

Author keywords

artificial talking head; speech driven facial animation; visual speech synthesis

Indexed keywords

CO-ARTICULATION; DYNAMICAL MODEL; FACIAL ANIMATION; GAUSSIAN PROCESSES; NON-PARAMETRIC; STATE SPACE MODEL; STATE-SPACE MODELS; SYNTHESIS TECHNIQUES; TALKING HEADS; VARIABLE-LENGTH MARKOV MODELS; VISUAL DATA; VISUAL SPEECH SYNTHESIS;

ANIMATION; GAUSSIAN DISTRIBUTION; GAUSSIAN NOISE (ELECTRONIC); LEARNING SYSTEMS; LINGUISTICS; MARKOV PROCESSES; SPEECH SYNTHESIS; STATE SPACE METHODS; SWITCHING;

INTERACTIVE COMPUTER SYSTEMS;

EID: 78650937887     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1891903.1891942     Document Type: Conference Paper

References (27)

1. BRAND, M. Voice puppetry. In SIGGRAPH '99: Proc. of the ACM Conference on Computer Graphics and Interactive Techniques (1999).
2. BREGLER, C., COVELL, M., AND SLANEY, M. Video rewrite: driving visual speech with audio. In SIGGRAPH '97: Proc. of the ACM Conference on Computer Graphics and Interactive Techniques (1997).
3. CAO, Y., FALOUTSOS, P., KOHLER, E., AND PIGHIN, F. Real-time speech motion synthesis from recorded motions. In SCA '04: Proc. of the ACM SIGGRAPH/Eurographics symposium on Computer animation (2004).
4. CHEN, J., KIM, M., WANG, Y., AND JI, Q. Switching Gaussian process dynamic models for simultaneous composite motion tracking and recognition. In CVPR'09: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (2009), pp. 2655-2662.
5. COOTES, T. F., EDWARDS, G. J., AND TAYLOR, C. J. Active Appearance Models. IEEE PAMI 23, 6 (2001), 681-685.
6. DEENA, S., AND GALATA, A. Speech-driven facial animation using a shared Gaussian process latent variable model. In ISVC'09: Proc. of the 5th International Symposium on Visual Computing (2009).
7. DIGALAKIS, O., OSTENDORF, M., AND DIGALAKIS, V. The stochastic segment model for continuous speech recognition. In Proc. of the Asilomar Conference on Signals, Systems and Computers (1991), pp. 964-968.
8. EK, C. H., RIHAN, J., TORR, P. H. S., ROGEZ, G., AND LAWRENCE, N. D. Ambiguity modeling in latent spaces. In MLMI'08: Proc. 5th International Workshop on Machine Learning for Multimodal Interaction (2008).
9. EK, C. H., TORR, P. H. S., AND LAWRENCE, N. D. Gaussian process latent variable models for human pose estimation. In MLMI'07: Proc of the 4th International Workshop on Machine Learning for Multimodal Interaction (2007).
10. ENGLEBIENNE, G., COOTES, T. F., AND RATTRAY, M. A probabilistic model for generating realistic lip movements from speech. In NIPS'07: Advances in Neural Information Processing Systems (2007).
11. EZZAT, T., GEIGER, G., AND POGGIO, T. Trainable videorealistic speech animation. In SIGGRAPH '02: Proc. of the ACM conference on Computer graphics and interactive techniques (2002).
12. GALATA, A., JOHNSON, N., AND HOGG, D. Learning variable length Markov models of behaviour. Computer Vision and Image Understanding 81, 3 (2001), 398-413.
13. Guyon, I., and Pereira, F. Design of a linguistic postprocessor using variable memory length Markov models. In ICDAR'95: Proc. of IEEE International Conference on Document Analysis and Recognition (1995), pp. 454-457.
14. HUANG, X., ACERO, A., AND HON, H.-W. Spoken Language Processing: A Guide to Theory, Algorithm and System Development. Prentice Hall PTR, 2001.
15. JONES, M. J., AND POGGIO, T. Multidimensional morphable models. In ICCV '98: Proc. of IEEE the International Conference on Computer Vision (1998).
16. LAWRENCE, N. D. Probabilistic non-linear principal component analysis with Gaussian process latent variable models. JMLR 6 (2005), 1783-1816.
17. LEHN-SCHIØLER, T., HANSEN, L. K., AND LARSEN, J. Mapping from speech to images using continuous state space models. In MLMI'04: Proc. of 1st International Workshop on Machine Learning for Multimodal Interaction (2005).
18. LINDBLOM, B. Speech Production and Speech Modeling. Kluwer Academic, Dordrecht, 1990, ch. Explaining phonetic variation: A sketch of the H&H theory, pp. 403-439.
19. MCGURK, H., AND MACDONALD. Hearing lips and seeing voices. Nature 264 (1976), 746-748.
20. RON, D., SINGER, Y., AND TISHBY, N. The power of amnesia: Learning probabilistic automata with variable memory length. Machine Learning 25 (1996), 117-149.
21. THEOBALD, B.-J. Visual speech synthesis using shape and appearance models. PhD thesis, School of Information Systems, University of East Anglia, 2003.
22. THEOBALD, B.-J., FAGEL, S., BAILLY, G., AND ELISEI, F. LIPS2008: Visual speech synthesis challenge. In Proc. of Interspeech (2008).
23. THEOBALD, B.-J., AND WILKINSON, N. A real-time speech-driven talking head using Active Appearance Models. In AVSP'07: Proc. of the International Conference on Auditory-Visual Speech Processing (2007).
24. URTASUN, R., FLEET, D. J., AND FUA, P. 3D people tracking with Gaussian process dynamical models. In CVPR'06: Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (2006), pp. 238-245.
25. WANG, J. M., FLEET, D. J., AND HERTZMANN, A. Gaussian process dynamical models for human motion. IEEE PAMI 30, 2 (2008), 283-298.
26. XIE, L., AND LIU, Z.-Q. A coupled HMM approach to video-realistic speech animation. Pattern Recognition 40, 8 (2007), 2325-2340.
27. YOUNG, S. The HTK Hidden Markov Model Toolkit: Design and philosophy. Tech. rep., University of Cambridge, Department of Engineering, 1993.