Data-driven speech animation synthesis focusing on realistic inside of the mouth

Journal of Information Processing

Volumn 22, Issue 2, 2014, Pages 401-409

  Kawai, Masahide ^a   Iwao, Tomoyori ^a   Mima, Daisuke ^a   Maejima, Akinobu ^a   Morishima, Shigeo ^a  

^a WASEDA UNIVERSITY   (Japan)

Author keywords

Detai lization; Inner mouth; Phoneme combination; Skull bone; Speech animation

Indexed keywords

EID: 84898663109     PISSN: 03875806     EISSN: 18826652     Source Type: Journal    
DOI: 10.2197/ipsjjip.22.401     Document Type: Article

References (27)

1. Alexander, O., Roger, M., Lambeth, W., Chiang, J.-Y., Ma, W.-C., Wang, C.-C. and Debevec, P.: The Digital Emily Project: Achieving a Photorealistic Digital Actor, IEEE Trans. CGA, pp.20-31 (2010).
2. Joshi, P., Tien, W.C., Desbrun, M. and Pighin, F.: Learning Contrals for Blend Shape Based Realistic Facial Animation, Proc. SCA'03, pp.187-192 (2003).
3. Tena, J.R., Torre, F.D. and Matthews, I.: Interactive Region-Based Linear 3D Face Models, Proc. SIGGRAPH'11, No.76 (2011).
4. Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Sumner, R.W. and Gross, M.: High-Quality Passive Facial Performance Capture using Anchor Frames, Proc. SIGGRAPH'11, No.75 (2011).
5. Seol, Y., Lewis, J.P., Seo, J., Choi, B., Anjyo, K. and Noh, J.: Spacetime Expression Cloning for Blendshapes, TOG'12 Journals, Vol.32 (Issue 2), No.14 (2012).
6. Deng, Z. and Neumann, U.: Expressive Speech Animation Synthesis with Phoneme-Level Controls, Computer Graphics Forum, Vol.27, No.8, pp.2096-2113 (2008).
7. Perez, P., Gangnet, M. and Blake, A.: Expressive Facial Animation Synthesis by Learning Speech Coarticulation and Expression Spaces, IEEE Trans. Visualization and Computer Graphics, Vol.12, No.6, pp.1523-1534 (2006).
8. Bregler, C., Covell, M. and Slaney, M.: Video Rewrite: Driving visual speech with audio, Proc. SIGGRAPH'97, pp.353-360 (1997).
9. Buck, I., Finkelstein, A., Jacobs, C., Klein, A., Salesin, H.A., Seims, J., Szeliski, R. and Toyama, K.: Performance-Driven Hand-Drawn Animation, Proc. 1st International Symposium on Non Photorealistic Animation and Rendering, pp.101-108 (2000).
10. Cosatto, E. and Graf, H.: Photo-realistic Talking-heads from Image Samples, IEEE Trans. Multimedia, Vol.2, No.3, pp.152-163 (2000).
11. Ezzat, T., Geiger, G. and POGGIO, T.: Trainable Videorealistic Speech Animation, Proc. SIGGRAPH'02, pp.388-398 (2002).
12. Chang, Y. and Ezzat, T.: Transferable Videorealistic Speech Animation, Proc. SCA'05, pp.143-151 (2005).
13. Mohammed, U., Prince, S.J.D. and Kautz, J.: Visio-lization: generating novel facial images, Proc. SIGGRAPH'09, No.57 (2009).
14. Perez, P., Gangnet, M. and Blake, A.: Poisson Image Editing, Proc. SIGGRAPH'03, pp.313-318 (2003).
15. Taylor, S.L., Mahler, M., Theobald, B.-J. and Matthews, I.: Dynamic Units of Visual Speech, Proc. SCA'12, pp.275-284 (2012).
16. Li, H., Weise, T. and Pauly, M.: Example-Based Facial Rigging, Proc. SIGGRAPH'10, No.32 (2010).
17. Huang, H., Chai, J., Tong, X. and Wu, H.-T.: Leveraging motion capture and 3D scanning for high-fidelity facial performance acquisition, Proc. SIGGRAPH'11, No.74 (2011).
18. Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfiste, H. and Gross, M.: Multi-scale capture of facial geometry and motion, Proc. SIGGRAPH'07, No.33 (2007).
19. King, A.S. and Parent, E.R.: A 3D Parametric Tongue Model for Animated Speech, The Journal of Visualization and Computer Animation, Vol.12, No.3, pp.107-115 (2001).
20. Vogt, F., Lloyd, E.J., Buchaillard, S., Perrier, P., Chabanas, M., Payan, Y. and Fels, S.S.: Efficient 3D Finite Element Modeling of a Muscle- Activated Tongue, The 3rd International Conference on Biomedical Simulation, pp.19-28 (2006).
21. Yang, Y., Guo, X., Vick, J., Torres, G.L. and Champbell, T.: Physics- Based Deformable Tongue Visualization, IEEE Trans. Visualization and Computer Graphics, Vol.19, No.5, pp.811-823 (2013).
22. Pelachaud, C., Overveld, C.W.A.M.V. and Seah, C.: Modeling and Animating the Human Tongue during Speech Production, Computer Animation'94, pp.40-49 (1994).
23. Comaniciu, D. and Meer, P.: Mean shift: A robust approach toward feature space analysis, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol.24, No.5, pp.603-619 (2002).
24. Irie, A., Takagiwa, M., Moriyama, K. and Yamashita, T.: Improvements to Facial Contour Detection by Hierarchical Fitting and Regression, The 1st Asian Conference on Pattern Recognition, pp.273-277 (2011).
25. Blanz, V., Basso, C., Poddio, T. and Vetter, T.: Reanimating faces in images and video, Eurographics'03, Vol.22 (2003).
26. Gibbon, F.E., Lee, A. and Yuen, I.: Tongue-Palate Contact During Selected Vowels in Normal Speech, The Cleft Palate-Craniofacial Journal, Vol.47, No.4, pp.405-412 (2010).
27. Stone, M., Morrish, K.A., Sonies, B.C. and Shawker, T.H.: Tongue curvature: A model of shape during vowel production, Folia Phoniat, Vol.39, pp.302-315 (1987).