3D lip shapes from video: A combined physical-statistical model

Speech Communication

Volumn 26, Issue 1-2, 1998, Pages 131-148

  Basu, Sumit ^a   Oliver, Nuria ^a   Pentland, Alex ^a  

^a MIT MEDIA LAB   (United States)

Author keywords

Analysis synthesis models; Deformable non rigid models; Finite element models; Lip models; Model based tracking; Training models from video

Indexed keywords

FINITE ELEMENT METHOD; MATHEMATICAL MODELS; SPEECH ANALYSIS; SPEECH SYNTHESIS; STATISTICAL METHODS; VIDEO SIGNAL PROCESSING;

AUDIOVISUAL SPEECH SYNTHESIS;

SPEECH COMMUNICATION;

EID: 0032180836     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0167-6393(98)00055-7     Document Type: Article

References (18)

1. Adjoudani, A., Benoît, C., 1995. On the integration of auditory and visual parameters in an HMM-based ASR. In: Stork, D., Hennecke, M. (Eds.), Speechreading by Man and Machine, Springer, Berlin, pp. 461-472.
2. Basu, S. 1997. A three-dimensional model of human lip motion. Master's Thesis, MIT Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge.
3. Basu, S., Essa, I., Pentland, A., 1996. Motion regularization for model-based head tracking. In: Proc. 13th Internat. Conf. on Pattern Recognition, Vienna, 25-29 August 1996. IEEE Computer Society Press, Los Alamitos, Vol. C, pp. 611-616.
4. Basu, S., Oliver, N., Pentland, A., 1998. 3D modeling and tracking of human lips. In: Proc. Internat. Conf. on Computer Vision, Bombay, 4-7 January 1998. IEEE Computer Society Press, Los Alamitos, pp. 337-343.
5. Bathe, K., 1982. Finite Element Procedures in Engineering Analysis, Prentice-Hall, Englewood Cliffs.
6. Bregler, C., Omohundro, S., 1995. Nonlinear image interpolation using manifold learning. In: Tesauro, G., Touretzky, D., Leen, T. (Eds.), Advances in Neural Information Processing Systems, Vol. 7. MIT Press, Cambridge, pp. 401-408.
7. Coianiz, T., Torresani, L., Caprile, B., 1995. 2D deformable models for visual speech analysis. In: Stork, D., Hennecke, M. (Eds.), Speechreading by Man and Machine. Springer, Berlin, pp. 391-398.
8. Essa, I., 1995. Analysis and interpretation and synthesis of facial expressions. Ph.D. Thesis, MIT Department of Media Arts and Sciences, Massachusetts Institute of Technology, Cambridge.
9. Gelb, A. (Ed.), 1974. Applied Optimal Estimation, MIT Press, Cambridge.
10. Jebara, T., Pentland, A., 1997. Parametrized structure from motion for 3D adaptive feedback tracking of faces, In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition '97, San Juan, 17-19 June 1997. IEEE Computer Society Press, Los Alamitos, pp. 144-150.
11. Kass, M., Witkin, A., Terzopoulos, D., 1988. Snakes: Active contour models. Internat. J. Computer Vision 1 (4), 321-331.
12. Lee, Y., Terzopoulos, D., Waters, K., 1995. Realistic modeling for facial animation. In: Proc. SIGGRAPH'95, Orlando, 6-11 August 1995. Addison-Wesley, Reading, MA, pp. 55-62.
13. Luettin, J., Thacker, N., Beet, S., 1996. Visual speech recognition using active shape models and hidden Markov models. In: Proc. Internat. Conf. on Acoustics Speech and Signal Processing 1996, Atlanta. IEEE Press, New York, pp. 817-820.
14. Martin, J., Pentland, A., Sclaroff, S., Kikinis, R., 1998. Characterization of neuropathological shape deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence 30 (2), 97-112.
15. Oliver, N., Pentland, A., Bérard, F., 1997. LAFTER: lips and face real time tracker. In: Proc. IEEE Conf. on Computer Vision and Pattern Recognition '97, San Juan, 17-19 June 1997. IEEE Computer Society Press, Los Alamitos, pp. 123-129.
16. Summerfield, Q., 1979. Use of visual information for phonetic perception. Phonetica 36, 314-331.
17. Waters, K., Frisbie, J., 1995. A coordinated muscle model for speech animation. In: Davis, W., Prusinkiewicz, P. (Eds.), Graphics Interface '95. Canadian Human-Computer Communications Society, Ontario, pp. 163-170.
18. Zienkiewicz, O.C., Cheung Y.K., 1967. The Finite Element Method in Structural and Continuum Mechanics. McGraw-Hill, London.