Speech-to-lip movement synthesis by maximizing audio-visual joint probability based on the EM algorithm

Journal of VLSI Signal Processing Systems for Signal, Image, and Video Technology

Volumn 27, Issue 1-2, 2001, Pages 119-126

  Nakamura, Satoshi ^a   Yamamoto, Eli ^b  

^a ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

^b WAKAYAMA UNIVERSITY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; DECODING; MARKOV PROCESSES; PARAMETER ESTIMATION; SPEECH RECOGNITION;

AUDIO-VISUAL JOINT PROBABILITY; EXPECTATION-MAXIMIZATION (EM) ALGORITHMS; SPEECH-TO-LIP MOVEMENT SYNTHESIS;

SPEECH SYNTHESIS;

EID: 0035251712     PISSN: 09225773     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1008179732362     Document Type: Article

References (15)

1. S. Morishima and H. Harashima, "A Media Conversion from Speech to Facial Image for Intelligent Man-Machine Interface," IEEE Journal on Sel. Areas in Communications, vol. 9, no. 4, 1991, pp. 594-600.
2. F. Lavagetto, "Converting Speech into Lip Movements: A Multimedia Telephone for Hard of Hearing People," IEEE Trans. on Rehabilitation Engineering, vol. 3, no. 1, 1995, pp. 90-102.
3. R.R. Rao and T. Chen, "Cross-Modal Prediction in Audio-Visual Communication," in Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, 1996, vol. 4, pp. 2056-2059.
4. W. Goldenthal, K. Waters, J.M. Van Thong, and O. Glickman, "Driving Synthetic Mouth Gestures: Phonetic Recognition for FaceMe!," in Eurospeech'97 Proceedings, 1997, vol. 4, pp. 1995-1998.
5. A. Simons and S. Cox, "Generation of Mouthshape for a Synthetic Talking Head," in Proc. of the Institute of Acoustics, 1990, vol. 12, no. 10.
6. W. Chou and H. Chen, "Speech Recognition for Image Animation and Coding," in ICASSP 95, 1995, pp. 2253-2256.
7. T. Chen and R. Rao, "Audio-Visual Interaction in Multimedia Communication," in ICASSP 97, 1997, pp. 179-182.
8. E. Yamamoto, S. Nakamura, and K. Shikano, "Speech-to-Lip Movement Synthesis by HMM," in ESCA Workshop of Audio Visual Speech Processing, 1997, pp. 137-140.
9. E. Yamamoto, S. Nakamura, and K. Shikano, "Lip Movement Synthesis from Speech Based on Hidden Markov Model," Speech Communication, vol. 26, 1998, pp. 105-115.
10. L.E. Baum, "An Inequality and Associated Maximization Technique in Statistical Estimation for Probabilistic Functions of Markov Processes," Inequalities, vol. 3, pp. 1-8.
11. T. Guiard-Marigny, T. Adjoudani, and C. Benoit, "3D Models of the Lips and Jaw for Visual Speech Synthesis," in Progress in Speech Synthesis, J. van Santen et al. (Eds.), Berlin: Springer-Verlag, 1996.
12. B.E. Walden, R.A. Prosek, A.A. Montogomery, C.K. Scherr, and C.J. Jones, "Effects of Training on the Visual Recognition of Consonants," Journal of Speech and Hearing Research, vol. 20, pp. 130-145.
13. K. Sekiyama, K. Joe, and M. Umeda, "Recognition of Japanese Mono-syllables by Lipreading," Institute of Electronics, Information and Communication Engineers, Technical Report, IE87-108, pp. 20-40.
14. J. Ostermann and A. Puri, "Natural and Synthetic Video in MPEG-4," in Proceeding of IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 6, pp. 3805-3808.
15. C. Reed-Ballreich et al., in VRML'98 Symposium, http: //ece.uwaterloo.ca:80/~h-anim/.