Real-time language independent lip synchronization method using a genetic algorithm

Signal Processing

Volumn 86, Issue 12, 2006, Pages 3644-3656

  Zorić, Goranka ^a   Pandžić, Igor S ^a  

^a UNIVERSITY OF ZAGREB   (Croatia)

Author keywords

Facial animation; Genetic algorithms; Human computer interaction; Lip sync; Lip synchronization; MPEG 4 FBA; Neural networks; Speech processing; Virtual characters

Indexed keywords

FACIAL ANIMATION; LIP SYNCHRONIZATION; VIRTUAL CHARACTERS;

ANIMATION; GENETIC ALGORITHMS; HUMAN COMPUTER INTERACTION; NEURAL NETWORKS; SPEECH PROCESSING; VIRTUAL REALITY;

SPEECH RECOGNITION;

EID: 33749429018     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2006.02.038     Document Type: Article

References (24)

1. Chen T., and Rao R. Audio-visual integration in multimodal communication. Proceedings of IEEE, Special Issue on Multimedia Signal Processing (1998) 837-852
2. D.F. McAllister, R.D. Rodman, D.L. Bitzer, A.S. Freeman, Lip synchronization for animation, Proceedings of SIGGRAPH 97, Los Angeles, CA, 1997.
3. Visage Technologies, www.visagetechnologies.com
4. F.J. Huang, T. Chen, Real-time lip-synch face animation driven by human voice, Proceedings of IEEE Multimedia Signal Processing Workshop, Los Angeles, CA, 1998.
5. In: Pandžić S., and Forchheimer R. (Eds). MPEG-4 Facial Animation-The Standard, Implementation and Applications (2002), Wiley, New York
6. Y. Li, F. Yu, Y. Xu, E. Chang, H. Shum, Speech-driven cartoon animation with emotions, Proceedings of the ninth ACM international conference on Multimedia, Ottawa, Canada, 2001.
7. S. Curinga, F. Lavagetto, F. Vignoli, Lip movements sythesis using Time Delay, Proceedings of EUSIPCO-96, Trieste, 1996.
8. Hong P., Wen Z., and Huang T.S. Real-time speech-driven face animation. In: Pandžić S., and Forchheimer R. (Eds). MPEG-4 Facial Animation-The Standard, Implementation and Applications (2002), Wiley, New York (Chapter 7)
9. P. Hong, Z. Wen, T.S. Huang, Real-time speech driven avatar with constant short time delay, Proceedings of International Conference on Augmented, Virtual Environments and 3D Imaging, Greece, 2001.
10. D.W. Massaro, J. Beskow, M.M. Cohen, C.L. Fry, T. Rodriguez, Picture my voice: Audio to visual speech synthesis using artificial neural networks, Proceedings of AVSP'99, Santa Cruz, CA, 1999.
11. Y. Huang, X. Ding, B. Guo, H. Shum, Real-time face synthesis driven by voice, Proceedings of Computer-Aided Design and Computer Graphics, Kunming, PRC, 2001.
12. M. Brand, Voice Puppetry, Proceedings of SIGGRAPH'99, 1999.
13. M. Tamura, T. Masuko, T. Kobayashi, K. Tokuda, Visual speech synthesis based on parameter generation from HMM: Speech-driven and text-and-speech driven approaches, Proceedings of Auditory-Visual Speech Processing AVSP'98, 1998.
14. Lewis. Automated Lip-Sync: background and techniques. Proc J. Visual. Comput. Anim. 2 (1991)
15. S. Kshirsagar, N. Magnenat-Thalmann, Lip synchronization using linear predictive analysis, Proceedings of IEEE International Conference on Multimedia and Expo, New York, 2000.
16. S. Kshirsagar, N. Magnenat-Thalmann, Viseme space for realistic speech animation, Proceedings of Auditory-Visual Speech Processing AVSP'01, 2001.
17. Axelsson, E. Björhall, Real time speech driven face animation, Master Thesis at The Image Coding Group, Department of Electrical Engineering at Linköping University, Linköping, 2003.
18. Fisher, Fisher linear discriminant and dataset transformation, http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/FISHER/FLD/fld.html
19. J.J. Dávila, Genetic optimization of neural networks for the task of natural language processing, dissertation, New York, 1999.
20. Rojas R. Neural networks, A Systematic Introduction (1996), Springer, Berlin Heidelberg
21. Jones J. Genetic algorithms and their applications to the design of neural networks. Neural Comput. Appl. 1 1 (1993) 32-45
22. Black Box Genetic algorithm, http://fdtd.rice.edu/GA/
23. Mathworks documentation for NNs, http://www.mathworks.com/access/helpdesk/help/toolbox/nnet/
24. 3ds max, http://www4.discreet.com/3dsmax/