Creating a speech enabled avatar from a single photograph

Proceedings - IEEE Virtual Reality

Volumn , Issue , 2008, Pages 107-110

  Bitouk, Dmitri ^a   Nayar, Shree K ^a  

^a Och Spine at New York Presbyterian Hospitals

Author keywords

H.5.2 information interfaces and presentation : Multimedia information systems animations; I.3.7 computer graphics : Three dimensional graphics and realism animation

Indexed keywords

CO-ARTICULATION; FACE GEOMETRY; FACIAL ANIMATION; FACIAL MOTIONS; H.5.2 [INFORMATION INTERFACES AND PRESENTATION]: MULTIMEDIA INFORMATION SYSTEMS-ANIMATIONS; I.3.7 [COMPUTER GRAPHICS]: THREE-DIMENSIONAL GRAPHICS AND REALISM-ANIMATION; SINGLE IMAGES;

ANIMATION; FACE RECOGNITION; IMAGING TECHNIQUES; MOTION ESTIMATION; PHOTOGRAPHY; VIRTUAL REALITY;

SPEECH;

EID: 50249153651     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/VR.2008.4480758     Document Type: Conference Paper

References (22)

1. Crazy talk. http://www.reallusion.com/crazytalk.
2. Motion portrait, http://www.motionportrait.com.
3. D. Bitouk. Head-pose and Illumination Invariant 3-D Audio-Visual Speech Recognition. PhD thesis, The Johns Hopkins University, 2006.
4. D. Bitouk and S. Nayar. Speech Enabled Avatar from a Single Photograph. Technical report, CUCS-045-07, Department of Computer Science, Columbia University, 2007.
5. V. Blanz, C. Basso, T. Poggio, and T. Vetter. Reanimating Faces in Images and Video. In Eurographics, volume 22 (3), pages 641-650, 2003.
6. F. Bookstein. Principal Warps: Thin Plate Splines and the Decomposition of Deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence. 11:567-585 1989.
7. M. Brand. Voice Puppetry. In SIGGRAPH. pages 21-28, 1999.
8. C. Bregler. M. Covell, and M. Slaney. Video Rewrite: Driving Visual Speech with Audio. Computer Graphics, 31:353-360, 1997.
9. Y. Cao, P. Faloutsos. E. Kohler and F. Pighin. Real-time Speech Motion Synthesis from Recorded Motions. In Eurographics Symposium on Computer Animation, pages 347-355, 2004.
10. A. Efros and T. K. Leung. Texture Synthesis by Non-parametric Sampling. In IEEE International Conference on Computer Vision, volume 2, pages 1033-1038. 1999.
11. T. Ezzat, G. Geiger, and T. Poggio. Trainable Videorealistic Speech Animation. ACM Transactions on Graphics, 21(3):388-398. 2002.
12. K. Grant, V. van Wassenhoveb, and D. Poeppelb. Detection of Auditory (Cross-Spectral) and Auditory-Visual (Cross-Modal) Synchrony. Speech Communication. 44:43-54, 2004.
13. C. Kimme. D. Ballard, and J. Slansky. Finding Circles by an Array of Accumulators. Communications of the ACM, 18(2): 120-122, 1975.
14. S. P. Lee, J. P. Badler, and N. I. Badler. Eyes Alive. ACM Transactions on Graphics, 21 (3):637-644, 2002.
15. T. Masuko. T. Kobayashi, M. Tamura, J. Masubuchi, and K. Tokuda. Text-to-Visual Speech Synthesis Based on Parameter Generation from hmm. In IEEE International Conference of Acoustics, Speech and Signal Processing, volume 6, pages 3745-3748, 1998.
16. P. Muller, G. Kalberer, M. Proesmans, and L. V. Gool. Realistic Speech Animation Based on Observed 3D Face Dynamics. Vision, Image & Signal Processing, 152:491-500, 2005.
17. F.I.Parke. Computer Generated Animation of Faces. In ACM National Conference, volume 1, pages 451-457, 1972.
18. L. Rabiner and B. Juang. Fundamentals of Speech Recognition. Prentice Hall, 1993.
19. D. Terzopoulos and K. Waters. Analysis and Synthesis of Facial Image Sequences using Physical and Anatomical Models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15:569-579, 1993.
20. P. Viola and M. Jones. Rapid Object Detection using a Boosted Cascade of Simple Features. In Computer Vision and Pattern Recognition, pages 511-518, 2001.
21. K. Wampler, D. Sasaki, L. Zhang, and Z. Popovic. Dynamic, Expressive Speech Animation from a Single Mesh. Eurographics Symposium on Computer Animation, pages 53-62, 2007.
22. S. Young, J. Jansen, J. Odell, D. Ollason, and P. Woodland. The HTK Book. Cambridge University Press, 2005.