Natural head motion synthesis driven by acoustic prosodic features

Computer Animation and Virtual Worlds

Volumn 16, Issue 3-4, 2005, Pages 283-290

  Busso, Carlos ^a,b,c   Deng, Zhigang ^a,b   Neumann, Ulrich ^d,e   Narayanan, Shrikanth ^a,b,d,f,g,h  

^a University of Southern California   (United States)

^b IEEE   (United States)

^c Div of Biokinesiology and Physical Therapy   (United States)

^d UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^e Department of Electrical Engineering

^f Signal and Image Processing Institute   (United States)

^g ACOUSTICAL SOCIETY OF AMERICA   (United States)

^h Acoustical Society of America ^*

Author keywords

Data driven; Facial animation; Head motion synthesis; HMM; Prosody; Spherical cubic interpolation

Indexed keywords

DATA DRIVEN; FACIAL ANIMATION; HEAD MOTION SYNTHESIS; HIDDEN MARKOV MODELS (HMM); PROSODY; SPHERICAL CUBIC INTERPOLATION;

ACOUSTIC WAVE PROPAGATION; ANIMATION; DATA REDUCTION; HUMAN COMPUTER INTERACTION; MARKOV PROCESSES; SAMPLING; VECTOR QUANTIZATION;

MOTION PLANNING;

EID: 27144506606     PISSN: 15464261     EISSN: None     Source Type: Journal    
DOI: 10.1002/cav.80     Document Type: Conference Paper

References (27)

1. Cohen MM, Massaro DW. Modeling coarticulation in synthetic visual speech. Models and Techniques in Computer Animation: Magnenat-Thalmann N, Thalmann D (eds). Springer Verlag: Tokyo, Japan, 1993; 139-156.
2. Bregler C, Covell M, Slaney M. Video rewrite: driving visual speech with audio. Proceedings of ACM SIGGRAPH'97, 1997; pp. 353-360.
3. Brand M. Voice puppetry. Proceedings of ACM SIGGRAPH' 99, 1999; pp. 21-28.
4. Kshirsagar S, Thalmann NM. Visyllable based speech animation. Computer Graphics Forum (Proceedings of Eurographics'03) 2003; 22(3): 631-939.
5. Ezzat T, Geiger G, Poggio T. Trainable videorealistic speech animation. ACM Transaction on Graphics (Proceedings of ACM SIGGRAPH'02) 2002; 388-398.
6. Deng Z, Lewis JP, Neumann U. Synthesizing speech animation by learning compact speech co-articulation models. In Computer Graphics International (CGI 2005). Stony Brook: NY, USA, June 2005; pp. 19-25.
7. Liu W, Yin B, Jia X. Audio to visual signal mappings with hmm. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP' 4). Quebec, Canada, May 2004; pp. 885-888.
8. Costa M, Chen T, Lavagetto F. Visual prosody analysis for realistic motion synthesis of 3d head models. In International Conference On Augmented, Virtual Environments and Three Dimensional Imaging, ICAV3D, Ornos, Mykonos, Greece, May-June 2001.
9. Kuratate T, Munhall KG, Rubin PE, Bateson EV, Yehia H. Audio-visual synthesis of talking faces from speech production correlates. In Sixth European Conference on Speech Communication and Technology, Eurospeech 1999. Budapest, Hungary, September 1999; pp. 1279-1282.
10. Munhall KG, Jones JA, Callan DE, Kuratate T, Bateson EV. Visual prosody and speech intelligibility: head movement improves auditory speech perception. Psychological Science 2004; 15(2): 133-137.
11. Yehia H, Kuratate T, Bateson EV. Facial animation and head motion driven by speech acoustics. In 5th Seminar on Speech Production: Models and Data, 2000; pp. 265-268.
12. Pelachaud C, Badler N, Steedman M. Generating facial expressions for speech. Cognitive Science 1996; 20(1): 1-46.
13. Ekman P, Friesen WV. Unmasking the Face: A Guide to Recognizing Emotions from Facial Clues. Prentice-Hall: Englewood Cliffs, New Jersey, USA, 1975.
14. Cassell J, Pelachaud C, Badler N, Steedman M, Achorn B, Bechet T, Douville B, Prevost S, Stone M. Animated conversation: ruled-based generation of facial expression gesture and spoken intonation for multiple conversational agents. In Computer Graphics (Proceedings of ACM SIGGRAPH'94), 1994; pp. 413-420.
15. Graf HP, Cosatto E, Strom V, Huang FJ. Visual prosody: facial movements accompanying speech. In Proceedings of IEEE International Conference on Automatic Faces and Gesture Recognition, 2002.
16. Chuang E, Bregler C. Head emotion. Stanford University Computer Science Technical Report, CS-TR-2003-02, 2003.
17. Deng Z, Busso C, Narayanan S, Neumann U. Audio-based head motion synthesis for avatar-based telepresence systems. In ACM SIGMM 2004 Workshop on Effective Telepresence (ETP 2004). ACM Press: New York, NY, 2004; 24-30.
18. Stegmann MB, Gomez DD. A brief introduction to statistical shape analysis, Informatics and Mathematical Modelling, Technical University of Denmark, 2002.
19. Dehon C, Filzmoser P, Croux C. Robust methods for canonical correlation analysis. In Data Analysis, Classification, and Related Methods. Springer-Verlag: Berlin, 2000; pp. 321-326.
20. Young S, Evermann G, Hain T, et al. The HTK Book. Entropic Cambridge Research Laboratory: Cambridge, England, 2002.
21. Linde Y, Buzo A, Gray R. An algorithm for vector quantizer design. IEEE Transactions on Communications 1980; 28(1): 84-95.
22. Rabiner LR. A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE 1989; 77(2): 257-286.
23. Huang X, Acero A, Hon H-W. Spoken Language Processing: A guide to theory, Algorithm and System Development. Prentice-Hall: Upper Saddle River, New Jersey, USA, 2001.
24. Shoemake K. Animating rotation with quaternion curves. Computer Graphics (Proceedings of SIGGRAPH85) 1985; 19(3): 245-254.
25. Eberly D. 3D Game Engine Design: A Practical Approach to Real-Time Computer Graphics. Morgan Kaufmann Publishers: San Francisco, California, USA, 2000.
26. Deng Z, Bulut M, Neumann U, Narayanan S. Automatic dynamic expression synthesis for speech animation. In IEEE 17th International Conference on Computer Animation and Social Agents (CASA 2004), Geneva, Switzerland, 2004; pp. 267-274.
27. Deng Z, Lewis PJ, Neumann U. Automated eye motion using texture synthesis. IEEE Computer Graphics and Applications 2005; 25(2): 24-30.