HMM-based synthesis of emotional facial expressions during speech in synthetic talking heads

ICMI'06: 8th International Conference on Multimodal Interfaces, Conference Proceeding

Volumn , Issue , 2006, Pages 380-387

  Mana, Nadia ^a   Pianesi, Fabio ^a  

^a ITC IRST   (Italy)

Author keywords

Emotional facial expression modeling; Face synthesis; Hidden Markov models; MPEG4 facial animation; Talking heads

Indexed keywords

HIDDEN MARKOV MODELS; HUMAN COMPUTER INTERACTION; LARGE SCALE SYSTEMS; MOTION PICTURE EXPERTS GROUP STANDARDS; SPEECH SYNTHESIS;

EMOTIONAL FACIAL EXPRESSION MODELING; FACE SYNTHESIS; MPEG4 FACIAL ANIMATION; TALKING HEADS;

GESTURE RECOGNITION;

EID: 34547151545     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1180995.1181065     Document Type: Conference Paper

References (23)

1. Bowman, B., Debray, S. K., and Peterson, L. L. Reasoning about naming systems. ACM Trans. Program. Lang. Syst., 15, 5 (Nov. 1993), 795-825.
2. K. Balci. Xface: Open Source Toolkit for Creating 3D Faces of an Embodied Conversational Agent. In Proceedings of Smart Graphics, 2005.
3. J. Beskow, L. Cerrato, P. Cosi, E. Costantini, M. Nordstrand, F. Pianesi, M. Prete, and G. Svanfeldt. Preliminary Cross-cultural Evaluation of Expressiveness in Synthetic Faces. In E. Andre, L. Dybkiaer, W. Minker, and P. Heisterkamp, editors, Affective Dialogue Systems ADS '04, Springer Verlag, 2004.
4. M. Brand. Voice Puppetry. In Proceedings of ACM SIGGRAPH'99, pp. 21-28, 1999.
5. th Australian Joint Conference on Artificial Intelligence (AI 2001), Adelaide, Australia, December 2001.
6. Cassell J., Bickmore T., Cambell L., Vilhjalmsson H., and Yan H. Human Conversation as a System Framework: Designing Embodied Conversational Agents. In J. Cassel, J. Sullivan, S. Prevost, and E. Churchill, editors, Embodied Conversational Agents, MIT Press, 22-63, 2000.
7. J. Cassell, V. Pelachuad, N. Badler, M. Steedman, B. Achorn, T. Becket, B. Douville, S. Prevost, and M. Stone. ANIMATED CONVERSATION: Rule-based Generation of Facial Expression, Gesture & Spoken Intonation for Multiple Conversational Agents. In Proceeding of SIGGRAPH'94, 1994.
8. I. Cohen, A. Garg, and T. Huang. Emotion recognition from facial expressions using multilevel HMM, 2000.
9. E. Costantini, F. Pianesi, and P. Cosi. Evaluation of Synthetic Faces: Human Recognition of Emotional Facial Displays. In E. Andre, L. Dybkiaer, W. Minker, and P. Heisterkamp, editors, Affective Dialogue Systems ADS '04, Springer-Verlag, 2004.
10. F. deRosis, C Pelachaud, I. Poggi, V. Carofiglio, and B.D. Carolis. From Greta's Mind to her Face: Modelling the Dynamics of Affective States in a Conversational Embodied Agent. International Journal of Human-Computer Studies. Special Issue on Applications of Affective Computing in HCI, 59(1) pp. 81-118, 2003.
11. P. Doenges, F. Lavagetto, J. Ostermann, I.S. Pandzic, and E. Petajan. MPEG-4: Audio/Video and Synthetic Graphics/Audio for Mixed Media. In Image Communications Journal, 5(4), May 1997.
12. P. Ekman. An Argument for Basic Emotions. In N.L. Stein, and K. Oatley, editors, Basic Emotions, pp 169-200, 1992.
13. P. Ekman, and W. Friesen. Manual for the Facial Action Coding System. Consulting Psychologists Press, 1978.
14. G. Ferrigno, and A. Pedotti. ELITE: A Digital Dedicated Hardware System for Movement Analysis via Real-Time TV Signal Processing. In IEEE Transactions on Biomedical Engineering, BME-32, pp 943-950, 1985.
15. X. D. Huang, Y. Ariki, and M. Jack, M. Hidden Markov Models for Speech Recognition. Edinburgh University Press, Edinburgh, 1990.
16. E. Magno Caldognetto, C Zmarich, P. Cosi and F. Ferrero. Italian Consonantal Visemes: Relationships Between Spatial/temporal Articulatory Characteristics and Coproduced Acoustic Signal. In Proceedings of AVSP-97, Tutorial & Research Workshop on Audio-Visual Speech Processing: Computational & Cognitive Science Approaches, Rhodes (Greece), pp. 5-8, 26-27 September 1997.
17. N. Mana, P. Cosi, G. Tisato, F. Cavicchio, E. Magno and F. Pianesi. An Italian Database of Emotional Speech and Facial Expressions. Proceedings of "Workshop on Emotion: Corpora for Research on Emotion and Affect", in association with "5th International Conference on Language, Resources and Evaluation (LREC2006), Genoa, Italy, May 2006.
18. McBreen, H., Jack, M. (2001). Evaluating Humanoid Synthetic Agents in e-retail Applications. IEEE Transactions on Systems, Man and Cybernetics, vol. 31 (5), 2001.
19. McBreen, H. M., Shade, P., Jack, M. A., & Wyard, P. J. (2000). Experimental Assessment of the Effectiveness of Synthetic Personae for Multi-Modal E-Retail Applications. In Proceedings of the Fourth International Conference on Autonomous Agents, Barcelona, Spain, 2000.
20. S. Narayanan and A.Alwan. Text to Speech Synthesis: New Paradigms and Advances. Prentice Hall Press, 2004.
21. L.R. Rabiner. A tutorial on Hidden Markov Models and Selected Applications in Speech Recognition. In Proceedings of the IEEE, 77(2), pp. 257-286, 1989.
22. R. Rickenberg and B. Reeves. The Effects of Animated Characters on Anxiety, Task Performance, and Evaluations of User Interfaces. In Proceedings of CHI 2000, 2000.
23. D. Sankoff and J.B. Kruskal. Time warps, string edits, and macromolecules: The theory and practice of sequence comparison. Addison-Wesley Publishing Company, Reading, MA, 1983.