Specifying and animating facial signals for discourse in embodied conversational agents

Computer Animation and Virtual Worlds

Volumn 15, Issue 1, 2004, Pages 27-38

  DeCorlo, Doug ^a   Stone, Matthew ^b,c   Revilla, Corey ^e   Venditti, Jennifer J ^d  

^a RUTGERS UNIVERSITY   (United States)

^b RUTGERS UNIVERSITY   (United States)

^c Center for Cognitive Science   (United States)

^d Columbia University ^*  (United States)

^e NONE

Author keywords

Embodied conversational agents; Facial animation

Indexed keywords

EMBODIED CONVERSATIONAL AGENTS; FACIAL ANIMATION; NON-VERBAL SIGNALS; SYLLABLES;

DATA REDUCTION; LINGUISTICS; MATHEMATICAL MODELS; MULTI AGENT SYSTEMS; REAL TIME SYSTEMS; SPEECH SYNTHESIS;

ANIMATION;

EID: 23444440179     PISSN: 15464261     EISSN: None     Source Type: Journal    
DOI: 10.1002/cav.5     Document Type: Article

References (54)

1. Cassell J, Bickmore T, Billinghurst M, Campbell L, Chang K, Vilhjálmsson H, Yan H. Embodiment in conversational characters: Rea. In Proceedings of CHI, 1999; pp 520-527.
2. Cassell J, Thórisson K. The power of a nod and a glance: envelope vs. emotional feedback in animated conversational agents. Applied Artificial Intelligence 1999; 13(3).
3. Nagao K, Takeuchi A. Speech dialogue with facial displays: multimodal human-computer conversation. In Proceedings of ACL 1994; pp 102-109.
4. Poggi I, Pelachaud C. Performative facial expressions in animated faces. In Embodied Conversational Agents, Cassell J, Sullivan J, Prevost S, Churchill E (eds). MIT: Cambridge, MA, 2000; 155-188.
5. Pelachaud C, Poggi I. Subtleties of facial expressions in embodied agents. Journal of Visualization and Computer Animation 2002; 13(5): 301-312.
6. André E, Rist T, van Mulken S, Klesen M, Baldes S. The automated design of believable dialogues for animated presentation teams. In Embodied Conversational Agents, Cassell J, Sullivan J, Prevost S, Churchill E (eds). MIT: Cambridge, MA, 2000; 220-255.
7. Ball G, Breese J. Emotion and personality in a conversational agent. In Embodied Conversational Agents, Cassell J, Sullivan J, Prevost S, Churchill E (eds). MIT: Cambridge, MA, 2000; 189-219.
8. De Carolis B, Pelachaud C, Poggi I, de Rosis F. Behavior planning for a reflexive agent. In Proceedings of IJCAI, 2001.
9. Lester JC, Towns SG, Callaway CB, Voerman JL, FitzGerald PJ. Deictic and emotive communication in animated pedagogical agents. In Embodied Conversational Agents, Cassell J, Sullivan J, Prevost S, Churchill E (eds). MIT: Cambridge, MA, 2000; 123-154.
10. Marsella SC. Sympathy for the agent: controlling an agent's nonverbal repertoire. In Proceedings of Agents, 2000.
11. Beckman M, Elam G A. Guidelines for ToBI labelling, version 3.0. Technical report, Ohio State University, 1997. http://lmg.ohio-state.edu/ Phonetics/etobi_homepage.html.
12. Silverman KEA, Beckman M, Pitrelli JF, Ostendorf M, Wightman C, Price P, Pierrehumbert J. ToBI: a standard for labeling English prosody. In Proceedings of the International Conference on Spoken Language Processing, 1992; pp 867-870.
13. Ekman P. About brows: emotional and conversational signals. In Human Ethology: Claims and Limits of a New Discipline: Contributions to the Colloquium, von Cranach M, Foppa K, Lepenies W, Ploog D (eds). Cambridge University Press: Cambridge, UK, 1979; 169-202.
14. McNeill D. Hand and Mind: What Gestures Reveal about Thought. University of Chicago Press: Chicago, 1992.
15. Bull P, Connelly G. Body movement and emphasis in speech. Journal of Nonverbal Behavior 1985; 9(3): 169-187.
16. Engle RA. Toward a theory of multimodal communication: combining speech, gestures, diagrams and demonstrations in instructional explanations. PhD thesis, Stanford University, 2000.
17. Cohen MM, Massaro DW. Modeling coarticulation in synthetic visual speech. In Models and Techniques in Computer Animation, Thalmann NM, Thalmann D (eds). Springer: Berlin, 1993; 139-156.
18. King SA. A facial model and animation techniques for animated speech. PhD thesis, Ohio State University, 2001.
19. Löfqvist A. Speech as audible gestures. In Speech Production and Speech Modeling, Hardcastle WJ, Marchal A (eds). Kluwer: Dordrecht, 1990; 289-322.
20. Black A, Taylor P. Festival speech synthesis system. Technical Report HCRC/TR-83, Human Communication Research Center, 1997.
21. Bavelas JB, Chovil N. Visible acts of meaning: an integrated message model of language in face-to-face dialogue. Journal of Language and Social Psychology 2000; 19(2): 163-194.
22. Perlin K. Layered compositioning of facial expression. In SIGGRAPH, 1997. Technical Sketch.
23. Perlin K. Noise, hypertexture, antialiasing and gestures. In Texturing and Modeling: A Procedural Approach (2nd edn), Ebert D (ed.). Academic Press: London, 1998; 209-274.
24. Brand M. Voice puppetry. In SIGGRAPH, 1999; pp 21-28.
25. Smid K, Pandzic IS. Conversational virtual character for the web. In Computer Animation, 2002; 240-247.
26. Cassell J, Vilhjálmsson H, Bickmore T. BEAT: the behavioral expression animation toolkit. In SIGGRAPH, 2001; pp 477-486.
27. Nass C, Isbister K, Lee E-J. Truth is beauty: resarching embodied conversational agents. In Embodied Conversational Agents, Cassell J, Sullivan J, Prevost S, Churchill E (eds). MIT: Cambridge, MA, 2000; 374-402.
28. Jilka M, Möhler G, Dogil G. Rules for the generation of ToBI-based American English intonation. Speech Communication 1999; 28: 83-108.
29. Möhler G, Mayer J. A discourse model for pitch-range control. In Proceedings of the 4th ISCA Tutorial and Research Workshop on Speech Synthesis, 2001.
30. Macon M, Cronk A, Kain A, Wouters J. OGIresLPC: diphone synthesiser using residual linear prediction coding. Technical Report CSE-97-007, Oregon Graduate Institute, 1997.
31. Pierrehumbert J, Hirschberg J. The meaning of intonational contours in the interpretation of discourse. In Intentions in Communication, Cohen P, Morgan J, Pollack M (eds). MIT Press: Cambridge, MA, 1990; 271-311.
32. Hirschberg J. Pitch accent in context: predicting intonational prominence from text. Artificial Intelligence 1993; 63(1-2): 305-340.
33. Hirschberg J, Nakatani C. A prosodic analysis of discourse segments in direction-giving monologues. In Proceedings of ACL, 1996.
34. Ladd DR, Silverman K, Tolkmitt F, Bergmann G, Scherer K. Evidence for the independent function of intonation contour type, voice quality and F0 range in signaling speaker affect. Journal of the Acoustical Society of America 1985; 78: 435-444.
35. Möhler G, Mayer J. A method for the analysis of prosodic registers. In Proceedings of the 6th European Conference on Speech Communication and Technology (EUROSPEECH), 1999.
36. Hadar U, Steiner TJ, Grant EC, Rose FC. Head movement correlates of juncture and stress at sentence level. Language and Speech 1983; 26(2): 117-129.
37. Krahmer E, Ruttkay Z, Swerts M, Wesselink W. Pitch, eye-brows and the perception of focus. In Symposium on Speech Prosody, 2002.
38. Krahmer E, Ruttkay Z, Swerts M, Wesselink W. Audiovisual cues to prominence. In International Conference on Spoken Lanugage Processing, 2002.
39. Weizenbaum J. ELIZA: a computer program for the study of natural language communication between man and machine. Communcations of the ACM 1966; 9(1): 36-45.
40. Stone M, DeCarlo D. Crafting the illusion of meaning: template-based specification of embodied conversational behavior. In Computer Animation and Social Agents, 2003.
41. Norvig P. Paradigms of Artificial Intelligence: Case Studies in Common Lisp. Morgan Kaufmann: San Francisco, 1992.
42. Platt SM. A structural model of the human face. PhD thesis, University of Pennsylvania, 1985.
43. Waters K. A muscle model for animating three-dimensional facial expressions. Computer Graphics 1987; 21(4): 17-24.
44. Terzopoulos D, Waters K. Physically-based facial modeling, analysis and animation. Journal of Visualization and Computer Animation 1990; 1(2): 73-80.
45. Cassell J, Stone M, Douville B, Prevost S, Achorn B, Steedman M, Badler N, Pelachaud C. Modeling the interaction between speech and gesture. In Proceedings of the Cognitive Science Society, 1994.
46. Cassell J, Stone M, Yan H. Coordination and context-dependence in the generation of embodied conversation. In First International Conference on Natural Language Generation, 2000; pp 171-178.
47. Pelachaud C, Badler N, Steedman M. Generating facial expressions for speech. Cognitive Science 1996; 20(1): 1-46.
48. Poggi I, Pelachaud C. Eye communication in a conversational 3D synthetic agent. AI Communications 2000; 13(3): 169-181.
49. Massaro DW. Perceiving Talking Faces: From Speech Perception to a Behavioral Principle. MIT: Cambridge, MA, 1998.
50. Cahn JE. The generation of affect in synthesized speech. Journal of the American Voice I/O Society 1990; 8: 1-19.
51. Chi D, Costa M, Zhao L, Badler N. The EMOTE model for effort and shape. In SIGGRAPH, 2000; pp 173-182.
52. Badler N, Allbeck J, Zhao L, Byun M. Representing and parameterizing agent behaviors. In Computer Animation, 2002; pp 133-143.
53. Byun M, Badler N. FacEMOTE: qualitative parametric modifiers for facial animations. In ACM SIGGRAPH Symposium on Computer Animation, 2002; pp 65-71.
54. Cassell J, Sullivan J, Prevost S, Churchill E (eds). Embodied Conversational Agents. MIT: Cambridge, MA, 2000.