AUTOMATIC GENERATION of GAZE and GESTURES for DIALOGUES between EMBODIED CONVERSATIONAL AGENTS

International Journal of Semantic Computing

Volumn 2, Issue 1, 2008, Pages 71-90

  Breitfuss, Werner ^a   Prendinger, Helmut ^b   Ishizuka, Mitsuru ^a  

^a UNIVERSITY OF TOKYO   (Japan)

^b NATIONAL INSTITUTE OF INFORMATICS   (Japan)

Author keywords

multi modal presentation; Multimodal input and output interfaces; processing of language and action patterns

Indexed keywords

MARKUP LANGUAGES; SPEECH SYNTHESIS;

AUTOMATIC GENERATION; EMBODIED CONVERSATIONAL AGENT; EMPIRICAL STUDIES; LINGUISTIC AND CONTEXTUAL INFORMATIONS; MULTI-MODAL; MULTIMODAL INPUTS; MULTIMODAL PRESENTATION MARKUP LANGUAGES; NONVERBAL BEHAVIOR;

CONSUMER BEHAVIOR;

EID: 70350351037     PISSN: 1793351X     EISSN: 17937108     Source Type: Journal    
DOI: 10.1142/S1793351X0800035X     Document Type: Article

References (28)

1. E. André, J. Müller and T. Rist: The PPP Persona: A multipurpose animated presentation agent, in T. Catarci, M. F. Costabile, S. Levialdi and G. Santucci (eds.), Advanced Visual Interfaces, ACM Press, 1996, pp. 245-247.
2. W. Breitfuss, H. Prendinger and M. Ishizuka, Automated generation of non-verbal behavior for virtual embodied characters, in Proceedings of the 9th International Conference on Multimodal Interfaces, 2007, pp. 199-202.
3. J. Cassell, H. Vilhjálmsson and T. Bickmore, BEAT: The behavior expression animation toolkit, in Proceedings of SIGGRAPH 2001, 2001, pp. 477-486.
4. R. Cox, J. McKendree, R. Tobin, J. Lee and T. Mayes, Vicarious learning from dialogue and discourse: A controlled comparison, Instructional Science 27 (1999) 431-458.
5. S. Craig, B. Gholson and D. Driscoll, Animated pedagogical agents in multimedia educational environments: Effects of agent properties, picture features, and redundance, Journal of Educational Psychology 94(2) (2002) 428-434.
6. S. Craig, B. Gholson, M. Ventura, A. Graesser and the Tutoring Research Group, Overhearing dialogues and monologues in virtual tutoring sessions: Effects on questioning and vicarious learning, International Journal of Artificial Intelligence in Education 11 (2000) 242-253.
7. T. Eichner, H. Prendinger, E. André and M. Ishizuka, Attentive presentation agents, in Proceedings of the 7th International Conference on Intelligent Virtual Agents (IVA'07), Springer, 2007, pp. 283-295.
8. M. Garau, M. Slater, S. Bee and M. A. Sasse, The impact of eye gaze on communication using humanoid avatars, in Proceedings of the SIGCHI Conference on Human factors in Computing Systems (CHI'01), ACM Press, 2001, pp. 309-316.
9. B. Hartmann, M. Mancini, S. Buisine and C. Pelachaud, Design and evaluation of expressive gesture synthesis for embodied conversational agents, in Proceedings of the Fourth International Joint Conference on Autonomous Agents and Multiagent Systems, ACM Press, 2005.
10. D. Heylen, Head gestures, gaze and the principles of conversational structure, International Journal of Humanoid Robotics 3(3) (2006) 241-226.
11. A. Kendon, Some functions of gaze-direction in social interaction, Acta Psychologica 26 (1967) 22-63.
12. P. Kenny, A. Hartholt, J. Gratch, D. Traum, S. Marsella and B. Swartout, The more the merrier: Multi-party negotiation with virtual humans, in Proceedings of the 22nd AAAI Conference on Artificial Intelligence, AAAI Press, 2007, pp. 1970-1971.
13. M. Kipp, Creativity meets automation: Combining nonverbal action authoring with rules and machine learning, in Proceedings of the 6th International Conference on Intelligent Virtual Agents (IVA 2006), Springer, 2006, pp. 217-242.
14. S. Kopp, P. Tepper and J. Cassell, Towards integrated microplanning of language and iconic gesture for multimodal output, in Proceedings of the International Conference on Multimodal Interfaces 2004, ACM Press, 2004, pp. 97-104.
15. B. Krenn, M. Grice, P. Piwek, M. Schroeder, M. Klesen, S. Baumann, H. Pirker, K. van Deemter and E. Gstrein, Generation of multi-modal dialogue for net environments, in Proceedings of KONVENS-02, 2002, pp. 91-98.
16. J. Lee and S. Marsella, Nonverbal behavior generator for embodied conversational agents, in Proceedings of the 6th International Conference on Intelligent Virtual Agents, 2006, pp. 243-255.
17. Loquendo Vocal Technologies and Services, URL (2008), www.loquendo.com.
18. D. McNeill, Hand and Mind: What Gestures Reveal about Thought, The University of Chicago Press (1992).
19. Y. Nakano, M. Okamoto, D. Kawahara, Q. Li and T. Nishida, Converting text into agent animations: Assigning gestures to text, Journal of Humanoid Robotics 3(3) (2006) 241-26.
20. M. Nischt, H. Prendinger, E. André and M. Ishizuka, MPML3D: A reactive framework for the Multimodal Presentation Markup Language, in Proceedings of the 6th International Conference on Intelligent Virtual Agents, Springer, 2006, pp. 218-229.
21. C. Peters, C. Pelachaud, E. Bevacqua and M. Mancini, A model of attention and interest using gaze behavior, in Proceedings of the 5th International Conference on Intelligent Virtual Agents, 2005, pp. 229-240.
22. H. Prendinger and M. Ishizuka (eds.), Life-Like Characters. Tools, Affective Functions, and Applications. Cognitive Technologies, Springer, Berlin-Heidelberg, 2004.
23. T. Rist, E. André, S. Baldes, P. Gebhard, M. Klesen, M. Kipp, P. Rist and M. Schmitt, A review of the development of embodied presentation agents and their application fields, in Prendinger and Ishizuka [22], pp. 377-404.
24. M. Shaikh, H. Prendinger and M. Ishizuka, A cognitively based approach to affect sensing from text, in Proceedings of 10th International Conference on Intelligent User Interfaces, ACM Press, 2006, pp. 349-351.
25. I. van Es, D. Heylen, B. van Dijk and A. Nijholt, Gaze behavior of talking faces makes a difference, in Conference on Human Factors in Computing Systems (CHI'02), Extended Abstracts, ACM Press, 2002, pp. 734-735.
26. R. Vertegaal and Y. Ding, Explaining of eye gaze on mediated group conversations: Amount or synchronization, in Proceedings of the 2002 ACM Conference on Computer Supported Cooperative Work, ACM Press, 2002, pp. 41-48.
27. R. Vertegaal, I. Weevers, C. Sohn and C. Cheung, Gaze-2: Conveying eye contact in group video conferencing using eye-controlled camera direction, in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'03), ACM Press, 2003, pp. 521-528.
28. S. Williams, P. Piwek, and R. Power, Generating monologue and dialogue to present personalised medical information to patients, in Proceedings of the 11th European Workshop on Natural Language Generation, 2007, pp. 167-170.