The effects of speech-gesture cooperation in animated agents' behavior in multimedia presentations

Interacting with Computers

Volumn 19, Issue 4, 2007, Pages 484-493

  Buisine, Stéphanie ^a   Martin, Jean Claude ^b,c  

^a ARTS ET MÉTIERS PARISTECH   (France)

^b CNRS   (France)

^c IUT de Montreuil   (France)

Author keywords

Embodied conversational agents; Experimental evaluation; Multimodal behavior; Redundancy

Indexed keywords

ANIMATION; GESTURE RECOGNITION; REDUNDANCY; SEMANTICS; SOFTWARE AGENTS; SPEECH RECOGNITION;

EMBODIED CONVERSATIONAL AGENTS; EXPERIMENTAL EVALUATION; MULTIMODAL BEHAVIOR;

MULTIMEDIA SERVICES;

EID: 34347268792     PISSN: 09535438     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.intcom.2007.04.002     Document Type: Article

References (35)

1. Abrilian, S., Buisine, S., Rendu, C., Martin, J.C., 2002. Specifying cooperation between modalities in lifelike animated agents. In: Proceedings of PRICAI'2002 Workshop on Lifelike Animated Agents Tools, Affective Functions, and Applications, pp. 3-8.
2. André E., Rist T., and Müller J. Employing AI methods to control the behavior of animated interface agents. Applied Artificial Intelligence 13 (1999) 415-448
3. Beun R.J., de Vos E., and Witteman C. Embodied conversational agents: effects on memory performance and anthropomorphisation. In: Rist T., Aylett R., Ballin D., and Rickel J. (Eds). IVA'2003 International Conference on Intelligent Virtual Agents. LNCS vol. 2792 (2003), Springer, Berlin 315-319
4. Buisine S., Abrilian S., and Martin J.C. Evaluation of multimodal behaviour of embodied agents. In: Ruttkay Z., and Pelachaud C. (Eds). From Brows to Trust: Evaluating Embodied Conversational Agents (2004), Kluwer Academic Publishers 217-238
5. Cassell J. Embodied conversational agents: representation and intelligence in user interface. AI Magazine 22 (2001) 67-83
6. Cassell J., McNeill D., and McCullough K.E. Speech-gesture mismatches: evidence for one underlying representation of linguistic and non-linguistic information. Pragmatics and Cognition 7 (1999) 1-33
7. Cassell, J., Prevost, S., 1996. Distribution of semantic features across speech and gesture by humans and computers. In: Proceedings of Workshop on the Integration of Gesture in Language and Speech, pp. 253-270.
8. Cassell, J., Stone, M., Yan, H., 2000. Coordination and context-dependence in the generation of embodied conversation. In: Proceedings of International Natural Language Generation Conference, pp. 171-178.
9. Cowell A.J., and Stanney K.M. Embodiment and interaction guidelines for designing credible, trustworthy ECAs. In: Rist T., Aylett R., Ballin D., and Rickel J. (Eds). IVA'2003 International Conference on Intelligent Virtual Agents. LNCS vol. 2792 (2003), Springer, Berlin 301-309
10. Craig S.D., Gholson B., and Driscoll D. Animated pedagogical agents in multimedia educational environments: effects of agent properties, picture features, and redundancy. Journal of Educational Psychology 94 (2002) 428-434
11. Dubois, V., Gyselinck, V., Choplin, H., 2003. Multimodalité et mémoire de travail [Multimodality and working memory]. In: Proceedings of EIAH'03 French-speaking conference on Environnements Informatiques pour l'Apprentissage Humain, pp. 187-198.
12. Goldin-Meadow S. The role of gesture in communication and thinking. Trends in Cognitive Sciences 3 (1999) 419-429
13. Goldin-Meadow S., Kim S., and Singer M. What the teacher's hands tell the student's mind about math. Journal of Educational Psychology 91 (1999) 720-730
14. Graesser A.C., Chipman P., Haynes B.C., and Olney A. AutoTutor: an intelligent tutoring system with mixed-initiative dialogue. IEEE Transactions in Education 48 (2005) 612-618
15. Johnson W.L., Mayer R.E., André E., and Rehm M. Cross-cultural evaluation of politeness in tactics for pedagogical agents. In: Looi C.K., McCalla G., Bredeweg B., and Breuker J. (Eds). AIED'05 International Conference on Artificial Intelligence in Education (2005), IOS Press, Amsterdam 298-305
16. Johnson W.L., Rickel J., and Lester J. Animated pedagogical agents: face-to-face interaction in interactive learning environments. International Journal of Artificial Intelligence in Education 11 (2000) 47-78
17. Johnson W.L., Shaw E., Marshall A., and LaBore C. Evolution of user interaction: the case of agent Adele. Proceedings of IUI'2003 International Conference on Intelligent User Interfaces (2003), ACM Press 93-100
18. Kalyuga S., Chandler P., and Sweller J. Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology 13 (1999) 351-371
19. Kimura D. Sex and Cognition (1999), MIT Press, Cambridge
20. Knapp M.L. Nonverbal communication in human interaction (2002), Thomson Learning, Florence Wadsworth
21. Kohar H., and Ginn I. Mediators: guides through online TV services. Proceedings of Demo Session in CHI'97 International Conference on Human Factors in Computing Systems (1997), ACM Press 38-39
22. Kopp, S., Gesellensetter, L., Krämer, N.C., Wachsmuth, I., 2005. A conversational agent as museum guide - design and evaluation of a real-world application. In: Panayiotopoulos, T., Gratch, J., Aylett, R., Ballin, D., Olivier, P., Rist, T., (Eds.), IVA'2005 International Conference on Intelligent Virtual Agents, LNCS, vol. 3661. Springer, Berlin, pp. 329-343.
23. Krämer N.C. Social communicative effects of a virtual program guide. In: Panayiotopoulos T., Gratch J., Aylett R., Ballin D., Olivier P., and Rist T. (Eds). IVA'2005 International Conference on Intelligent Virtual Agents. LNCS vol. 3661 (2005), Springer, Berlin 442-453
24. Le Bohec O., and Jamet E. Les effets de redondance dans l'apprentissage à partir de documents multimédia [Redundancy effect and the multimedia learning process]. Le Travail Humain 68 (2005) 97-124
25. Lester J., Towns S., and FitzGerald P. Achieving affective impact: visual emotive communication in lifelike pedagogical agents. International Journal of Artificial Intelligence in Education 10 (1999) 278-291
26. Lester J., Voerman J., Towns S., and Callaway C. Deictic believability: coordinating gesture, locomotion, and speech in lifelike pedagogical agents. Applied Artificial Intelligence 13 (1999) 383-414
27. McBreen, H., Anderson, J., Jack, M. 2001. Evaluating 3D embodied conversational agents in contrasting VRML retail applications. In: Proceedings of International Conference on Autonomous Agents Workshop on Multimodal Communication and Context in Embodied Agents, pp. 83-87.
28. McNeill D. Hand and Mind (1992), University of Chicago Press
29. Moreno R., and Mayer R.E. Verbal redundancy in multimedia learning: when reading helps listening. Journal of Educational Psychology 94 (2002) 156-163
30. Moreno R., Mayer R.E., Spires H., and Lester J. The case for social agency in computer-based teaching: do students learn more deeply when they interact with animated pedagogical agents?. Cognition and Instruction 19 (2001) 177-213
31. Myers J.L. Fundamentals of Experimental Design. third ed. (1979), Allyn & Bacon, Inc., Boston
32. Rickel J., and Johnson W.L. Animated agents for procedural training in virtual reality: perception, cognition, and motor control. Applied Artificial Intelligence 13 (1999) 343-382
33. Rimé B., and Schiaratura L. Gesture and speech. In: Feldman R.S., and Rimé B. (Eds). Fundamentals of Nonverbal Behavior (1991), Cambridge University Press 239-284
34. Stone, B., Lester, J., 1996. Dynamically sequencing an animated pedagogical agent. In: Proceedings of National Conference on Artificial Intelligence, pp. 424-431.
35. Van Mulken S., André E., and Müller J. The persona effect: how substantial is it?. Proceedings of HCI'98 International Conference on Human-Computer Interaction (1998), Springer 53-66