Discovering eye gaze behavior during human-agent conversation in an interactive storytelling application

International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction, ICMI-MLMI 2010

Volumn , Issue , 2010, Pages

  Bee, Nikolaus ^a   Wagner, Johannes ^a   André, Elisabeth ^a   Vogt, Thurid ^a   Charles, Fred ^b   Pizzi, David ^b   Cavazza, Marc ^b  

^a UNIVERSITY OF AUGSBURG   (Germany)

^b TEESSIDE UNIVERSITY   (United Kingdom)

Author keywords

eye gaze; human agent conversation; interactive storytelling; virtual agent

Indexed keywords

EMBODIED CONVERSATIONAL AGENT; EYE TRACKERS; EYE-GAZE; GAZE BEHAVIOR; HUMAN-AGENT CONVERSATION; INTERACTIVE MODELS; INTERACTIVE STORYTELLING; NATURAL LANGUAGES; NON-INTERACTIVE; USER RATING; VIRTUAL AGENT; VIRTUAL CHARACTER;

INTERACTIVE COMPUTER SYSTEMS; LEARNING SYSTEMS; TEACHING; VIRTUAL REALITY;

BEHAVIORAL RESEARCH;

EID: 78650965179     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1891903.1891915     Document Type: Conference Paper

References (35)

1. Argyle and Cook. Gaze & Mutual Gaze. Cambridge University Press, 1976.
2. Augsburg University. Horde3D GameEngine. http://mm-werkstatt.informatik. uni-augsburg.de/projects/GameEngine/.
3. R. Aylett, S. Louchart, J. Dias, A. Paiva, M. Vala, S. Woods, and L. Hall. Unscripted narrative for affectively driven characters. IEEE Comput. Graph. Appl., 26(3):42-52, 2006.
4. R. Aylett, N. Vannini, E. André, A. Paiva, S. Enz, and L. Hall. But that was in another country: agents and intercultural empathy. In AAMAS '09: Proc. of The 8th International Conference on Autonomous Agents and Multiagent Systems, pages 329-336, Richland, SC, 2009.
5. N. Bee, E. André, and S. Tober. Breaking the ice in human-agent communication: Eye-gaze based initiation of contact with an embodied conversational agent. In 9th International Conference on Intelligent Virtual Agents (IVA), pages 229-242. Springer, 2009.
6. N. Bee, B. Falk, and E. André. Simplified facial animation control utilizing novel input devices: A comparative study. In International Conference on Intelligent User Interfaces (IUI '09), pages 197-206, 2009.
7. M. Cavazza, D. Pizzi, F. Charles, T. Vogt, and E. André. Emotional input for character-based interactive storytelling. In AAMAS '09: Proc. of The 8th International Conference on Autonomous Agents and Multiagent Systems, pages 313-320, 2009.
8. A. Colburn, M. Cohen, and S. Drucker. The role of eye gaze in avatar mediated conversational interfaces. Technical report, Microsoft Research, 2000.
9. S. Dow, M. Mehta, E. Harmon, B. MacIntyre, and M. Mateas. Presence and engagement in an interactive drama. In CHI '07: Proc. of the SIGCHI conference on Human factors in computing systems, pages 1475-1484, New York, NY, USA, 2007. ACM.
10. T. Eichner, H. Prendinger, E. André, and M. Ishizuka. Attentive presentation agents. In Intelligent Virtual Agents (IVA 2007), pages 283-295, 2007.
11. P. Ekman and W. Friesen. Unmasking the Face. Prentice Hall, 1975.
12. Facial Expression Repertoire. Filmakademie Baden-Württemberg.
13. G. Flaubert. La revue de Paris. France, 1856.
14. A. Fukayama, T. Ohno, N. Mukawa, M. Sawaki, and N. Hagita. Messages embedded in gaze of interface agents - impression management with agent's gaze. In CHI '02: Proc. of the SIGCHI conference on Human factors in computing systems, pages 41-48, New York, NY, USA, 2002. ACM Press.
15. M. Garau, M. Slater, S. Bee, and M. A. Sasse. The impact of eye gaze on communication using humanoid avatars. In CHI '01: Proc. of the SIGCHI conference on Human factors in computing systems, pages 309-316. ACM Press, 2001.
16. E. T. Hall. A system for notation of proxemic behavior. American Anthropologist, 65:1003-1026, 1963.
17. A. Kendon. Some functions of gaze direction in social interaction. Acta Psychologica, 26:22-63, 1967.
18. M. Kipp and P. Gebhard. IGaze: Studying Reactive Gaze Behavior in Semi-immersive Human-Avatar Interactions. In Intelligent Virtual Agents (IVA '08), pages 191-199, 2008.
19. C. L. Kleinke. Gaze and eye contact: A research review. Psychological Bulletin, 100(1):78-100, 1986.
20. S. P. Lee, J. B. Badler, and N. I. Badler. Eyes alive. ACM Transactions on Graphics, 21(3):637-644, 2002.
21. L. P. Morency, M. C. Christoudias, and T. Darrell. Recognizing gaze aversion gestures in embodied conversational discourse. In ICMI '06: Proceedings of the 8th international conference on Multimodal interfaces, pages 287-294, New York, NY, USA, 2006. ACM Press.
22. Y. I. Nakano, G. Reinstein, T. Stocky, and J. Cassell. Towards a model of face-to-face grounding. In ACL '03: Proc. of the 41st Annual Meeting on Association for Computational Linguistics, pages 553-561. Association for Computational Linguistics, 2003.
23. Y. I. Nakano and Y. Yamaoka. Information state based multimodal dialogue management: Estimating conversational engagement from gaze information. In Z. Ruttkay, M. Kipp, A. Nijholt, and H. H. Vilhjálmsson, editors, Intelligent Virtual Agents, 9th International Conference, IVA 2009, Amsterdam, The Netherlands, September 14-16, 2009, Proceedings, volume 5773 of Lecture Notes in Computer Science, pages 531-532. Springer, 2009.
24. C. Peters, C. Pelachaud, E. Bevacqua, M. Mancini, and I. Poggi. A model of attention and interest using gaze behavior. In Intelligent Virtual Agents (IVA' 05), pages 229-240, London, UK, 2005. Springer-Verlag.
25. M. Rehm and E. André. From chatterbots to natural interaction - face to face communication with embodied conversational agents. IEICE Transactions on Information and Systems, Special Issue on Life-Like Agents and Communication, 88-D(11):2445-2452, 2005.
26. M. Sagar. Facial performance capture and expressive translation for king kong. In SIGGRAPH '06: ACM SIGGRAPH 2006 Sketches, page 26. ACM, 2006.
27. D. D. Salvucci and J. H. Goldberg. Identifying fixations and saccades in eye-tracking protocols. In ETRA '00: Proc. of the symposium on Eye tracking research & applications, pages 71-78. ACM Press, 2000.
28. C. L. Sidner, C. D. Kidd, C. Lee, and N. Lesh. Where to look: a study of human-robot engagement. In IUI '04: Proc. of the 9th international conference on Intelligent user interfaces, pages 78-84. ACM Press, 2004.
29. I. Starker and R. A. Bolt. A gaze-responsive self-disclosing display. In CHI '90: Proc. of the SIGCHI conference on Human factors in computing systems, pages 3-10. ACM, 1990.
30. W. Steptoe, R. Wolff, A. Murgia, E. Guimaraes, J. Rae, P. Sharkey, D. Roberts, and A. Steed. Eye-tracking for avatar eye-gaze and interactional analysis in immersive collaborative virtual environments. In CSCW '08: Proc. of the ACM 2008 conference on Computer supported cooperative work, pages 197-200. ACM, 2008.
31. Valve. Facial Expressions Primer from Half-Life 2. http://developer. valvesoftware.com/wiki/Facial-Expressions-Primer.
32. R. Vertegaal, R. Slagter, G. van der Veer, and A. Nijholt. Eye gaze patterns in conversations: there is more to conversational agents than meets the eyes. In CHI '01: Proc. of the SIGCHI conference on Human factors in computing systems, pages 301-308. ACM Press, 2001.
33. V. Vinayagamoorthy, M. Garau, A. Steed, and M. Slater. An Eye Gaze Model for Dyadic Interaction in an Immersive Virtual Environment: Practice and Experience. Computer Graphics Forum, 23(1):1-11, 2004.
34. T. Vogt, E. André, and N. Bee. Emovoice - a framework for online recognition of emotions from voice. In Proc. of Workshop on Perception and Interactive Technologies for Speech-Based Systems. Springer, 2008.
35. J. Wagner, E. André, and F. Jung. Smart sensor integration: A framework for multimodal emotion recognition in real-time. In Affective Computing and Intelligent Interaction (ACII 2009), 2009.