Human physiology as a basis for designing and evaluating affective communication with life-like characters

IEICE Transactions on Information and Systems

Volumn E88-D, Issue 11, 2005, Pages 2453-2460

  Prendinger, Helmut ^a,b   Ishizuka, Mitsuru ^{b,c,d,e,f,g,h}  

^a NATIONAL INSTITUTE OF INFORMATICS   (Japan)

^b UNIVERSITY OF TOKYO   (Japan)

^c UNIVERSITY OF TOKYO   (Japan)

^d INSTITUTE OF INDUSTRIAL SCIENCE   (Japan)

^e IEEE

^f AAAI ^*

^g Japanese Society for AI   (Japan)

^h IPS Japan   (Japan)

Author keywords

Affective communication; Bio signal; Evaluation; Eye tracking; Interface agents; Markup languages

Indexed keywords

EVALUATION; INTELLIGENT AGENTS; PHYSIOLOGY;

AFFECTIVE COMMUNICATION; BIO-SIGNAL; EYE TRACKING; INTERFACE AGENTS; MARKUP LANGUAGES; MOMENT-TO-MOMENT BASIS;

INTERFACES (COMPUTER);

EID: 29144486081     PISSN: 09168532     EISSN: 17451361     Source Type: Journal    
DOI: 10.1093/ietisy/e88-d.11.2453     Document Type: Article

References (45)

1. J. Cassell, J. Sullivan, S. Prevost, and E. Churchill, eds., Embodied Conversational Agents, The MIT Press, Cambridge, MA, 2000.
2. H. Prendinger and M. Ishizuka, eds., Life-Like Characters. Tools, Affective Functions, and Applications, Cognitive Technologies, Springer Verlag, Berlin Heidelberg, 2004.
3. T. Rist, E. André, S. Baldes, P. Gebhard, M. Klesen, M. Kipp, P. Rist, and M. Schmitt, "A review of the development of embodied presentation agents and their appication fields," in Life-Like Characters. Tools, Affective Functions, and Applications, ed. H. Prendinger and M. Ishizuka, Cognitive Technologies, pp.377-404, Springer Verlag, Berlin Heidelberg, 2004.
4. H. Prendinger, S. Descamps, and M. Ishizuka, "MPML: A markup language for controlling the behavior of life-like characters," J. Visual Languages and Computing, vol.15, no.2, pp. 183-203, 2004.
5. Microsoft, Developing for Microsoft Agent, Microsoft Press, Redmond, WA, 1998.
6. K. Kushida, Y. Nishimura, H. Dohi, M. Ishizuka, J. Takeuchi, and H. Tsujino, "Humanoid robot presentation through multimodal presentation markup language MPML-HR," AAMAS-05 Workshop on Creating Bonds with Humanoids, 2005.
7. Y. Arafa, K. Kamyab, and E. Mamdani, "Towards a unified scripting language. Lessons learned from developing CML & AML," in Life-Like Characters. Tools, Affective Functions, and Applications, ed. H. Prendinger and M. Ishizuka, Cognitive Technologies, pp.39-63, Springer Verlag, Berlin Heidelberg, 2004.
8. A. Mariott and J. Stallo, "VHML - Uncertainties and problems. A discussion," Proc. AAMAS-02 Workshop on Embodied Conversational Agents - let's specify and evaluate them!, 2002.
9. J. Cassell, H. Vilhjálmsson, and T. Bickmore, "BEAT: the Behavior Expression Animation Toolkit," in Life-Like Characters. Tools, Affective Functions, and Applications, Cognitive Technologies, ed. H. Prendinger, and M. Ishizuka, pp.163-185, Springer Verlag, Berlin Heidelberg, 2004.
10. H. Prendinger, S. Descamps, and M. Ishizuka, "Scripting affective communication with life-like characters in web-based interaction systems," Applied Artificial Intelligence, vol.16, no.7-8, pp.519-553, 2002.
11. W.S.N. Reilly, Believable Social and Emotional Agents, Ph.D. Thesis, Carnegie Mellon University, 1996. CMU-CS-96-138.
12. C. Elliott, The Affective Reasoner. A process model of emotions in a multi-agent system, Ph.D. Thesis, Northwestern University, Institute for the Learning Sciences, 1992.
13. J.C. Lester, S.A. Converse, S.E. Kahler, S.T. Barlow, B.A. Stone, and R.S. Bhogal, "The Persona effect: Affective impact of animated pedagogical agents," Proc. CHI-97, pp.359-366, ACM Press, 1997.
14. D.M. Dehn and S. van Mulken, "The impact of animated interface agents: A review of empirical research," Int. J. Human-Computer Studies, no.52, pp. 1-22, 2000.
15. R.E. Nisbett and T.D. Wilson, "Telling more than we know: Verbal reports on mental processes," Psychological Review, vol.84, pp.231-259, 1977.
16. R.W. Picard, Affective Computing, The MIT Press, Cambridge, MA, 1997.
17. R.W. Levenson, "Autonomic specifity and emotion," in Handbook of Affective Sciences, ed. R.J. Davidson, K.R. Scherer, and H.H. Goldsmith, pp.212-224, Oxford University Press, Oxford, 2003.
18. J.L. Andreassi, Psychophysiology. Human Behavior & Physiological Response, 4 ed., Lawrence Erlbaum Associates, Mahwah, NJ, 2000.
19. J. Schreirer, R. Fernandez, J. Klein, and R.W. Picard, "Frustrating the user on purpose: A step toward building an affective computer," Interacting with Computers, vol.14, pp.93-118, 2002.
20. A.T. Duchowski, Eye Tracking Methodology: Theory and Practice, Springer, London, UK, 2003.
21. C. Stevenson, "Affective gaming: Using pupil dilation to determine player arousal," The 2005 International Conference on Changing Views: Worlds in Play, 2005.
22. D. Bersak, G. McDarby, N. Augenblick, P. McDarby, D. McDonnell, B. McDonald, and R. Karkun, "Intelligent biofeedback using an immersive competitive environment," Online Proc. Ubicomp 2001 Workshop on Designing Ubiquitous Computer Games, 2001.
23. R. Burke, "Breathing new life into accessibility," CodeZone Magazine, Dec. 2003.
24. H. Prendinger and M. Ishizuka, "The Empathic Companion: A character-based interface that addresses users' affective states," Int. J. Applied Artificial Intelligence, vol.19, no.3, pp.267-285, 2005.
25. C. Becker, H. Prendinger, M. Ishizuka, and I. Wachsmuth, "Evaluating affective feedback of the 3D agent Max in a competitive cards game," Proc. First International Conference on Affective Computing and Intelligent Interaction (ACII-05), LNCS 3784, pp.466-473, Springer, 2005.
26. W. Bosma and E. André, "Exploiting emotions to disambiguate dialogue acts," Proc. 9th International Conference on Intelligent User Interfaces (IUI-04), pp.85-92, ACM Press, New York, 2004.
27. R.J.K. Jacob, "The use of eye movements in human-computer interaction techniques: What you look at is what you get," ACM Trans. Information Systems, vol.9, no.3, pp.152-169, 1991.
28. T. Selker, "Visual attentive interfaces," BT Technology Journal, vol.22, no.4, pp.146-150, 2004.
29. L. Qu, N. Wang, and W.L. Johnson, "Pedagogical agents that interact with learners," AAMAS-04 Workshop on Balanced Perception and Action in ECAs, 2004.
30. Y.I. Nakano, G. Reinstein, T. Stocky, and J. Cassell, "Towards a model of face-to-face grounding," Proc. Association for Computational Linguistics (ACL-03), pp.553-561, 2003.
31. A. Ortony, G.L. Clore, and A. Collins, The Cognitive Structure of Emotions, Cambridge University Press, Cambridge, 1988.
32. H. Prendinger and M. Ishizuka, "Social role awareness in animated agents," Proc. 5th International Conference on Autonomous Agents (Agents-01), pp.270-277, ACM Press, New York, 2001.
33. A. Ortony, "Value and emotion," in Memories, thoughts, and emotions: Essays in the honor of George Mandler, ed. W. Kessen, A. Ortony, and F. Craik, pp.337-353, Erlbaum, Hillsdale, NJ, 1991.
34. H. Prendinger, J. Mori, and M. Ishizuka, "Using human physiology to evaluate subtle expressivity of a virtual quizmaster in a mathematical game," Int. J. Human-Computer Studies, vol.62, no.2, pp.231-245, 2005.
35. H. Prendinger, C. Ma, J. Yingzi, A. Nakasone, and M. Ishizuka, "Understanding the effect of life-like interface agents through eye users' eye movements," Proc. Seventh International Conference on Multimodal Interfaces (ICMI-05), ACM Press, 2005.
36. "Hottolink Inc.," URL: http://www.hottolink.com
37. R.M. Cooper, "The control of eye fixation by the meaning of spoken language: A new methodology for the real-time investigation of speech perception, memory, and language processing," Cognitive Psychology, vol.6, pp. 84-107, 1974.
38. P.J. Lang, "The emotion probe: Studies of motivation and attention," American Psychologist, vol.50, no.5, pp.372-385, 1995.
39. S. Kopp, B. Jung, N. Lessmann, and I. Wachsmuth, "Max - A multimodal assistant in virtual reality construction," KI Zeitschift (German Magazine of Artificial Intelligence), Special Issue on Embodied Conversational Agents, vol.4, pp. 11-17, 2003.
40. C. Becker, S. Kopp, and I. Wachsmuth, "Simulating the emotion dynamics of a multimodal conversational agent," Proc. Tutorial and Research Workshop on Affective Dialogue Systems (ADS-04), LNAI3068, pp.154-165, Springer, Berlin Heidelberg, 2004.
41. U. Spierling, D. Grasbon, N. Braun, and I. Iurgel, "Setting the scene: Playing digital director in interactive storytelling and creation," Comput. Graph., vol.26, pp.31-44, 2002.
42. "A.L.I.C.E AI Foundation," URL: http://www.alicebot.org.
43. H. Umemuro and J. Yamashita, "Detection of user's confusion and surprise based on pupil dilation," Japanese Journal of Ergonomics, vol.39, no.4, pp. 153-161, 2003.
44. A. Kapoor, R.W. Picard, and Y. Ivanov, "Probabilistic combination of multiple modalities to detect interest," Proc. International Conference on Pattern Recognition, pp.969-972, 2005.
45. H. Liu, H. Lieberman, and T. Selker, "A model of textual affect sensing using real-world knowledge," Proc. International Conference on Intelligent User Interfaces (IUI-03), pp.125-132, 2003.