Adaptive eye gaze patterns in interactions with human and artificial agents

ACM Transactions on Interactive Intelligent Systems

Volumn 1, Issue 2, 2012, Pages

  Yu, Chen ^a   Schermerhorn, Paul ^a   Scheutz, Matthias ^b  

^a INDIANA UNIVERSITY   (United States)

^b TUFTS UNIVERSITY   (United States)

Author keywords

Gaze based interaction; Human robot interaction; Multimodal interface

Indexed keywords

DIGITAL STORAGE; EYE MOVEMENTS; INTERACTIVE COMPUTER SYSTEMS;

BEHAVIORAL PATTERNS; DATA-DRIVEN APPROACH; GAZE-BASED INTERACTION; HUMAN-AGENT INTERACTION; HUMAN-HUMAN COMMUNICATION; INTERACTING AGENTS; MULTI-MODAL INTERACTIONS; MULTI-MODAL INTERFACES;

VIRTUAL REALITY;

EID: 84983577454     PISSN: 21606455     EISSN: 21606463     Source Type: Journal    
DOI: 10.1145/2070719.2070726     Document Type: Article

References (47)

1. ALLOPENNA, P.,MAGNUSON, J., AND TANENHAUS, M. 1998. Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models. J. Mem. Lang. 38, 4, 419-439.
2. ARGYLE, M. AND DEAN, J. 1965. Eye contact, distance and affiliation. Sociometry 28, 2, 89-304.
3. ARGYLE, M. AND INGHAM, J. 1972. Gaze, mutual gaze, and proximity. Semiotica 6, 32-49.
4. BALDWIN, D. 1993. Early referential understanding: Infants' ability to recognize referential acts for what they are. Dev. Psych. 29, 5, 832-843.
5. BALLARD, D., HAYHOE, M., POOK, P., AND RAO, R. 1997. Deictic codes for the embodiment of cognition. Behav. Brain Sci. 20, 4, 723-742.
6. BEE, N.,WAGNER, J., ANDRE, E., VOGT, T., CHARLES, F., PIZZI, D., AND CAVAZZA, M. 2010a. Discovering eye gaze behavior during human-agent conversation in an interactive storytelling application. In Proceedings of the International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction.
7. BEE, N.,WAGNER, J., ANDRE, E., VOGT, T., CHARLES, F., PIZZI, D., AND CAVAZZA, M. 2010b. Multimodal interaction with a virtual character in interactive storytelling. In Proceedings of the International Conference on Autonomous Agents and Multiagent Systems. 1535-1536.
8. BREAZEAL, C. AND SCASSELLATI, B. 2000. Infant-like social interactions between a robot and a human caregiver. Adapt. Behav. 8, 1, 49.
9. BRENNAN, S., CHEN, X., DICKINSON, C., NEIDER, M., AND ZELINSKY, G. 2008. Coordinating cognition: The costs and benefits of shared gaze during collaborative search. Cognition 106, 3, 1465-1477.
10. BRICK, T., AND SCHEUTZ, M. 2007. Incremental natural language processing for HRI. In Proceedings of the 2nd ACM IEEE International Conference on Human-Robot Interaction.
11. BUTTERWORTH, G. 2001I. Joint visual attention in infancy. In Blackwell Handbook of Infant Development 213-240.
12. CLARK, H. AND KRYCH, M. 2004. Speaking while monitoring addressees for understanding. J. Mem. Lang. 50, 1, 62-81.
13. GERGLE, D. AND CLARK, A. T. 2011. Seewhat I'm saying? Using dyadicmobile eye tracking to study collaborative reference. In Proceedings of the ACM Conference on Computer Supported Cooperative Work. 435-444.
14. GRIFFIN, Z. 2004. Why look? Reasons for eye movements related to language production. In The Interface of Language, Vision, and Action: Eye Movements and the Visual World. 213-247.
15. GRIFFIN, Z. AND BOCK, K. 2000. What the eyes say about speaking. Psych. Sci. 11, 4, 274.
16. GU, E. AND BADLER, N.I. 2006. Visual Attention and Eye Gaze During Multiparty Conversations with Distractions. In Proceedings of the 6th International Conference on Intelligent Virtual Agents. 193-204.
17. HANNA, J. AND BRENNAN, S. 2007. Speakers' eye gaze disambiguates referring expressions early during faceto-face conversation. J. Mem. Lang. 57, 4, 596-615.
18. HASEGAWA, D., CASSELL, J., AND ARAKI, K. 2010. The role of embodiment and perspective in direction-giving systems. In Proceedings of the AAAI Fall Workshop on Dialog with Robots.
19. HAYHOE, M. AND BALLARD, D. 2005. Eye movements in natural behavior. Trends Cogn. Sci. 9, 4, 188-194.
20. HAYHOE, M. M., SHRIVASTAVA, A., MRUCZEK, R., AND PELZ, J. B. 2003. Visual memory and motor planning in a natural task. J. Vis. 3, 1, 49-63.
21. KATZENMAIER, K., STIEFELHAGEN, R., AND SCHULTZ, T. 2004. Identifying the addressee in human-human-robot interactions based on head pose and speech. In Proceedings of the 6th ACM International Conference on Multimodal Interfaces. 144-151.
22. KIPP, M. AND GEBHARD, P. 2008. IGaze: Studying reactive gaze behavior in semi-immersive human-avatar interactions, In Proceedings of the 8th International Conference on Intelligent Virtual Agents. 191-199.
23. KRAMER, J. AND SCHEUTZ,M. 2007. ADE: A framework for robust complex robotic architectures. In Proceedings of the International Conference on Intelligent Robots and Systems.
24. LAND, M.,MENNIE, N., AND RUSTED, J. 1999. The roles of vision and eye movements in the control of activities of daily living. Perception 28, 11, 1311-1328.
25. MACDORMAN, K. AND ISHIGURO, H. 2006. The uncanny advantage of using androids in cognitive and social science research. Interact. Stud. 7, 3, 297-337.
26. MEYER, A., SLEIDERINK, A., AND LEVELT, W. 1998. Viewing and naming objects: Eye movements during noun phrase production. Cognition, 66, 2, 25-33.
27. MORENCY, L.-P., CHRISTOUDIAS, C., AND DARRELL, T. 2006. Recognizing gaze aversion gestures in embodied conversational discourse. In Proceedings of the 8th International Conference on Multimodal Interfaces.
28. MUTLU, B.,FORLIZZI, J., AND HODGINS, J. 2006. A storytelling robot:modeling and evaluation of human-like gaze behavior. In Proceedings of the 6th IEEE-RAS International Conference on Humanoid Robots. 518-523.
29. NAKANO, Y., REINSTEIN, G., STOCKY, T., AND CASSELL, J. 2003. Towards a model of face-to-face grounding. In Proceedings of the 41st Meeting of the Association for Computational Linguistics. 553-561.
30. PELZ, J., HAYHOE, M., AND LOEBER, R. 2001. The coordination of eye, head, and hand movements in a natural task. Exper. Brain Resear. 139, 3, 266-277.
31. QU, S. AND CHAI, J. 2010. Context-based word acquisition for situated dialogue in a virtual world. J. Artif. Intel. Resear. 37, 1, 247-278.
32. RAYNER, K. 1998. Eye movements in reading and information processing: 20 years of research. Psych. Bull. 124, 372-422.
33. REHM, M. AND ANDRE, E. 2005. Where do they look? Gaze behaviors of multiple users interacting with an embodied conversational agent. In Proceedings of the 5th International Conference on Intelligent Virtual Agents.
34. SCHEUTZ, M., SCHERMERHORN, P., KRAMER, J., AND ANDERSON, D. 2007. First steps toward natural human-like HRI. Auton. Robots, 22, 4, 411-423.
35. SHINTEL, H. AND KEYSAR, B. 2009. Less is more: A minimalist account of joint action in communication. Topics Cogn. Sci. 1, 2, 260-273.
36. SHOCKLEY, K., SANTANA, M., AND FOWLER, C. 2003. Mutual interpersonal postural constraints are involved in cooperative conversation. J. Exper. Psych. 29, 2, 326-332.
37. STAUDTE, M. AND CROCKER, M. 2009. Visual attention in spoken human-robot interaction. In Proceedings of the 4th ACM/IEEE Conference on Human-Robot Interaction.
38. TANENHAUS, M., SPIVEY-KNOWLTON, M., EBERHARD, K., AND SEDIVY, J. 1995. Integration of visual and linguistic information in spoken language comprehension. Science, 268, 5217, 1632-1634.
39. TRAFTON, J.,CASSIMATIS, N.,BUGAJSKA,M.,BROCK, D.,MINTZ, F., AND SCHULTZ, A. 2005.Enabling effective humanrobot interaction using perspective-taking in robots. IEEE Trans Syst. Man Cyber. 35, 4,460-470.
40. VERTEGAAL, R. 2003. Attentive user interfaces. Comm. ACM 46, 3, 3133.
41. YAMAZAKI, A.,YAMAZAKI, K.,KUNO, Y.,BURDELSKI, M.,KAWASHIMA, M., AND KUZUOKA, H. 2008. Precision timing in human-robot interaction: coordination of head movement and utterance. In Proceedings of the Conference on Human Factors in Computing Systems.
42. YU, C. AND BALLARD, D. 2004. A multi modal learning interface for grounding spoken language in sensory perceptions. ACM Trans. Appl. Percept. 1, 1, 57-80.
43. YU, C., SCHEUTZ, M., AND SCHERMERHORN, P. 2010a. Investigating multimodal real-time patterns of joint attention in an hri word learning task. In Proceedings of the 5th ACM/IEEE International Conference on Human-Robot Interaction.
44. YU, C., SMITH, L., SHEN, H., PEREIRA, A., AND SMITH, T. 2009a. Active information selection: Visual attention through the hands. IEEE Trans. Auton. Ment. Devel. 2, 141-151.
45. YU, C., SMITH, T., HIDAKA, S., SCHEUTZ, M., AND SMITH, L. 2010b. A data-driven paradigm to understand multimodal communication in human-human and human-robot interaction. In Advances in Intelligent Data Analysis, vol. 9, 232-244.
46. YU, C., ZHONG, Y., SMITH, T., PARK, I., AND HUANG, W. 2009b. Visual data mining of multimedia data for social and behavioral studies. Inf. Visual. 8, 1, 56-70.
47. ZHANG, H., FRICKER, D., SMITH, T., AND YU, C. 2010. Real-time adaptive behaviors in multimodal human-avatar interactions. In Proceedings of the International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction.