Gaze, conversational agents and face-to-face communication

Speech Communication

Volumn 52, Issue 6, 2010, Pages 598-612

  Bailly, Gérard ^a   Raidt, Stephan ^a   Elisei, Frédéric ^a  

^a GIPSA LAB   (France)

Author keywords

Conversational agents; Face to face communication; Gaze

Indexed keywords

COGNITIVE STATE; COMMUNICATIVE FUNCTIONS; CONVERSATIONAL AGENTS; EXPERIMENTAL KNOWLEDGE; EYE TRACKERS; FACE-TO-FACE COMMUNICATIONS; FACE-TO-FACE INTERACTION; HUMAN FACES; NON-INVASIVE; SPEECH ACTIVITY;

PINHOLE CAMERAS;

VIRTUAL REALITY;

EID: 77951114320     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2010.02.015     Document Type: Article

References (55)

1. Argyle M., and Cook M. Gaze and Mutual Gaze (1976), Cambridge University Press, London
2. Bailly G., Bérar M., Elisei F., and Odisio M. Audiovisual speech synthesis. Int. J. Speech Technol. 6 (2003) 331-346
3. Bailly, G., Elisei, F., Raidt, S., 2005. Multimodal face-to-face interaction with a talking face: mutual attention and deixis. Human-Computer Interaction, Las Vegas.
4. Baron-Cohen S., Leslie A., and Frith U. Does the autistic child have a "theory of mind"?". Cognition 21 (1985) 37-46
5. Benoît C., Grice M., and Hazan V. The SUS test: a method for the assessment of text-to-speech synthesis intelligibility using Semantically Unpredictable Sentences. Speech Comm. 18 (1996) 381-392
6. Blais C., Jack R.E., Scheepers C., Fiset D., and Caldara R. Culture shapes how we look at faces. PLoS ONE 3 8 (2008) e3022
7. Breazeal, C., 2000. Sociable machines: expressive social exchange between humans and robots. Sc.D. Dissertation. Department of Electrical Engineering and Computer Science, MIT, Boston, MA.
8. Buchan J.N., Paré M., and Munhall K.G. Spatial statistics of gaze Fixations during dynamic face processing. Social Neurosci. 2 1 (2007) 1-13
9. Buchan J.N., Paré M., and Munhall K.G. The effect of varying talker identity and listening conditions on gaze behavior during audiovisual speech perception. Brain Res. 1242 (2008) 162-171
10. Carpenter, M., Tomasello, M., 2000. Joint attention, cultural learning and language acquisition: implications for children with autism. In: Wetherby, A.M., Prizant, B.M. (Eds.), Communicative and Language Intervention Series. Autism Spectrum Disorders: A Transactional Perspective, Vol. 9, Paul H. Brooks Publishing, Baltimore, pp. 30-54.
11. Cassell J., Sullivan J., Prevost S., and Churchill E. Embodied Conversational Agents (2000), Cambridge, MIT Press
12. Castiello U., Paulignan Y., and Jeannerod M. Temporal dissociation of motor responses and subjective awareness. Brain 114 (1991) 2639-2655
13. Chen, M., 2002. Leveraging the asymmetric sensitivity of eye contact for video conference. In: SIGCHI Conf. on Human Factors in Computing Systems: Changing our World, Changing Ourselves, Minneapolis, Minnesota, pp. 49-56.
14. Driver J., Davis G., Ricciardelli P., Kidd P., Maxwell E., and Baron-Cohen S. Gaze perception triggers reflexive visuospatial orienting. Visual Cognit. 6 (1999) 509-540
15. Duncan S. Some signals and rules for taking speaking turns in conversations. J. Personality Social Psychol. 23 2 (1972) 283-292
16. Elisei F., Bailly G., and Casari A. Towards Eyegaze-Aware Analysis and Synthesis of Audiovisual Speech. Auditory-visual Speech Processing (2007), Hilvarenbeek, The Netherlands pp. 120-125
17. Evinger C., Manning K., Pellegrini J., Basso M., Powers A., and Sibony P. Not looking while leaping: the linkage of blinking and saccadic gaze shifts. Exp. Brain Res. 100 (1994) 337-344
18. Fujie, S., Fukushima, K., Kobayashi, T., 2005. Back-channel Feedback Generation Using Linguistic and Nonlinguistic Information and its Application to Spoken Dialogue System. Interspeech, Lisbon, Portugal, pp. 889-892.
19. Geiger G., Ezzat T., and Poggio T. Perceptual Evaluation of Video-Realistic Speech (2003), Massachusetts Institute of Technology, Cambridge, MA p. 224
20. Giles H., and Clair R. Language and Social Psychology (1979), Blackwell, Oxford
21. Goodwin C. Restarts, pauses, and the achievement of a state of mutual gaze at turn-beginning. Sociolog. Inquiry 50 3-4 (1980) 272-302 Special double issue on language and social interaction
22. Haddington, P., 2002. Congruent gaze points, mutual gaze and evasive gaze: some ways of using gaze in stance-taking sequences in a conversation. Studia Linguistica et Litteria Septentrionalia. Studies presented to Heikki Nyyssönen. E. Kärkäinen, J. Haines and T. Lauttamus, Department of English, University of Oulu, pp. 107-125.
23. Itti, L., Dhavale, N., Pighin, F., 2003. Realistic avatar eye and head animation using a neurobiological model of visual attention. In: SPIE 48th Annual Internat. Symp. on Optical Science and Technology, San Diego, CA, pp. 64-78.
24. Kaur, M., Tremaine, M., Huang, N., Wilder, J., Gacovski, Z., Flippo, F., Sekhar Mantravadi, C., 2003. Where is "it"? Event synchronization in gaze-speech input systems. In: Internat. Conf. on Multimodal Interfaces, Vancouver, BC, pp. 151-158.
25. Kendon A. Some functions of gaze-direction in social interaction. Acta Psychol. 26 (1967) 22-63
26. Langton S., and Bruce V. Reflexive visual orienting in response to the social attention of others. Visual Cognit. 6 5 (1999) 541-567
27. Langton S., Watt J., and Bruce V. Do the eyes have it? Cues to the direction of social attention. Trends Cognit. Sci. 4 2 (2000) 50-59
28. Lee S.P., Badler J.B., and Badler N. Eyes alive. ACM Trans. Graph. 21 3 (2002) 637-644
29. Leslie, A.M., 1994. ToMM, ToBY, and agency: core architecture and domain specificity. In: Hirschfeld, L.A., Gelman, S.A. (Eds.), Mapping the Mind: Domain Specificity in Cognition and Culture, Cambridge University Press, Cambridge, pp. 119-148.
30. Lewkowicz D.J. Perception of auditory-visual temporal synchrony in human infants. J. Exp. Psychol.: Human Perception Perform. 22 (1996) 1094-1106
31. Matsusaka, Y., Tojo, T., Kobayashi, T., 2003. Conversation robot participating in group conversation. IEICE Trans. Inform. Syst. E86-D(1), 26-36.
32. Miller L.M., and D'Esposito M. Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. J. Neurosci. 25 25 (2005) 5884-5893
33. Morgan J.L., and Demuth K. Signal to Syntax: An Overview (1996), Lawrence Erlbaum Associates, Mahwah, NJ, USA
34. Novick, D.G., Hansen, B., Ward, K., 1996. Coordinating Turn-Taking With Gaze, ICSLP, Philadelphia, PA, pp. 1888-1891.
35. Os E.d., Boves L., Rossignol S., Bosch L.t., and Vuurpijlc L. Conversational agent or direct manipulation in human-system interaction. Speech Comm. 47 1-2 (2005) 194-207
36. Peters C., and O'Sullivan C. Attention-Driven Eye Gaze And Blinking For Virtual Humans (2003), Siggraph, San Diego, CA
37. Peters, C., Pelachaud, C., Bevacqua, E., Mancini, M., Poggi, I., 2005. A model of attention and interest using gaze behavior. Intelligent Virtual Agents, Kos, Greece, Springer-Verlag, pp. 229-240.
38. Picot, A., Bailly, G., Elisei, F., Raidt, S., 2007. Scrutinizing natural scenes: controlling the gaze of an embodied conversational agent. International Conference on Intelligent Virtual Agents (IVA), Paris, pp. 272-282.
39. Posner M., and Peterson S. The attention system of the human brain. Annu. Rev. Neurosci. 13 (1990) 25-42
40. Posner M.I. Orienting of attention. Quart. J. Exp. Psychol. 32 (1980) 3-25
41. Pourtois G., Sander D., Andres M., Grandjean D., Revéret L., Olivier E., and Vuilleumier P. Dissociable roles of the human somatosensory and superior temporal cortices for processing social face signals. Eur. J. Neurosci. 20 (2004) 3507-3515
42. Povinelli R.J., and Feng X. A new temporal pattern identification method for characterization and prediction of complex time series events. IEEE Trans. Knowledge Data Eng. 15 2 (2003) 339-352
43. Premack D., and Woodruff G. Does the chimpanzee have a theory of mind?. Behav. Brain Sci. 1 (1978) 515-526
44. Raidt, S., 2008. Gaze and face-to-face communication between a human speaker and an embodied conversational agent. Mutual attention and multimodal deixis. Ph.D. Thesis. GIPSA-Lab. Speech & Cognition Dpt. Institute National Polytechnique Grenoble, France, 175 pp.
45. Raidt, S., Bailly, G., Elisei,. F., 2006. Does a virtual talking face generate proper multimodal cues to draw user's attention towards interest points? In: Language Resources and Evaluation Conference (LREC), Genova, Italy, pp. 2544-2549.
46. Revéret, L., Bailly, G., Badin, P., 2000. MOTHER: a new generation of talking heads providing a flexible articulatory control for video-realistic speech animation. in: International Conference on Speech and Language Processing, Beijing, China, pp. 755-758.
47. Riva G., Davide F., and Ijsselsteijn W.A. Being There: Concepts, Effects and Measurements of User Presence in Synthetic Environments (2003), IOS Press, Amsterdam
48. Rochet-Capellan A., Laboissière R., Galvan A., and Schwartz J.L. The speech focus effect on jaw-finger coordination in a pointing task. J. Speech Language Hearing Res. 51 (2008) 1507-1521
49. Rutter D.R., and Durkin K. Turn-taking in mother-infant interaction: an examination of vocalizations and gaze. Dev. Psychol. 23 1 (1987) 54-61
50. Salvucci, D.D., Goldberg, J.H., 2000. Identifying fixations and saccades in eye-tracking protocols. In: Eye Tracking Research and Applications Symposium, Palm Beach Gardens, FL, pp. 71-78.
51. Scassellati, B., 2001. Foundations for a theory of mind for a humanoid robot. Department of Computer Science and Electrical Engineering. MIT, Boston, MA, 174 pp.
52. Thórisson K. Natural turn-taking needs no manual: computational theory and model from perception to action. In: Granström B., House D., and Karlsson I. (Eds). Multimodality in Language and Speech Systems (2002), Kluwer Academic, Dordrecht, The Netherlands 173-207
53. Vatikiotis-Bateson E., Eigsti I.-M., Yano S., and Munhall K.G. Eye movement of perceivers during audiovisual speech perception. Percept. Psychophys. 60 (1998) 926-940
54. Wallbott H.G., and Scherer K.R. Stress specifics: differential effects of coping style, gender, and type of stressor on automatic arousal, facial expression, and subjective feeling. J. Personality Social Psychol. 61 (1991) 147-156
55. Yarbus A.L. Eye movements during perception of complex objects. In: Riggs L.A. (Ed). Eye Movements and Vision Vol. II (1967), Plenum Press, New York 171-196