Investigating joint attention mechanisms through spoken human-robot interaction

Cognition

Volumn 120, Issue 2, 2011, Pages 268-291

  Staudte, Maria ^a   Crocker, Matthew W ^a  

^a SAARLAND UNIVERSITY   (Germany)

Author keywords

Gaze; Human robot interaction; Joint attention; Reference resolution; Referential gaze; Referential intention; Situated language processing; Utterance comprehension

Indexed keywords

ARTICLE; ASSOCIATION; ATTENTION; COMPREHENSION; EYE TRACKING; GAZE; HUMAN; HUMAN EXPERIMENT; INTERPERSONAL COMMUNICATION; LANGUAGE PROCESSING; PRIORITY JOURNAL; ROBOTICS; SPEECH; TASK PERFORMANCE; VERBAL BEHAVIOR;

ADULT; ATTENTION; COMMUNICATION; COMPREHENSION; EYE MOVEMENTS; HUMANS; LANGUAGE; USER-COMPUTER INTERFACE;

EID: 79959201003     PISSN: 00100277     EISSN: 18737838     Source Type: Journal    
DOI: 10.1016/j.cognition.2011.05.005     Document Type: Article

References (70)

1. Adams R.B., Kleck R.E. Perceived gaze direction and the processing of facial displays of emotions. Psychological Science 2003, 14:644-647.
2. Allopenna P., Magnuson J., Tanenhaus M. Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models. Journal of Memory and Language 1998, 38:419-439.
3. Altmann G., Kamide Y. Incremental interpretation at verbs: Restricting the domain of subsequent reference. Cognition 1999, 73:247-264.
4. Altmann G., Kamide Y. Now you see it, now you don't: Mediating the mapping between language and the visual world. The interface of language, vision, and action: Eye movements and the visual world 2004, 347-386. Psychology Press, NY. J. Henderson, F. Ferreira (Eds.).
5. Argyle M., Dean J. Eye-contact, distance and affiliation. Sociometry 1965, 28:289-304.
6. Baayen R., Davidson D., Bates D. Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language 2008, 59:390-412.
7. Bangerter A. Using pointing and describing to achieve joint focus of attention in dialogue. Psychological Science 2004, 15:415-419.
8. Baron-Cohen S. Mindblindness: An essay on autism and theory of mind 1995, MIT Press/Bradford Books, MA.
9. Baron-Cohen S., Baldwin D., Crowson M. Do children with autism use the speaker's direction of gaze strategy to crack the code of language?. Child Development 1997, 68:48-57.
10. Baron-Cohen S., Campbell R., Karmiloff-Smith A., Grant J., Walker J. Are children with autism blind to the mentalistic significance of the eyes?. British Journal of Developmental Psychology 1995, 13:379-398.
11. Baron-Cohen S., Wheelwright S., Jolliffe T. Is there a language of the eyes? Evidence from normal adults, and adults with autism or Asperger syndrom. Visual Cognition 1997, 4:311-331.
12. Bates D. Fitting linear mixed models in R. R News 2005, 5:27-30.
13. Bayliss A., Paul M., Cannon P., Tipper S. Gaze cueing and affective judgments of objects: I like what you look at. Psychonomic Bulletin & Review 2006, 13:1061-1066.
14. Becchio C., Bertone C., Castiello U. How the gaze of others influences object processing. Trends in Cognitive Science 2008, 12:254-258.
15. Beun R., Cremers A. Object reference in a shared domain of conversation. Pragmatics and Cognition 1998, 6:111-142.
16. Breazeal, C., Kidd, C., Thomaz, A., Hoffman, G., & Berlin, M. (2005). Effects of nonverbal communication on efficiency and robustness in human-robot teamwork. In Proceedings of IEEE/RSJ international conference on intelligent robots and systems (IROS'05) (pp. 708-713).
17. Bundesen C. A theory of visual attention. Psychological Review 1990, 97:523-547.
18. Cassell, J., Bickmore, T., Billinghurst, M., Campbell, L., Chang, K., Vilhjalmsson, H., et al. (1999). Embodiment in conversational interfaces: Rea. In Proceedings of the ACM conference on human factors in computing systems (CHI'99) (pp. 520-527).
19. Cassell J., Thórisson K. The power of a nod and a glance: Envelope vs. emotional feedback in animated conversational agents. Applied Artificial Intelligence 1999, 13:519-538.
20. Cassell J., Torres O., Prevost S. Turn taking vs. discourse structure: How best to model multimodal conversation. Machine Conversations 1999, 143-154. Kluwer. Y. Wilks (Ed.).
21. Castiello U. Understanding other people's actions: Intention and attention. Journal of Experimental Psychology 2003, 29:416-430.
22. Clark H.H., Krych M.A. Speaking while monitoring addressees for understanding. Journal of Memory and Language 2004, 50:62-81.
23. Cooper R.M. The control of eye fixation by the meaning of spoken language: A new methodology for the real-time investigation of speech perception, memory, and language processing. Cognitive Psychology 1974, 6:84-107.
24. Crawley M.J. The R book 2007, Wiley Publishing.
25. Crocker M.W., Knoeferle P., Mayberry M. Situated sentence processing: The coordinated interplay account and a neurobehavioral model. Brain and Language 2010, 112:189-201.
26. Desimone R., Duncan J. Neural mechanisms of selective visual attention. Annual Review of Neuroscience 1995, 18:193-222.
27. Dovidio J.F., Ellyson S.L. Decoding visual dominance: Attributions of power based on relative percentages of looking while speaking and looking while listening. Social Psychology Quarterly 1982, 45:106-113.
28. Driver J., Davis G., Ricciardelli P., Kidd P., Maxwell E., Baron-Cohen S. Gaze perception triggers reflexive visuospatial orienting. Visual Cognition 1999, 6:509-540.
29. Duncan S. Some signals and rules for taking speaking turns in conversations. Journal of Personality and Social Psychology 1972, 23:283-292.
30. Eberhard K.M., Spivey-Knowlton M.J., Sedivy J.C., Tanenhaus M.K. Eye movements as a window into real-time spoken language comprehension in natural contexts. Journal of Psycholinguistic Research 1995, 24:409-436.
31. Emery N. The eyes have it: The neuroethology, function and evolution of social gaze. Neuroscience and Biobehavioral Reviews 2000, 24:581-604.
32. Gaze-following: Its development and significance 2007, Lawrence Erlbaum Associates Publishers, Mahwah, NJ, US. R. Flom, K. Lee, D. Muir (Eds.).
33. Friesen C., Kingstone A. The eyes have it! Reflexive orienting is triggered by nonpredictive gaze. Psychonomic Bulletin & Review 1998, 5:490-495.
34. Gleitman L.R., January D., Nappa R., Trueswell J.C. On the give and take between event apprehension and utterance formulation. Journal of Memory and Language 2007, 57:544-569.
35. Griffin Z.M. Gaze durations during speech reflect word selection and phonological encoding. Cognition 2001, 82:B1-B14.
36. Griffin Z.M., Bock K. What the eyes say about speaking. Psychological Science 2000, 11:274-279.
37. Hanna J., Brennan S. Speakers' eye gaze disambiguates referring expressions early during face-to-face conversation. Journal of Memory and Language 2007, 57:596-615.
38. Heider F., Simmel M. An experimental study of apparent behavior. American Journal of Psychology 1944, 57:243-259.
39. Henderson J.M. Human gaze control during real-world scene perception. Trends in Cognitive Sciences 2003, 7:498-504.
40. Jaeger T.F. Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language 2008, 59:434-446. (special issue: emerging data analysis).
41. Kanda, T., Ishiguro, H., & Ishida, T. (2001). Psychological analysis on human-robot interaction. In Proceedings of IEEE international conference on robotics and automation (ICRA'01) (pp. 4166-4173).
42. Kaplan F., Hafner V.V. The challenges of joint attention. Interaction Studies 2006, 7:135-169.
43. Kendon A. Some functions of gaze-direction in social interaction. Acta Psychologica 1967, 26:22-63.
44. Kiesler S., Powers A., Fussell S., Torrey C. Anthropomorphic interactions with a robot and robot-like agent. Social Cognition 2008, 26:169-181.
45. Kliegl R., Masson M.E., Richter E. A linear mixed model analysis of masked repetition priming. Visual Cognition 2010, 18:655-681.
46. Kliegl R., Risse S., Laubrock J. Preview benefit and parafoveal-on-foveal effects from word n +2. Journal of Experimental Psychology: Human Perception and Performance 2007, 33:1250-1255.
47. Knoeferle P., Crocker M.W. Constituent order and semantic parallelism in online comprehension: Eye-tracking evidence from German. The Quarterly Journal of Experimental Psychology 2009, 62:2338-2371.
48. Knoeferle P., Crocker M.W., Pickering M., Scheepers C. The influence of the immediate visual context on incremental thematic role-assignment: Evidence from eye-movements in depicted events. Cognition 2005, 95:95-127.
49. Langton S.R., Bruce V. Reflexive visual orienting in response to the social attention of others. Visual Cognition 1999, 6:541-567.
50. Langton S.R., Watt R.J., Bruce V. Do the eyes have it? Cues to the direction of social attention. Trends in Cognitive Science 2000, 4:50-59.
51. Meltzoff A.N., Brooks R. Eyes wide shut: The importance of eyes in infant gaze-following and understanding of other minds. Gaze-following. Its development and significance 2007, 217-241. Lawrence Erlbaum Associates, Publishers, Mahwah, NJ. R. Flom, K. Lee, D. Muir (Eds.).
52. Meltzoff A.N., Brooks R., Shon A.P., Rao R.P. Social robots are psychological agents for infants: A test of gaze following. Neural Networks 2010, 23:966-972. (Social cognition: From babies to robots).
53. Meyer A., Sleiderink A., Levelt W. Viewing and naming objects: Eye movements during noun phrase production. Cognition 1998, 66:B25-B33.
54. Joint attention: Its origins and role in development 1995, Lawrence Erlbaum Associates, Publishers, Hillsdale, NJ. C. Moore, P. Dunham (Eds.).
55. Mutlu, B., Hodgins, J., & Forlizzi, J. (2006). A storytelling robot: Modeling and evaluation of human-like gaze behavior. In Proceedings of the 2006 IEEE-RAS international conference on humanoid robots (HUMANOIDS'06). Genova, Italy.
56. Nappa R., Wessel A., McEldoon K.L., Gleitman L.R., Trueswell J.C. Use of speaker's gaze and syntax in verb learning. Language Learning and Development 2009, 5:203-234.
57. Nass C., Moon Y. Machines and mindlessness: Social responses to computers. Journal of Social Issues 2000, 56:81-103.
58. Ristic J., Friesen C.K., Kingstone A. Are eyes special? It depends on how you look at it. Psychonomic Bulletin & Review 2002, 9:507-513.
59. Ristic J., Wright A., Kingstone A. Attentional control and reflexive orienting to gaze and arrow cues. Psychonomic Bulletin & Review 2007, 14:964-969.
60. Schroeder M., Trouvain J. The German text-to-speech synthesis system MARY: A tool for research, development and teaching. International Journal of Speech Technology 2003, 6:365-377.
61. Sedivy .J.C., Tanenhaus M.K., Chambers C.G., Carlson G.N. Achieving incremental semantic interpretation through contextual representation. Cognition 1999, 71:109-147.
62. Sidner C.L., Lee C., Kidd C., Lesh N., Rich C. Explorations in engagement for humans and robots. Artificial Intelligence 2005, 166:140-164.
63. Staudte, M. (2010). Joint Attention in Spoken Human-Robot Interaction. Ph.D. Thesis Saarland University Germany. http://scidok.sulb.uni-saarland.de/volltexte/2010/3242/.
64. Tanenhaus M.K., Spivey-Knowlton M., Eberhard K., Sedivy J. Integration of visual and linguistic information in spoken language comprehension. Science 1995, 268:1632-1634.
65. Tomasello M., Carpenter M. Shared intentionality. Developmental Science 2007, 10:121-125.
66. Tomasello M., Carpenter M., Call J., Behne T., Moll H. Understanding and sharing intentions: The origins of cultural cognition. Behavioral and Brain Sciences 2005, 28:675-691.
67. Tomlin R.S. Mapping conceptual representations into linguistic representations: The role of attention in grammar. Language and conceptualization 1997, 162-189. Cambridge University Press, Cambridge. J. Nuyts, E. Pederson (Eds.).
68. Van der Meulen F.F., Meyer A.S., Levelt W.J.M. Eye movements during the production of nouns and pronouns. Memory & Cognition 2001, 29:512-521.
69. Vecera S., Rizzo M. Eye gaze does not produce reflexive shifts of attention: Evidence from frontal-lobe damage. Neuropsychologia 2006, 44:150-159.
70. Woods, S., Walters, M., Koay, K.L., & Dautenhahn, K. (2006). Comparing human robot interaction scenarios using live and video based methods: Towards a novel methodological approach. In Proceedings of the 9th international workshop on advanced motion control (AMC'06).