I see what you're saying: The integration of complex speech and scenes during language comprehension

Acta Psychologica

Volumn 137, Issue 2, 2011, Pages 208-216

  Andersson, Richard ^a   Ferreira, Fernanda ^b   Henderson, John M ^b  

^a LUND UNIVERSITY COGNITIVE SCIENCE   (Sweden)

^b UNIVERSITY OF SOUTH CAROLINA   (United States)

Author keywords

Attention; Eye movements; Language comprehension; Scene perception; Visual world

Indexed keywords

ADOLESCENT; ADULT; ARTICLE; AUDITORY STIMULATION; COMPREHENSION; EYE MOVEMENT; FEMALE; HUMAN; LANGUAGE; MALE; PHOTOSTIMULATION; PHYSIOLOGY; SPEECH PERCEPTION; VISION;

ACOUSTIC STIMULATION; ADOLESCENT; ADULT; COMPREHENSION; EYE MOVEMENTS; FEMALE; HUMANS; LANGUAGE; MALE; PHOTIC STIMULATION; SPEECH PERCEPTION; VISUAL PERCEPTION;

EID: 79957940673     PISSN: 00016918     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.actpsy.2011.01.007     Document Type: Article

References (34)

1. Allopenna P.D., Magnuson J.S., Tanenhaus M.K. Tracking the time course of spoken word recognition using eye movements: Evidence for continuous mapping models. Journal of Memory and Language 1998, 38:419-439.
2. Altmann G.T., Kamide Y. Incremental interpretation at verbs: Restricting the domain of subsequent reference. Cognition 1999, 73:247-264.
3. Altmann G.T.M., Kamide Y. The real-time mediation of visual attention by language and world knowledge: Linking anticipatory (and other) eye movements to linguistic processing. Journal of Memory and Language 2007, 57:502-518.
4. Baayen R.H. languageR: Data sets and functions with "analyzing linguistic data: A practical introduction to statistics" 2007, (R package version 1.0).
5. Barr D.J. Analyzing "visual world" eyetracking data using multilevel logistic regression. Journal of Memory and Language 2008, 59:457-474.
6. Bates D., Maechler M. lme4: Linear mixed-effects models using S4 classes R package version 0.999375-37. http://CRAN.R-project.org/package=lme4.
7. Boersma P., Weenink D. Praat: Doing phonetics by computer (Version 4.5.17) [Computer program]. http://www.praat.org.
8. Brown-Schmidt S., Campana E., Tanenhaus M.K. Real-time reference resolution in a referential communication task. Processing world-situated language: Bridging the language-as-action and language-as-product traditions 2005, MIT Press. J.C. Trueswell, M.K. Tanenhaus (Eds.).
9. Brown-Schmidt S., Tanenhaus M.K. Real-time investigation of referential domains in unscripted conversation: A targeted language game approach. Cognitive Science. 2008, 32(4):643-684.
10. Castelhano M.S., Henderson J.M. Initial scene representations facilitate eye movement guidance in visual search. Journal of Experimental Psychology: Human Perception and Performance 2007, 33(4):753-763.
11. Dahan D., Gaskell M.G. Temporal dynamics of ambiguity resolution: Evidence from spoken-word recognition. Journal of Memory and Language 2007, 57:483-501.
12. Dahan D., Tanenhaus M.K. Looking at the rope when looking for the snake: Conceptually mediated eye movements during spoken-word recognition. Psychological Bulletin & Review 2005, 12:455-459.
13. Eberhard K.M., Spivey-Knowlton M.J., Sedivy J.C., Tanenhaus M.K. Eye movements as a window into real-time spoken language comprehension in natural contexts. Journal of Psycholinguistic Research 1995, 24(6):409-436.
14. Ferreira F., Apel J., Henderson J.M. Taking a new look at looking at nothing. Trends in Cognitive Sciences 2008, 12:405-410.
15. Ferreira F., Tanenhaus M.K. Introduction to the special issue on language-vision interactions. Journal of Memory and Language 2007, 57:455-459.
16. Gibson B.S., Eberhard K.M., Bryant T.A. Linguistically mediated visual search: The critical role of speech rate. Psychonomic Bulletin & Review 2005, 12:276-281.
17. Hanna J.E., Tanenhaus M.K. Pragmatic effects on reference resolution in a collaborative task: Evidence from eye movements. Cognitive Science 2004, 28:105-115.
18. Henderson J.M., Ferreira F. Scene perception for psycholinguists. The interface of language, vision, and action: Eye movements and the visual world 2004, Psychology Press, New York. J.M. Henderson, F. Ferreira (Eds.).
19. Huettig F., Altmann G.T.M. Word meaning and the control of eye fixation: Semantic competitor effects and the visual world paradigm. Cognition 2005, 96(1):B23-B32. 10.1016/j.cognition.2004.10.003.
20. Huettig F., Altmann G.T.M. Visual-shape competition during language-mediated attention is based on lexical input and not modulated by contextual appropriateness. Visual Cognition 2007, 15(8):985-1018. 10.1080/13506280601130875.
21. Huettig F., McQueen J.M. The tug of war between phonological, semantic and shape information in language-mediated visual search. Journal of Memory and Language 2007, 57(4):460-482. 10.1016/j.jml.2007.02.001.
22. Kamide Y., Altmann G.T., Haywood S.L. The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye movements. Journal of Memory and Language 2003, 49:133-156.
23. Knoeferle P., Crocker M.W., Scheepers C., Pickering M. The influence of the immediate visual context on incremental thematic role-assignment: Evidence from eye-movements in depicted events. Cognition 2005, 95:95-127.
24. Kronmüller E., Barr D. Perspective-free pragmatics: Broken precedents and the recovery-from-preemption hypothesis. The Journal of Memory and Language 2007, 56:436-455.
25. R Development Core Team R: A language and environment for statistical computing (Version 2.8.1) [Computer program]. Retrieved Dec 23, 2008, from. http://www.r-project.org.
26. Rayner K. Eye movements in reading and information processing: 20 years of research. Psychological Bulletin 1998, 124(3):372-422.
27. Richardson D.C., Altmann G.T.M., Spivey M.J., Hoover M.A. Much ado about eye movements to nothing: A reply to "Taking a new look at looking at nothing" by Ferreira, Apel & Henderson. Trends in Cognitive Science 2009, 13:235-236.
28. Scheipl F. RLRsim: Exact (restricted) likelihood ratio tests for mixed and additive models R package version 1.0. http://cran.r-project.org/web/packages/RLRsim/index.html.
29. Tanenhaus M.K., Brown-Schmidt S. Language processing in the natural world. Philosophical transactions of the royal society B: Biological sciences 2008, 363:1105-1122. B.C.M. Moore, L.K. Tyler, W.D. Marslen-Wilson (Eds.).
30. Tanenhaus M.K., Magnuson J.S., Dahan D., Chambers C. Eye movements and lexical access in spoken language comprehension: Evaluating a linking hypothesis between fixations and linguistic processing. Journal of Psycholinguistic Research 2000, 29:557-580.
31. Tanenhaus M.K., Spivey M.J., Eberhard K.M., Sedivy J.C. Integration of visual and linguistic information in spoken language comprehension. Science 1995, 268:1632-1634.
32. Torralba A., Oliva A., Castelhano M., Henderson J.M. Contextual guidance of attention in natural scenes: The role of global features on object search. Psychological Review 2006, 113(4):766-786.
33. Yee E., Sedivy J. Eye movements reveal transient semantic activation during spoken word recognition. Journal of Experimental Psychology: Learning Memory & Cognition 2006, 32(1):1-14.
34. Zelinsky G.J., Murphy G.L. Synchronizing visual and language processing: An effect of object name length on oculomotor behavior. Psychological Science 2000, 11:125-131.