Comprehending how visual context influences incremental sentence processing: Insights from ERPs and picture-sentence verification

Psychophysiology

Volumn 48, Issue 4, 2011, Pages 495-506

  Knoeferle, Pia ^a   Urbach, Thomas P ^b,c   Kutas, Marta ^b,c,d,e  

^a BIELEFELD UNIVERSITY   (Germany)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

^d UNIVERSITY OF CALIFORNIA   (United States)

^e UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Event related brain potentials; Picture sentence verification; Situated comprehension

Indexed keywords

EID: 79952049109     PISSN: 00485772     EISSN: 14698986     Source Type: Journal    
DOI: 10.1111/j.1469-8986.2010.01080.x     Document Type: Article

References (52)

1. Altmann, G. T. M. (2004). Language-mediated eye movements in the absence of a visual world: The blank screen paradigm. Cognition, 93, B79-B87.
2. Barrett, S. E., & Rugg, M. D. (1990). Event-related potentials and the semantic matching of pictures. Brain and Cognition, 14, 201-212.
3. Carpenter, P. A., & Just, M. A. (1975). Sentence comprehension: A psycholinguistic model of verification. Psychological Review, 82, 45-76.
4. Carpenter, P. A., & Just, M. A. (1976). Models of sentence verification and linguistic comprehension. Psychological Review, 83, 318-322.
5. Chambers, C. G., Tanenhaus, M. K., & Magnuson, J. S. (2004). Actions and affordances in syntactic ambiguity resolution. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30, 687-696.
6. Chwilla, D. J., Brown, C., & Hagoort, P. (1995). The N400 as a function of the level of processing. Psychophysiology, 32, 274-285.
7. Clark, H. H., & Chase, W. G. (1972). On the process of comparing sentences against pictures. Cognitive Psychology, 3, 472-517.
8. Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
9. Conway, A. R. A., Kane, M. J., Bunting, M. F., Hambrick, D. Z., Wilhelm, O., & Engle, R. W. (2005). Working memory span tasks: A methodological review and user's guide. Psychonomic Bulletin & Review, 12, 769-786.
10. Dale, A. M. (1994). Source localization and spatial discriminant analysis of event-related potentials: Linear approaches. Unpublished doctoral dissertation, University of California, San Diego.
11. Daneman, M., & Carpenter, P. A. (1980). Individual differences in working memory and reading. Journal of Verbal Learning and Verbal Behavior, 19, 450-466.
12. Daneman, M., & Merikle, P. M. (1996). Working memory and language comprehension: A meta-analysis. Psychonomic Bulletin & Review, 3, 422-433.
13. D'Arcy, R. C. N., & Connolly, J. F. (1999). An event-related brain potential study of receptive speech comprehension using a modified Token Test. Neuropsychologia, 37, 1477-1489.
14. Fasteneau, P. S. (2003). Extended complex figure test. Los Angeles, CA: Western Psychological Services.
15. Federmeier, K. D., & Kutas, M. (1999). A rose by any other name: Long-term memory structure and sentence processing. Journal of Memory and Language, 41, 469-495.
16. Federmeier, K. D., & Kutas, M. (2002). Picture the difference: Electrophysiological investigations of picture processing in the cerebral hemispheres. Neuropsychologia, 40, 730-747.
17. Fischler, I., Bloom, P. A., Childers, D. G., Roucos, S. E., & Perry, N. W. J. (1983). Brain potentials related to stages of sentence verification. Psychophysiology, 20, 400-409.
18. Goolkasian, P. (1996). Picture-word differences in a sentence verification task. Memory and Cognition, 24, 584-594.
19. Gough, P. B. (1965). Grammatical transformations and speed of understanding. Journal of Verbal Learning & Verbal Behavior, 4, 107-111.
20. Griffin, D., Murray, S., & Gonzalez, R. (1999). Difference score correlations in relationship research: A conceptual primer. Personal Relationships, 6, 505-518.
21. Holcomb, P. J., & Neville, H. J. (1991). Natural speech processing: An analysis using event-related brain potentials. Psychobiology, 19, 286-300.
22. Hutcheson, G., & Sofroniou, N. (1999). The multivariate social scientist: Introductory statistics by using generalized linear models. Thousand Oaks, CA: Sage.
23. Johns, G. (1981). Difference score measures of organizational behavior variables: A critique. Organizational Behavior and Human Performance, 27, 443-463.
24. Just, M. A., & Carpenter, P. A. (1992). A capacity theory of comprehension: Individual differences in working memory. Psychological Review, 99, 122-149.
25. Knoeferle, P., Crocker, M. W., Scheepers, C., & Pickering, M. J. (2005). The influence of the immediate visual context on incremental thematic role assignment: Evidence from eye movements in depicted events. Cognition, 95, 95-127.
26. Knoeferle, P., & Crocker, M. W. (2006). The coordinated interplay of scene, utterance, and world knowledge: Evidence from eye tracking. Cognitive Science, 30, 481-529.
27. Knoeferle, P., & Crocker, M. W. (2007). The influence of recent scene events on spoken comprehension: Evidence from eye movements. Journal of Memory and Language, 57, 519-543.
28. Knoeferle, P., Habets, B., Crocker, M. W., & Münte, T. F. (2008). Visual scenes trigger immediate syntactic reanalysis. Cerebral Cortex, 18, 789-795.
29. Kounios, J., & Holcomb, P. J. (1992). Structure and process in semantic memory: Evidence from event-related brain potentials and reaction times. Journal of Experimental Psychology: General, 121, 459-479.
30. Kutas, M. (1993). In the company of other words: Electrophysiological evidence for single word versus sentence context effects. Language and Cognitive Processes, 8, 533-572.
31. Kutas, M., & Hillyard, S. A. (1980). Reading senseless sentences: Brain potentials reflect semantic incongruity. Science, 207, 203-205.
32. Kutas, M., & Hillyard, S. A. (1984). Brain potentials during reading reflect word expectancy and semantic association. Nature, 307, 161-163.
33. Kutas, M., & Van Petten, C. (1988). Event-related brain potential studies of language. In P. K. Ackles, J. R. Jennings, & M. G. H. Coles (Eds.), Advances in psychophysiology (Vol. 3, pp. 139-187). Greenwich, Connecticut: JAI Press.
34. Kutas, M., Van Petten, C. K., & Kluender, R. (2006). Psycholinguistics electrified II. In M. A. Gernsbacher & M. Traxler (Eds.), Handbook of psycholinguistics (2nd edition, pp. 659-724). New York: Elsevier Press.
35. Mayberry, M., Crocker, M. W., & Knoeferle, P. (2009). Learning to attend: A connectionist model of situated language comprehension. Cognitive Science, 33, 449-496.
36. McCallum, W., Farmer, S. F., & Pocock, P. V. (1984). The effects of physical and semantic incongruities on auditory event-related potentials. Electroencephalography and Clinical Neurophysiology/Evoked Potentials Section, 5, 477-488.
37. Meyer, D. E., & Schvaneveldt, R. W. (1971). Facilitation in recognizing pairs of words: Evidence of a dependence between retrieval operations. Journal of Experimental Psychology, 90, 227-234.
38. Otten, M., & Van Berkum, J. J. A. (2007). What makes a discourse constraining? Comparing the effects of discourse message and scenario fit on the discourse-dependent N400 effect. Brain Research, 1153, 166-177.
39. Reichle, E. D., Carpenter, P. A., & Just, M. A. (2000). The neural basis of strategy and skill in sentence-picture verification. Cognitive Psychology, 40, 261-295.
40. Sedivy, J. C., Tanenhaus, M. K., Chambers, C. G., & Carlson, G. N. (1999). Achieving incremental semantic interpretation through contextual representation. Cognition, 71, 109-148.
41. Singer, M. (2006). Verification of text ideas during reading. Journal of Memory and Language, 54, 574-591.
42. Spivey, M., Tanenhaus, M. K., Eberhard, K., & Sedivy, J. (2002). Eye movements and spoken language comprehension: Effects of visual context on syntactic ambiguity resolution. Cognitive Psychology, 45, 447-481.
43. Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18, 643-662.
44. Szucs, D., Soltész, F., Czigler, I., & Csépe, V. (2007). Electroencephalography effects to semantic and non-semantic mismatch in properties of visually presented single-characters: The N2b and the N400. Neuroscience Letters, 412, 18-23.
45. Tabachnik, B., & Fidell, L. S. (2007). Using multivariate statistics. Pearson Allyn & Bacon: Boston.
46. Tanenhaus, M. K., Carroll, J. M., & Bever, T. G. (1976). Sentence-picture verification models as theories of sentence comprehension: A critique of Carpenter and Just. Psychological Review, 83, 310-317.
47. Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995). Integration of visual and linguistic information in spoken language comprehension. Science, 268, 1632-1634.
48. Underwood, G., Jebbett, L., & Roberts, K. (2004). Inspecting pictures for information to verify a sentence: Eye movements in general encoding and in focused search. The Quarterly Journal of Experimental Psychology, 56, 165-182.
49. Van Berkum, J. J. A., Hagoort, P., & Brown, C. M. (1999). Semantic integration in sentences and discourse: Evidence from the N400. Journal of Cognitive Neuroscience, 11, 657-671.
50. Vissers, C., Kolk, H., Van de Meerendonk, N., & Chwilla, D. (2008). Monitoring in language perception: Evidence from ERPs in a picture-sentence matching task. Neuropsychologia, 46, 967-982.
51. Wassenaar, M., & Hagoort, P. (2007). Thematic role assignment in patients with Broca's aphasia: Sentence-picture matching electrified. Neuropsychologia, 45, 716-740.
52. Waters, G. S., & Caplan, D. (2001). Age, working memory, and online syntactic processing in sentence comprehension. Psychology and Aging, 16, 128-144.