Semantic integration of differently asynchronous audio-visual information in videos of real-world events in cognitive processing: An ERP study

Neuroscience Letters

Volumn 498, Issue 1, 2011, Pages 84-88

  Liu, Baolin ^a   Wu, Guangning ^a   Wang, Zhongning ^a   Ji, Xiang ^a  

^a TSINGHUA UNIVERSITY   (China)

Author keywords

Asynchronous; Multisensory; N400; P600; Video

Indexed keywords

ADULT; ARTICLE; AUDITORY SYSTEM PARAMETERS; COGNITION; ELECTROENCEPHALOGRAM; EVENT RELATED POTENTIAL; FEMALE; HUMAN; LIFE EVENT; MALE; PRIORITY JOURNAL; SEMANTICS; VIDEORECORDING; VIRTUAL REALITY; VISUAL INFORMATION; WAVEFORM;

ACOUSTIC STIMULATION; AUDITORY PERCEPTION; BRAIN; BRAIN MAPPING; ELECTROENCEPHALOGRAPHY; EVOKED POTENTIALS; FEMALE; HUMANS; MALE; PHOTIC STIMULATION; VISUAL PERCEPTION; YOUNG ADULT;

EID: 79958099783     PISSN: 03043940     EISSN: 18727972     Source Type: Journal    
DOI: 10.1016/j.neulet.2011.04.068     Document Type: Article

References (28)

1. Calvert G.A. Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb. Cortex 2001, 11:1110-1123.
2. Federmeier K.D., Wlotko E.W., De Ochoa-Dewald E., Kutas M. Multiple effects of sentential constraint on word processing. Brain Res. 2007, 1146:75-84.
3. Kelly S., Kravitz C., Hopkins M. Neural correlates of bimodal speech and gesture comprehension. Brain Lang. 2004, 89:243-260.
4. Kim A., Osterhout L. The independence of combinatory semantic processing: evidence from event-related potentials. J. Mem. Lang. 2005, 52:205-225.
5. Kutas M., Hillyard S.A. Reading between the lines: event-related brain potentials during natural sentence processing. Brain Lang. 1980, 11:354-373.
6. Li T. Handedness questionnaire (Chinese version). Acta Psychol. 1983, 15:268-276.
7. Liu B., Jin Z., Li W., Li Y., Wang Z. The pragmatic meanings conveyed by function words in Chinese sentences: an ERP study. J. Neurolinguist. 2009, 22:548-562.
8. Liu B., Jin Z., Qing Z., Wang Z. The processing of phonological, orthographical, and lexical information of Chinese characters in sentence contexts: an ERP study. Brain Res. 2011, 1372:81-91.
9. Liu B., Jin Z., Wang Z., Gong C. The influence of temporal asynchrony on multisensory integration in the processing of asynchronous audio-visual stimuli of real-world events: an event-related potential study. Neuroscience 2011, 176:254-264.
10. Liu B., Jin Z., Wang Z., Hu Y. The interaction between pictures and words: evidence from positivity offset and negativity bias. Exp. Brain Res. 2010, 201:141-153.
11. Liu B., Jin Z., Wang Z., Wu G. Chinese function words grammaticalized from content words: evidence from ERPs. J. Neurolinguist. 2010, 23:663-675.
12. Liu B., Wang Z., Jin Z. The integration processing of the visual and auditory information in videos of real-world events: an ERP study. Neurosci. Lett. 2009, 461:7-11.
13. Liu B., Wang Z., Jin Z. The effects of punctuations in Chinese sentence comprehension: an ERP study. J. Neurolinguist. 2010, 23:66-80.
14. Liu B., Wang Z., Jin Z., Song G. The influence on cognitive processing from the switches of shooting angles in videos of real-world events: an ERP study. Neurosci. Lett. 2010, 469:370-374.
15. Liu B., Wang Z., Song G., Wu G. Cognitive processing of traffic signs in immersive virtual reality environment: an ERP study. Neurosci. Lett. 2010, 485:43-48.
16. Liu B., Wang Z., Wu G., Meng X. Cognitive integration of asynchronous natural or non-natural auditory and visual information in videos of real-world events: an event-related potential study. Neuroscience 2011, 180:181-190.
17. Liu B., Xin S., Jin Z., Hu Y., Li Y. Emotional facilitation effect in the picture-word interference task: an ERP study. Brain Cogn. 2010, 72:289-299.
18. McCarthy G., Donchin E. A metric for thought: a comparison of P300 latency and response time. Science 1981, 211:77-80.
19. Moreno E.M., Federmeier K.D., Kutas M. Switching languages, switching palabras (words): an electrophysiological study of code switching. Brain Lang. 2002, 80:188-207.
20. Pijnacker J, Geurts B., van Lambalgen M., Buitelaar J., Hagoort P. Exceptions and anomalies: an ERP study on context sensitivity in autism. Neuropsychologia 2010, 48:2940-2951.
21. Reid V.M., Hoehl S., Grigutsch M., Groendahl A., Parise E., Striano T. The neural correlates of infant and adult goal prediction: evidence for semantic processing systems. Dev. Psychol. 2009, 45:620-629.
22. Reid V.M., Striano T. N400 involvement in the processing of action sequences. Neurosci. Lett. 2008, 433:93-97.
23. Schneider T.R., Debener S., Oostenveld R., Engel A.K. Enhanced EEG gamma-band activity reflects multisensory semantic matching in visual-to-auditory object priming. Neuroimage 2008, 42:1244-1254.
24. Schroeder C.E., Smiley J., Fu K.G., McGinnis T., O'Connell M.N., Hackett T.A. Anatomical mechanisms and functional implications of multisensory convergence in early cortical processing. Int. J. Psychophysiol. 2003, 50:5-17.
25. Senkowski D., Schneider T.R., Tandler F., Engel A.K. Gamma-band activity reflects multisensory matching in working memory. Exp. Brain Res. 2009, 198:363-372.
26. Sitnikova T., Holcomb P.J., Kuperberg G. Two neurocognitive mechanisms of semantic integration during the comprehension of visual real-world events. J. Cogn. Neurosci. 2008, 20:2037-2057.
27. Soltani M., Knight R.T. Neural origins of the P300. Crit. Rev. Neurobiol. 2000, 14:199-224.
28. Wu Y.C., Coulson S. Meaningful gestures: electrophysiological indices of iconic gesture comprehension. Psychophysiology 2005, 42:654-667.