Look who's talking: The deployment of visuo-spatial attention during multisensory speech processing under noisy environmental conditions

NeuroImage

Volumn 43, Issue 2, 2008, Pages 379-387

  Senkowski, Daniel ^a,b   Saint Amour, Dave ^a,c,d   Gruber, Thomas ^e   Foxe, John J ^a,f  

^a The Cognitive Neurophysiology Laboratory   (United States)

^b UNIVERSITY MEDICAL CENTER HAMBURG EPPENDORF   (Germany)

^c UNIVERSITY OF MONTREAL   (Canada)

^d UNIVERSITÉ DE MONTRÉAL   (Canada)

^e UNIVERSITY OF LEIPZIG   (Germany)

^f CITY COLLEGE OF NEW YORK   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ARTICLE; ATTENTION; CONTROLLED STUDY; DEPTH PERCEPTION; EVOKED RESPONSE; FEMALE; HUMAN; MALE; NOISE; NORMAL HUMAN; PERFORMANCE; PRIORITY JOURNAL; SPEECH; SPEECH DISCRIMINATION; STEADY STATE; STIMULATION;

ADULT; AROUSAL; ATTENTION; BRAIN MAPPING; EVOKED POTENTIALS, VISUAL; FEMALE; HUMANS; MALE; SPACE PERCEPTION; SPEECH PERCEPTION; YOUNG ADULT;

EID: 53049094490     PISSN: 10538119     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2008.06.046     Document Type: Article

References (46)

1. Ahveninen J., Jaaskelainen I.P., Raij T., Bonmassar G., Devore S., Hamalainen M., Levanen S., Lin F.H., Sams M., Shinn-Cunningham B.G., Witzel T., and Belliveau J.W. Task-modulated "what" and "where" pathways in human auditory cortex. Proc. Natl. Acad. Sci. U. S. A. 103 (2006) 14608-14613
2. Alsius A., Navarra J., Campbell R., and Soto-Faraco S. Audiovisual integration of speech falters under high attention demands. Curr. Biol. 15 (2005) 839-843
3. Alsius A., Navarra J., and Soto-Faraco S. Attention to touch weakens audiovisual speech integration. Exp. Brain Res. 183 (2007) 399-404
4. Asari H., Pearlmutter B.A., and Zador A.M. Sparse representations for the cocktail party problem. J. Neurosci. 26 (2006) 7477-7490
5. Beck D.M., and Kastner S. Stimulus context modulates competition in human extrastriate cortex. Nat. Neurosci. 8 (2005) 1110-1116
6. Besle J., Fort A., Delpuech C., and Giard M.H. Bimodal speech: early suppressive visual effects in human auditory cortex. Eur. J. Neurosci. 20 (2004) 2225-2234
7. Busse L., and Woldorff M.G. The ERP omitted stimulus response to "no-stim" events and its implications for fast-rate event-related fMRI designs. NeuroImage 18 (2003) 856-864
8. Busse L., Roberts K.C., Crist R.E., Weissman D.H., and Woldorff M.G. The spread of attention across modalities and space in a multisensory object. Proc. Natl. Acad. Sci. U. S. A. 102 (2005) 18751-18756
9. Callan D.E., Jones J.A., Munhall K., Kroos C., Callan A.M., and Vatikiotis-Bateson E. Multisensory integration sites identified by perception of spatial wavelet filtered visual speech gesture information. J. Cogn. Neurosci. 16 (2004) 805-816
10. Calvert G.A., Campbell R., and Brammer M.J. Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr. Biol. 10 (2000) 649-657
11. Carlyon R.P. How the brain separates sounds. Trends. Cogn. Sci. 8 (2004) 465-471
12. Conway A.R., Cowan N., and Bunting M.F. The cocktail party phenomenon revisited: the importance of working memory capacity. Psychon. Bull. Rev. 8 (2001) 331-335
13. Desimone R., and Duncan J. Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18 (1995) 193-222
14. Escera C., Alho K., Schroger E., and Winkler I. Involuntary attention and distractibility as evaluated with event-related brain potentials. Audiol. Neuro-otol. 5 (2000) 151-166
15. Foxe J.J., and Schroeder C.E. The case for feedforward multisensory convergence during early cortical processing. NeuroReport 16 (2005) 419-423
16. Foxe J.J., and Simpson G.V. Biasing the brain's attentional set: II. effects of selective intersensory attentional deployments on subsequent sensory processing. Exp. Brain Res. 166 (2005) 393-401
17. Foxe J.J., Simpson G.V., and Ahlfors S.P. Parieto-occipital approximately 10 Hz activity reflects anticipatory state of visual attention mechanisms. NeuroReport 9 (1998) 3929-3933
18. Ghazanfar A.A., and Schroeder C.E. Is neocortex essentially multisensory?. Trends Cogn. Sci. 10 (2006) 278-285
19. Ghazanfar A.A., Maier J.X., Hoffman K.L., and Logothetis N.K. Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. J. Neurosci. 25 (2005) 5004-5012
20. Haykin S., and Chen Z. The cocktail party problem. Neural. Comput. 17 (2005) 1875-1902
21. Herrmann C.S. Human EEG responses to 1-100 Hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena. Exp. Brain Res. 137 (2001) 346-353
22. Herrmann C.S., Mecklinger A., and Pfeifer E. Gamma responses and ERPs in a visual classification task. Clin. Neurophysiol. 110 (1999) 636-642
23. Hertrich I., Mathiak K., Lutzenberger W., Menning H., and Ackermann H. Sequential audiovisual interactions during speech perception: a whole-head MEG study. Neuropsychologia 45 (2007) 1342-1354
24. Hillyard S.A., Hinrichs H., Tempelmann C., Morgan S.T., Hansen J.C., Scheich H., and Heinze H.J. Combining steady-state visual evoked potentials and fMRI to localize brain activity during selective attention. Hum. Brain Mapp. 5 (1997) 287-292
25. Hopf J.M., Boehler C.N., Luck S.J., Tsotsos J.K., Heinze H.J., and Schoenfeld M.A. Direct neurophysiological evidence for spatial suppression surrounding the focus of attention in vision. Proc. Natl. Acad. Sci. U. S. A. 103 (2006) 1053-1058
26. Kastner S., De Weerd P., Desimone R., and Ungerleider L.G. Mechanisms of directed attention in the human extrastriate cortex as revealed by functional MRI. Science 282 (1998) 108-111
27. Kidd Jr. G., Arbogast T.L., Mason C.R., and Gallun F.J. The advantage of knowing where to listen. J. Acoust. Soc. Am. 118 (2005) 3804-3815
28. Luck S.J., Chelazzi L., Hillyard S.A., and Desimone R. Neural mechanisms of spatial selective attention in areas V1, V2, and V4 of macaque visual cortex. J. Neurophysiol. 77 (1997) 24-42
29. McGurk H., and MacDonald J. Hearing lips and seeing voices. Nature 264 (1976) 746-748
30. Miller L.M., and D'Esposito M. Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. J. Neurosci. 25 (2005) 5884-5893
31. Morgan S.T., Hansen J.C., and Hillyard S.A. Selective attention to stimulus location modulates the steady-state visual evoked potential. Proc. Natl. Acad. Sci. U. S. A. 93 (1996) 4770-4774
32. Muller M.M., Picton T.W., Valdes-Sosa P., Riera J., Teder-Salejarvi W.A., and Hillyard S.A. Effects of spatial selective attention on the steady-state visual evoked potential in the 20-28 Hz range. Brain Res. Cogn. Brain Res. 6 (1998) 249-261
33. Muller M.M., Teder-Salejarvi W., and Hillyard S.A. The time course of cortical facilitation during cued shifts of spatial attention. Nat. Neurosci. 1 (1998) 631-634
34. Muller M.M., Malinowski P., Gruber T., and Hillyard S.A. Sustained division of the attentional spotlight. Nature 424 (2003) 309-312
35. Muller N.G., Mollenhauer M., Rosler A., and Kleinschmidt A. The attentional field has a Mexican hat distribution. Vision Res. 45 (2005) 1129-1137
36. Ross L.A., Saint-Amour D., Leavitt V.M., Javitt D.C., and Foxe J.J. Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cereb. Cortex 17 (2007) 1147-1153
37. Ross L.A., Saint-Amour D., Leavitt V.M., Molholm S., Javitt D.C., and Foxe J.J. Impaired multisensory processing in schizophrenia: deficits in visual enhancement of speech comprehension under noisy environmental conditions. Schizophr. Res. 97 (2007) 173-183
38. Saint-Amour D., De Sanctis S.P., Molholm S., Ritter W., and Foxe J.J. Seeing voices: high-density electrical mapping and source-analysis of the multisensory mismatch negativity evoked during the McGurk illusion. Neuropsychologia 45 (2007) 587-597
39. Senkowski D., Talsma D., Herrmann C.S., and Woldorff M.G. Multisensory processing and oscillatory gamma responses: effects of spatial selective attention. Exp. Brain Res. 3-4 (2005) 411-426
40. Senkowski D., Molholm S., Gomez-Ramirez M., and Foxe J.J. Oscillatory beta activity predicts response speed during a multisensory audiovisual reaction time task: a high-density electrical mapping study. Cereb. Cortex 16 (2006) 1556-1565
41. Senkowski D., Talsma D., Grigutsch M., Herrmann C.S., and Woldorff M.G. Good times for multisensory integration: effects of the precision of temporal synchrony as revealed by gamma-band oscillations. Neuropsychologia 45 (2007) 561-571
42. Shomstein S., and Yantis S. Control of attention shifts between vision and audition in human cortex. J. Neurosci. 24 (2004) 10702-10706
43. Sumby W.H., and Pollack I. Visual contribution to speech intelligibility in noise. J. Acoust. Soc. Am. 26 (1954) 212-215
44. Talsma D., and Woldorff M.G. Selective attention and multisensory integration: multiple phases of effects on the evoked brain activity. J. Cogn. Neurosci. 17 (2005) 1098-1114
45. Woldorff M.G. Distortion of ERP averages due to overlap from temporally adjacent ERPs: analysis and correction. Psychophysiology 30 (1993) 98-119
46. Wood N.L., and Cowan N. The cocktail party phenomenon revisited: attention and memory in the classic selective listening procedure of Cherry (1953). J. Exp. Psychol. Gen. 124 (1995) 243-262