Top-down attention regulates the neural expression of audiovisual integration

NeuroImage

Volumn 119, Issue , 2015, Pages 272-285

  Morís Fernández, Luis ^a   Visser, Maya ^b   Ventura Campos, Noelia ^b   Ávila, César ^b   Soto Faraco, Salvador ^a,c  

^a UNIVERSITAT POMPEU FABRA   (Spain)

^b Universidad Jaume I de Castellón   (Spain)

^c INSTITUCIÓ CATALANA DE RECERCA I ESTUDIS AVANÇATS ICREA   (Spain)

Author keywords

Attention; Audiovisual; FMRI; Multisensory; Speech perception; STS

Indexed keywords

ADULT; ANTERIOR CINGULATE; ANTERIOR INSULA; ARTICLE; AUDIOVISUAL INTEGRATION; AUDITORY STIMULATION; BEHAVIOR; CONFLICT; CONTROLLED STUDY; EXTRASTRIATE BODY AREA; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HEARING; HUMAN; HUMAN EXPERIMENT; MALE; MOTOR CORTEX; NERVE POTENTIAL; NORMAL HUMAN; PITCH; PRIORITY JOURNAL; SELECTIVE ATTENTION; SPEECH PERCEPTION; STIMULUS RESPONSE; SUPERIOR TEMPORAL GYRUS; SUPERIOR TEMPORAL SULCUS; VISION; YOUNG ADULT; ATTENTION; BRAIN; BRAIN MAPPING; NUCLEAR MAGNETIC RESONANCE IMAGING; PHOTOSTIMULATION; PHYSIOLOGY;

ACOUSTIC STIMULATION; ADULT; ATTENTION; BRAIN; BRAIN MAPPING; FEMALE; HUMANS; MAGNETIC RESONANCE IMAGING; MALE; PHOTIC STIMULATION; PITCH PERCEPTION; SPEECH PERCEPTION; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84937795802     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2015.06.052     Document Type: Article

References (87)

1. Alsius A., Soto-Faraco S. Searching for audiovisual correspondence in multiple speaker scenarios. Exp. Brain Res. 2011, 213(2-3):175-183. 10.1007/s00221-011-2624-0.
2. Alsius A., Navarra J., Campbell R., Soto-Faraco S. Audiovisual integration of speech falters under high attention demands. Curr. Biol. CB 2005, 15(9):839-843. 10.1016/j.cub.2005.03.046.
3. Alsius A., Navarra J., Soto-Faraco S. Attention to touch weakens audiovisual speech integration. Exp. Brain Res. 2007, 183(3):399-404. (http://dx.doi.org/10.1007/s00221-007-1110-1).
4. Alsius A., Möttönen R., Sams M.E., Soto-Faraco S., Tiippana K. Effect of attentional load on audiovisual speech perception: evidence from ERPs. Front. Psychol. 2014, 5(July):727. 10.3389/fpsyg.2014.00727.
5. Andersen Tobias S., Tiippana K., Laarni J., Kojo I., Sams M. The role of visual spatial attention in audiovisual speech perception. Speech Comm. 2009, 51(2):184-193. 10.1016/j.specom.2008.07.004.
6. Arnal L.H., Giraud A.-L. Cortical oscillations and sensory predictions. Trends Cogn. Sci. 2012, 16(7):390-398. 10.1016/j.tics.2012.05.003.
7. Beauchamp M.S. Statistical criteria in FMRI studies of multisensory integration. Neuroinformatics 2005, 3(2):93-113. 10.1385/NI:3:2:093.
8. Beauchamp M., Lee K., Argall B. Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 2004, 41:809-823. (Retrieved from http://www.sciencedirect.com/science/article/pii/S0896627304000704).
9. Beauchamp M.S., Argall B.D., Bodurka J., Duyn J.H., Martin A. Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat. Neurosci. 2004, 7(11):1190-1192. 10.1038/nn1333.
10. Bertelson P., Vroomen J., de Gelder B., Driver J. The ventriloquist effect does not depend on the direction of deliberate visual attention. Percept. Psychophys. 2000, 62(2):321-332. (Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/11436735).
11. Brett Matthew, Anton Jean-Luc, Romain Valabregue J.-B.P. Region of interest analysis using an SPM toolbox. 8th International Conference on Functional Mapping of the Human Brain. Sendai, Japan 2002, (Retrieved from http://marsbar.sourceforge.net/).
12. Buchan J.N., Munhall K.G. The influence of selective attention to auditory and visual speech on the integration of audiovisual speech information. Perception 2011, 40(10):1164-1182. 10.1068/p6939.
13. Buchan J.N., Munhall K.G. The effect of a concurrent working memory task and temporal offsets on the integration of auditory and visual speech information. Seeing Perceiving 2012, 25(1):87-106. 10.1163/187847611X620937.
14. Burnham D., Dodd B. Auditory-visual speech integration by prelinguistic infants: perception of an emergent consonant in the McGurk effect. Dev. Psychobiol. 2004, 45(4):204-220. 10.1002/dev.20032.
15. Calvert G.A. Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb. Cortex 2001, 11(12):1110-1123. 10.1093/cercor/11.12.1110.
16. Calvert G.a, Campbell R., Brammer M.J. Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr. Biol. CB 2000, 10(11):649-657. (Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/10837246).
17. Calvert G., Spence C., Stein B.E. The Handbook of Multisensory Processes 2004, MIT Press.
18. Campbell R., De Gelder B., De Haan E. The lateralization of lip-reading: a second look. Neuropsychologia 1996, 34(12):1235-1240. (Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/8951835).
19. Cherry E.C. Some experiments on the recognition of speech, with one and with two ears. J. Acoust. Soc. Am. 1953, 25(5):975. 10.1121/1.1907229.
20. Cox R.W. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput. Biomed. Res. Int. J. 1996, 29(3):162-173.
21. Déry C., Campbell N.K.J., Lifshitz M., Raz A. Suggestion overrides automatic audiovisual integration. Conscious. Cogn. 2014, 24:33-37. 10.1016/j.concog.2013.12.010.
22. Driver J. Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading. Nature 1996, 381(6577):66-68. 10.1038/381066a0.
23. Driver J., Noesselt T. Multisensory interplay reveals crossmodal influences on "sensory-specific" brain regions, neural responses, and judgments. Neuron 2008, 57(1):11-23. 10.1016/j.neuron.2007.12.013.
24. Driver J., Noesselt T. Multisensory interplay reveals crossmodal influences on "sensory-specific" brain regions, neural responses, and judgments. Neuron 2008, 57(1):11-23. 10.1016/j.neuron.2007.12.013.
25. Driver J., Spence C. Attention and the crossmodal construction of space. Trends Cogn. Sci. 1998, 2(7):254-262. (http://dx.doi.org/10.1016/S1364-6613(98)01188-7).
26. Ernst M.D. Permutation methods: a basis for exact inference. Stat. Sci. 2004, 19(4):676-685. 10.1214/088342304000000396.
27. Fairhall S.L., Macaluso E. Spatial attention can modulate audiovisual integration at multiple cortical and subcortical sites. Eur. J. Neurosci. 2009, 29(6):1247-1257. 10.1111/j.1460-9568.2009.06688.x.
28. Fox P.T., Huang a, Parsons L.M., Xiong J.H., Zamarippa F., Rainey L., Lancaster J.L. Location-probability profiles for the mouth region of human primary motor-sensory cortex: model and validation. NeuroImage 2001, 13(1):196-209. 10.1006/nimg.2000.0659.
29. Fujisaki W., Koene A., Arnold D., Johnston A., Nishida S. Visual search for a target changing in synchrony with an auditory signal. Proc. Biol. Sci. R. Soc. 2006, 273(1588):865-874. 10.1098/rspb.2005.3327.
30. Hill K.T., Miller L.M. Auditory attentional control and selection during cocktail party listening. Cereb. Cortex 2010, 20(3):583-590. (New York, N.Y.: 1991).
31. Jones J.a, Callan D.E. Brain activity during audiovisual speech perception: an fMRI study of the McGurk effect. Neuroreport 2003, 14(8):1129-1133. (http://dx.doi.org/10.1097/01.wnr.0000074343.81633.2a).
32. Koelewijn T., Bronkhorst A., Theeuwes J. Attention and the multiple stages of multisensory integration: a review of audiovisual studies. Acta Psychol. 2010, 134(3):372-384. 10.1016/j.actpsy.2010.03.010.
33. Kushnerenko E., Teinonen T., Volein A., Csibra G. Electrophysiological evidence of illusory audiovisual speech percept in human infants 2008, 105(32).
34. Lakens D. Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs. Front. Psychol. 2013, 4(November):863. 10.3389/fpsyg.2013.00863.
35. Lavie N. Perceptual load as a necessary condition for selective attention. J. Exp. Psychol. Hum. Percept. Perform. 1995, 21(3):451-468. (Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/7790827).
36. Lee A.K.C., Larson E., Maddox R.K., Shinn-Cunningham B.G. Using neuroimaging to understand the cortical mechanisms of auditory selective attention. Hear. Res. 2014, 307:111-120. 10.1016/j.heares.2013.06.010.
37. Macaluso E. Modulation of human visual cortex by crossmodal spatial attention. Science 2000, 289(5482):1206-1208. 10.1126/science.289.5482.1206.
38. Macaluso E., Driver J. Multisensory spatial interactions: a window onto functional integration in the human brain. Trends Neurosci. 2005, 28(5):264-271. 10.1016/j.tins.2005.03.008.
39. McGurk H., MacDonald J. Hearing lips and seeing voices. Nature 1976, 264(5588):746-748. (Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/1012311).
40. Miller L.M., D'Esposito M. Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. J. Neurosci. Off. J. Soc. Neurosci. 2005, 25(25):5884-5893. 10.1523/JNEUROSCI.0896-05.2005.
41. Nahorna O., Berthommier F., Schwartz J.-L. Binding and unbinding the auditory and visual streams in the McGurk effect. J. Acoust. Soc. Am. 2012, 132(2):1061-1077. 10.1121/1.4728187.
42. Nath A.R., Beauchamp M.S. A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion. NeuroImage 2012, 59(1):781-787. 10.1016/j.neuroimage.2011.07.024.
43. Navarra J., Alsius A., Soto-Faraco S., Spence C. Assessing the role of attention in the audiovisual integration of speech. Inf. Fusion 2010, 11(1):4-11. 10.1016/j.inffus.2009.04.001.
44. Noesselt T., Rieger J.W., Schoenfeld M.A., Kanowski M., Hinrichs H., Heinze H.-J., Driver J. Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. J. Neurosci. Off. J. Soc. Neurosci. 2007, 27(42):11431-11441. 10.1523/JNEUROSCI.2252-07.2007.
45. Noesselt T., Bergmann D., Heinze H.-J., Münte T., Spence C. Coding of multisensory temporal patterns in human superior temporal sulcus. Front. Integr. Neurosci. 2012, 6(August):64. 10.3389/fnint.2012.00064.
46. Noppeney U., Josephs O., Hocking J., Price C.J., Friston K.J. The effect of prior visual information on recognition of speech and sounds. Cereb. Cortex 2008, 18(3):598-609. (New York, N.Y.: 1991). 10.1093/cercor/bhm091.
47. Noppeney U., Ostwald D., Werner S. Perceptual decisions formed by accumulation of audiovisual evidence in prefrontal cortex. J. Neurosci. 2010, 30(21):7434-7446. 10.1523/JNEUROSCI.0455-10.2010.
48. Orr J.M., Weissman D.H. Anterior cingulate cortex makes 2 contributions to minimizing distraction. Cereb. Cortex 2009, 19(3):703-711. (New York, N.Y.: 1991). 10.1093/cercor/bhn119.
49. Pekkola J., Ojanen C.A.V., Autti T., Jììskelìinen I.P., Riikka M., Tarkiainen A., Sams M. Primary auditory cortex activation by visual speech[U+202F]: an fMRI study at 3T 2005, 16(2):5-8.
50. Pekkola J., Laasonen M., Ojanen V., Autti T., Jääskeläinen I.P., Kujala T., Sams M. Perception of matching and conflicting audiovisual speech in dyslexic and fluent readers: an fMRI study at 3T. NeuroImage 2006, 29(3):797-807. 10.1016/j.neuroimage.2005.09.069.
51. Ridderinkhof K.R., Ullsperger M., Crone E.a, Nieuwenhuis S. The role of the medial frontal cortex in cognitive control. Science (New York, N.Y.) 2004, 306(5695):443-447. 10.1126/science.1100301.
52. Roberts K.L., Hall D.a. Examining a supramodal network for conflict processing: a systematic review and novel functional magnetic resonance imaging data for related visual and auditory Stroop tasks. J. Cogn. Neurosci. 2008, 20(6):1063-1078. 10.1162/jocn.2008.20074.
53. Rosenblum L.D., Schmuckler M.a, Johnson J.a. The McGurk effect in infants. Percept. Psychophys. 1997, 59(3):347-357. (Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/9136265).
54. Ross L.a, Saint-Amour D., Leavitt V.M., Javitt D.C., Foxe J.J. Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cereb. Cortex 2007, 17(5):1147-1153. (New York, N.Y.: 1991). 10.1093/cercor /bhl024.
55. Sánchez-García C., Alsius A., Enns J.T., Soto-Faraco S. Cross-modal prediction in speech perception. PLoS ONE 2011, 6(10):e25198. 10.1371/journal.pone.0025198.
56. Schroeder C.E., Foxe J. Multisensory contributions to low-level, "unisensory" processing. Curr. Opin. Neurobiol. 2005, 15(4):454-458. 10.1016/j.conb.2005.06.008.
57. Schroeder C.E., Lakatos P. Low-frequency neuronal oscillations as instruments of sensory selection. Trends Neurosci. 2009, 32(1):9-18. 10.1016/j.tins.2008.09.012.
58. Sebastián-Gallés N., Martí M.A., Cuetos F., Carreiras M. LEXESP: Léxico Informatizado del Español 2000, Edicions de la Universitat de Barcelona, Barcelona.
59. Senkowski D., Talsma D., Herrmann C.S., Woldorff M.G. Multisensory processing and oscillatory gamma responses: effects of spatial selective attention. Exp. Brain Res. 2005, 166(3-4):411-426. 10.1007/s00221-005-2381-z.
60. Shenhav A., Botvinick M.M., Cohen J.D. The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron 2013, 79(2):217-240. 10.1016/j.neuron.2013.07.007.
61. Shinn-Cunningham B.G. Object-based auditory and visual attention. Trends Cogn. Sci. 2008, 12(5):182-186. 10.1016/j.tics.2008.02.003.
62. Skipper J.I., Nusbaum H.C., Small S.L. Listening to talking faces: motor cortical activation during speech perception. NeuroImage 2005, 25(1):76-89. 10.1016/j.neuroimage.2004.11.006.
63. Skipper J.I., van Wassenhove V., Nusbaum H.C., Small S.L. Hearing lips and seeing voices: how cortical areas supporting speech production mediate audiovisual speech perception. Cereb. Cortex 2007, 17(10):2387-2399. (New York, N.Y.: 1991). 10.1093/cercor/bhl147.
64. Soto-Faraco S., Navarra J., Alsius A. Assessing automaticity in audiovisual speech integration: evidence from the speeded classification task. Cognition 2004, 92(3):B13-B23. 10.1016/j.cognition.2003.10.005.
65. Stevenson R.a, James T.W. Audiovisual integration in human superior temporal sulcus: inverse effectiveness and the neural processing of speech and object recognition. NeuroImage 2009, 44(3):1210-1223. 10.1016/j.neuroimage.2008.09.034.
66. Stevenson R.a, Altieri N.a, Kim S., Pisoni D.B., James T.W. Neural processing of asynchronous audiovisual speech perception. NeuroImage 2010, 49(4):3308-3318. 10.1016/j.neuroimage.2009.12.001.
67. Stevenson R.a, VanDerKlok R.M., Pisoni D.B., James T.W. Discrete neural substrates underlie complementary audiovisual speech integration processes. NeuroImage 2011, 55(3):1339-1345. 10.1016/j.neuroimage.2010.12.063.
68. Sumby W.H., Pollack I. Visual contribution to speech intelligibility in noise. J. Acoust. Soc. Am. 1954, 26(2):212. 10.1121/1.1907309.
69. Szycik G.R., Jansma H., Münte T.F. Audiovisual integration during speech comprehension: an fMRI study comparing ROI-based and whole brain analyses. Hum. Brain Mapp. 2009, 30(7):1990-1999. 10.1002/hbm.20640.
70. Talsma D. Predictive coding and multisensory integration: an attentional account of the multisensory mind. Front. Integr. Neurosci. 2015, 09(March):1-13. 10.3389/fnint.2015.00019.
71. Talsma D., Woldorff M.G. Selective attention and multisensory integration: multiple phases of effects on the evoked brain activity. J. Cogn. Neurosci. 2005, 17(7):1098-1114. 10.1162/0898929054475172.
72. Talsma D., Senkowski D., Soto-Faraco S., Woldorff M.G. The multifaceted interplay between attention and multisensory integration. Trends Cogn. Sci. 2010, 14(9):400-410. (Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/20675182).
73. Teinonen T., Aslin R.N., Alku P., Csibra G. Visual speech contributes to phonetic learning in 6-month-old infants. Cognition 2008, 108(3):850-855. 10.1016/j.cognition.2008.05.009.
74. Tiippana K., Andersen T.S., Sams M. Visual attention modulates audiovisual speech perception. Eur. J. Cogn. Psychol. 2004, 16(3):457-472. 10.1080/09541440340000268.
75. Tiippana K., Puharinen H., Möttönen R., Sams M. Sound location can influence audiovisual speech perception when spatial attention is manipulated. Seeing Perceiving 2011, 24(1):67-90. 10.1163/187847511X557308.
76. Tzourio-Mazoyer N., Landeau B., Papathanassiou D., Crivello F., Etard O., Delcroix N., Joliot M., et al. Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. NeuroImage 2002, 15(1):273-289. 10.1006/nimg.2001.0978.
77. Van der Burg E., Olivers C.N.L., Bronkhorst A.W., Theeuwes J. Pip and pop: nonspatial auditory signals improve spatial visual search. J. Exp. Psychol. Hum. Percept. Perform. 2008, 34(5):1053-1065. 10.1037/0096-1523.34.5.1053.
78. Van Ee R., van Boxtel J.J.a, Parker A.L., Alais D. Multisensory congruency as a mechanism for attentional control over perceptual selection. J. Neurosci. Off. J. Soc. Neurosci. 2009, 29(37):11641-11649. 10.1523/JNEUROSCI.0873-09.2009.
79. Van Essen D.C. A population-average, landmark- and surface-based (PALS) atlas of human cerebral cortex. NeuroImage 2005, 28:635-662.
80. Van Essen D.C., Dickson J., Harwell J., Hanlon D., Anderson C.H., Drury H.A. An integrated software system for surface-based analyses of cerebral cortex. J. Am. Med. Inform. Assoc. 2001, 8(5):443-459.
81. Vroomen J., Bertelson P., de Gelder B. The ventriloquist effect does not depend on the direction of automatic visual attention. Percept. Psychophys. 2001, 63(4):651-659. (Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/11436735).
82. Wassenhove V.Van, Grant K.W., Poeppel D. Visual Speech Speeds Up the Neural Processing of Auditory Speech 2004.
83. Weissman D.H., Warner L.M., Woldorff M.G. The neural mechanisms for minimizing cross-modal distraction. J. Neurosci. Off. J. Soc. Neurosci. 2004, 24(48):10941-10949. 10.1523/JNEUROSCI.3669-04.2004.
84. Wilson S.M., Saygin A.P., Sereno M.I., Iacoboni M. Listening to speech activates motor areas involved in speech production. Nat. Neurosci. 2004, 7(7):701-702. 10.1038/nn1263.
85. Yu A.J., Dayan P., Cohen J.D. Dynamics of attentional selection under conflict: toward a rational Bayesian account. J. Exp. Psychol. Hum. Percept. Perform. 2009, 35(3):700-717. 10.1037/a0013553.
86. Zimmer U., Roberts K.C., Harshbarger T.B., Woldorff M.G. Multisensory conflict modulates the spread of visual attention across a multisensory object. NeuroImage 2010, 52(2):606-616. 10.1016/j.neuroimage.2010.04.245.
87. Zion Golumbic E., Cogan G.B., Schroeder C.E., Poeppel D. Visual input enhances selective speech envelope tracking in auditory cortex at a "cocktail party". J. Neurosci. Off. J. Soc. Neurosci. 2013, 33(4):1417-1426. 10.1523/JNEUROSCI.3675-12.2013.