Brain activation during audiovisual exposure anticipates future perception of ambiguous speech

NeuroImage

Volumn 57, Issue 4, 2011, Pages 1601-1607

  Kilian Hütten, Niclas ^a   Vroomen, Jean ^b   Formisano, Elia ^a  

^a MAASTRICHT UNIVERSITY   (Netherlands)

^b TILBURG UNIVERSITY   (Netherlands)

Author keywords

Audiovisual; Auditory perception; Cross modal recalibration; FMRI; Implicit learning; McGurk

Indexed keywords

ADULT; ANTICIPATION; ARTICLE; AUDITORY DISCRIMINATION; AUDITORY STIMULATION; BRAIN FUNCTION; BRAIN REGION; CONTROLLED STUDY; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; INTERLOBAR SULCUS; MALE; MIDDLE TEMPORAL GYRUS; NORMAL HUMAN; PARIETAL LOBE; PHONEME; PRIORITY JOURNAL; SPEECH DISCRIMINATION; VISUAL DISCRIMINATION; VISUAL STIMULATION;

BRAIN; BRAIN MAPPING; FEMALE; HUMANS; ILLUSIONS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MALE; SPEECH PERCEPTION; VISUAL PERCEPTION; YOUNG ADULT;

EID: 79960225356     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2011.05.043     Document Type: Article

References (48)

1. Baart M., Vroomen J. Phonetic recalibration does not depend on working memory. Exp. Brain Res. 2010, 203:575-582.
2. Baldo J.V., Dronkers N.F. The role of inferior parietal and inferior frontal cortex in working memory. Neuropsychology 2006, 20:529-538.
3. Beauchamp M.S., Ro T. Neural substrates of sound-touch synesthesia after a thalamic lesion. J. Neurosci. 2008, 28:13696-13702.
4. Beauchamp M.S., Nath A.R., Pasalar S. fMRI-guided transcranial magnetic stimulation reveals that the superior temporal sulcus is a cortical locus of the McGurk effect. J. Neurosci. 2010, 30:2414-2417.
5. Benoit M.M., Raij T., Lin F.H., Jaaskelainen I.P., Stufflebeam S. Primary and multisensory cortical activity is correlated with audiovisual percepts. Hum. Brain Mapp. 2010, 31:526-538.
6. Bertelson P., Vroomen J., De Gelder B. Visual recalibration of auditory speech identification: a McGurk aftereffect. Psychol. Sci. 2003, 14:592-597.
7. Calvert G.A. Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb. Cortex 2001, 11:1110-1123.
8. Calvert G.A., Campbell R. Reading speech from still and moving faces: the neural substrates of visible speech. J. Cogn. Neurosci. 2003, 15:57-70.
9. Calvert G.A., Thesen T. Multisensory integration: methodological approaches and emerging principles in the human brain. J. Physiol. Paris 2004, 98:191-205.
10. Calvert G.A., Campbell R., Brammer M.J. Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr. Biol. 2000, 10:649-657.
11. Derrfuss J., Brass M., Neumann J., von Cramon D.Y. Involvement of the inferior frontal junction in cognitive control: meta-analyses of switching and Stroop studies. Hum. Brain Mapp. 2005, 25:22-34.
12. Fingelkurts A.A., Fingelkurts A.A., Krause C.M., Mottonen R., Sams M. Cortical operational synchrony during audio-visual speech integration. Brain Lang. 2003, 85:297-312.
13. Forman S.D., Cohen J.D., Fitzgerald M., Eddy W.F., Mintun M.A., Noll D.C. Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): use of a cluster-size threshold. Magn. Reson. Med. 1995, 33:636-647.
14. Forster B., Cavina Pratesi C., Aglioti S.M., Berlucchi G. Redundant target effect and intersensory facilitation from visual-tactile interactions in simple reaction time. Exp. Brain Res. 2002, 143:480-487.
15. Frassinetti F., Bolognini N., Ladavas E. Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp. Brain Res. 2002, 147:332-343.
16. Friston K.J., Buechel C., Fink G.R., Morris J., Rolls E., Dolan R.J. Psychophysiological and modulatory interactions in neuroimaging. Neuroimage 1997, 6:218-229.
17. Friston K.J., Fletcher P., Josephs O., Holmes A., Rugg M.D., Turner R. Event-related fMRI: characterizing differential responses. Neuroimage 1998, 7:30-40.
18. Friston K.J., Holmes A.P., Price C.J., Buchel C., Worsley K.J. Multisubject fMRI studies and conjunction analyses. Neuroimage 1999, 10:385-396.
19. Gilbert C.D., Sigman M., Crist R.E. The neural basis of perceptual learning. Neuron 2001, 31:681-697.
20. Goebel R., Esposito F., Formisano E. Analysis of functional image analysis contest (FIAC) data with brainvoyager QX: from single-subject to cortically aligned group general linear model analysis and self-organizing group independent component analysis. Hum. Brain Mapp. 2006, 27:392-401.
21. Hertrich I., Dietrich S., Ackermann H. Cross-modal interactions during perception of audiovisual speech and nonspeech signals: an fMRI study. J. Cogn. Neurosci. 2011, 23:221-237.
22. Hughes H.C., Reuter Lorenz P.A., Nozawa G., Fendrich R. Visual-auditory interactions in sensorimotor processing: saccades versus manual responses. J. Exp. Psychol. Hum. Percept. Perform. 1994, 20:131-153.
23. Jancke L., Wustenberg T., Scheich H., Heinze H.J. Phonetic perception and the temporal cortex. Neuroimage 2002, 15:733-746.
24. Jones J.A., Callan D.E. Brain activity during audiovisual speech perception: an fMRI study of the McGurk effect. Neuroreport 2003, 14:1129-1133.
25. Jones J.A., Jarick M. Multisensory integration of speech signals: the relationship between space and time. Exp. Brain Res. 2006, 174:588-594.
26. Kilian-Hütten N., Valente G., Vroomen J., Formisano E. Auditory cortex encodes the perceptual interpretation of ambiguous sound. J. Neurosci. 2011, 31:1715-1720.
27. McGurk H., MacDonald J. Hearing lips and seeing voices. Nature 1976, 264:746-748.
28. Meienbrock A., Naumer M.J., Doehrmann O., Singer W., Muckli L. Retinotopic effects during spatial audio-visual integration. Neuropsychologia 2007, 45:531-539.
29. Mottonen R., Krause C.M., Tiippana K., Sams M. Processing of changes in visual speech in the human auditory cortex. Brain Res. Cogn. Brain Res. 2002, 13:417-425.
30. Myers E.B., Blumstein S.E., Walsh E., Eliassen J. Inferior frontal regions underlie the perception of phonetic category invariance. Psychol. Sci. 2009, 20:895-903.
31. Naumer M.J., van den Bosch J.J. Touching sounds: thalamocortical plasticity and the neural basis of multisensory integration. J. Neurophysiol. 2009, 102:7-8.
32. Naumer M.J., Doehrmann O., Muller N.G., Muckli L., Kaiser J., Hein G. Cortical plasticity of audio-visual object representations. Cereb. Cortex 2009, 19:1641-1653.
33. Nishitani N., Hari R. Viewing lip forms: cortical dynamics. Neuron 2002, 36:1211-1220.
34. Ojanen V., Mottonen R., Pekkola J., Jaaskelainen I.P., Joensuu R., Autti T., Sams M. Processing of audiovisual speech in Broca's area. Neuroimage 2005, 25:333-338.
35. Olson I.R., Gatenby J.C., Gore J.C. A comparison of bound and unbound audio-visual information processing in the human cerebral cortex. Brain Res. Cogn. Brain Res. 2002, 14:129-138.
36. Raizada R.D., Poldrack R.A. Selective amplification of stimulus differences during categorical processing of speech. Neuron 2007, 56:726-740.
37. Saito D.N., Yoshimura K., Kochiyama T., Okada T., Honda M., Sadato N. Cross-modal binding and activated attentional networks during audio-visual speech integration: a functional MRI study. Cereb. Cortex 2005, 15:1750-1760.
38. Schroger E., Widmann A. Speeded responses to audiovisual signal changes result from bimodal integration. Psychophysiology 1998, 35:755-759.
39. Sekiyama K., Kanno I., Miura S., Sugita Y. Auditory-visual speech perception examined by fMRI and PET. Neurosci. Res. 2003, 47:277-287.
40. Sekuler R., Sekuler A.B., Lau R. Sound alters visual motion perception. Nature 1997, 385:308.
41. Shimojo S., Shams L. Sensory modalities are not separate modalities: plasticity and interactions. Curr. Opin. Neurobiol. 2001, 11:505-509.
42. Stevenson R.A., James T.W. Audiovisual integration in human superior temporal sulcus: inverse effectiveness and the neural processing of speech and object recognition. Neuroimage 2009, 44:1210-1223.
43. van Atteveldt N., Formisano E., Goebel R., Blomert L. Integration of letters and speech sounds in the human brain. Neuron 2004, 43:271-282.
44. van Atteveldt N.M., Formisano E., Blomert L., Goebel R. The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. Cereb. Cortex 2007, 17:962-974.
45. Vroomen J., van Linden S., de Gelder B., Bertelson P. Visual recalibration and selective adaptation in auditory-visual speech perception: contrasting build-up courses. Neuropsychologia 2007, 45:572-577.
46. Werner S., Noppeney U. Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Cereb. Cortex 2009.
47. Wiersinga Post E., Tomaskovic S., Slabu L., Renken R., de Smit F., Duifhuis H. Decreased BOLD responses in audiovisual processing. Neuroreport 2010, 21:1146-1151.
48. Wright T.M., Pelphrey K.A., Allison T., McKeown M.J., McCarthy G. Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cereb. Cortex 2003, 13:1034-1043.