A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion

NeuroImage

Volumn 59, Issue 1, 2012, Pages 781-787

  Nath, Audrey R ^a   Beauchamp, Michael S ^a  

^a UNIVERSITY OF TEXAS MEDICAL SCHOOL   (United States)

Author keywords

Audiovisual integration; fMRI; McGurk; Speech; STS

Indexed keywords

ADULT; ARTICLE; AUDITORY CORTEX; BOLD SIGNAL; BRAIN REGION; CONTROLLED STUDY; CORRELATION ANALYSIS; ELECTROENCEPHALOGRAM; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; ILLUSION; INFERIOR FRONTAL GYRUS; MALE; MCGURK EFFECT; NORMAL HUMAN; PERCEPTION; PRIORITY JOURNAL; SPEECH; SUPERIOR TEMPORAL SULCUS;

ADULT; BRAIN; BRAIN MAPPING; FEMALE; HUMANS; ILLUSIONS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MALE; SPEECH PERCEPTION; VISUAL PERCEPTION;

EID: 80054124202     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2011.07.024     Document Type: Article

References (58)

1. Argall B.D., Saad Z.S., Beauchamp M.S. Simplified intersubject averaging on the cortical surface using SUMA. Hum. Brain Mapp. 2006, 27:14-27.
2. Aziz-Zadeh L., Sheng T., Gheytanchi A. Common premotor regions for the perception and production of prosody and correlations with empathy and prosodic ability. PLoS One 2010, 5:e8759.
3. Barraclough N.E., Xiao D., Baker C.I., Oram M.W., Perrett D.I. Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions. J. Cogn. Neurosci. 2005, 17:377-391.
4. Baynes K., Funnell M.G., Fowler C.A. Hemispheric contributions to the integration of visual and auditory information in speech perception. Percept. Psychophys. 1994, 55:633-641.
5. Beauchamp M.S. Statistical criteria in FMRI studies of multisensory integration. Neuroinformatics 2005, 3:93-114.
6. Beauchamp M.S., Lee K.E., Argall B.D., Martin A. Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 2004, 41:809-823.
7. Beauchamp M.S., Yasar N.E., Kishan N., Ro T. Human MST but not MT responds to tactile stimulation. J. Neurosci. 2007, 27:8261-8267.
8. Beauchamp M.S., Yasar N.E., Frye R.E., Ro T. Touch, sound and vision in human superior temporal sulcus. NeuroImage 2008, 41:1011-1020.
9. Beauchamp M.S., Nath A.R., Pasalar S. fMRI-guided transcranial magnetic stimulation reveals that the superior temporal sulcus is a cortical locus of the McGurk effect. J. Neurosci. 2010, 30:2414-2417.
10. Belin P., Fecteau S., Bedard C. Thinking the voice: neural correlates of voice perception. Trends Cogn. Sci. 2004, 8:129-135.
11. Benoit M.M., Raij T., Lin F.-H., Jaaskelainen I.P., Stufflebeam S. Primary and multisensory cortical activity is correlated with audiovisual percepts. Hum. Brain Mapp. 2010, 31:526-538.
12. Bernstein L.E., Jiang J., Pantazis D., Lu Z.-L., Joshi A. Visual phonetic processing localized using speech and nonspeech face gestures in video and point-light displays. Hum. Brain Mapp. 2011, 10.1002/hbm.21139.
13. Bovo R., Ciorba A., Prosser S., Martini A. The McGurk phenomenon in Italian listeners. Acta Otorhinolaryngol. Ital. 2009, 29:203-208.
14. Callan D.E., Jones J.A., Munhall K., Kroos C., Callan A.M., Vatikiotis-Bateson E. Multisensory integration sites identified by perception of spatial wavelet filtered visual speech gesture information. J. Cogn. Neurosci. 2004, 16:805-816.
15. Calvert G.A., Campbell R., Brammer M.J. Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr. Biol. 2000, 10:649-657.
16. Campbell R. The processing of audio-visual speech: empirical and neural bases. Philos. Trans. R. Soc. B 2008, 363:1001-1010.
17. Colin C., Radeau M., Deltenre P. Top-down and bottom-up modulation of audiovisual integration in speech. Eur. J. Cogn. Psychol. 2005, 17:541-560.
18. Cox R.W. AFNI: Software for analysis and visualization of functional magnetic resonance neuroimages. Comput. Biomed. Res. 1996, 29:162-173.
19. Dahl C.D., Logothetis N.K., Kayser C. Spatial organization of multisensory responses in temporal association cortex. J. Neurosci. 2009, 29:11924-11932.
20. Dale A.M., Fischl B., Sereno M.I. Cortical surface-based analysis. I. Segmentation and surface reconstruction. NeuroImage 1999, 9:179-194.
21. Eisner F., McGettigan C., Faulkner A., Rosen S., Scott S.K. Inferior frontal gyrus activation predicts individual differences in perceptual learning of cochlear-implant simulations. J. Neurosci. 2010, 30:7179-7186.
22. Fischl B., Sereno M.I., Dale A.M. Cortical surface-based analysis. II: Inflation, flattening, and a surface-based coordinate system. NeuroImage 1999, 9:195-207.
23. Fischl B., van der Kouwe A., Destrieux C., Halgren E., Segonne F., Salat D.H., Busa E., Seidman L.J., Goldstein J., Kennedy D., Caviness V., Makris N., Rosen B., Dale A.M. Automatically parcellating the human cerebral cortex. Cereb. Cortex 2004, 14:11-22.
24. Genovese C.R., Lazar N.A., Nichols T. Thresholding of statistical maps in functional neuroimaging using the false discovery rate. NeuroImage 2002, 15:870-878.
25. Gentilucci M., Cattaneo L. Automatic audiovisual integration in speech perception. Exp. Brain Res. 2005, 167:66-75.
26. Grant K.W., Seitz P.F. The use of visible speech cues for improving auditory detection of spoken sentences. J. Acoust. Soc. Am. 2000, 108:1197-1208.
27. Hall D.A., Fussell C., Summerfield A.Q. Reading fluent speech from talking faces: typical brain networks and individual differences. J. Cogn. Neurosci. 2005, 17:939-953.
28. Hasson U., Skipper J.I., Nusbaum H.C., Small S.L. Abstract coding of audiovisual speech: beyond sensory representation. Neuron 2007, 56:1116-1126.
29. Hickok G., Poeppel D. The cortical organization of speech perception. Nat. Rev. Neurosci. 2007, 8:393-402.
30. Jones J.A., Callan D.E. Brain activity during audiovisual speech perception: an fMRI study of the McGurk effect. Neuroreport 2003, 14:1129-1133.
31. Laurienti P.J., Burdette J.H., Wallace M.T., Yen Y.F., Field A.S., Stein B.E. Deactivation of sensory-specific cortex by cross-modal stimuli. J. Cogn. Neurosci. 2002, 14:420-429.
32. Ludman C.N., Summerfield A.Q., Hall D., Elliott M.R., Foster J., Hykin J.L., Bowtell R., Morris P.G. Lip-reading ability and patterns of cortical activation studied using fMRI. Br. J. Audiol. 2000, 34:225-230.
33. Macaluso E., George N., Dolan R., Spence C., Driver J. Spatial and temporal factors during processing of audiovisual speech: a PET study. NeuroImage 2004, 21:725-732.
34. MacDonald J.D., Andersen S., Bachmann T. Hearing by eye: how much spatial degradation can be tolerated?. Perception 2000, 29:1155-1168.
35. McGurk H., MacDonald J.W. Hearing lips and seeing voices. Nature 1976, 264:746-748.
36. Mei L., Chen C., Xue G., He Q., Li T., Xue F., Yang Q., Dong Q. Neural predictors of auditory word learning. Neuroreport 2008, 19:215-219.
37. Miller L.M., D'Esposito M. Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. J. Neurosci. 2005, 25:5884-5893.
38. Nath A.R., Beauchamp M.S. Dynamic changes in superior temporal sulcus connectivity during perception of noisy audiovisual speech. J. Neurosci. 2011, 31:1704-1714.
39. Noesselt T., Rieger J.W., Schoenfeld M.A., Kanowski M., Hinrichs H., Heinze H.J., Driver J. Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. J. Neurosci. 2007, 27:11431-11441.
40. Norrix L.W., Plante E., Vance R. Auditory-visual speech integration by adults with and without language-learning disabilities. J. Commun. Disord. 2006, 39:22-36.
41. Oden G.C., Massaro D.W. Integration of featural information in speech perception. Psychol. Rev. 1978, 85:172-191.
42. Olson I.R., Gatenby J.C., Gore J.C. A comparison of bound and unbound audio-visual information processing in human cerebral cortex. Cogn. Brain Res. 2002, 14:129-138.
43. Poeppel D., Wharton C., Fritz J., Guillemin A., San Jose L., Thompson J., Bavelier D., Braun A. FM sweeps, syllables and word stimuli differentially modulate left and right non-primary auditory areas. Neuropsychologia 2004, 42:183-200.
44. Poeppel D., Idsardi W.J., van Wassenhove V. Speech perception at the interface of neurobiology and linguistics. Philos. Trans. R. Soc. B 2008, 363:1071-1086.
45. Saad Z.S., Glen D.R., Chen G., Beauchamp M.S., Desai R., Cox R.W. A new method for improving functional-to-structural MRI alignment using local Pearson correlation. NeuroImage 2009, 44:839-848.
46. Sams M., Manninen P., Surakka V., Helin P., Katto R. McGurk effect in Finnish syllables, isolated words, and words in sentences: effects of word meaning and sentence context. Speech Commun. 1998, 26:75-87.
47. Samuel A.G. Speech perception. Annu. Rev. Psychol. 2011, 62:49-72.
48. Scott S.K., Johnsrude I.S. The neuroanatomical and functional organization of speech perception. Trends Neurosci. 2003, 26:100-107.
49. Sekiyama K. McGurk effect and incompatibility: a cross-language study on auditory-visual speech perception. Studies and essays. Behav. Sci. Philos. 1994, 14:29-62.
50. Sekiyama, K., Tohkura, Y., 1991. McGurk effect in non-English listeners: few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility. J. Acoust. Soc. Am. 90 (4 Pt 1), 1797-1805.
51. Sekiyama K., Kanno I., Miura S., Sugita Y. Auditory-visual speech perception examined by fMRI and PET. Neurosci. Res. 2003, 47:277-287.
52. Stein B.E., Meredith M.A. The Merging of the Senses 1993, MIT Press.
53. Stevenson R.A., James T.W. Audiovisual integration in human superior temporal sulcus: inverse effectiveness and the neural processing of speech and object recognition. NeuroImage 2009, 44:1210-1223.
54. Van Atteveldt N., Formisano E., Goebel R., Blomert L. Integration of letters and speech sounds in the human brain. Neuron 2004, 43:271-282.
55. Werner S., Noppeney U. Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Cereb. Cortex 2010, 20:1829-1842.
56. Wilson M. The MRC Psycholinguistic Database: Machine Readable Dictionary, Version 2. Behav. Res. Methods Instrum. Comput. 1988, 20:6-11.
57. Wong P.C., Perrachione T.K., Parrish T.B. Neural characteristics of successful and less successful speech and word learning in adults. Hum. Brain Mapp. 2007, 28:995-1006.
58. Wright T.M., Pelphrey K.A., Allison T., McKeown M.J., McCarthy G. Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cereb. Cortex 2003, 13:1034-1043.