Dynamic changes in superior temporal sulcus connectivity during perception of noisy audiovisual speech

Journal of Neuroscience

Volumn 31, Issue 5, 2011, Pages 1704-1714

  Nath, Audrey R ^a   Beauchamp, Michael S ^a  

^a UNIVERSITY OF TEXAS   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ARTICLE; AUDIOVISUAL EQUIPMENT; AUDITORY CORTEX; AUDITORY STIMULATION; BEHAVIOR; BLOOD OXYGENATION; BRAIN REGION; CONTROLLED STUDY; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HEARING; HUMAN; HUMAN EXPERIMENT; MALE; NEUROIMAGING; NEUROPHYSIOLOGY; NOISE; OCCIPITAL CORTEX; PRIORITY JOURNAL; RELIABILITY; SPEECH; STIMULUS RESPONSE; SUPERIOR TEMPORAL SULCUS; TASK PERFORMANCE; VISUAL CORTEX;

ACOUSTIC STIMULATION; ADULT; AUDITORY CORTEX; AUDITORY PERCEPTION; FEMALE; HUMANS; MAGNETIC RESONANCE IMAGING; MALE; NEUROPSYCHOLOGICAL TESTS; NOISE; OXYGEN; PHOTIC STIMULATION; SPEECH; TEMPORAL LOBE; VISUAL CORTEX; VISUAL PERCEPTION;

EID: 79551508307     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.4853-10.2011     Document Type: Article

References (113)

1. Alais D, Burr D (2004) The ventriloquist effect results from near-optimal bimodal integration. Curr Biol 14:257-262.
2. Argall BD, Saad ZS, Beauchamp MS (2006) Simplified intersubject averaging on the cortical surface using SUMA. Hum Brain Mapp 27:14-27.
3. Beauchamp MS (2005a) Statistical criteria in FMRI studies of multisensory integration. Neuroinformatics 3:93-113.
4. Beauchamp MS (2005b) See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex. Curr Opin Neurobiol 15:145-153.
5. Beauchamp MS, Cox RW, DeYoe EA (1997) Graded effects of spatial and featural attention on human area MT and associated motion processing areas. J Neurophysiol 78:516-520.
6. Beauchamp MS, Lee KE, Haxby JV, Martin A (2002) Parallel visual motion processing streams for manipulable objects and human movements. Neuron 34:149-159.
7. Beauchamp MS, Lee KE, Haxby JV, Martin A (2003) fMRI responses to video and point-light displays of moving humans and manipulable objects. J Cogn Neurosci 15:991-1001.
8. Beauchamp MS, Lee KE, Argall BD, Martin A (2004) Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41:809-823.
9. Beauchamp MS, Yasar NE, Frye RE, Ro T (2008) Touch, sound and vision in human superior temporal sulcus. Neuroimage 41:1011-1020.
10. Beauchamp MS, Pasalar S, Ro T (2010a) Neural substrates of reliability-weighted visual-tactile multisensory integration. Front Syst Neurosci 4:25.
11. Beauchamp MS, Nath AR, Pasalar S (2010b) fMRI-guided transcranial magnetic stimulation reveals that the superior temporal sulcus is a cortical locus of the McGurk effect. J Neurosci 30:2414-2417.
12. Belin P, Fecteau S, Bédard C (2004) Thinking the voice: neural correlates of voice perception. Trends Cogn Sci 8:129-135.
13. Bernstein LE, Lu ZL, Jiang J (2008) Quantified acoustic-optical speech signal incongruity identifies cortical sites of audiovisual speech processing. Brain Res 1242:172-184.
14. Bernstein LE, Jiang J, Pantazis D, Lu ZL, Joshi A (2010) Visual phonetic processing localized using speech and nonspeech face gestures in video and point-light displays. Hum Brain Mapp. Advance online publication. Retrieved September 30, 2010. doi:10.1002/hbm.21139.
15. Binder JR, Frost JA, Hammeke TA, Cox RW, Rao SM, Prieto T (1997) Human brain language areas identified by functional magnetic resonance imaging. J Neurosci 17:353-362.
16. Branch C, Milner B, Rasmussen T (1964) Intracarotid sodium amytal for the lateralization of cerebral speech dominance; observations in 123 patients. J Neurosurg 21:399-405.
17. Büchel C, Friston K (2001) Interactions among neuronal systems assessed with functional neuroimaging. Rev Neurol (Paris) 157:807-815.
18. Büchel C, Friston KJ (1997) Modulation of connectivity in visual pathways by attention: cortical interactions evaluated with structural equation modelling and fMRI. Cereb Cortex 7:768-778.
19. Callan DE, Jones JA, Munhall K, Kroos C, Callan AM, Vatikiotis-Bateson E (2004) Multisensory integration sites identified by perception of spatial wavelet filtered visual speech gesture information. J Cogn Neurosci 16:805-816.
20. Calvert GA, Campbell R, Brammer MJ (2000) Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr Biol 10:649-657.
21. Campanella S, Belin P (2007) Integrating face and voice in person perception. Trends Cogn Sci 11:535-543.
22. Campbell R (2008) The processing of audio-visual speech: empirical and neural bases. Philos Trans R Soc Lond B Biol Sci 363:1001-1010.
23. Chen G, Glen DR, Stein JL, Meyer-Lindenberg AS, Saad ZS, Cox RW (2007) Model validation and automated search in fMRI path analysis: a fast open-source tool for structural equation modeling. Paper presented at the Human Brain Mapping Conference, Chicago, IL, June.
24. Cox RW (1996) AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput Biomed Res 29:162-173.
25. Dahan D, Mead RL (2010) Context-conditioned generalization in adaptation to distorted speech. J Exp Psychol Hum Percept Perform 36:704-728.
26. Dahl CD, Logothetis NK, Kayser C (2009) Spatial organization of multisensory responses in temporal association cortex. J Neurosci 29:11924-11932.
27. Dale AM (1999) Optimal experimental design for event-related fMRI. Hum Brain Mapp 8:109-114.
28. Dale AM, Fischl B, Sereno MI (1999) Cortical surface-based analysis. I. Segmentation and surface reconstruction. Neuroimage 9:179-194.
29. Davis MH, Johnsrude IS, Hervais-Adelman A, Taylor K, McGettigan C (2005) Lexical information drives perceptual learning of distorted speech: evidence from the comprehension of noise-vocoded sentences. J Exp Psychol Gen 134:222-241.
30. de la Mothe LA, Blumell S, Kajikawa Y, Hackett TA (2006) Cortical connections of the auditory cortex in marmoset monkeys: core and medial belt regions. J Comp Neurol 496:27-71.
31. de Marco G, Vrignaud P, Destrieux C, de Marco D, Testelin S, Devauchelle B, Berquin P (2009) Principle of structural equation modeling for exploring functional interactivity within a putative network of interconnected brain areas. Magn Reson Imaging 27:1-12.
32. Downing PE, Jiang Y, Shuman M, Kanwisher N (2001) A cortical area selective for visual processing of the human body. Science 293:2470-2473.
33. Dumoulin SO, Bittar RG, Kabani NJ, Baker CL Jr, Le Goualher G, Bruce Pike G, Evans AC (2000) A new anatomical landmark for reliable identification of human area V5/MT: a quantitative analysis of sulcal patterning. Cereb Cortex 10:454-463.
34. Dupont S, Luettin J (2000) Audio-visual speech modeling for continuous speech recognition. IEEE Trans Multimedia 2:141-151.
35. Ellmore TM, Beauchamp MS, Breier JI, Slater JD, Kalamangalam GP, O'Neill TJ, Disano MA, Tandon N (2010) Temporal lobe white matter asymmetry and language laterality in epilepsy patients. Neuroimage 49:2033-2044.
36. Ernst MO, Banks MS (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415:429-433.
37. Felleman DJ, Van Essen DC (1991) Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex 1:1-47.
38. Fischl B, Sereno MI, Dale AM (1999) Cortical surface-based analysis. II: Inflation, flattening, and a surface-based coordinate system. Neuroimage 9:195-207.
39. Friston KJ, Büchel C (2000) Attentional modulation of effective connectivity from V2 to V5/MT in humans. Proc Natl Acad Sci U S A 97:7591-7596.
40. Friston KJ, Buechel C, Fink GR, Morris J, Rolls E, Dolan RJ (1997) Psychophysiological and modulatory interactions in neuroimaging. Neuroimage 6:218-229.
41. Fu CHY, McIntosh AR, Kim J, Chau W, Bullmore ET, Williams SCR, Honey GD, McGuire PK (2006) Modulation of effective connectivity by cognitive demand in phonological verbal fluency. Neuroimage 30:266-271.
42. Genovese CR, Lazar NA, Nichols T (2002) Thresholding of statistical maps in functional neuroimaging using the false discovery rate. Neuroimage 15:870-878.
43. Ghose GM (2009) Attentional modulation of visual responses by flexible input gain. J Neurophysiol 101:2089-2106.
44. Giraud AL, Kell C, Thierfelder C, Sterzer P, Russ MO, Preibisch C, Kleinschmidt A (2004) Contributions of sensory input, auditory search and verbal comprehension to cortical activity during speech processing. Cereb Cortex 14:247-255.
45. Girin L, Schwartz J-L, Feng G (2001) Audio-visual enhancement of speech in noise. J Acoust Soc Am 109:3007-3020.
46. Gordon MS, Allen S (2009) Audiovisual speech in older and younger adults: integrating a distorted visual signal with speech in noise. Exp Aging Res 35:202-219.
47. Grant KW, Seitz PF (2000) The use of visible speech cues for improving auditory detection of spoken sentences. J Acoust Soc Am 108:1197-1208.
48. Gruber O, Müller T, Falkai P (2007) Dynamic interactions between neural systems underlying different components of verbal working memory. J Neural Transm 114:1047-1050.
49. Hampson M, Peterson BS, Skudlarski P, Gatenby JC, Gore JC (2002) Detection of functional connectivity using temporal correlations in MR images. Hum Brain Mapp 15:247-262.
50. Hauser MD (1996) The evolution of communication. Cambridge, MA: MIT Press.
51. Hein G, Knight RT (2008) Superior temporal sulcus - it's my area: or is it? J Cogn Neurosci 20:2125-2136.
52. Helbig HB, Ernst MO (2008) Visual-haptic cue weighting is independent of modality-specific attention. J Vis 8:21 21-16.
53. Hesselmann G, Sadaghiani S, Friston KJ, Kleinschmidt A (2010) Predictive coding or evidence accumulation? False inference and neuronal fluctuations. PLoS ONE 5:e9926.
54. Hickok G, Poeppel D (2007) The cortical organization of speech perception. Nat Rev Neurosci 8:393-402.
55. Horwitz B, Braun AR (2004) Brain network interactions in auditory, visual and linguistic processing. Brain Lang 89:377-384.
56. Horwitz B, Warner B, Fitzer J, Tagamets MA, Husain FT, Long TW (2005) Investigating the neural basis for functional and effective connectivity. Application to fMRI. Philos Trans R Soc Lond B Biol Sci 360:1093-1108.
57. Ihaka R, Gentleman R (1996) R: a language for data analysis and graphics. J Comput Graph Stat 5:299-314.
58. Kanwisher N, Yovel G (2006) The fusiform face area: a cortical region specialized for the perception of faces. Philos Trans R Soc Lond B Biol Sci 361:2109-2128.
59. Kersten D, Mamassian P, Yuille A (2004) Object perception as Bayesian inference. Annu Rev Psychol 55:271-304.
60. Kreifelts B, Ethofer T, Grodd W, Erb M, Wildgruber D (2007) Audiovisual integration of emotional signals in voice and face: an event-related fMRI study. Neuroimage 37:1445-1456.
61. Kriegeskorte N, Simmons WK, Bellgowan PS, Baker CI (2009) Circular analysis in systems neuroscience: the dangers of double dipping. Nat Neurosci 12:535-540.
62. Kryter KD (1996) Handbook of hearing and the effects of noise. New York: New York Academic.
63. Lee J, Maunsell JH (2009) Anormalization model of attentional modulation of single unit responses. PLoS ONE 4:e4651.
64. Lewis JW, Van Essen DC (2000) Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey. J Comp Neurol 428:112-137.
65. Ma WJ, Zhou X, Ross LA, Foxe JJ, Parra LC (2009) Lip-reading aids word recognition most in moderate noise: a Bayesian explanation using high-dimensional feature space. PLoS ONE 4:e4638.
66. Macaluso E, Frith C, Driver J (2000) Selective spatial attention in vision and touch: unimodal and multimodal mechanisms revealed by PET. J Neurophysiol 83:3062-3075.
67. MacDonald J, Andersen S, Bachmann T (2000) Hearing by eye: how much spatial degradation can be tolerated? Perception 29:1155-1168.
68. Mazziotta J, Toga A, Evans A, Fox P, Lancaster J, Zilles K, Woods R, Paus T, Simpson G, Pike B, Holmes C, Collins L, Thompson P, MacDonald D, Iacoboni M, Schormann T, Amunts K, Palomero-Gallagher N, Geyer S, Parsons L, et al. (2001) A probabilistic atlas and reference system for the human brain: International Consortium for Brain Mapping (ICBM). Philos Trans R Soc Lond B Biol Sci 356:1293-1322.
69. McIntosh AR, Gonzalez-Lima F (1994) Structural equation modeling and its application to network analysis in functional brain imaging. Hum Brain Mapp 2:2-22.
70. Miller LM, D'Esposito M (2005) Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. J Neurosci 25:5884-5893.
71. Mozolic JL, Joyner D, Hugenschmidt CE, Peiffer AM, Kraft RA, Maldjian JA, Laurienti PJ (2008) Cross-modal deactivations during modality-specific selective attention. BMC Neurol 8:35.
72. Munhall KG, Kroos C, Jozan G, Vatikiotis-Bateson E (2004) Spatial frequency requirements for audiovisual speech perception. Percept Psychophys 66:574-583.
73. Murray SO, Kersten D, Olshausen BA, Schrater P, Woods DL (2002) Shape perception reduces activity in human primary visual cortex. Proc Natl Acad Sci U S A 99:15164-15169.
74. Narain C, Scott SK, Wise RJS, Rosen S, Leff A, Iversen SD, Matthews PM (2003) Defining a left-lateralized response specific to intelligible speech using fMRI. Cereb Cortex 13:1362-1368.
75. Noesselt T, Rieger JW, Schoenfeld MA, Kanowski M, Hinrichs H, Heinze HJ, Driver J (2007) Audiovisual temporal correspondence modulates human multisensory superior temporal sulcus plus primary sensory cortices. J Neurosci 27:11431-11441.
76. Noppeney U, Josephs O, Hocking J, Price CJ, Friston KJ (2008) The effect of prior visual information on recognition of speech and sounds. Cereb Cortex 18:598-609.
77. Obleser J, Wise RJS, Dresner MA, Scott SK (2007) Functional integration across brain regions improves speech perception under adverse listening conditions. J Neurosci 27:2283-2289.
78. Olman CA, Ugurbil K, Schrater P, Kersten D (2004) BOLD fMRI and psychophysical measurements of contrast response to broadband images. Vision Res 44:669-683. (Pubitemid 38125108)
79. Park JC, Zhang X, Ferrera J, Hirsch J, Hood DC (2008) Comparison of contrast-response functions from multifocal visual-evoked potentials (mfVEPs) and functional MRI responses. J Vis 8:1-12.
80. Patel RS, Bowman FD, Rilling JK (2006) Determining hierarchical functional networks from auditory stimuli fMRI. Hum Brain Mapp 27:462-470.
81. Patterson RD, Johnsrude IS (2008) Functional imaging of the auditory processing applied to speech sounds. Philos Trans R Soc Lond B Biol Sci 363:1023-1035.
82. Pelphrey KA, Morris JP, Michelich CR, Allison T, McCarthy G (2005) Functional anatomy of biological motion perception in posterior temporal cortex: an fMRI study of eye, mouth and hand movements. Cereb Cortex 15:1866-1876. (Pubitemid 41628328)
83. Poeppel D, Idsardi WJ, van Wassenhove V (2008) Speech perception at the interface of neurobiology and linguistics. Philos Trans R Soc Lond B Biol Sci 363:1071-1086. (Pubitemid 351158935)
84. Price CJ (2000) The anatomy of language: contributions from functional neuroimaging. J Anat 197:335-359.
85. Raichle ME, MacLeod AM, Snyder AZ, Powers WJ, Gusnard DA, Shulman GL (2001) A default mode of brain function. Proc Natl Acad Sci U S A 98:676-682.
86. Reynolds JH, Heeger DJ (2009) The normalization model of attention. Neuron 61:168-185.
87. Saad ZS, Glen DR, Chen G, Beauchamp MS, Desai R, Cox RW (2009) A new method for improving functional-to-structural MRI alignment using local Pearson correlation. Neuroimage 44:839-848.
88. Scott SK, Johnsrude IS (2003) The neuroanatomical and functional organization of speech perception. Trends Neurosci 26:100-107. (Pubitemid 36110544)
89. Scott SK, Blank CC, Rosen S, Wise RJ (2000) Identification of a pathway for intelligible speech in the left temporal lobe. Brain 123:2400-2406.
90. Sekiyama K, Kanno I, Miura S, Sugita Y (2003) Auditory-visual speech perception examined by fMRI and PET. Neurosci Res 47:277-287. (Pubitemid 37267963)
91. Seltzer B, Pandya DN (1994) Parietal, temporal, and occipital projections to cortex of the superior temporal sulcus in the rhesus monkey: a retrograde tracer study. J Comp Neurol 343:445-463. (Pubitemid 24129553)
92. Seltzer B, Cola MG, Gutierrez C, Massee M, Weldon C, Cusick CG (1996) Overlapping and nonoverlapping cortical projections to cortex of the superior temporal sulcus in the rhesus monkey: double anterograde tracer studies. J Comp Neurol 370:173-190. (Pubitemid 26227144)
93. Shahin AJ, Miller LM (2009) Multisensory integration enhances phonemic restoration. J Acoust Soc Am 125:1744-1750.
94. Shannon RV, Zeng F-G, Kamath V, Wygonski J, Ekelid M (1995) Speech recognition with primarily temporal cues. Science 270:303-304.
95. Smiley JF, Hackett TA, Ulbert I, Karmas G, Lakatos P, Javitt DC, Schroeder CE (2007) Multisensory convergence in auditory cortex, I. Cortical connections of the caudal superior temporal plane in macaque monkeys. J Comp Neurol 502:894-923. (Pubitemid 46806627)
96. Stein BE, Meredith MA (1993) The merging of the senses. Cambridge, MA: MIT Press.
97. Stein JL, Wiedholz LM, Bassett DS, Weinberger DR, Zink CF, Mattay VS, Meyer-Lindenberg A (2007) A validated network of effective amygdala connectivity. Neuroimage 36:736-745.
98. Stevenson RA, James TW (2009) Audiovisual integration in human superior temporal sulcus: inverse effectiveness and the neural processing of speech and object recognition. Neuroimage 44:1210-1223.
99. Stevenson RA, Kim S, James TW (2009) An additive-factors design to disambiguate neuronal and areal convergence: measuring multisensory interactions between audio, visual and haptic sensory streams using fMRI. Exp Brain Res 198:183-194.
100. Sumby WH, Pollack I (1954) Visual contribution to speech intelligibility in noise. J Acoust Soc Am 26:212-215.
101. Thomas SM, Jordan TR (2002) Determining the influence of Gaussian blurring on inversion effects with talking faces. Percept Psychophys 64:932-944.
102. Tootell RB, Reppas JB, Kwong KK, Malach R, Born RT, Brady TJ, Rosen BR, Belliveau JW (1995) Functional analysis of human MT and related visual cortical areas using magnetic resonance imaging. J Neurosci 15:3215-3230.
103. Upadhyay J, Silver A, Knaus TA, Lindgren KA, Ducros M, Kim D-S, Tager-Flusberg H (2008) Effective and structural connectivity in the human auditory cortex. J Neurosci 28:3341-3349. (Pubitemid 351469322)
104. van Atteveldt NM, Blau VC, Blomert L, Goebel R (2010) fMR-adaptation indicates selectivity to audiovisual content congruency in distributed clusters of human superior temporal cortex. BMC Neurosci 11:11.
105. van Atteveldt N, Roebroeck A, Goebel R (2009) Interaction of speech and script in human auditory cortex: insights from neuro-imaging and effective connectivity. Hear Res 258:152-164.
106. Van Essen DC (2005) A population-average, landmark- and surface-based (PALS) atlas of human cerebral cortex. Neuroimage 28:635-662.
107. von Kriegstein K, Dogan O, Grüter M, Giraud AL, Kell CA, Grüter T, Kleinschmidt A, Kiebel SJ (2008) Simulation of talking faces in the human brain improves auditory speech recognition. Proc Natl Acad Sci U S A 105:6747-6752.
108. Werner S, Noppeney U (2010) Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Cereb Cortex 20:1829-1842.
109. Wilson M (1988) The MRC psycholinguistic database: machine readable dictionary, version 2. Behav Res Methods Instrum Comput 20:6-11.
110. Winer JA (2006) Decoding the auditory corticofugal systems. Hear Res 212:1-8.
111. Wolmetz M, Poeppel D, Rapp B (2011) What does the right hemisphere know about phoneme categories? J Cogn Neurosci 23:552-569.
112. Wright TM, Pelphrey KA, Allison T, McKeown MJ, McCarthy G (2003) Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cereb Cortex 13:1034-1043.
113. Zatorre RJ (2007) There's more to auditory cortex than meets the ear. Hear Res 229:24-30.