Discrete neural substrates underlie complementary audiovisual speech integration processes

NeuroImage

Volumn 55, Issue 3, 2011, Pages 1339-1345

  Stevenson, Ryan A ^a,b   VanDerKlok, Ross M ^a   Pisoni, David B ^a   James, Thomas W ^a  

^a INDIANA UNIVERSITY   (United States)

^b VANDERBILT UNIVERSITY   (United States)

Author keywords

Autism; Dyslexia; FMRI; Multisensory integration; Schizophrenia; Speech perception; STS; Temporal processing

Indexed keywords

ADULT; ARTICLE; BEHAVIOR; BRAIN REGION; CONTROLLED STUDY; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; IMAGE ANALYSIS; MALE; NEUROANATOMY; NORMAL HUMAN; PRIORITY JOURNAL; SPEECH PERCEPTION; STIMULUS RESPONSE; TEMPORAL CORTEX;

ACOUSTIC STIMULATION; AUDITORY PERCEPTION; BRAIN MAPPING; FEMALE; FUNCTIONAL LATERALITY; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MALE; OXYGEN; PHOTIC STIMULATION; SPEECH PERCEPTION; TEMPORAL LOBE; VISUAL PERCEPTION; YOUNG ADULT;

EID: 79952071953     PISSN: 10538119     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2010.12.063     Document Type: Article

References (69)

1. Bastien-Toniazzo M., Stroumza A., Cavé C. Audio-visual perception and integration in developmental dyslexia: an exploratory study using the McGurk effect. Curr. Psychol. Lett. 2009, 25.
2. Beauchamp M.S. Statistical criteria in FMRI studies of multisensory integration. Neuroinformatics 2005, 3:93-113.
3. Beauchamp M.S., Argall B.D., Bodurka J., Duyn J.H., Martin A. Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat. Neurosci. 2004, 7:1190-1192.
4. Bebko J.M., Weiss J.A., Demark J.L., Gomez P. Discrimination of temporal synchrony in intermodal events by children with autism and children with developmental disabilities without autism. J. Child Psychol. Psychiatry 2006, 47:88-98.
5. Bishop C.W., Miller L.M. A multisensory cortical network for understanding speech in noise. J. Cogn. Neurosci. 2009, 21:1790-1805.
6. Blau V., van Atteveldt N., Ekkebus M., Goebel R., Blomert L. Reduced neural integration of letters and speech sounds links phonological and reading deficits in adult dyslexia. Curr. Biol. 2009, 19:503-508.
7. Blau V., Reithler J., van Atteveldt N., Seitz J., Gerretsen P., Goebel R., Blomert L. Deviant processing of letters and speech sounds as proximate cause of reading failure: a functional magnetic resonance imaging study of dyslexic children. Brain 2010, 133:868-879.
8. Bleich-Cohen M., Hendler T., Kotler M., Strous R.D. Reduced language lateralization in first-episode schizophrenia: an fMRI index of functional asymmetry. Psychiatry Res. 2009, 171:82-93.
9. Boddaert N., Zilbovicius M. Functional neuroimaging and childhood autism. Pediatr. Radiol. 2002, 32:1-7.
10. Boddaert N., Chabane N., Gervais H., Good C.D., Bourgeois M., Plumet M.H., Barthelemy C., Mouren M.C., Artiges E., Samson Y., Brunelle F., Frackowiak R.S., Zilbovicius M. Superior temporal sulcus anatomical abnormalities in childhood autism: a voxel-based morphometry MRI study. Neuroimage 2004, 23:364-369.
11. Brainard D.H. The psychophysics toolbox. Spat. Vis. 1997, 10:433-436.
12. Brunelle F., Boddaert N., Zilbovicius M. Autism and brain imaging. Bull. Acad. Natl Méd. 2009, 193:287-297. discussion 297-288.
13. Bushara K.O., Hanakawa T., Immisch I., Toma K., Kansaku K., Hallett M. Neural correlates of cross-modal binding. Nat. Neurosci. 2003, 6:190-195.
14. Conrey B., Pisoni D.B. Auditory-visual speech perception and synchrony detection for speech and nonspeech signals. J. Acoust. Soc. Am. 2006, 119:4065-4073.
15. Dale A.M., Buckner R.L. Selective averaging of rapidly presented individual trials using fMRI. Hum. Brain Mapp. 1997, 5:329-340.
16. de Gelder B., Vroomen J., Annen L., Masthof E., Hodiamont P. Audio-visual integration in schizophrenia. Schizophr. Res. 2003, 59:211-218.
17. Doniger G.M., Foxe J.J., Murray M.M., Higgins B.A., Javitt D.C. Impaired visual object recognition and dorsal/ventral stream interaction in schizophrenia. Arch. Gen. Psychiatry 2002, 59:1011-1020.
18. Fairhall S.L., Macaluso E. Spatial attention can modulate audiovisual integration at multiple cortical and subcortical sites. Eur. J. Neurosci. 2009, 29:1247-1257.
19. Forman S.D., Cohen J.D., Fitzgerald M., Eddy W.F., Mintun M.A., Noll D.C. Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): use of a cluster-size threshold. Magn. Reson. Med. 1995, 33:636-647.
20. Foss-Feig J.H., Kwakye L.D., Cascio C.J., Burnette C.P., Kadivar H., Stone W.L., Wallace M.T. An extended multisensory temporal binding window in autism spectrum disorders. Exp. Brain Res. 2009, 203:381-389.
21. Foucher J.R., Lacambre M., Pham B.T., Giersch A., Elliott M.A. Low time resolution in schizophrenia Lengthened windows of simultaneity for visual, auditory and bimodal stimuli. Schizophr. Res. 2007, 97:118-127.
22. Gervais H., Belin P., Boddaert N., Leboyer M., Coez A., Sfaello I., Barthelemy C., Brunelle F., Samson Y., Zilbovicius M. Abnormal cortical voice processing in autism. Nat. Neurosci. 2004, 7:801-802.
23. Glover G.H. Deconvolution of impulse response in event-related BOLD fMRI. Neuroimage 1999, 9:416-429.
24. Hagmann P., Cammoun L., Gigandet X., Meuli R., Honey C.J., Wedeen V.J., Sporns O. Mapping the structural core of human cerebral cortex. PLoS Biol. 2008, 6:e159.
25. Hairston W.D., Burdette J.H., Flowers D.L., Wood F.B., Wallace M.T. Altered temporal profile of visual-auditory multisensory interactions in dyslexia. Exp. Brain Res. 2005, 166:474-480.
26. Hamilton R.H., Shenton J.T., Coslett H.B. An acquired deficit of audiovisual speech processing. Brain Lang. 2006, 98:66-73.
27. James T.W., Stevenson R.A. The use of fMRI to assess multisensory integration. Frontiers in the Neural Basis of Multisensory Processes 2011, Taylor & Francis, London. M.H. Wallace, M.M. Murray (Eds.).
28. James T.W., Stevenson R.A., Kim S. Assessing Multisensory Integration with Additive Factors and Functional MRI 2009, The International Society for Psychophysics, Dublin, Ireland.
29. James T.W., Stevenson R.A., Kim S. Inverse effectiveness in multisensory processing. The New Handbook of Multisensory Processes 2011, MIT Press, Cambridge, MA. B.E. Stein (Ed.).
30. Jones E.G., Powell T.P. An anatomical study of converging sensory pathways within the cerebral cortex of the monkey. Brain 1970, 93:793-820.
31. Lachs L., Hernandez L.R. Update: the Hoosier audiovisual multitalker database. Research on Spoken Language Processing 1998, 377-388. Speech Research Laboratory, Indiana University, Bloomington, IN. D.B. Pisoni (Ed.).
32. Levitt J.G., Blanton R.E., Smalley S., Thompson P.M., Guthrie D., McCracken J.T., Sadoun T., Heinichen L., Toga A.W. Cortical sulcal maps in autism. Cereb. Cortex 2003, 13:728-735.
33. Luce P.A., Pisoni D.B. Recognizing spoken words: the neighborhood activation model. Ear Hear. 1998, 19:1-36.
34. Macaluso E., George N., Dolan R., Spence C., Driver J. Spatial and temporal factors during processing of audiovisual speech: a PET study. Neuroimage 2004, 21:725-732.
35. McIntosh A.R., Korostil M. Interpretation of neuroimaging data based on network concepts. Brain Imaging Behav. 2008, 2:264-269.
36. Meredith M.A. On the neuronal basis for multisensory convergence: a brief overview. Brain Res. Cogn. Brain Res. 2002, 14:31-40.
37. Meredith M.A., Nemitz J.W., Stein B.E. Determinants of multisensory integration in superior colliculus neurons. I. Temporal factors. J. Neurosci. 1987, 7:3215-3229.
38. Miller L.M., D'Esposito M. Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. J. Neurosci. 2005, 25:5884-5893.
39. Mongillo E.A., Irwin J.R., Whalen D.H., Klaiman C., Carter A.S., Schultz R.T. Audiovisual processing in children with and without autism spectrum disorders. J. Autism Dev. Disord. 2008, 38:1349-1358.
40. Pandya D.N., Yeterian E.H. Architecture and connections of cortical association areas. Cerebral Cortex 1985, 3-61. Plenum, New York. A. Peters, E.G. Jones (Eds.).
41. Pearl D., Yodashkin-Porat D., Katz N., Valevski A., Aizenberg D., Sigler M., Weizman A., Kikinzon L. Differences in audiovisual integration, as measured by McGurk phenomenon, among adult and adolescent patients with schizophrenia and age-matched healthy control groups. Compr. Psychiatry 2009, 50:186-192.
42. Pekkola J., Laasonen M., Ojanen V., Autti T., Jaaskelainen I.P., Kujala T., Sams M. Perception of matching and conflicting audiovisual speech in dyslexic and fluent readers: an fMRI study at 3T. Neuroimage 2006, 29:797-807.
43. Pelli D.G. The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spat. Vis. 1997, 10:437-442.
44. Pelphrey K.A., Carter E.J. Brain mechanisms for social perception: lessons from autism and typical development. Ann. NY Acad. Sci. 2008, 1145:283-299.
45. Pelphrey K.A., Carter E.J. Charting the typical and atypical development of the social brain. Dev. Psychopathol. 2008, 20:1081-1102.
46. Poldrack R.A., Mumford J.A. Independence in ROI analysis: where is the voodoo?. Soc. Cogn. Affect. Neurosci. 2009, 4:208-213.
47. Ramirez J., Mann V. Using auditory-visual speech to probe the basis of noise-impaired consonant-vowel perception in dyslexia and auditory neuropathy. J. Acoust. Soc. Am. 2005, 118:1122-1133.
48. Richards T., Stevenson J., Crouch J., Johnson L.C., Maravilla K., Stock P., Abbott R., Berninger V. Tract-based spatial statistics of diffusion tensor imaging in adults with dyslexia. AJNR Am. J. Neuroradiol. 2008, 29:1134-1139.
49. Romanski L.M., Bates J.F., Goldman-Rakic P.S. Auditory belt and parabelt projections to the prefrontal cortex in the rhesus monkey. J. Comp. Neurol. 1999, 403:141-157.
50. Ross L.A., Saint-Amour D., Leavitt V.M., Molholm S., Javitt D.C., Foxe J.J. Impaired multisensory processing in schizophrenia: deficits in the visual enhancement of speech comprehension under noisy environmental conditions. Schizophr. Res. 2007, 97:173-183.
51. Seltzer B., Pandya D.N. Afferent cortical connections and architectonics of the superior temporal sulcus and surrounding cortex in the rhesus monkey. Brain Res. 1978, 149:1-24.
52. Serences J.T. A comparison of methods for characterizing the event-related BOLD timeseries in rapid fMRI. Neuroimage 2004, 21:1690-1700.
53. Sheffert S.M., Lachs L., Hernandez L.R. The Hooiser audiovisual multitalker database. Research on Spoken Language Processing 1996, 578-583. Speech Research Laboratory, Indiana University, Bloomington, IN. D.B. Pisoni (Ed.).
54. Shenton M.E., Dickey C.C., Frumin M., McCarley R.W. A review of MRI findings in schizophrenia. Schizophr. Res. 2001, 49:1-52.
55. Smith E.G., Bennetto L. Audiovisual speech integration and lipreading in autism. J. Child Psychol. Psychiatry 2007, 48:813-821.
56. Sporns O. Networks of the Brain 2010, MIT Press, Cambridge, MA.
57. Stevenson R.A., James T.W. Audiovisual integration in human superior temporal sulcus: inverse effectiveness and the neural processing of speech and object recognition. Neuroimage 2009, 44:1210-1223.
58. Stevenson R.A., Geoghegan M.L., James T.W. Superadditive BOLD activation in superior temporal sulcus with threshold non-speech objects. Exp. Brain Res. 2007, 179:85-95.
59. Stevenson R.A., Kim S., James T.W. An additive-factors design to disambiguate neuronal and areal convergence: measuring multisensory interactions between audio, visual, and haptic sensory streams using fMRI. Exp. Brain Res. 2009, 198:183-194.
60. Stevenson R.A., Altieri N.A., Kim S., Pisoni D.B., James T.W. Neural processing of asynchronous audiovisual speech perception. Neuroimage 2010, 49:3308-3318.
61. Surguladze S.A., Calvert G.A., Brammer M.J., Campbell R., Bullmore E.T., Giampietro V., David A.S. Audio-visual speech perception in schizophrenia: an fMRI study. Psychiatry Res. 2001, 106:1-14.
62. Szycik G.R., Munte T.F., Dillo W., Mohammadi B., Samii A., Emrich H.M., Dietrich D.E. Audiovisual integration of speech is disturbed in schizophrenia: an fMRI study. Schizophr. Res. 2009, 110:111-118.
63. Talaraich J., Tournoux P. Co-planar stereotaxic atlas of the human brain 1988, Theime Medical Publishers, New York, NY.
64. Thirion B., Pinel P., Meriaux S., Roche A., Dehaene S., Poline J.B. Analysis of a large fMRI cohort: statistical and methodological issues for group analyses. Neuroimage 2007, 35:105-120.
65. van Atteveldt N.M., Formisano E., Blomert L., Goebel R. The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. Cereb. Cortex 2007, 17:962-974.
66. van Wassenhove V., Grant K.W., Poeppel D. Temporal window of integration in auditory-visual speech perception. Neuropsychologia 2007, 45:598-607.
67. Wallace M.T. Dyslexia: bridging the gap between hearing and reading. Curr. Biol. 2009, 19:R260-R262.
68. Werner S., Noppeney U. Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Cereb. Cortex 2009, 20:1829-1842.
69. Zilbovicius M., Boddaert N., Belin P., Poline J.B., Remy P., Mangin J.F., Thivard L., Barthelemy C., Samson Y. Temporal lobe dysfunction in childhood autism: a PET study. Positron emission tomography. Am. J. Psychiatry 2000, 157:1988-1993.