A novel approach to study audiovisual integration in speech perception: Localizer fMRI and sparse sampling

Brain Research

Volumn 1220, Issue , 2008, Pages 142-149

  Szycik, Gregor Rafael ^a,b   Tausche, Peggy ^a   Münte, Thomas F ^a  

^a OTTO VON GUERICKE UNIVERSITY   (Germany)

^b HANNOVER MEDICAL SCHOOL   (Germany)

Author keywords

Audiovisual integration; Functional MRI; Multisensory integration; Speech comprehension; Superior temporal sulcus

Indexed keywords

ADULT; ARTICLE; AUDIOVISUAL INTEGRATION; AUDITORY STIMULATION; BRAIN FUNCTION; BRAIN REGION; COMPREHENSION; EVENT RELATED POTENTIAL; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; NOISE; NORMAL HUMAN; PRIORITY JOURNAL; SAMPLING; SPEECH PERCEPTION; SUPERIOR TEMPORAL SULCUS; TECHNIQUE; TEMPORAL LOBE; VISUAL STIMULATION;

ACOUSTIC STIMULATION; ADULT; ANALYSIS OF VARIANCE; BRAIN; BRAIN MAPPING; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; LIPREADING; MAGNETIC RESONANCE IMAGING; OXYGEN; PHOTIC STIMULATION; SPEECH PERCEPTION;

EID: 48949119475     PISSN: 00068993     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.brainres.2007.08.027     Document Type: Article

References (41)

1. Baayen R.H., Piepenbrock R., and Gulikers L. The CELEX Lexical Database [CD-ROM] (1995), University of Pennsylvania, Linguistic Data Consortium, Philadelphia
2. Baumgart F., Kaulisch T., Tempelmann C., Gaschler-Markefski B., Tegeler C., Schindler F., Stiller D., and Scheich H. Electrodynamic headphones and woofers for application in magnetic resonance imaging scanners. Med. Phys. 25 (1998) 2068-2070
3. Beauchamp M.S. See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex. Curr. Opin. Neurobiol. 15 (2005) 145-153
4. Beauchamp M.S. Statistical criteria in fMRI studies of multisensory integration. Neuroinformatics 3 (2005) 93-113
5. Beauchamp M.S., Argall B.D., Bodurka J., Duyn J.H., and Martin A. Unraveling multisensory integration: patchy organization within human STS multisensory cortex. Nat. Neurosci. 7 (2004) 1190-1192
6. Beauchamp M.S., Lee K.E., Argall B.D., and Martin A. Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41 (2004) 809-823
7. Bernstein L.E., Auer Jr. E.T., Moore J.K., Ponton C.W., Don M., and Singh M. Visual speech perception without primary auditory cortex activation. Neuroreport 13 (2002) 311-315
8. Boynton G.M., Engel S.A., Glover G.H., and Heeger D.J. Linear systems analysis of functional magnetic resonance imaging in human V1. J. Neurosci. 16 (1996) 4207-4221
9. Bunzeck N., Wuestenberg T., Lutz K., Heinze H., and Jäncke J. Scanning silence: mental imagery of complex sounds. Neuroimage 26 (2005) 1119-1127
10. Callan D.E., Jones J.A., Munhall K., Callan A.M., Kroos C., and Vatikiotis-Bateson E. Neural processes underlying perceptual enhancement by visual speech gestures. Neuroreport 14 (2003) 2213-2218
11. Calvert G.A., and Campbell R. Reading speech from still and moving faces: the neural substrates of visible speech. J. Cogn. Neurosci. 15 (2003) 57-70
12. Calvert G.A., and Thesen T. Multisensory integration: methodological approaches and emerging principles in the human brain. J. Physiol. Paris 98 (2004) 191-205
13. Calvert G.A., Bullmore E.T., Brammer M.J., Campbell R., Williams S.C., McGuire P.K., Woodruff P.W., Iversen S.D., and David A.S. Activation of auditory cortex during silent lipreading. Science 276 (1997) 593-596
14. Calvert G.A., Campbell R., and Brammer M.J. Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr. Biol. 10 (2000) 649-657
15. Calvert G.A., Hansen P.C., Iversen S.D., and Brammer M.J. Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect. Neuroimage 14 (2001) 427-438
16. Gaab N., Gaser C., Zaehle T., Jäncke L., and Schlaug G. Functional anatomy of pitch memory-an fMRI study with sparse temporal sampling. Neuroimage 19 (2003) 1417-1426
17. Hall D.A., Haggard M.P., Akeroyd M.A., Palmer A.R., Summerfield A.Q., Elliott M.R., Gurney E.M., and Bowtell R.W. "Sparse" temporal sampling in auditory fMRI. Hum. Brain Mapp. 7 (1999) 213-223
18. Hershenson M. Reaction time as a measure of intersensory facilitation. J. Exp. Psychol. 63 (1962) 289-293
19. Joseph J.E., Gathers A.D., Liu X., Corbly C.R., Whitaker S.K., and Bhatt R.S. Neural developmental changes in processing inverted faces. Cogn. Affect. Behav. Neurosci. 6 (2006) 223-235
20. Lancaster J.L., Woldorff M.G., Parsons L.M., Liotti M., Freitas C.S., Rainey L., Kochunov P.V., Nickerson D., Mikiten S.A., and Fox P.T. Automated Talairach atlas labels for functional brain mapping. Hum. Brain Mapp. 10 (2000) 120-131
21. Maurer D., O'Craven K.M., Le G.R., Mondloch C.J., Springer M.V., and Lewis T.L. Neural correlates of processing facial identity based on features versus their spacing. Neuropsychologia 45 (2007) 1438-1451
22. McGurk H., and MacDonald J. Hearing lips and seeing voices. Nature 264 (1976) 746-748
23. Miller L.M., and D'Esposito M. Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. J. Neurosci. 25 (2005) 5884-5893
24. Morrell L.K. Temporal characteristics of sensory interaction in choice reaction times. J. Exp. Psychol. 77 (1968) 14-18
25. Möttönen R., Schurmann M., and Sams M. Time course of multisensory interactions during audiovisual speech perception in humans: a magnetoencephalographic study. Neurosci. Lett. 363 (2004) 112-115
26. Ng M., Ciaramitaro V.M., Anstis S., Boynton G.M., and Fine I. Selectivity for the configural cues that identify the gender, ethnicity, and identity of faces in human cortex. Proc. Natl. Acad. Sci. U. S. A. 103 (2006) 19552-19557
27. Nishitani N., and Hari R. Viewing lip forms: cortical dynamics. Neuron 36 (2002) 1211-1220
28. Pelphrey K.A., Morris J.P., Michelich C.R., Allison T., and McCarthy G. Functional anatomy of biological motion perception in posterior temporal cortex: an fMRI study of eye, mouth and hand movements. Cereb. Cortex 15 (2005) 1866-1876
29. Raij T., Uutela K., and Hari R. Audiovisual integration of letters in the human brain. Neuron 28 (2000) 617-625
30. Reale R.A., Calvert G.A., Thesen T., Jenison R.L., Kawasaki H., Oya H., Howard M.A., and Brugge J.F. Auditory-visual processing represented in the human superior temporal gyrus. Neuroscience 145 (2007) 162-184
31. Ross L.A., Saint-Amour D., Leavitt V.M., Javitt D.C., and Foxe J.J. Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. Cereb. Cortex 17 (2007) 1147-1153
32. Schwarzlose R.F., Baker C.I., and Kanwisher N. Separate face and body selectivity on the fusiform gyrus. J. Neurosci. 25 (2005) 11055-11059
33. Sekiyama K., Kanno I., Miura S., and Sugita Y. Auditory-visual speech perception examined by fMRI and PET. Neurosci. Res. 47 (2003) 277-287
34. Skipper J.I., van Wassenhove V., Nusbaum H.C., and Small S.L. Hearing lips and seeing voices: how cortical areas supporting speech production mediate audiovisual speech perception. Cereb. Cortex (2007) [Electroni publication ahead of print]
35. Stein B.E., and Meredith M.A. The Merging of the Senses (1994), MIT Press, Cambridge, MA
36. Sumby W.H., and Pollack I. Visual contribution to speech intelligibility in noise. J. Acoust. Soc. Am. 26 (1954) 212-215
37. Talairach J., and Tornoux P. Co-Planar Stereotactic Atlas of the Human Brain (1988), Thieme, Stuttgart
38. van Atteveldt N., Formisano E., Goebel R., and Blomert L. Integration of letters and speech sounds in the human brain. Neuron 43 (2004) 271-282
39. van Wassenhove V., Grant K.W., and Poeppel D. Visual speech speeds up the neural processing of auditory speech. Proc. Natl. Acad. Sci. U. S. A. 102 (2005) 1181-1186
40. Wright T.M., Pelphrey K.A., Allison T., McKeown M.J., and McCarthy G. Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cereb. Cortex 13 (2003) 1034-1043
41. Zaehle T., Wustenberg T., Meyer M., and Jäncke L. Evidence for rapid auditory perception as the foundation of speech processing: a sparse temporal sampling fMRI study. Eur. J. Neurosci. 20 (2004) 2447-2456