Bimodal speech: Early suppressive visual effects in human auditory cortex

European Journal of Neuroscience

Volumn 20, Issue 8, 2004, Pages 2225-2234

  Besle, Julien ^a   Fort, Alexandra ^a   Delpuech, Claude ^a   Giard, Marie Hélène ^a  

^a INSERM   (France)

Author keywords

Audiovisual; Electrophysiology; Multisensory integration; Speech perception

Indexed keywords

ADULT; ARTICLE; AUDITORY CORTEX; AUDITORY RESPONSE; AUDITORY STIMULATION; CONTROLLED STUDY; EVENT RELATED POTENTIAL; FACILITATION; FEEDBACK SYSTEM; FEMALE; HUMAN; HUMAN EXPERIMENT; MALE; NORMAL HUMAN; PRIORITY JOURNAL; RECOGNITION; SPEECH PERCEPTION; TASK PERFORMANCE; VISION;

ACOUSTIC STIMULATION; ADULT; AUDITORY CORTEX; EVOKED POTENTIALS; FEMALE; HUMANS; LEAST-SQUARES ANALYSIS; MALE; PHOTIC STIMULATION; REACTION TIME; SPEECH;

EID: 4944263848     PISSN: 0953816X     EISSN: None     Source Type: Journal    
DOI: 10.1111/j.1460-9568.2004.03670.x     Document Type: Article

References (67)

1. Alcaini, M., Giard, M.H., Thevenet, M. & Pernier, J. (1994) Two separate frontal components in the N1 wave of the human auditory evoked response. Psychophysiology, 31, 611-615.
2. Arnold, P. & Hill, F. (2001) Bisensory augmentation: a speechreading advantage when speech is clearly audible and intact. Br. J. Psychol., 92, 339-355.
3. Barth, D.S.N., Goldberg, B., Brett, B. & Di, S. (1995) The spatiotemporal organization of auditory, visual and auditory-visual evoked potentials in rat cortex. Brain. Res., 678, 177-190.
4. Bense, S., Stephan, T., Yousry, T.A., Brandt, T. & Dieterich, M. (2001) Multisensory cortical signal increases and decreases during vestibular galvanic stimulation (fMRI). J. Neurophysiol., 85, 886-899.
5. Besle, J., Fort, A. & Giard, M.-H. (2004) Interest and validity of the additive model in electrophysiological studies of multisensory interactions. Cognitive Processing, 5, 189-192.
6. Calvert, G.A. (2001) Crossmodal processing in the human brain: Insights from functional neuroimaging studies. Cereb. Cortex, 11, 1110-1123.
7. Calvert, G.A., Brammer, M.J., Bullmore, E.T., Campbell, R., Iversen, S.D. & David, A.S. (1999) Response amplification in sensory-specific cortices during crossmodal binding. Neuroreport, 10, 2619-2623.
8. Calvert, G.A. & Campbell, R. (2003) Reading speech from still and moving faces: The neural substrates Divisible speech. J. Cogn. Neurosci., 15, 57-70.
9. Colin, C., Radeau, M., Soquet, A., Demolin, D., Colin, F. & Deltenre, P. (2002) Mismatch negativity evoked by the McGurk-MacDonald effect: a phonetic representation within short-term memory. Clin. Neurophysiol., 113, 495-506.
10. Falchier, A., Clavagnier, S., Barone, P. & Kennedy, H. (2002) Anatomical evidence of multimodal integration in primate striate cortex. J. Neurosci., 22, 5749-5759.
11. Fernandez-Duque, D & Posner, M.I. (1997) Relating the mechanisms of orienting and alerting. Neuropsychologia, 35, 477-486.
12. Fort, A., Delpuech, C., Pernier, J. & Giard, M.H. (2002a) Dynamics of cortico-subcorical crossmodal operations involved in audio-visual object detection in humans. Cereb. Cortex, 12, 1031-1039.
13. Fort, A., Delpuech, C., Pernier, J. & Giard, M.H. (2002b) Early auditory-visual interactions in human cortex during nonredundant target identification. Brain Res. Cogn. Brain Res., 14, 20-30.
14. Fort, A. & Giard, M.-H. (2004) Multiple electrophysiological mechanisms of audio-visual integration in human perception. In Calvert, G., Spence, C. & Stein, B. (eds), The Handbook of Multisensory Processes, MIT Press, Cambridge, pp. 503-514.
15. Foxe, J.J., Morocz, I.A., Murray, M.M., Higgins, B.A., Javitt, D.C. & Schroeder, C.E. (2000) Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Brain Res. Cogn. Brain Res., 10, 77-83.
16. Giard, M.-H., Fort, A., Mouchetant-Rostaing, Y. & Pernier, J. (2000) Neurophysiological mechanisms of auditory selective attention in humans. Front. Biosci., 5, 84-94.
17. Giard, M.H. & Peronnet, F. (1999) Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J. Cogn. Neurosci., 11, 473-490.
18. Giard, M.H., Perrin, F., Echallier, J.F., Thevenet, M., Froment, J.C. & Pernier, J. (1994) Dissociation of temporal and frontal components in the human auditory N1 wave: a scalp current density and dipole model analysis. Electroencephalogr. Clin. Neuro., 92, 238-252.
19. Grant, K.W. & Seitz., P.F. (2000) The use of visible speech cues for improving auditory detection of spoken sentences. J. Acoust. Soc. Am., 108, 1197-1208.
20. Guthrie, D. & Buchwald, J.S. (1991) Significance testing of difference potentials. Psychophysiology, 28, 240-244.
21. Hershenson, M. (1962) Reaction time as a measure of intersensory facilitation. J. Exp. Psychol., 63, 289-293.
22. Hillyard, S.A., Teder-Salejarvi, W.A. & Munte, T.F. (1998) Temporal dynamics of early perceptual processing. Curr. Opin. Neurobiol., 8, 202-210.
23. Holcomb, P.J. & Anderson, J.E. (1993) Cross-modal semantic priming - a time-course analysis using event-related brain potentials. Lang. Cogn. Proc., 8, 379-411.
24. Holcomb, P.J. & Neville, H.J. (1990) Auditory and visual semantic priming in lexical decision: a comparison using event-related brain potentials. Lang. Cogn. Proc., 5, 281-312.
25. Kawashima, R., O'Sullivan, B.T. & Roland, P.E. (1995) Positron-emission tomography studies of cross-modality inhibition in selective attentional tasks: Closing the 'mind's eye'. Proc. Natl. Acad. Sci. USA., 92, 5969-5972.
26. Laurienti, P.J., Burdette, J.H., Wallace, M.T., Yen, Y.F., Field, A.S. & Stein, B.E. (2002) Deactivation of sensory-specific cortex by cross-modal stimuli. J. Cogn. Neurosci., 14, 420-429.
27. Lebib, R., Papo, D., de Bode, S. & Baudonnière, P.M. (2003) Evidence of a visual-to-auditory cross-modal sensory gating phenomenon as reflected by the human P50 event-related brain potential modulation. Neurosci. Lett., 341, 185-188.
28. Lewis, J.W., Beauchamp, M.S. & DeYoe, E.A. (2000) A comparison of visual and auditory motion processing in human cerebral cortex. Cereb. Cortex, 10, 873-888.
29. Macaluso, E., Frith, C. & Driver, J. (2000) Modulation of human visual cortex by crossmodal spatial attention. Science, 289, 1206-1208.
30. Mangun, G.R., (1995) Neural mechanisms of visual selective attention. Psychophysiology, 32, 4-18.
31. Massaro, D.W. (1993) Perceiving asynchronous bimodal speech in consonant-vowel and vowel syllables. Speech Comm., 13, 127-134.
32. Massaro, D.W. & Cohen, M.M. (2000) Tests of auditory-visual integration efficiency within the framework of the fuzzy logical model of perception. J. Acoust. Soc. Am., 108, 784-789.
33. McGurk, H. & McDonald, J. (1976) Hearing lips and seeing voices. Nature, 264, 746-748.
34. Miki, K., Watanabe, S. & Kakigi, R. (2004) Interaction between auditory and visual stimulus relating to the vowel sounds in the auditory cortex in humans: a magnetoencephalographic study. Neurosci. Lett., 357, 199-202.
35. Miller, J.O. (1982) Divided attention: Evidence for coactivation with redundant signals. Cognit. Psychol., 14, 247-279.
36. Miller, J.O. (1986) Time course of coactivation in bimodal divided attention. Percept. Psychophys., 40, 331-343.
37. Miniussi, C., Girelli, M. & Marzi, C.A. (1998) Neural site of the redundant target effect: Electrophysiological evidence. J. Cogn. Neurosci., 10, 216-230.
38. Molholm, S., Ritter, W., Murray, M.M., Javitt, D.C., Schroeder, C.E. & Foxe, J.J. (2002) Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res. Cogn. Brain Res., 14, 115-128.
39. Möttönen, R., Krause, C.M., Tiippana, K. & Sams, M. (2002) Processing of changes in visual speech in the human auditory cortex. Brain Res. Cogn. Brain Res., 13, 417-425.
40. Näätänen, R. (2001) The perception of speech sounds by the human brain as reflected by the mismatch negativity (MMN) and its magnetic equivalent (MMNm). Psychophysiology, 38, 1-21.
41. Näätänen, R. (1992) Attention and Brain Function, Lawrence Erlbaum Associates, Hillsdale, NJ.
42. Näätänen, R. & Picton, T.W. (1987) The N1 wave of the human electric and magnetic response to sound: a review and an analysis of the component structure. Psychophysiology, 24, 375-425.
43. Näätänen, R. & Winkler, I. (1999) The concept of auditory stimulus representation in cognitive neuroscience. Psychol. Bull., 125, 826-859.
44. Nickerson, R.S. (1973) Intersensory facilitation of reaction time: Energy summation or preparation enhancement? Psychol. Rev., 80, 489-509.
45. Olson, I.R., Gatenby, J.C. & Gore, J.C. (2002) A comparison of bound and unbound audio-visual information processing in the human cerebral cortex. Brain Res. Cogn. Brain Res., 14, 129-138.
46. Pandya, D.N., Hallett, M. & Kmukherjee, S.K. (1969) Infra- and interhemispheric connections of the neocortical auditory system in the rhesus monkey. Brain. Res., 14, 49-65.
47. Pernier, J., Perrin, F. & Bertrand, O. (1988) Scalp current density fields: Concept and properties. Electroencephalogr. Clin. Neuro., 69, 385-389.
48. Perrin, F., Pernier, J., Bertrand, O. & Echallier, J.F. (1989) Spherical splines for scalp potential and current density mapping. Electroencephalogr. Clin. Neuro., 72, 184-187.
49. Perrin, F., Pernier, J., Bertrand, O. & Giard, M.H. (1987) Mapping of scalp potentials by surface spline interpolation. Electroencephalogr. Clin. Neuro., 66, 75-81.
50. Posner, M.I., Nissen, M.J. & Klein, R.M. (1976) Visual dominance: An information-processing account of its origins and significance. Psychol. Rev., 83, 157-171.
51. Raab, D.H. (1962) Statistical facilitation of simple reaction times. Trans. NY Acad. Sci., 24, 574-590.
52. Raij, T., Uutela, K. & Hari, R. (2000) Audiovisual integration of letters in the human brain. Neuron, 28, 617-625.
53. Ratcliff, R. (1979) Group reaction time distributions and an analysis of distribution statistics. Psychol. Bull., 86, 446-461.
54. Reisberg, D.J., McLean, J & Goldfield, A. (1987) Easy to hear but hard to understand: A lipreading advantage with intact auditory stimuli. In Dodd, B. & Campbell, R. (eds), Hearing by Eye: the Psychology of Lipreading. Lawrence Erlbaum Associates, London, pp. 93-113.
55. Sams, M., Aulanko, R., Hamalainen, H., Hari, R., Lounasmaa, O.V., Lu, S.T. & Simola, J. (1991) Seeing speech: Visual information from lip movements modifies activity in the human auditory cortex. Neurosci. Lett., 127, 141-145.
56. Scherg, M. (1990) Fundamentals of dipole source potential analysis. In Grandori, F., Hoke, M. & Romani, G.L. (eds), Auditory Evoked Magnetic Fields and Electric Potentials. Advances in Audiology, Vol. 5. Karger, Basel, pp. 40-69.
57. Scherg, M. & Von Cramon, D. (1985) A new interpretation of the generators of BAEP waves I-V: Results of a spatio-temporal dipole model. Electroencephalogr. Clin. Neuro., 62, 290-299.
58. Scherg, M. & Von Cramon, D. (1986) Evoked dipole source potentials of the human auditory cortex. Electroencephalogr. Clin. Neuro., 65, 344-360.
59. Schroeder, C.E., Lindsley, R.W., Specht, C., Marcovici, A., Smiley, J.F. & Javitt, D.C. (2001) Somatosensory input to auditory association cortex in the macaque monkey. J. Neurophysiol., 85, 1322-1327.
60. Schroeder, C., Molholm, S., Lakatos, P., Ritter, W. & Foxe, J.J. (2004) Human-simian correspondence in the early cortical processing of multisensory cues. Cognitive Processing, 5, 140-151.
61. Schroeder, CE & Foxe, J.J. (2002) The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Res. Cogn. Brain Res., 14, 187-198.
62. Seltzer, B. & Pandya, D.N. (1978) Afferent cortical connections and architectonics of the superior temporal sulcus and surrounding cortex in the rhesus monkey. Brain. Res., 149, 1-24.
63. Sumby, W.H. & Pollack, I. (1954) Visual contribution to speech intelligibility in noise. J. Acoust. Soc. Am., 26, 212-215.
64. Summerfield, Q. (1987) Some preliminaries to a comprehensive account of audio-visual speech perception. In Dodd, B. & Campbell, R., eds. Hearing by Eye: the Psychology of Lipreading. Lawrence Erlbaum Associates, Location, pp. 3-52.
65. Thiel, C.M., Zilles, K. & Fink, G.R. (2004) Cerebral correlates of alerting, orienting and reorienting of visuospatial attention: An event-related fMRI study. Neuroimage, 21, 318-328.
66. Vaughan, H.G. & Ritter, W. (1970) The sources of auditory evoked responses recorded from the human scalp. Electroencephalogr. Clin. Neuro., 28, 360-367.
67. Vogel, E.K. & Luck, S.J. (2000) The visual N1 component as an index of a discrimination process. Psychophysiology, 37, 190-203.