Time course of early audiovisual interactions during speech and nonspeech central auditory processing: A magnetoencephalography study

Journal of Cognitive Neuroscience

Volumn 21, Issue 2, 2009, Pages 259-274

  Hertrich, Ingo ^a   Mathiak, Klaus ^b   Lutzenberger, Werner ^a   Ackermann, Hermann ^a  

^a UNIVERSITY OF TÜBINGEN   (Germany)

^b RWTH AACHEN UNIVERSITY   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ACOUSTICS; ADULT; ARTICLE; AUDITORY CORTEX; AUDITORY NERVOUS SYSTEM; AUDITORY STIMULATION; COMPUTER PROGRAM; DEVICE; DIPOLE; FEMALE; HUMAN; HUMAN EXPERIMENT; INFORMATION PROCESSING; LANGUAGE; MAGNETIC FIELD; MAGNETOENCEPHALOGRAPHY; MALE; MOTION; NORMAL HUMAN; NUCLEAR MAGNETIC RESONANCE IMAGING; PRIORITY JOURNAL; SENSOR; SPEECH; SPEECH ARTICULATION; TIME; VISUAL NERVOUS SYSTEM; VISUAL STIMULATION;

ACOUSTIC STIMULATION; ADULT; AUDITORY CORTEX; CEREBRAL CORTEX; FEMALE; FUNCTIONAL LATERALITY; HUMANS; MAGNETIC RESONANCE IMAGING; MAGNETOENCEPHALOGRAPHY; MALE; PHONETICS; PHOTIC STIMULATION; REACTION TIME; SPEECH; SPEECH PERCEPTION; TEMPORAL LOBE; VISUAL PERCEPTION; YOUNG ADULT;

EID: 59949094483     PISSN: 0898929X     EISSN: 15308898     Source Type: Journal    
DOI: 10.1162/jocn.2008.21019     Document Type: Article

References (52)

1. Ackermann, H., Hertrich, I., Mathiak, K., & Lutzenberger, W.(2001). Contralaterality of cortical auditory processing at the level of the M50/M100 complex and the mismatch field: A whole-head magnetoencephalography study. NeuroReport, 12, 1683-1687.
2. Ahlfors, S. P., Simpson, G. V., Dale, A. M., Belliveau, J. W., Liu, A. K., Korvenoja, A., et al.(1999). Spatiotemporal activity of a cortical network for processing visual motion revealed by MEG and fMRI. Journal of Neurophysiology, 82, 2545-2555.
3. Asahi, T., Uwano, T., Eifuku, S., Tamura, R., Endo, S., Ono, T., et al.(2006). Neuronal responses to a delayed-response delayed-reward go/nogo task in the monkey posterior insular cortex. Neuroscience, 143, 627-639.
4. Avery, P., & Rice, K.(1989). Segment structure and coronal underspecification. Phonology, 6, 179-200.
5. Besle, J., Fort, A., Delpuech, C., & Giard, M. H.(2004). Bimodal speech: Early suppressive visual effects in human auditory cortex. European Journal of Neuroscience, 20, 2225-2234.
6. Bizley, J. K., Nodal, F. R., Bajo, V. M., Nelken, I., & King, A. J.(2007). Physiological and anatomical evidence for multisensory interactions in auditory cortex. Cerebral Cortex, 17, 2172-2189.
7. Boutros, N. N., & Belger, A.(1999). Midlatency evoked potentials attenuation and augmentation reflect different aspects of sensory gating. Biological Psychiatry, 45, 917-922.
8. Brosch, M., Selezneva, E., & Scheich, H.(2005). Nonauditory events of a behavioral procedure activate auditory cortex of highly trained monkeys. Journal ofNeuroscience, 25, 6797-6806.
9. Bunzeck, N., Wuestenberg, T., Lutz, K., Heinze, H. J., & jancke, L.(2005). Scanning silence: Mental imagery of complex sounds. Neuroimage, 26, 1119-1127.
10. Bushara, K. O., Grafman, J., & Hallett, M.(2001). Neural correlates of auditory-visual stimulus onset asynchrony detection. Journal of Neuroscience, 21, 300-304.
11. Calvert, G. A.(2001). Crossmodal processing in the human brain: Insights from functional neuroimaging studies. Cerebral Cortex, 11, 1110-1123.
12. Calvert, G. A., Bullmore, E. T., Brammer, M. J., Campbell, R., Williams, S. C., McGuire, P. K., et al.(1997). Activation of auditory cortex during silent lipreading. Science, 276, 593-596.
13. Calvert, G. A., Hansen, P. C., Iversen, S. D., & Brammer, M. J.(2001). Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect. Neuroimage, 14, 427-438.
14. Colin, C., Radeau, M., Soquet, A., & Deltenre, P.(2004). Generalization of the generation of an MMN by illusory McGurk percepts: Voiceless consonants. Clinical Neurophysiology, 115, 1989-2000.
15. De Lacy, P.(2006). Markedness: Reduction and preservation in phonology(Cambridge Studies in Linguistics, Vol. 112). Cambridge: Cambridge University Press.
16. Driver, J., & Frith, C.(2000). Shifting baseline in attention research. Nature Reviews Neuroscience, 1, 147-148.
17. Fort, A., & Giard, M. H.(2004). Multiple electrophysiological mechanisms of audio-visual integration in human perception. In G. A. Calvert, C. Spence, & B. E. Stein(Eds.), The handbook of multisensory processes(pp. 503-513).Cambridge: MIT.
18. Fujisaki, W., Shimojo, S., Kashino, M., & Nishida, S.(2003). Recalibration of audiovisual simultaneity by adaptation to a constant time lag. Journal of Vision, 3, 34a.
19. Fujisaki, W., Shimojo, S., Kashino, M., & Nishida, S.(2004). Recalibration of audiovisual simultaneity. Nature Neuroscience, 7, 773-778.
20. Ghazanfar, A. A., Maier, J. X., Hoffman, K. L., & Logothetis, N. K.(2005). Multisensory integration of dynamic faces and voices in rhesus monkey auditory cortex. Journal of Neuroscience, 25, 5004-5012.
21. Giard, M. H., & Peronnet, F.(1999). Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. Journal of Cognitive Neuroscience, 11, 473-490.
22. Harris, J., & Lindsey, G.(1995). The elements of phonological representation. In J. Durand & F. Katamba(Eds.), Frontiers of phonology: Atoms, structures, derivations(pp. 34-79).
23. Harlow, Essex: Longman.
24. Hertrich, I., & Ackermann, H.(1999). A vowel synthesizer based on formant sinusoids modulated by fundamental frequency. Journal of the Acoustical Society of America, 106, 2988-2990.
25. Hertrich, I., & Ackermann, H.(2007). Modelling voiceless speech segments by means of an addditive procedure based on the computation of formant sinusoids. In P. Wagner, J. Abresch, S. Breuer, & W. Hess(Eds.), Proceedings of the 6th ISCA Workshop on Speech Synthesis(pp. 178-181). Bonn: University of Bonn.
26. Hertrich, I., Mathiak, K., Lutzenberger, W., & Ackermann, H.(2004). Transient and phase-locked evoked magnetic fields in response to periodic acoustic signals. NeuroReport, 15, 1687-1690.
27. Hertrich, I., Mathiak, K., Lutzenberger, W., Menning, H., & Ackermann, H.(2007). Sequential audiovisual interactions during speech perception: A whole-head MEG study. Neuropsychologia, 45, 1342-1354.
28. Jäncke, L., & Shah, N. J.(2004). Hearing syllables by seeing visual stimuli. European Journal of Neuroscience, 19, 2603-2608.
29. Kastner, S., Pinsk, M. A., De Weerd, P., Ungerleider, L. G., & Desimone, R.(1999). Increased activity in human visual cortex during directed attention in the absence of visual stimulation. Neuron, 22, 751-761.
30. Lebib, R., Papo, D., De Bode, S., & Baudonniere, P. M.(2003). Evidence of a visual-to-auditory cross-modal sensory gating phenomenon as reflected by the human P50 event-related brain potential modulation. Neuroscience Letters, 341, 185-188.
31. MacDonald, J., & McGurk, H.(1978). Visual influences on speech perception processes. Perception & Psychophysics, 24, 253-257.
32. MacSweeney, M., Amaro, E., Calvert, G. A., Campbell, R., David, A. S., McGuire, P., et al.(2000). Silent speechreading in the absence of scanner noise: An event-related fMRI study. NeuroReport, 11, 1729-1733.
33. Miki, K., Watanabe, S., & Kakigi, R.(2004). Interaction between auditory and visual stimulus relating to the vowel sounds in the auditory cortex in humans: A magnetoencephalographic study. Neuroscience Letters, 357, 199-202.
34. Miki, K., Watanabe, S., Kakigi, R., & Puce, A.(2004). Magnetoencephalographic study of occipitotemporal activity elicited by viewing mouth movements. Clinical Neurophysiology, 115, 1559-1574.
35. Mottonen, R., Krause, C. M., Tiippana, K., & Sams, M.(2002). Processing of changes in visual speech in the human auditory cortex. Cognitive Brain Research, 13, 417-425.
36. Näätänen, R., Lehtokoski, A., Lennes, M., Cheour, M., Huotilainen, M., Iivonen, A., et al.(1997). Language-specific phoneme representations revealed by electric and magnetic brain responses. Nature, 385, 432-434.
37. Näätänen, R., & Winkler, I.(1999). The concept of auditory stimulus representation in cognitive neuroscience. Psychological Bulletin, 125, 826-859.
38. Noesselt, T., Shah, N. J., & Jäncke, L.(2003). Top-down and bottom-up modulation of language related areas-An fMRI study. BMC Neuroscience, 4, 13.
39. Oldfield, R. C.(1971). The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia, 9, 97-113.
40. Pekkola, J., Ojanen, V., Autti, T., Jaaskelainen, I. P., Mottonen, R., & Sams, M.(2006). Attention to visual speech gestures enhances hemodynamic activity in the left planum temporale. Human Brain Mapping, 27, 471-477.
41. Petitto, L. A., Zatorre, R. J., Gauna, K., Nikelski, E. J., Dostie, D., & Evans, A. C.(2000). Speech-like cerebral activity in profoundly deaf people processing signed languages: Implications for the neural basis of human language. Proceedings of the National Academy of Sciences, U.S.A., 97, 13961-13966.
42. Phillips, C., Pellathy, T., Marantz, A., Yellin, E., Wexler, K., Poeppel, D., et al.(2000). Auditory cortex accesses phonological categories: An MEG mismatch study. Journal of Cognitive Neuroscience, 12, 1038-1055.
43. Pujol, J., Deus, J., Losilla, J. M., & Capdevila, A.(1999). Cerebral lateralization of language in normal left-handed people studied by functional MRI. Neurology, 52, 1038-1043.
44. Schroeder, C. E., & Foxe, J.(2005). Multisensory contributions to low-level, "unisensory" processing. Current Opinion in Neurobiology, 15, 454-458.
45. Sekiyama, K., Kanno, I., Miura, S., & Sugita, Y.(2003). Auditory-visual speech perception examined by fMRI and PET. Neuroscience Research, 47, 277-287.
46. Sekiyama, K., & Tohkura, Y.(1991). McGurk effect in non-English listeners: Few visual effects for Japanese subjects hearing Japanese syllables of high auditory intelligibility. Journal of the Acoustical Society of America, 90, 1797-1805.
47. Sumby, W. H., & Pollack, I.(1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212-215.
48. Tiitinen, H., Mäkelä, A. M., Mäkinen, V., May, P. J., & Alku, P.(2005). Disentangling the effects of phonation and articulation: Hemispheric asymmetries in the auditory N1m response of the human brain. BMC Neuroscience [electronic resource], 6, 62.
49. Tsuda, R., & Ueno, S.(2000). Source localization of visually evoked magnetic fields to stimuli in apparent motion. IEEE Transactions on Magnetics, 36, 3727-3729.
50. Van Wassenhove, V., Grant, K. W., & Poeppel, D.(2005). Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences, U.S.A. 102 1181-1186.
51. Van Wassenhove, V., Grant, K. W., & Poeppel, D.(2007). Temporal window of integration in auditory-visual speech perception. Neuropsychologia, 45, 598-607.
52. Wheeldon, L., & Waksler, R.(2004). Phonological underspecification and mapping mechanisms in the speech recognition lexicon. Brain and Language, 90, 401-412.