The neural correlates of cross-modal interaction in speech perception during a semantic decision task on sentences: A PET study

NeuroImage

Volumn 32, Issue 1, 2006, Pages 423-431

  Kang, Eunjoo ^a,b   Lee, Dong Soo ^a   Kang, Hyejin ^a   Hwang, Chan Ho ^a   Oh, Seung Ha ^a   Kim, Chong Sun ^a   Chung, June Key ^a   Lee, Myung Chul ^a  

^a Seoul National University   (South Korea)

^b Kangwon National University   (South Korea)

Author keywords

Audiovisual; Auditory language; Bimodal; Broca's area; Crossmodal interaction; Inferior prefrontal; Lipreading; Superior temporal gyrus (STG)

Indexed keywords

ADULT; ARTICLE; BRAIN; BRAIN MAPPING; DECISION MAKING; EDUCATIONAL STATUS; FEMALE; HUMAN; MALE; MOTOR ACTIVITY; MOUTH; NOISE; POSITRON EMISSION TOMOGRAPHY; REACTION TIME; SCINTISCANNING; SPEECH PERCEPTION;

ADULT; BRAIN; BRAIN MAPPING; DECISION MAKING; EDUCATIONAL STATUS; FEMALE; HUMANS; MALE; MOTOR ACTIVITY; MOUTH; NOISE; POSITRON-EMISSION TOMOGRAPHY; REACTION TIME; SPEECH PERCEPTION;

EID: 33748647240     PISSN: 10538119     EISSN: 10959572     Source Type: Journal    
DOI: 10.1016/j.neuroimage.2006.03.016     Document Type: Article

References (56)

1. Allison, T., Puce, A., McCarthy, G., 2000. Social perception from visual cues: role of the STS region. Trends Cogn. Sci. 4, 267-278.
2. Badre, D., Poldrack, R.A., Pare-Blagoev, E.J., Insler, R.Z., Wagner, A.D., 2005. Dissociable controlled retrieval and generalized selection mechanisms in ventrolateral prefrontal cortex. Neuron 47, 907-918.
3. Ban, T., Shiwa, T., Kawamura, K., 1991. Cortico-cortical projections from the prefrontal cortex to the superior temporal sulcal area (STs) in the monkey studied by means of HRP method. Arch. Ital. Biol. 129, 259-272.
4. Baron-Cohen, S., Ring, H., Moriarty, J., Schmitz, B., Costa, D., Ell, P., 1994. Recognition of mental state terms. Clinical findings in children with autism and a functional neuroimaging study of normal adults. Br. J. Psychiatry 165, 640-649.
5. Bernstein, L.E., Auer Jr., E.T., Moore, J.K., Ponton, C.W., Don, M., Singh, M., 2002. Visual speech perception without primary auditory cortex activation. NeuroReport 13, 311-315.
6. Calder, A.J., Lawrence, A.D., Keane, J., Scott, S.K., Owen, A.M., Christoffels, I., Young, A.W., 2002. Reading the mind from eye gaze. Neuropsychologia 40, 1129-1138.
7. Callan, D., Jones, J., Munhall, K., Kroos, C., Callan, C., Vatikiotis-Bateson, E., 2002. Mirror neuron system activity and audiovisual speech perception. 8th Annual meeting of Organization of Human Brain Mapping, No. 10056.
8. Calvert, G.A., Bullmore, E.T., Brammer, M.J., Campbell, R., Williams, S.C., McGuire, P.K., Woodruff, P.W., Iversen, S.D., David, A.S., 1997. Activation of auditory cortex during silent lipreading. Science 276, 593-596.
9. Calvert, G.A., Brammer, M.J., Bullmore, E.T., Campbell, R., Iversen, S.D., David, A.S., 1999. Response amplification in sensory-specific cortices during crossmodal binding. NeuroReport 10, 2619-2623.
10. Calvert, G.A., Campbell, R., Brammer, M.J., 2000. Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr. Biol. 10, 649-657.
11. Campbell, R., 1998. How brains see speech: the cortical localization of speechreading in hearing people. In: Campbell, R., Dodd, B., Burnham, D. (Eds.), Hearing by Eye II: Advances in Psychology of Speechreading and Auditory-Visual Speech. Psychology Press, Hover, pp. 177-193.
12. Campbell, R., MacSweeney, M., Surguladze, S., Calvert, G., McGuire, P., Suckling, J., Brammer, M.J., David, A.S., 2001. Cortical substrates for the perception of face actions: an fMRI study of the specificity of activation for seen speech and for meaningless lower-face acts (gurning). Brain Res. Cogn. Brain Res. 12, 233-243.
13. Carruthers, P., Smith, P.K., 1996. Theories of Theories of Mind. Cambridge University, Cambridge.
14. Deutschländer, A., Bense, S., Stephan, T., Schwaiger, M., Brandt, T., Dieterich, M., 2002. Sensory system interactions during simultaneous vestibular and visual stimulation in PET. Hum. Brain Mapp. 16, 92-103.
15. Duvernoy, H.M., 1999. The Human Brain: Surface, Blood Supply, and Three-Dimensional Sectional Anatomy, 2nd edR SpringerWien, New York.
16. Ettlinger, G., Wilson, W.A., 1990. Cross-modal performance: behavioural processes, phylogenetic considerations and neural mechanisms. Behav. Brain Res. 40, 169-192.
17. Fadiga, L., Craighero, L., Buccino, G., Rizzolatti, G., 2002. Speech listening specifically modulates the excitability of tongue muscles: a TMS study. Eur. J. Neurosci. 15, 399-402.
18. Fletcher, P.C., Happe, F., Frith, U., Baker, S.C., Dolan, R.J., Frackowiak, R.S., Frith, C.D., 1995. Other minds in the brain: a functional imaging study of "theory of mind"in story comprehension. Cognition 57, 109-128.
19. Fort, A., Delpuech, C., Pernier, J., Giard, M.H., 2002. Dynamics of corticosubcortical cross-modal operations involved in audio-visual object detection in humans. Cereb. Cortex 12, 1031-1039.
20. Friederici, A.D., Meyer, M., von Cramon, D.Y., 2000. Auditory language comprehension: an event-related fMRI study on the processing of syntactic and lexical information. Brain Lang. 74, 289-300.
21. Friederici, A.D., Ruschemeyer, S.A., Hahne, A., Fiebach, C.J., 2003. The role of left inferior frontal and superior temporal cortex in sentence comprehension: localizing syntactic and semantic processes. Cereb. Cortex 13, 170-177.
22. Gallese, V., Fadiga, L., Fogassi, L., Rizzolatti, G., 1996. Action recognition in the premotor cortex. Brain 119, 593-609.
23. Giard, M.H., Peronnet, F., 1999. Auditory-visual integration during multimodal object recognition in humans: a behavioral and electrophysiological study. J. Cogn. Neurosci. 11, 473-490.
24. Goel, V., Grafman, J., Sadato, N., Hallett, M., 1995. Modeling other minds. NeuroReport 6, 1741-1746.
25. Haxby, J.V., Horwitz, B., Ungerleider, L.G., Maisog, J.M., Pietrini, P., Grady, C.L., 1994. The functional organization of human extrastriate cortex: a PET-rCBF study of selective attention to faces and locations. J. Neurosci. 14, 6336-6353.
26. Hickok, G., Poeppel, D., 2000. Towards a functional neuroanatomy of speech perception. Trends Cogn. Sci. 4, 131-138.
27. Hoffman, E.A., Haxby, J.V., 2000. Distinct representations of eye gaze and identity in the distributed human neural system for face perception. Nat. Neurosci. 3, 80-84.
28. Howard, R.J., Brammer, M., Wright, I., Woodruff, P.W., Bullmore, E.T., Zeki, S., 1996. A direct demonstration of functional specialization within motion-related visual and auditory cortex of the human brain. Curr. Biol. 6, 1015-1019.
29. Jacquemot, C., Pallier, C., LeBihan, D., Dehaene, S., Dupoux, E., 2003. Phonological grammar shapes the auditory cortex: a functional magnetic resonance imaging study. J. Neurosci. 23, 9541-9546.
30. Jones, J.A., Callan, D.E., 2003. Brain activity during audiovisual speech perception: an fMRI study of the McGurk effect. NeuroReport 14, 1129-1133.
31. Laurienti, P.J., Burdette, J.H., Wallace, M.T., Yen, Y.F., Field, A.S., Stein, B.E., 2002. Deactivation of sensory-specific cortex by cross-modal stimuli. J. Cogn. Neurosci. 14, 420-429.
32. Lebib, R., Papo, D., de Bode, S., Baudonniere, P.M., 2003. Evidence of a visual-to-auditory cross-modal sensory gating phenomenon as reflected by the human P50 event-related brain potential modulation. Neurosci. Lett. 341, 185-188.
33. Macaluso, E., George, N., Dolan, R., Spence, C., Driver, J., 2004. Spatial and temporal factors during processing of audiovisual speech: a PET study. NeuroImage 21, 725-732.
34. MacSweeney, M., Calvert, G.A., Campbell, R., McGuire, P.K., David, A.S., Williams, S.C., Woll, B., Brammer, M.J., 2002a. Speechreading circuits in people born deaf. Neuropsychologia 40, 801-807.
35. MacSweeney, M., Woll, B., Campbell, R., McGuire, P.K., David, A.S., Williams, S.C., Suckling, J., Calvert, G.A., Brammer, M.J., 2002b. Neural systems underlying British Sign Language and audio-visual English processing in native users. Brain 125, 1583-1593.
36. McGurk, H., MacDonald, J., 1976. Hearing lips and seeing voices. Nature 264, 746-748.
37. Nishitani, N., Hari, R., 2000. Temporal dynamics of cortical representation for action. Proc. Natl. Acad. Sci. U. S. A. 97, 913-918.
38. Nishimura, H., Hashikawa, K., Doi, K., Iwaki, T., Watanabe, Y., Kusuoka, H., Nishimura, T., Kubo, T., 1999. Sign language Fheard_ in the auditory cortex. Nature 397, 116.
39. Olson, I.R., Gatenby, J.C., Gore, J.C., 2002. A comparison of bound and unbound audio-visual information processing in the human cerebral cortex. Brain Res. Cogn. Brain Res. 14, 129-138.
40. Petitto, L.A., Zatorre, R.J., Gauna, K., Nikelski, E.J., Dostie, D., Evans, A.C., 2000. Speech-like cerebral activity in profoundly deaf people processing signed languages: implications for the neural basis of human language. Proc. Natl. Acad. Sci. U. S. A. 97, 13961-13966.
41. Puce, A., Allison, T., Bentin, S., Gore, J.C., McCarthy, G., 1998. Temporal cortex activation in humans viewing eye and mouth movements. J. Neurosci. 18, 2188-2199.
42. Rademacher, J., Morosan, P., Schleicher, A., Freund, H.J., Zilles, K., 2001. Human primary auditory cortex in women and men. NeuroReport 12, 1561-1565.
43. Rahm, B., Opwis, K., Kaller, C.P., Spreer, J., Schwarzwald, R., Seifritz, E., Halsband, U., Unterrainer, J.M., 2006. Tracking the subprocesses of decision-based action in the human frontal lobes. NeuroImage 30, 656-667.
44. Rauschecker, J.P., 1995. Compensatory plasticity and sensory substitution in the cerebral cortex. Trends Neurosci. 18, 36-43.
45. Rizzolatti, G., Arbib, M.A., 1998. Language within our grasp. Trends Neurosci. 21, 188-194.
46. Rizzolatti, G., Fadiga, L., Gallese, V., Fogassi, L., 1996. Premotor cortex and the recognition of motor actions. Brain Res. Cogn. Brain Res. 3, 131-141.
47. Rizzolatti, G., Fogassi, L., Gallese, V., 2002. Motor and cognitive functions of the ventral premotor cortex. Curr. Opin. Neurobiol. 12, 149-154.
48. Sams, M., Aulanko, R., Hamalainen, M., Hari, R., Lounasmaa, O.V., Lu, S.T., Simola, J., 1991. Seeing speech: visual information from lip movements modifies activity in the human auditory cortex. Neurosci. Lett. 127, 141-145.
49. Shulman, G.L., Corbetta, M., Buckner, R.L., Raichle, M.E., Fiez, J.A., Miezin, F.M., Petersen, S.E., 1997. Top-down modulation of early sensory cortex. Cereb. Cortex 7, 193-206.
50. Talairach, J., Tournoux, P., 1988. Co-Planar Stereotaxic Atlas of the Human Brain: 3-Dimensional Proportional System: An Approach to Cerebral Imaging. Thieme, New York.
51. van Atteveldt, N., Formisano, E., Goebel, R., Blomert, L., 2004. Integration of letters and speech sounds in the human brain. Neuron 43, 271-282.
52. Wallace, M.T., Stein, B.E., 1996. Sensory organization of the superior colliculus in cat and monkey. Prog. Brain Res. 112, 301-311.
53. Wallace, M.T., Meredith, M.A., Stein, B.E., 1993. Converging influences from visual, auditory, and somatosensory cortices onto output neurons of the superior colliculus. J. Neurophysiol. 69, 1797-1809.
54. Watkins, K.E., Strafella, A.P., Paus, T., 2003. Seeing and hearing speech excites the motor system involved in speech production. Neuropsychologia 41, 989-994.
55. Wright, T.M., Pelphrey, K.A., Allison, T., McKeown, M.J., McCarthy, G., 2003. Polysensory interactions along lateral temporal regions evoked by audiovisual speech. Cereb. Cortex 13, 1034-1043.
56. Zatorre, R.J., Evans, A.C., Meyer, E., Gjedde, A., 1992. Lateralization of phonetic and pitch discrimination in speech processing. Science 256, 846-849.