Activation of auditory cortex during silent lipreading

Science

Volumn 276, Issue 5312, 1997, Pages 593-596

  Calvert, Gemma A ^a   Bullmore, Edward T ^b   Brammer, Michael J ^b   Campbell, Ruth ^c   Williams, Steven C R ^b   McGuire, Philip K ^b   Woodruff, Peter W R ^b   Iversen, Susan D ^a   David, Anthony S ^b,d  

^a UNIVERSITY OF OXFORD   (United Kingdom)

^b KING'S COLLEGE LONDON   (United Kingdom)

^c UNIVERSITY COLLEGE LONDON   (United Kingdom)

^d KING'S COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; AUDITORY CORTEX; CONVERSATION; HUMAN; LIP READING; LOUDNESS RECRUITMENT; NOISE; PRIORITY JOURNAL; SPEECH PERCEPTION;

AUDITORY CORTEX; AUDITORY PERCEPTION; BRAIN MAPPING; CUES; FACIAL EXPRESSION; FRONTAL LOBE; GESTURES; HUMANS; LIPREADING; MAGNETIC RESONANCE IMAGING; SPEECH PERCEPTION; TEMPORAL LOBE; VISUAL PERCEPTION;

EID: 0030999865     PISSN: 00368075     EISSN: None     Source Type: Journal    
DOI: 10.1126/science.276.5312.593     Document Type: Article

References (35)

1. D. Reisberg, J. McLean, A. Goldfield, in Hearing by Eye, B. Dodd and R. Campbell, Eds. (Lawrence Erlbaum, London, UK, 1987), pp. 97-114.
2. W. H. Sumby and I. Pollack, J. Acoustic Soc. Am. 26, 212 (1954).
3. H. McGurk and J. MacDonald, Nature 263, 747 (1976). The McGurk effect is elicited when a listener's perceptual report of a heard syllable (for example, "ba") is influenced by the sight of the speaker mouthing a different syllable (for example, "ga"), inducing the report of another syllable (typically "da").
4. J. R. Bider et al., Ann. Neurol. 35, 662 (1994); A. S. David et al., NeuroReport 7, 932 (1996).
5. J. R. Bider et al., Ann. Neurol. 35, 662 (1994); A. S. David et al., NeuroReport 7, 932 (1996).
6. K. Green and P. Kuhl, J. Exp. Psychol. Hum. Percept. Performance 17, 278 (1991); Q. Summerfield and M. McGrath, Q. J. Exp. Psychol. 36A, 51 (1984).
7. K. Green and P. Kuhl, J. Exp. Psychol. Hum. Percept. Performance 17, 278 (1991); Q. Summerfield and M. McGrath, Q. J. Exp. Psychol. 36A, 51 (1984).
8. The brain regions reported active in each experiment were those demonstrating periodic signal change, in phase either with the ON or OFF condition. The OFF condition was always presented first. There was no auditory stimulation in experiments 2 to 4 except for the scanner noise. As this was the same during all ON and OFF conditions, any signal change related to the background noise would not be periodic and would therefore not contribute to the estimated experimental effect. For all experiments reported, all participants gave informed and written consent.
9. -4, that voxel was considered activated and represented in color on the gray scale background of a coregistered (spoiled GRASS) MR image. By significance testing at this level, false-positive activation should account for less than 10 colored voxels over the whole generic brain activation map.
10. -4, that voxel was considered activated and represented in color on the gray scale background of a coregistered (spoiled GRASS) MR image. By significance testing at this level, false-positive activation should account for less than 10 colored voxels over the whole generic brain activation map.
11. -4, that voxel was considered activated and represented in color on the gray scale background of a coregistered (spoiled GRASS) MR image. By significance testing at this level, false-positive activation should account for less than 10 colored voxels over the whole generic brain activation map.
12. -4, that voxel was considered activated and represented in color on the gray scale background of a coregistered (spoiled GRASS) MR image. By significance testing at this level, false-positive activation should account for less than 10 colored voxels over the whole generic brain activation map.
13. -4, that voxel was considered activated and represented in color on the gray scale background of a coregistered (spoiled GRASS) MR image. By significance testing at this level, false-positive activation should account for less than 10 colored voxels over the whole generic brain activation map.
14. The stimuli in experiments 1 and 2 comprised spoken or mouthed numbers ranging from 1 to 10, uttered in random order at 0.5 Hz. The stimuli were selected to be equally comprehensive when presented in either the auditory or visual domain. All participants were shown examples of the stimuli before the scanning session and were able to lipread the numbers. In experiment 1 no visual stimulus was present during either condition.
15. Throughout the experiment the participant's field of view was restricted to the lower half of the face to minimize the influence of direction of gaze and facial identity processing. The same face was used for both moving-and static-face stimuli.
16. J. D. Watson et al., Cereb. Cortex 3 (no. 2), 79 (1994); R. J. Howard et al., Curr. Biol. 6, 1015 (1996).
17. J. D. Watson et al., Cereb. Cortex 3 (no. 2), 79 (1994); R. J. Howard et al., Curr. Biol. 6, 1015 (1996).
18. J.-F. Demonet et al., Brain 115, 1753 (1992); H. Hecaen and H. Kremin, in Studies in Neurolinguistics, H. Whitaker and H. A. Whitaker, Eds. (Academic Press, New York, 1979), vol. 2, pp. 269-329.
19. J.-F. Demonet et al., Brain 115, 1753 (1992); H. Hecaen and H. Kremin, in Studies in Neurolinguistics, H. Whitaker and H. A. Whitaker, Eds. (Academic Press, New York, 1979), vol. 2, pp. 269-329.
20. In a further experiment with the same individuals reported in experiments 1 and 2, we contrasted hearing speech with concordant lip movements (face-to-face conversation) with the same lip movements in the absence of sound. This comparison resulted in marked attenuation of the expected signal from auditory cortical sites in the first condition, presumably because they were canceled out by activation of these areas during the second condition (silent lipreading). This concurs with our conclusion that silent lipreading stimulates those auditory cortical sites activated when listening to speech.
21. S. E. Blumstein, in The Cognitive Neurosciences, M. S. Gazzaniga, Ed. (MIT Press, Cambridge, MA, 1995), pp. 915-929.
22. P. K. Kuhl and A. N. Meltzoff, Science 218, 1138 (1982); M. Papousek, in Nonverbal Vocal Communication: Comparative and Developmental Approaches, H. Papousek, U. Jurgens, M. Papousek, Eds. (Cambridge Univ. Press, New York, 1992). Such cross-modal processes might also be relevant to compensatory plasticity after sensory loss [see J. P. Rauschecker, Trends Neurosci. 18, 36 (1995)].
23. P. K. Kuhl and A. N. Meltzoff, Science 218, 1138 (1982); M. Papousek, in Nonverbal Vocal Communication: Comparative and Developmental Approaches, H. Papousek, U. Jurgens, M. Papousek, Eds. (Cambridge Univ. Press, New York, 1992). Such cross-modal processes might also be relevant to compensatory plasticity after sensory loss [see J. P. Rauschecker, Trends Neurosci. 18, 36 (1995)].
24. P. K. Kuhl and A. N. Meltzoff, Science 218, 1138 (1982); M. Papousek, in Nonverbal Vocal Communication: Comparative and Developmental Approaches, H. Papousek, U. Jurgens, M. Papousek, Eds. (Cambridge Univ. Press, New York, 1992). Such cross-modal processes might also be relevant to compensatory plasticity after sensory loss [see J. P. Rauschecker, Trends Neurosci. 18, 36 (1995)].
25. To address the possibility that activation of auditory cortex during silent lipreading in experiment 2 was due to greater articulatory demands in this ON condition (repeating back numbers), we repeated the experiment but modified the task instructions in the OFF condition by asking participants to count silently from one, rather than simply repeating "one," to match more closely the articulatory demands of the ON (lipreading) condition. This comparison again demonstrated activation of auditory cortical areas in response to lipreading (Fig. 1B), as shown initially in experiment 2. The largest cluster of (13) voxels in temporal cortex that were coincidentally activated by auditory speech perception (experiment 1), and by silent lipreading in this replication of experiment 2, had a diameter of 9 mm and was centered at Talairach coordinates x = -56, y = -26, z = 9.5. This center point lies in BA 42 and the cluster extends superiorly into BA 41 and posteriorly and interiorly into BA 22, thus replicating the findings of experiment 2. In addition, our previous PET study of silent articulation reveals prominent activation of the inferior frontal gyrus but only minimal activation of temporal cortex and no activation in BA 41 during internal rehearsal [P. K. McGuire et al., Psychol. Med. 26, 29 (1996)]. Hence, if in our initial study the temporal cortical activation had been due to increased articulation, we would have also expected to see substantial differential activation in the inferior frontal gyrus during the lipreading condition. In fact there were only a few voxels activated in this region in experiment 2, suggesting that the two conditions in this experiment did not differ importantly in terms of their articulatory demands.
26. J. D. C. Mellers et al., NeuroReport 7, 109 (1995).
27. The stimuli comprised a videotape of the same speaker mouthing a random string of syllables using a wide range of phonological structures: vowels included a range of non-English diphthongs (such as the German "ü"); consonants included some combinations that are not common in English (such as "pf-," "fth-," and "plm-"). We have classified these as examples of pseudospeech that were perceived as phonetically plausible but were not recognizable as words.
28. E. Paulesu, C. D. Frith, R. S. J. Frackowiak, Nature 362, 342 (1993); M. Habib et al., Neuropsychologia 33, 327 (1995).
29. E. Paulesu, C. D. Frith, R. S. J. Frackowiak, Nature 362, 342 (1993); M. Habib et al., Neuropsychologia 33, 327 (1995).
30. P. Ekman, Ed., Darwin and Facial Expression (Academic Press, New York, 1973).
31. E. Bonda, M. Petrides, D. Ostry, A. Evans, J. Neurosci. 16 (no. 11), 3737 (1996).
32. D. W. Massaro, Ed., Speech Perception by Ear and by Eye (Erlbaum, Hillsdale, NJ, 1988).
33. E. T. Bullmore et al., Magn. Reson. Med. 35, 261 (1996).
34. Areas of activation shown are those corresponding to the largest clusters. A complete list of activations can be obtained from the corresponding author. Within each cluster the coordinates of the voxel with the maximum FPQ are shown.
35. Supported by an Oxford University Pump Priming grant and the Bethlem and Maudsley Research Fund. E.T.B. and P.K.M. were supported by the Wellcome Trust, G.A.C. by the Medical Research Council of Great Britain, and P.W.R.W. by British Telecom. We are grateful to I. C. Wright and R. Howard and to two anonymous reviewers for their helpful comments on this manuscript.