Audiovisual speech integration does not rely on the motor system: Evidence from articulatory suppression, the McGurk effect, and fMRI

Journal of Cognitive Neuroscience

Volumn 26, Issue 3, 2014, Pages 606-620

  Matchin, William ^a   Groulx, Kier ^a   Hickok, Gregory ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; AUDIOVISUAL AID; BEHAVIOR; BRAIN REGION; DIAGONAL BAND OF BROCA; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HEARING; MCGURK EFFECT; MOTOR SYSTEM; NEUROIMAGING; PRIORITY JOURNAL; SPEECH; SPEECH ARTICULATION; SPEECH INTEGRATION;

ACOUSTIC STIMULATION; ADOLESCENT; ADULT; AUDITORY PERCEPTION; BRAIN; BRAIN MAPPING; FEMALE; HUMANS; MAGNETIC RESONANCE IMAGING; MALE; MOTOR ACTIVITY; NEURAL PATHWAYS; PHOTIC STIMULATION; SPEECH; SPEECH ACOUSTICS; SPEECH PERCEPTION; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84893396245     PISSN: 0898929X     EISSN: 15308898     Source Type: Journal    
DOI: 10.1162/jocn_a_00515     Document Type: Article

References (53)

1. Baddeley, A., Eldridge, M., & Lewis, V. (1981). The role of subvocalization in reading. Quarterly Journal of Experimental Psychology: Section A-Human Experimental Psychology, 33, 439-454.
2. Beauchamp, M. S. (2005). Statistical criteria in fMRI studies of multisensory integration. Neuroinformatics, 3, 93-113.
3. Beauchamp, M. S., Argall, B. D., Bodurka, J., Duyn, J. H., & Martin, A. (2004). Unraveling multisensory integration: Patchy organization within human STS multisensory cortex. Nature Neuroscience, 7, 1190-1192.
4. Beauchamp, M. S., Lee, K. E., Argall, B. D., & Martin, A. (2004). Integration of auditory and visual information about objects in superior temporal sulcus. Neuron, 41, 809-823.
5. Beauchamp, M. S., Nath, A. R., & Pasalar, S. (2010). fMRI-guided transcranial magnetic stimulation reveals that the superior temporal sulcus is a cortical locus of the McGurk effect. Journal of Neuroscience, 30, 2414-2417.
6. Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10, 433-436.
7. Buchsbaum, B. R., Olsen, R. K., Koch, P., & Berman, K. F. (2005). Human dorsal and ventral auditory streams subserve rehearsal-based and echoic processes during verbal working memory. Neuron, 48, 687-697.
8. Buckner, R. L., Andrews-Hanna, J. R., & Schacter, D. L. (2008). The brain's default network. Annals of the New York Academy of Sciences, 1124, 1-38.
9. Callan, D. E., Jones, J. A., Munhall, K., Callan, A. M., Kroos, C., & Vatikiotis-Bateson, E. (2003). Neural processes underlying perceptual enhancement by visual speech gestures. NeuroReport, 14, 2213-2218.
10. Callan, D. E., Jones, J. A., Munhall, K., Kroos, C., Callan, A. M., & Vatikiotis-Bateson, E. (2004). Multisensory integration sites identified by perception of spatial wavelet filtered visual speech gesture information. Journal of Cognitive Neuroscience, 16, 805-816.
11. Calvert, G., Brammer, M., & Campbell, R. (2001). Cortical subtrates of seeing speech: Still and moving faces. Neuroimage, 13, S513.
12. Calvert, G. A., & Campbell, R. (2003). Reading speech from still and moving faces: The neural substrates of visible speech. Journal of Cognitive Neuroscience, 15, 57-70.
13. Calvert, G. A., Campbell, R., & Brammer, M. J. (2000). Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Current Biology, 10, 649-657.
14. Campbell, R., MacSweeney, M., Surguladze, S., Calvert, G., McGuire, P., Suckling, J., et al. (2001). Cortical substrates for the perception of face actions: An fMRI study of the specificity of activation for seen speech and for meaningless lower-face acts (gurning). Cognitive Brain Research, 12, 233-243.
15. Fisher, C. G. (1968). Confusions among visually perceived consonants. Journal of Speech, Language and Hearing Research, 11, 796.
16. Fridriksson, J., Moss, J., Davis, B., Baylis, G. C., Bonilha, L., & Rorden, C. (2008). Motor speech perception modulates the cortical language areas. Neuroimage, 41, 605-613.
17. Hickok, G. (2012). Computational neuroanatomy of speech production. Nature Reviews Neuroscience, 13, 135-145.
18. Hickok, G., Buchsbaum, B., Humphries, C., & Muftuler, T. (2003). Auditory-motor interaction revealed by fMRI: Speech, music, and working memory in area Spt. Journal of Cognitive Neuroscience, 15, 673-682.
19. Hickok, G., Houde, J., & Rong, F. (2011). Sensorimotor integration in speech processing: Computational basis and neural organization. Neuron, 69, 407-422.
20. Houde, J. F., & Nagarajan, S. S. (2011). Speech production as state feedback control. Frontiers in Human Neuroscience, 5, 82.
21. Jones, E. G., & Powell, T. P. S. (1970). An anatomical study of converging sensory pathways in the cerebral cortex of the monkey. Brain Behavior and Evolution, 93, 793-820.
22. Kleiner, M., Brainard, D., & Pelli, D. (2007). What's new in Psychtoolbox-3? Perception, 36, 1-1.
23. Liberman, A. M., & Mattingly, I. G. (1985). The motor theory of speech perception revised. Cognition, 21, 1-36.
24. Macmillan, N. A., & Creelman, C. D. (2005). Detection theory: A user's guide. Mahwah, NJ: Erlbaum.
25. MacSweeney, M., Amaro, E., Calvert, G. A., Campbell, R., David, A. S., McGuire, P., et al. (2000). Silent speechreading in the absence of scanner noise: An event-related fMRI study. NeuroReport, 11, 1729-1733.
26. McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746-748.
27. Mesulam, M., & Mufson, E. J. (1982). Insula of the old world monkey. III: Efferent cortical output and comments on function. Journal of Comparative Neurology, 212, 38-52.
28. Miller, L. M., & D'Esposito, M. (2005). Perceptual fusion and stimulus coincidence in the cross-modal integration of speech. Journal of Neuroscience, 25, 5884-5893.
29. Nath, A. R., & Beauchamp, M. S. (2012). A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion. Neuroimage, 59, 781-787.
30. Novick, J. M., Trueswell, J. C., & Thompson-Schill, S. L. (2005). Cognitive control and parsing: Reexamining the role of Broca's area in sentence comprehension. Cognitive Affective & Behavioral Neuroscience, 5, 263-281.
31. Ojanen, V., Mottonen, R., Pekkola, J., Jaaskelainen, I. P., Joensuu, R., Autti, T., et al. (2005). Processing of audiovisual speech in Broca's area. Neuroimage, 25, 333-338.
32. Okada, K., & Hickok, G. (2009). Two cortical mechanisms support the integration of visual and auditory speech: A hypothesis and preliminary data. Neuroscience Letters, 452, 219-223.
33. Paulesu, E., Perani, D., Blasi, V., Silani, G., Borghese, N. A., De Giovanni, U., et al. (2003). A functional-anatomical model for lipreading. Journal of Neurophysiology, 90, 2005-2013.
34. Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10, 437-442.
35. Rosenblum, L. D., Schmuckler, M. A., & Johnson, J. A. (1997). The McGurk effect in infants. Perception & Psychophysics, 59, 347-357.
36. Sams, M., Mottonen, R., & Sihvonen, T. (2005). Seeing and hearing others and oneself talk. Cognitive Brain Research, 23, 429-435.
37. Sato, M., Buccino, G., Gentilucci, M., & Cattaneo, L. (2010). On the tip of the tongue: Modulation of the primary motor cortex during audiovisual speech perception. Speech Communication, 52, 533-541.
38. Sato, M., Grabski, K., Glenberg, A. M., Brisebois, A., Basirat, A., Menard, L., et al. (2011). Articulatory bias in speech categorization: Evidence from use-induced motor plasticity. Cortex, 47, 1001-1003.
39. Sekiyama, K., Kanno, I., Miura, S., & Sugita, Y. (2003). Auditory-visual speech perception examined by fMRI and PET. Neuroscience Research, 47, 277-287.
40. Seltzer, B., & Pandya, D. N. (1978). Afferent cortical connections and architectonics of superior temporal sulcus and surrounding cortex in rhesus-monkey. Brain Research, 149, 1-24.
41. Seltzer, B., & Pandya, D. N. (1980). Converging visual and somatic sensory cortical input to the intraparietal sulcus of the rhesus-monkey. Brain Research, 192, 339-351.
42. Skipper, J. I., Nusbaum, H. C., & Small, S. L. (2005). Listening to talking faces: Motor cortical activation during speech perception. Neuroimage, 25, 76-89.
43. Skipper, J. I., van Wassenhove, V., Nusbaum, H. C., & Small, S. L. (2007). Hearing lips and seeing voices: How cortical areas supporting speech production mediate audiovisual speech perception. Cerebral Cortex, 17, 2387-2399.
44. Snyder, H. R., Feigenson, K., & Thompson-Schill, S. L. (2007). Prefrontal cortical response to conflict during semantic and phonological tasks. Journal of Cognitive Neuroscience, 19, 761-775.
45. Stevens, K. N., & Halle, M. (1967). Remarks on analysis by synthesis and distinctive features. In W. Wathen-Dunn (Ed.), Models for the perception of speech and visual form (pp. 88-102). Cambridge, MA: MIT Press.
46. Stevenson, R. A., VanDerKlok, R. M., Pisoni, D. B., & James, T. W. (2011). Discrete neural substrates underlie complementary audiovisual speech integration processes. Neuroimage, 55, 1339-1345.
47. Sumby, W. H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. Journal of the Acoustical Society of America, 26, 212-215.
48. Talairach, J., & Tournoux, P. (1988). Co-planar stereotaxic atlas of the human brain. New York: Thieme.
49. Thompson-Schill, S. L., D'Esposito, M., Aguirre, G. K., & Farah, M. J. (1997). Role of left inferior prefrontal cortex in retrieval of semantic knowledge: A reevaluation. Proceedings of the National Academy of Sciences, U.S.A., 94, 14792-14797.
50. Tourville, J. A., & Guenther, F. H. (2011). The DIVA model: A neural theory of speech acquisition and production. Language and Cognitive Processes, 26, 952-981.
51. Venezia, J. H., Saberi, K., Chubb, C., & Hickok, G. (2012). Response bias modulates the speech motor system during syllable discrimination. Frontiers in Psychology, 3, 157.
52. Watkins, K. E., Strafella, A. P., & Paus, T. (2003). Seeing and hearing speech excites the motor system involved in speech production. Neuropsychologia, 41, 989-994.
53. Yeterian, E. H., & Pandya, D. N. (1985). Corticothalamic connections of the posterior parietal cortex in the rhesus-monkey. Journal of Comparative Neurology, 237, 408-426.