Lip movements entrain the observers’ low-frequency brain oscillations to facilitate speech intelligibility

eLife

Volumn 5, Issue MAY2016, 2016, Pages

  Park, Hyojin ^a   Kayser, Christoph ^a   Thut, Gregor ^a   Gross, Joachim ^a  

^a UNIVERSITY OF GLASGOW   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ATTENTION; BRAIN FUNCTION; CLINICAL STUDY; COMPREHENSION; HUMAN; LIP; MOTOR CORTEX; OSCILLATION; SPEECH INTELLIGIBILITY; VISUAL CORTEX; ADOLESCENT; ADULT; AUDITORY STIMULATION; ELECTROENCEPHALOGRAM; FEMALE; MALE; MOVEMENT (PHYSIOLOGY); NORMAL HUMAN; PHYSIOLOGY; PSYCHOMOTOR PERFORMANCE; YOUNG ADULT;

ACOUSTIC STIMULATION; ADOLESCENT; ADULT; BRAIN WAVES; FEMALE; HEALTHY VOLUNTEERS; HUMANS; LIP; MALE; MOTOR CORTEX; MOVEMENT; PSYCHOMOTOR PERFORMANCE; SPEECH INTELLIGIBILITY; VISUAL CORTEX; YOUNG ADULT;

EID: 84975256242     PISSN: None     EISSN: 2050084X     Source Type: Journal    
DOI: 10.7554/eLife.14521     Document Type: Article

References (63)

1. Ahissar E, Nagarajan S, Ahissar M, Protopapas A, Mahncke H, Merzenich MM. 2001. Speech comprehension is correlated with temporal response patterns recorded from auditory cortex. Proceedings of the National Academy of Sciences of the United States of America 98: 13367-13372. doi: 10.1073/pnas.201400998
2. Arnal LH, Giraud AL. 2012. Cortical oscillations and sensory predictions. Trends in Cognitive Sciences 16: 390-398. doi: 10.1016/j.tics.2012.05.003
3. Beauchamp MS, Lee KE, Argall BD, Martin A. 2004. Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41: 809-823. doi: 10.1016/S0896-6273(04)00070-4
4. Behne D, Wang Y, Belsby SO, Kaasa S, Simonsen L, Back K. 2008. Visual field advantage in the perception of audiovisual speech segments. International Conference on Auditory-Visual Speech Processing. AVSP-2008. Queensland, Australia. pp. 47- 50.
5. Besl PJ, McKay HD. 1992. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence 14: 239-256. doi: 10.1109/34.121791
6. Blank H, von Kriegstein K. 2013. Mechanisms of enhancing visual-speech recognition by prior auditory information. NeuroImage 65: 109-118. doi: 10.1016/j.neuroimage.2012.09.047
7. Bornkessel-Schlesewsky I, Schlesewsky M, Small SL, Rauschecker JP. 2015. Neurobiological roots of language in primate audition: Common computational properties. Trends in Cognitive Sciences 19: 142-150. doi: 10.1016/j. tics.2014.12.008
8. Brainard DH. 1997. The psychophysics toolbox. Spatial Vision 10: 433-436. doi: 10.1163/156856897X00357
9. Campbell R, De Gelder B, De Haan E. 1996. The lateralization of lip-reading: A second look. Neuropsychologia 34: 1235-1240. doi: 10.1016/0028-3932(96)00046-2
10. Chandrasekaran C, Trubanova A, Stillittano S, Caplier A, Ghazanfar AA. 2009. The natural statistics of audiovisual speech. PLoS Computational Biology 5: e1000436. doi: 10.1371/journal.pcbi.1000436
11. Crosse MJ, Butler JS, Lalor EC. 2015. Congruent visual speech enhances cortical entrainment to continuous auditory speech in noise-free conditions. The Journal of Neuroscience 35: 14195-14204. doi: 10.1523/JNEUROSCI.1829-15.2015
12. Ding N, Simon JZ. 2012. Neural coding of continuous speech in auditory cortex during monaural and dichotic listening. Journal of Neurophysiology 107: 78-89. doi: 10.1152/jn.00297.2011
13. Ding N, Simon JZ. 2014. Cortical entrainment to continuous speech: Functional roles and interpretations. Frontiers in Human Neuroscience 8: 311. doi: 10.3389/fnhum.2014.00311
14. Evans S, Davis MH. 2015. Hierarchical organization of auditory and motor representations in speech perception: Evidence from searchlight similarity analysis. Cerebral Cortex 25. doi: 10.1093/cercor/bhv136
15. Genovese CR, Lazar NA, Nichols T. 2002. Thresholding of statistical maps in functional neuroimaging using the false discovery rate. NeuroImage 15: 870-878. doi: 10.1006/nimg.2001.1037
16. Giraud AL, Kleinschmidt A, Poeppel D, Lund TE, Frackowiak RS, Laufs H. 2007. Endogenous cortical rhythms determine cerebral specialization for speech perception and production. Neuron 56: 1127-1134. doi: 10.1016/j. neuron.2007.09.038
17. Giraud AL, Poeppel D. 2012. Cortical oscillations and speech processing: Emerging computational principles and operations. Nature Neuroscience 15: 511-517. doi: 10.1038/nn.3063
18. Grant KW, Seitz PF. 2000. The use of visible speech cues for improving auditory detection of spoken sentences. The Journal of the Acoustical Society of America 108: 1197-1208. doi: 10.1121/1.1288668
19. Gross J, Baillet S, Barnes GR, Henson RN, Hillebrand A, Jensen O, Jerbi K, Litvak V, Maess B, Oostenveld R, Parkkonen L, Taylor JR, van Wassenhove V, Wibral M, Schoffelen J-M. 2013a. Good practice for conducting and reporting MEG research. NeuroImage 65: 349-363. doi: 10.1016/j.neuroimage.2012.10.001
20. Gross J, Hoogenboom N, Thut G, Schyns P, Panzeri S, Belin P, Garrod S. 2013b. Speech rhythms and multiplexed oscillatory sensory coding in the human brain. PLoS Biology 11: e1001752: e1001752. doi: 10.1371/ journal.pbio.1001752
21. Gross J, Kujala J, Hamalainen M, Timmermann L, Schnitzler A, Salmelin R. 2001. Dynamic imaging of coherent sources: Studying neural interactions in the human brain. Proceedings of the National Academy of Sciences of the United States of America 98: 694-699. doi: 10.1073/pnas.98.2.694
22. Hasson U, Ghazanfar AA, Galantucci B, Garrod S, Keysers C. 2012. Brain-to-brain coupling: A mechanism for creating and sharing a social world. Trends in Cognitive Sciences 16: 114-121. doi: 10.1016/j.tics.2011.12.007
23. Hocking J, Price CJ. 2008. The role of the posterior superior temporal sulcus in audiovisual processing. Cerebral Cortex 18: 2439-2449. doi: 10.1093/cercor/bhn007
24. Kayser SJ, Ince RA, Gross J, Kayser C. 2015. Irregular speech rate dissociates auditory cortical entrainment, evoked responses, and frontal alpha. The Journal of Neuroscience 35: 14691-14701. doi: 10.1523/JNEUROSCI. 2243-15.2015
25. Kim J, Davis C. 2003. Hearing foreign voices: Does knowing what is said affect visual-masked-speech detection? Perception 32: 111-120. doi: 10.1068/p3466
26. Lakatos P, Musacchia G, O’Connel MN, Falchier AY, Javitt DC, Schroeder CE. 2013. The spectrotemporal filter mechanism of auditory selective attention. Neuron 77: 750-761. doi: 10.1016/j.neuron.2012.11.034
27. Likert R. 1932. A technique for the measurement of attitudes. Archives of Psychology 22: 1-55.
28. Luo H, Liu Z, Poeppel D. 2010. Auditory cortex tracks both auditory and visual stimulus dynamics using low-frequency neuronal phase modulation. PLoS Biology 8: e1000445. doi: 10.1371/journal.pbio.1000445
29. Luo H, Poeppel D. 2007. Phase patterns of neuronal responses reliably discriminate speech in human auditory cortex. Neuron 54: 1001-1010. doi: 10.1016/j.neuron.2007.06.004
30. Meister IG, Wilson SM, Deblieck C, Wu AD, Iacoboni M. 2007. The essential role of premotor cortex in speech perception. Current Biology: CB 17: 1692-1696. doi: 10.1016/j.cub.2007.08.064
31. Morillon B, Hackett TA, Kajikawa Y, Schroeder CE. 2015. Predictive motor control of sensory dynamics in auditory active sensing. Current Opinion in Neurobiology 31: 230-238. doi: 10.1016/j.conb.2014.12.005
32. Morillon B, Schroeder CE, Wyart V. 2014. Motor contributions to the temporal precision of auditory attention. Nature Communications 5. doi: 10.1038/ncomms6255
33. Möttönen R, Dutton R, Watkins KE. 2013. Auditory-motor processing of speech sounds. Cerebral Cortex 23: 1190-1197. doi: 10.1093/cercor/bhs110
34. Ng BS, Logothetis NK, Kayser C. 2013. EEG phase patterns reflect the selectivity of neural firing. Cerebral Cortex 23: 389-398. doi: 10.1093/cercor/bhs031
35. Nicholls ME, Searle DA. 2006. Asymmetries for the visual expression and perception of speech. Brain and Language 97: 322-331. doi: 10.1016/j.bandl.2005.11.007
36. Noesselt T, Bergmann D, Heinze HJ, Münte T, Spence C. 2012. Coding of multisensory temporal patterns in human superior temporal sulcus. Frontiers in Integrative Neuroscience 6: 64. doi: 10.3389/fnint.2012.00064
37. Nolte G. 2003. The magnetic lead field theorem in the quasi-static approximation and its use for magnetoencephalography forward calculation in realistic volume conductors. Physics in Medicine and Biology 48: 3637-3652. doi: 10.1088/0031-9155/48/22/002
38. Oldfield RC. 1971. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9: 97-113. doi: 10.1016/0028-3932(71)90067-4
39. Oostenveld R, Fries P, Maris E, Schoffelen JM. 2011. Fieldtrip: Open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Computational Intelligence and Neuroscience 2011: 156869. doi: 10.1155/2011/156869
40. Panzeri S, Macke JH, Gross J, Kayser C. 2015. Neural population coding: Combining insights from microscopic and mass signals. Trends in Cognitive Sciences 19: 162-172. doi: 10.1016/j.tics.2015.01.002
41. Park H, Ince RA, Schyns PG, Thut G, Gross J. 2015. Frontal top-down signals increase coupling of auditory low-frequency oscillations to continuous speech in human listeners. Current Biology 25: 1649-1653. doi: 10.1016/j. cub.2015.04.049
42. Peelle JE, Gross J, Davis MH. 2013. Phase-locked responses to speech in human auditory cortex are enhanced during comprehension. Cerebral Cortex 23: 1378-1387. doi: 10.1093/cercor/bhs118
43. Peelle JE, Sommers MS. 2015. Prediction and constraint in audiovisual speech perception. Cortex 68: 169-181. doi: 10.1016/j.cortex.2015.03.006
44. Pickering MJ, Garrod S. 2013. An integrated theory of language production and comprehension. The Behavioral and Brain Sciences 36: 329-347. doi: 10.1017/S0140525X12001495
45. Rauschecker JP, Scott SK. 2009. Maps and streams in the auditory cortex: Nonhuman primates illuminate human speech processing. Nature Neuroscience 12: 718-724. doi: 10.1038/nn.2331
46. Rosenberg JR, Amjad AM, Breeze P, Brillinger DR, Halliday DM. 1989. The fourier approach to the identification of functional coupling between neuronal spike trains. Progress in Biophysics and Molecular Biology 53: 1-31. doi: 10.1016/0079-6107(89)90004-7
47. Rosenberg JR, Halliday DM, Breeze P, Conway BA. 1998. Identification of patterns of neuronal connectivity-partial spectra, partial coherence, and neuronal interactions. Journal of Neuroscience Methods 83: 57-72. doi: 10.1016/S0165-0270(98)00061-2
48. Schroeder CE, Lakatos P, Kajikawa Y, Partan S, Puce A. 2008. Neuronal oscillations and visual amplification of speech. Trends in Cognitive Sciences 12: 106-113. doi: 10.1016/j.tics.2008.01.002
49. Schyns PG, Thut G, Gross J. 2011. Cracking the code of oscillatory activity. PLoS Biology 9: e1001064. doi: 10. 1371/journal.pbio.1001064
50. Smeele PM, Massaro DW, Cohen MM, Sittig AC. 1998. Laterality in visual speech perception. Journal of Experimental Psychology. Human Perception and Performance 24: 1232-1242. doi: 10.1037/0096-1523.24.4. 1232
51. Smith ZM, Delgutte B, Oxenham AJ. 2002. Chimaeric sounds reveal dichotomies in auditory perception. Nature 416: 87-90. doi: 10.1038/416087a
52. Sumby WH., Pollack I. 1954. Visual contribution to speech intelligibility in noise. The Journal of the Acoustical Society of America 26: 212. doi: 10.1121/1.1907309
53. Swerts M, Krahmer E. 2008. Facial expression and prosodic prominence: Effects of modality and facial area. Journal of Phonetics 36: 219-238. doi: 10.1016/j.wocn.2007.05.001
54. Thompson LA, Malmberg J, Goodell NK, Boring RL. 2004. The Distribution of Attention Across a Talker’s Face. Discourse Processes 38: 145-168. doi: 10.1207/s15326950dp3801_6
55. van Wassenhove V, Grant KW, Poeppel D. 2005. Visual speech speeds up the neural processing of auditory speech. Proceedings of the National Academy of Sciences of the United States of America 102: 1181-1186. doi: 10.1073/pnas.0408949102
56. Watkins KE, Strafella AP, Paus T. 2003. Seeing and hearing speech excites the motor system involved in speech production. Neuropsychologia 41: 989-994. doi: 10.1016/S0028-3932(02)00316-0
57. Werner S, Noppeney U. 2010. Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Cerebral Cortex 20: 1829-1842. doi: 10.1093/cercor/bhp248
58. Wiesmann UN, DiDonato S, Herschkowitz NN, Schwartz, J.L., Savariaux, C. 1975. Effect of chloroquine on cultured fibroblasts: Release of lysosomal hydrolases and inhibition of their uptake. Biochemical and Biophysical Research Communications 66: e1003743. doi: 10.1371/journal.pcbi.1003743
59. Wilson SM, Molnar-Szakacs I, Iacoboni M. 2008. Beyond superior temporal cortex: Intersubject correlations in narrative speech comprehension. Cerebral Cortex 18: 230-242. doi: 10.1093/cercor/bhm049
60. Wilson SM, Saygin AP, Sereno MI, Iacoboni M. 2004. Listening to speech activates motor areas involved in speech production. Nature Neuroscience 7: 701-702. doi: 10.1038/nn1263
61. Ylinen S, Nora A, Leminen A, Hakala T, Huotilainen M, Shtyrov Y, MäkeläJP, Service E. 2015. Two distinct auditory-motor circuits for monitoring speech production as revealed by content-specific suppression of auditory cortex. Cerebral Cortex 25: 1576-1586. doi: 10.1093/cercor/bht351
62. Zion Golumbic E, Cogan GB, Schroeder CE, Poeppel D. 2013. Visual input enhances selective speech envelope tracking in auditory cortex at a "cocktail party". The Journal of Neuroscience 33: 1417-1426. doi: 10.1523/JNEUROSCI.3675-12.2013
63. Zion-Golumbic E, Schroeder CE. 2012. Attention modulates ‘speech-tracking’ at a cocktail party. Trends in Cognitive Sciences 16: 363-364. doi: 10.1016/j.tics.2012.05.004