Coupled neural systems underlie the production and comprehension of naturalistic narrative speech

Proceedings of the National Academy of Sciences of the United States of America

Volumn 111, Issue 43, 2014, Pages E4687-E4696

  Silbert, Lauren J ^a   Honey, Christopher J ^a,b   Simony, Erez ^a   Poeppel, David ^c   Hasson, Uri ^a  

^a PRINCETON UNIVERSITY   (United States)

^b UNIVERSITY OF TORONTO   (Canada)

^c NEW YORK UNIVERSITY   (United States)

Author keywords

Brain to brain coupling; Intersubject correlation; Speech comprehension; Speech production

Indexed keywords

ADULT; ARTICLE; BOLD SIGNAL; BRAIN REGION; COMPREHENSION; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; LANGUAGE PROCESSING; LEFT HEMISPHERE; NARRATIVE; NEUROIMAGING; NORMAL HUMAN; SPEECH; BLOOD; BRAIN; NERVE CELL; NUCLEAR MAGNETIC RESONANCE IMAGING; PHYSIOLOGY; REPRODUCIBILITY; SIGNAL PROCESSING; TIME; VERBAL COMMUNICATION; YOUNG ADULT;

OXYGEN;

ADULT; BRAIN; COMPREHENSION; HUMANS; MAGNETIC RESONANCE IMAGING; NARRATION; NEURONS; OXYGEN; REPRODUCIBILITY OF RESULTS; SIGNAL PROCESSING, COMPUTER-ASSISTED; SPEECH; TIME FACTORS; YOUNG ADULT;

EID: 84908287054     PISSN: 00278424     EISSN: 10916490     Source Type: Journal    
DOI: 10.1073/pnas.1323812111     Document Type: Article

References (114)

1. Schiller NO, Bles M, Jansma BM (2003) Tracking the time course of phonological encoding in speech production: An event-related brain potential study. Brain Res Cogn Brain Res 17(3):819-831.
2. Abdel Rahman R, Sommer W (2003) Does phonological encoding in speech production always follow the retrieval of semantic knowledge? Electrophysiological evidence for parallel processing. Brain Res Cogn Brain Res 16(3):372-382.
3. Rodriguez-Fornells A, Schmitt BM, Kutas M, Münte TF (2002) Electrophysiological estimates of the time course of semantic and phonological encoding during listening and naming. Neuropsychologia 40(7):778-787.
4. Hanulová J, Davidson DJ, Indefrey P (2011) Where does the delay in L2 picture naming come from? Psycholinguistic and neurocognitive evidence on second language word production. Lang Cogn Process 26(7):902-934.
5. Camen C, Morand S, Laganaro M (2010) Re-evaluating the time course of gender and phonological encoding during silent monitoring tasks estimated by ERP: Serial or parallel processing? J Psycholinguist Res 39(1):35-49.
6. Rahman RA, Sommer W (2008) Seeing what we know and understand: How knowledge shapes perception. Psychon Bull Rev 15(6):1055-1063.
7. Aristei S, Melinger A, Abdel Rahman R (2011) Electrophysiological chronometry of semantic context effects in language production. J Cogn Neurosci 23(7):1567-1586.
8. Huettig F, Hartsuiker RJ (2008) When you name the pizza you look at the coin and the bread: Eye movements reveal semantic activation during word production. Mem Cognit 36(2):341-360.
9. Haller S, Radue EW, Erb M, Grodd W, Kircher T (2005) Overt sentence production in event-related fMRI. Neuropsychologia 43(5):807-814.
10. Meyer M, Alter K, Friederici AD, Lohmann G, von Cramon DY (2002) FMRI reveals brain regions mediating slow prosodic modulations in spoken sentences. Hum Brain Mapp 17(2):73-88.
11. Garrod S, Pickering MJ (2004) Why is conversation so easy? Trends Cogn Sci 8(1):8-11.
12. Indefrey P (2011) The spatial and temporal signatures of word production components: a critical update. Front Psychol 2:255.
13. Indefrey P, Levelt WJ (2004) The spatial and temporal signatures of word production components. Cognition 92(1-2):101-144.
14. Braun AR, Guillemin A, Hosey L, Varga M (2001) The neural organization of discourse: An H2 15O-PET study of narrative production in English and American sign language. Brain 124(Pt 10):2028-2044.
15. Bavelier D, et al. (1997) Sentence reading: A functional MRI study at 4 tesla. J Cogn Neurosci 9(5):664-686.
16. Jung-Beeman M (2005) Bilateral brain processes for comprehending natural language. Trends Cogn Sci 9(11):512-518.
17. Hickok G, Poeppel D (2007) The cortical organization of speech processing. Nat Rev Neurosci 8(5):393-402.
18. Lerner Y, Honey CJ, Silbert LJ, Hasson U (2011) Topographic mapping of a hierarchy of temporal receptive windows using a narrated story. J Neurosci 31(8):2906-2915.
19. Xu J, Kemeny S, Park G, Frattali C, Braun A (2005) Language in context: Emergent features of word, sentence, and narrative comprehension. Neuroimage 25(3):1002-1015.
20. Federmeier KD, Wlotko EW, Meyer AM (2008) What's "right" in language comprehension: ERPs reveal right hemisphere language capabilities. Lang Linguist Compass 2(1):1-17.
21. Friederici AD (2012) The cortical language circuit: From auditory perception to sentence comprehension. Trends Cogn Sci 16(5):262-268.
22. Hagoort P, Levelt WJ (2009) Neuroscience. The speaking brain. Science 326(5951):372-373.
23. Pickering MJ, Garrod S (2004) Toward a mechanistic psychology of dialogue. Behav Brain Sci 27(2):169-190, discussion 190-226.
24. Okada K, Hickok G (2006) Left posterior auditory-related cortices participate both in speech perception and speech production: Neural overlap revealed by fMRI. Brain Lang 98(1):112-117.
25. Wilson SM, Saygin AP, Sereno MI, Iacoboni M (2004) Listening to speech activates motor areas involved in speech production. Nat Neurosci 7(7):701-702.
26. Menenti L, Gierhan SM, Segaert K, Hagoort P (2011) Shared language: Overlap and segregation of the neuronal infrastructure for speaking and listening revealed by functional MRI. Psychol Sci 22(9):1173-1182.
27. Segaert K, Menenti L, Weber K, Petersson KM, Hagoort P (2012) Shared syntax in language production and language comprehension - an FMRI study. Cereb Cortex 22(7):1662-1670.
28. Heim S (2008) Syntactic gender processing in the human brain: A review and a model. Brain Lang 106(1):55-64.
29. Price CJ (2012) A review and synthesis of the first 20 years of PET and fMRI studies of heard speech, spoken language and reading. Neuroimage 62(2):816-847.
30. Indefrey P, Hellwig F, Herzog H, Seitz RJ, Hagoort P (2004) Neural responses to the production and comprehension of syntax in identical utterances. Brain Lang 89(2):312-319.
31. Price CJ (2010) The anatomy of language: A review of 100 fMRI studies published in 2009. Ann N Y Acad Sci 1191:62-88.
32. Awad M, Warren JE, Scott SK, Turkheimer FE, Wise RJ (2007) A common system for the comprehension and production of narrative speech. J Neurosci 27(43):11455-11464.
33. Stephens GJ, Silbert LJ, Hasson U (2010) Speaker-listener neural coupling underlies successful communication. Proc Natl Acad Sci USA 107(32):14425-14430.
34. Long MA, Fee MS (2008) Using temperature to analyse temporal dynamics in the songbird motor pathway. Nature 456(7219):189-194.
35. Gupta L, Molfese DL, Tammana R, Simos PG (1996) Nonlinear alignment and averaging for estimating the evoked potential. IEEE Trans Biomed Eng 43(4):348-356.
36. Baldo JV, Wilkins DP, Ogar J, Willock S, Dronkers NF (2011) Role of the precentral gyrus of the insula in complex articulation. Cortex 47(7):800-807.
37. Kotz SA, Schwartze M, Schmidt-Kassow M (2009) Non-motor basal ganglia functions: A review and proposal for a model of sensory predictability in auditory language perception. Cortex 45(8):982-990.
38. Thompson-Schill SL, D'Esposito M, Aguirre GK, Farah MJ (1997) Role of left inferior prefrontal cortex in retrieval of semantic knowledge: A reevaluation. Proc Natl Acad Sci USA 94(26):14792-14797.
39. Sahin NT, Pinker S, Cash SS, Schomer D, Halgren E (2009) Sequential processing of lexical, grammatical, and phonological information within Broca's area. Science 326(5951):445-449.
40. Friederici AD (2009) Pathways to language: Fiber tracts in the human brain. Trends Cogn Sci 13(4):175-181.
41. Liakakis G, Nickel J, Seitz RJ (2011) Diversity of the inferior frontal gyrus - a meta-analysis of neuroimaging studies. Behav Brain Res 225(1):341-347.
42. Seghier ML (2008) Laterality index in functional MRI: Methodological issues. Magn Reson Imaging 26(5):594-601.
43. Wilke M, Lidzba K (2007) LI-tool: A new toolbox to assess lateralization in functional MR-data. J Neurosci Methods 163(1):128-136.
44. Hasson U, Nir Y, Levy I, Fuhrmann G, Malach R (2004) Intersubject synchronization of cortical activity during natural vision. Science 303(5664):1634-1640.
45. Hasson U, Malach R, Heeger D (2010) Reliability of cortical activity during natural stimulation. Trends Cogn Sci 14(1):40-48.
46. Honey CJ, Thompson CR, Lerner Y, Hasson U (2012) Not lost in translation: Neural responses shared across languages. J Neurosci 32(44):15277-15283.
47. Galantucci B, Fowler CA, Turvey MT (2006) The motor theory of speech perception reviewed. Psychon Bull Rev 13(3):361-377.
48. Liberman AM, Mattingly IG (1985) The motor theory of speech perception revised. Cognition 21(1):1-36.
49. Pickering MJ, Garrod S (2007) Do people use language production to make predictions during comprehension? Trends Cogn Sci 11(3):105-110.
50. Poeppel D, Emmorey K, Hickok G, Pylkkänen L (2012) Towards a new neurobiology of language. J Neurosci 32(41):14125-14131.
51. Zatorre RJ, Belin P, Penhune VB (2002) Structure and function of auditory cortex: Music and speech. Trends Cogn Sci 6(1):37-46.
52. Giraud AL, et al. (2007) Endogenous cortical rhythms determine cerebral specialization for speech perception and production. Neuron 56(6):1127-1134.
53. Ben-Yakov A, Honey CJ, Lerner Y, Hasson U (2012) Loss of reliable temporal structure in event-related averaging of naturalistic stimuli. Neuroimage 63(1):501-506.
54. Geschwind N (1970) The organization of language and the brain. Science 170(3961):940-944.
55. Geschwind N (1979) Anatomical and functional specialization of the cerebral hemispheres in the human. Bull Mem Acad R Med Belg 134(6):286-297.
56. Amunts K, et al. (2010) Broca's region: Novel organizational principles and multiple receptor mapping. PLoS Biol 8(9):e1000489.
57. Shergill SS, et al. (2002) Modulation of activity in temporal cortex during generation of inner speech. Hum Brain Mapp 16(4):219-227.
58. Curtis VA, et al. (1999) Attenuated frontal activation in schizophrenia may be task dependent. Schizophr Res 37(1):35-44.
59. Papoutsi M, et al. (2009) From phonemes to articulatory codes: An fMRI study of the role of Broca's area in speech production. Cereb Cortex 19(9):2156-2165.
60. Moss HE, Tyler LK (1995) Investigating semantic memory impairments: The contribution of semantic priming. Memory 3(3-4):359-395.
61. Xiao Z, et al. (2005) Differential activity in left inferior frontal gyrus for pseudowords and real words: An event-related fMRI study on auditory lexical decision. Hum Brain Mapp 25(2):212-221.
62. Ben-Shachar M, Hendler T, Kahn I, Ben-Bashat D, Grodzinsky Y (2003) The neural reality of syntactic transformations: Evidence from functional magnetic resonance imaging. Psychol Sci 14(5):433-440.
63. Ben-Shachar M, Palti D, Grodzinsky Y (2004) Neural correlates of syntactic movement: Converging evidence from two fMRI experiments. Neuroimage 21(4):1320-1336.
64. Decety J, Chaminade T (2003) Neural correlates of feeling sympathy. Neuropsychologia 41(2):127-138.
65. Hennenlotter A, et al. (2005) A common neural basis for receptive and expressive communication of pleasant facial affect. Neuroimage 26(2):581-591.
66. Shin LM, et al. (2000) Activation of anterior paralimbic structures during guilt-related script-driven imagery. Biol Psychiatry 48(1):43-50.
67. Rämä P, et al. (2001) Working memory of identification of emotional vocal expressions: An fMRI study. Neuroimage 13(6 Pt 1):1090-1101.
68. Honey GD, Bullmore ET, Sharma T (2000) Prolonged reaction time to a verbal working memory task predicts increased power of posterior parietal cortical activation. Neuroimage 12(5):495-503.
69. Rogalsky C, Matchin W, Hickok G (2008) Broca's area, sentence comprehension, and working memory: An fMRI Study. Front Hum Neurosci 2:14.
70. Rogalsky C, Hickok G (2011) The role of Broca's area in sentence comprehension. J Cogn Neurosci 23(7):1664-1680.
71. Hickok G, Poeppel D (2000) Towards a functional neuroanatomy of speech perception. Trends Cogn Sci 4(4):131-138.
72. Bates E, et al. (2003) Voxel-based lesion-symptom mapping. Nat Neurosci 6(5):448-450.
73. Dronkers NF (1996) A new brain region for coordinating speech articulation. Nature 384(6605):159-161.
74. Eimas PD, Miller JL, Jusczyk PW (1987) On infant speech perception and the acquisition of language. Categorical Perception: The Groundwork of Cognition, ed Harnad S (Cambridge Univ Press, Cambridge, UK), pp 161-195.
75. Hickok G, Poeppel D (2004) Dorsal and ventral streams: A framework for understanding aspects of the functional anatomy of language. Cognition 92(1-2):67-99.
76. Pulvermüller F, Fadiga L (2010) Active perception: Sensorimotor circuits as a cortical basis for language. Nat Rev Neurosci 11(5):351-360.
77. Dronkers NF, Wilkins DP, Van Valin RD, Redfern BB, Jaeger JJ (1994) A reconsideration of the brain areas involved in the disruption of morphosyntactic comprehension. Brain Lang 47(3):461-463.
78. Dronkers NF, Redfern BB, Knight RT (2000) The neural architecture of language disorders. The New Cognitive Neurosciences, ed Gazzaniga MS (MIT Press, Cambridge, MA), pp 949-958.
79. Dronkers NF, Wilkins DP, Van Valin RD, Jr, Redfern BB, Jaeger JJ (2004) Lesion analysis of the brain areas involved in language comprehension. Cognition 92(1-2):145-177.
80. Caplan D, Hildebrandt N, Makris N (1996) Location of lesions in stroke patients with deficits in syntactic processing in sentence comprehension. Brain 119(Pt 3):933-949.
81. Caplan D, et al. (2007) A study of syntactic processing in aphasia II: Neurological aspects. Brain Lang 101(2):151-177.
82. Damasio AR, Damasio H (2002) Principles of Behavioral and Cognitive Neurology (Oxford Univ Press, New York).
83. Binder JR, et al. (1997) Human brain language areas identified by functional magnetic resonance imaging. J Neurosci 17(1):353-362.
84. Binder JR, Desai RH, Graves WW, Conant LL (2009) Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. Cereb Cortex 19(12):2767-2796.
85. Tyler LK, Marslen-Wilson W (2008) Fronto-temporal brain systems supporting spoken language comprehension. Philos Trans R Soc Lond B Biol Sci 363(1493):1037-1054.
86. Price CJ (2000) The anatomy of language: Contributions from functional neuroimaging. J Anat 197(Pt 3):335-359.
87. Bookheimer S (2002) Functional MRI of language: New approaches to understanding the cortical organization of semantic processing. Annu Rev Neurosci 25:151-188.
88. Friederici AD (2002) Towards a neural basis of auditory sentence processing. Trends Cogn Sci 6(2):78-84.
89. Vigneau M, et al. (2006) Meta-analyzing left hemisphere language areas: Phonology, semantics, and sentence processing. Neuroimage 30(4):1414-1432.
90. Ferstl EC, Neumann J, Bogler C, von Cramon DY (2008) The extended language network: A meta-analysis of neuroimaging studies on text comprehension. Hum Brain Mapp 29(5):581-593.
91. Peelle JE, Johnsrude IS, Davis MH (2010) Hierarchical processing for speech in human auditory cortex and beyond. Front Hum Neurosci 4:51.
92. Turken AU, Dronkers NF (2011) The neural architecture of the language comprehension network: Converging evidence from lesion and connectivity analyses. Front Syst Neurosci 5:1.
93. McGuire PK, Silbersweig DA, Frith CD (1996) Functional neuroanatomy of verbal self-monitoring. Brain 119(Pt 3):907-917.
94. Hirano S, et al. (1997) Cortical processing mechanism for vocalization with auditory verbal feedback. Neuroreport 8(9-10):2379-2382.
95. Mar RA (2011) The neural bases of social cognition and story comprehension. Annu Rev Psychol 62:103-134.
96. Amodio DM, Frith CD (2006) Meeting of minds: The medial frontal cortex and social cognition. Nat Rev Neurosci 7(4):268-277.
97. Gallagher HL, et al. (2000) Reading the mind in cartoons and stories: An fMRI study of 'theory of mind' in verbal and nonverbal tasks. Neuropsychologia 38(1):11-21.
98. Cavanna AE, Trimble MR (2006) The precuneus: A review of its functional anatomy and behavioural correlates. Brain 129(Pt 3):564-583.
99. Buckner RL, Carroll DC (2007) Self-projection and the brain. Trends Cogn Sci 11(2):49-57.
100. Adolphs R (2009) The social brain: Neural basis of social knowledge. Annu Rev Psychol 60:693-716.
101. Decety J, Sommerville JA (2003) Shared representations between self and other: A social cognitive neuroscience view. Trends Cogn Sci 7(12):527-533.
102. Saxe R (2006) Uniquely human social cognition. Curr Opin Neurobiol 16(2):235-239.
103. Fletcher PC, et al. (1995) Other minds in the brain: A functional imaging study of "theory of mind" in story comprehension. Cognition 57(2):109-128.
104. Friston KJ, Rotshtein P, Geng JJ, Sterzer P, Henson RN (2006) A critique of functional localisers. Neuroimage 30(4):1077-1087.
105. Numminen J, Salmelin R, Hari R (1999) Subject's own speech reduces reactivity of the human auditory cortex. Neurosci Lett 265(2):119-122.
106. Tian X, Poeppel D (2013) The effect of imagination on stimulation: The functional specificity of efference copies in speech processing. J Cogn Neurosci 25(7):1020-1036.
107. Hasson U, Ghazanfar AA, Galantucci B, Garrod S, Keysers C (2012) Brain-to-brain coupling: A mechanism for creating and sharing a social world. Trends Cogn Sci 16(2):114-121.
108. Brainard DH (1997) The Psychophysics Toolbox. Spat Vis 10(4):433-436.
109. Behzadi Y, Restom K, Liau J, Liu TT (2007) A component based noise correction method (CompCor) for BOLD and perfusion based fMRI. Neuroimage 37(1):90-101.
110. Ellis D Dynamic time warp (DTW) in Matlab (Columbia Univ Laboratory for Recognition and Organization of Speech and Audio, New York). Available at http://labrosa.ee.columbia.edu/matlab/dtw/).
111. Turetsky RJ, Ellis DPW Ground-truth transcriptions of real music from force-aligned MIDI syntheses (Columbia Univ, New York). Available at www.ee.columbia.edu/~dpwe/pubs/ismir03-midi.pdf.
112. Zarahn E, Aguirre GK, D'Esposito M (1997) Empirical analyses of BOLD fMRI statistics. I. Spatially unsmoothed data collected under null-hypothesis conditions. Neuroimage 5(3):179-197.
113. Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: A practical and powerful approach to multiple testing. J R Stat Soc, B 57(1):289-300.
114. Benjamini Y, Yekutieli D (2001) The control of the false discovery rate in multiple testing under dependency. Ann Stat 29(4):1165-1188.