The hierarchical cortical organization of human speech processing

Journal of Neuroscience

Volumn 37, Issue 27, 2017, Pages 6539-6557

  de Heer, Wendy A ^a   Huth, Alexander G ^a   Griffiths, Thomas L ^a   Gallant, Jack L ^a   Theunissen, Frédéric E ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

FMRI; Natural stimuli; Regression; Speech

Indexed keywords

ACOUSTIC ANALYSIS; ADULT; ARTICLE; AUDITORY STIMULATION; AUDITORY SYSTEM FUNCTION; BOLD SIGNAL; BROCA AREA; COMPREHENSION; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HEMISPHERE; HUMAN; HUMAN EXPERIMENT; MALE; NORMAL HUMAN; PRIMARY AUDITORY CORTEX; PRIORITY JOURNAL; SEMANTIC MEMORY; SIGNAL DETECTION; SPECTROSCOPY; SPEECH; SPEECH ARTICULATION; STATISTICAL MODEL; TEMPORAL LOBE; BIOLOGICAL MODEL; BRAIN CORTEX; COMPUTER SIMULATION; NERVE CELL NETWORK; NERVE TRACT; PHYSIOLOGY; SPEECH PERCEPTION;

ADULT; CEREBRAL CORTEX; COMPUTER SIMULATION; FEMALE; HUMANS; MALE; MODELS, NEUROLOGICAL; NERVE NET; NEURAL PATHWAYS; SPEECH PERCEPTION;

EID: 85022072305     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.3267-16.2017     Document Type: Article

References (78)

1. Abrams DA, Nicol T, Zecker S, Kraus N (2008) Right-hemisphere auditory cortex is dominant for coding syllable patterns in speech. J Neurosci 28:3958–3965.CrossRef Medline
2. Belin P, Zatorre RJ (2000) “What”, “where” and “how” in auditory cortex. Nat Neurosci 3:965–966.CrossRef Medline
3. Belin P, Zatorre RJ, Hoge R, Evans AC, Pike B (1999) Event-related fMRI of the auditory cortex. Neuroimage 10:417–429.CrossRef Medline
4. Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc Series B Stat Methodol 57:289–300
5. Binder JR, Frost JA, Hammeke TA, Bellgowan PS, Springer JA, Kaufman JN, Possing ET (2000) Human temporal lobe activation by speech and nonspeech sounds. Cereb Cortex 10:512–528.CrossRef Medline
6. Binder JR, Desai RH, Graves WW, Conant LL (2009) Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. Cereb Cortex 19:2767–2796.CrossRef Medline
7. Boemio A, Fromm S, Braun A, Poeppel D (2005) Hierarchical and asymmetric temporal sensitivity in human auditory cortices. Nat Neurosci 8:389–395.CrossRef Medline
8. Bornkessel-Schlesewsky I, Schlesewsky M, Small SL, Rauschecker JP (2015) Neurobiological roots of language in primate audition: common computational principles. Trends Cogn Sci 19:142–150.CrossRef Medline
9. Bouchard KE, Mesgarani N, Johnson K, Chang EF (2013) Functional organization of human sensorimotor cortex for speech articulation. Nature 495:327–332.CrossRef Medline
10. Brant-Zawadzki M, Gillan GD, Nitz WR (1992) MP RAGE - A 3-dimensional, T1-Weighted, Gradient-Echo Sequence—initial experience in the brain. Radiology 182:769–775.CrossRef Medline
11. Chi T, Gao Y, Guyton MC, Ru P, Shamma S (1999) Spectro-temporal modulation transfer functions and speech intelligibility. J Acoust Soc Am 106: 2719–2732.CrossRef Medline
12. C¸ ukur T, Nishimoto S, Huth AG, Gallant JL (2013) Attention during natural vision warps semantic representation across the human brain. Nat Neurosci 16:763–770.CrossRef Medline
13. Dale AM, Fischl B, Sereno MI (1999) Cortical surface-based analysis - I. Segmentation and surface reconstruction. Neuroimage 9:179–194.CrossRef Medline
14. David SV, Gallant JL (2005) Predicting neuronal responses during natural vision. Network 16:239–260.CrossRef Medline
15. Deerwester S, Dumais ST, Furnas GW, Landauer TK, Harshman R (1990) Indexing by latent semantic analysis. J Am Soc Inf Sci 41:391–407. CrossRef
16. Depireux DA, Simon JZ, Klein DJ, Shamma SA (2001) Spectro-temporal response field characterization with dynamic ripples in ferret primary auditory cortex. J Neurophysiol 85:1220–1234. Medline
17. Desai R, Liebenthal E, Waldron E, Binder JR (2008) Left posterior temporal regions are sensitive to auditory categorization. J Cogn Neurosci 20: 1174–1188.CrossRef Medline
18. DeWitt I, Rauschecker JP (2012) Phoneme and word recognition in the auditory ventral stream. Proc Natl Acad Sci U S A 109:E505–E514.CrossRef Medline
19. Di Liberto GM, O’Sullivan JA, Lalor EC (2015) Low-frequency cortical entrainment to speech reflects phoneme-level processing. Curr Biol 25: 2457–2465.CrossRef Medline
20. Elliott TM, Theunissen FE (2009) The modulation transfer function for speech intelligibility. PLoS Comput Biol 5:e1000302.CrossRef Medline
21. Escabi MA, Schreiner CE (2002) Nonlinear spectrotemporal sound analysis by neurons in the auditory midbrain. J Neurosci 22:4114–4131. Medline
22. Fedorenko E, Nieto-Castañón A, Kanwisher N (2012) Syntactic processing in the human brain: what we know, what we don’t know, and a suggestion for how to proceed. Brain Lang 120:187–207.CrossRef Medline
23. Flinker A, Korzeniewska A, Shestyuk AY, Franaszczuk PJ, Dronkers NF, Knight RT, Crone NE (2015) Redefining the role of Broca’s area in speech. Proc Natl Acad Sci U S A 112:2871–2875.CrossRef Medline
24. Fritz JB, Elhilali M, David SV, Shamma SA (2007) Does attention play a role in dynamic receptive field adaptation to changing acoustic salience in Al? Hear Res 229:186–203.CrossRef Medline
25. Gill P, Zhang J, Woolley SM, Fremouw T, Theunissen FE (2006) Sound representation methods for spectro-temporal receptive field estimation. J Comput Neurosci 21:5–20.CrossRef Medline
26. Holdgraf CR, de Heer W, Pasley B, Rieger J, Crone N, Lin JJ, Knight RT, Theunissen FE (2016) Rapid tuning shifts in human auditory cortex enhance speech intelligibility. Nat Commun 7:13654.CrossRef Medline
27. Hsu A, Borst A, Theunissen FE (2004) Quantifying variability in neural responses and its application for the validation of model predictions. Network 15:91–109.CrossRef Medline
28. Hullett PW, Hamilton LS, Mesgarani N, Schreiner CE, Chang EF (2016) Human superior temporal gyrus organization of spectrotemporal modulation tuning derived from speech stimuli. J Neurosci 36:2014–2026.CrossRef Medline
29. Huth AG, Nishimoto S, Vu AT, Gallant JL (2012) A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron 76:1210–1224.CrossRef Medline
30. Huth AG, de Heer WA, Griffiths TL, Theunissen FE, Gallant JL (2016) Natural speech reveals the semantic maps that tile human cerebral cortex. Nature 532:453–458.CrossRef Medline
31. Jenkinson M, Smith S (2001) A global optimisation method for robust affine registration of brain images. Med Image Anal 5:143–156.CrossRef Medline
32. Jones E, Oliphant T, Peterson P (2007) SciPy: open source scientific tools for Python. Beaverton, OR: Python Software Foundation. Available at: http://www.scipy.org/.
33. Just MA, Cherkassky VL, Aryal S, Mitchell TM (2010) A neurosemantic theory of concrete noun representation based on the underlying brain codes. PLoS One 5:e8622.CrossRef Medline
34. Knecht S, Flöel A, Dräger B, Breitenstein C, Sommer J, Henningsen H, Ringelstein EB, Pascual-Leone A (2002) Degree of language lateralization determines susceptibility to unilateral brain lesions. Nat Neurosci 5:695–699.CrossRef Medline
35. Leaver AM, Rauschecker JP (2010) Cortical representation of natural complex sounds: effects of acoustic features and auditory object category. J Neurosci 30:7604–7612.CrossRef Medline
36. Lerner Y, Honey CJ, Silbert LJ, Hasson U (2011) Topographic mapping of a hierarchy of temporal receptive windows using a narrated story. J Neurosci 31:2906–2915.CrossRef Medline
37. Lescroart MD, Stansbury DE, Gallant JL (2015) Fourier power, subjective distance, and object categories all provide plausible models of BOLD responses in scene-selective visual areas. Front Comput Neurosci 9:135.CrossRef Medline
38. Levelt WJ (1993) Speaking: from intention to articulation. Cambridge, MA: MIT.
39. Lund K, Burgess C (1996) Hyperspace analogue to language (HAL): a general model semantic representation. Brain Cogn 30:5.
40. Lyon RF (1982) A computational model of filtering, detection and compression in the cochlea. Paper presented at IEEE International Conference on Acoustics, Speech, and Signal Processing. Paris, France, May.
41. Mesgarani N, Chang EF (2012) Selective cortical representation of attended speaker in multi-talker speech perception. Nature 485:233–236.CrossRef Medline
42. Mesgarani N, Slaney M, Shamma SA (2006) Discrimination of speech from nonspeech based on multiscale spectro-temporal modulations. IEEE Trans Audio Speech Lang Process 14:920–930. CrossRef
43. Mesgarani N, Cheung C, Johnson K, Chang EF (2014) Phonetic feature encoding in human superior temporal gyrus. Science 343:1006–1010.CrossRef Medline
44. Miller LM, Escabí MA, Read HL, Schreiner CE (2002) Spectrotemporal receptive fields in the lemniscal auditory thalamus and cortex. J Neurophysiol 87:516–527.CrossRef Medline
45. Mitchell TM, Shinkareva SV, Carlson A, Chang KM, Malave VL, Mason RA, Just MA (2008) Predicting human brain activity associated with the meanings of nouns. Science 320:1191–1195.CrossRef Medline
46. Möttönen R, Dutton R, Watkins KE (2013) Auditory-Motor Processing of Speech Sounds. Cereb Cortex 23:1190–1197.CrossRef Medline
47. Moore RC, Lee T, Theunissen FE (2013) Noise-invariant neurons in the avian auditory cortex: hearing the song in noise. PLoS Comput Biol 9:e1002942.CrossRef Medline
48. Nishimoto S, Gallant JL (2011) A three-dimensional spatiotemporal receptive field model explains responses of area MT neurons to naturalistic movies. J Neurosci 31:14551–14564.CrossRef Medline
49. Nishimoto S, Vu AT, Naselaris T, Benjamini Y, Yu B, Gallant JL (2011) Reconstructing visual experiences from brain activity evoked by natural movies. Curr Biol 21:1641–1646.CrossRef Medline
50. Norman-Haignere S, Kanwisher NG, McDermott JH (2015) Distinct cortical pathways for music and speech revealed by hypothesis-free voxel decomposition. Neuron 88:1281–1296.CrossRef Medline
51. Oliphant TE (2006) Aguide to NumPy. Austin, TX: Continuum Publishing.
52. Overath T, McDermott JH, Zarate JM, Poeppel D (2015) The cortical analysis of speech-specific temporal structure revealed by responses to sound quilts. Nat Neurosci 18:903–911.CrossRef Medline
53. Peelle JE, Gross J, Davis MH (2013) Phase-locked responses to speech in human auditory cortex are enhanced during comprehension. Cereb Cortex 23:1378–1387.CrossRef Medline
54. Poeppel D, Emmorey K, Hickok G, Pylkkänen L (2012) Towards a new neurobiology of language. J Neurosci 32:14125–14131.CrossRef Medline
55. Price CJ (2010) The anatomy of language: a review of 100 fMRI studies published in 2009. Ann N Y Acad Sci 1191:62–88.CrossRef Medline
56. Pujol J, Deus J, Losilla JM, Capdevila A (1999) Cerebral lateralization of language in normal left-handed people studied by functional MRI. Neurology 52:1038–1043.CrossRef Medline
57. Rauschecker JP, Scott SK (2009) Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing. Nat Neurosci 12:718–724.CrossRef Medline
58. Rauschecker JP, Tian B, Hauser M (1995) Processing of complex sounds in the macaque nonprimary auditory cortex. Science 268:111–114.CrossRef Medline
59. Rodd JM, Davis MH, Johnsrude IS (2005) The neural mechanisms of speech comprehension: fMRI studies of semantic ambiguity. Cereb Cortex 15: 1261–1269. Medline
60. Sahani M, Linden J (2003) Evidence optimization techniques for estimating stimulus-response functions. In: Advances in neural infomation processing systems (Becker S, Thrun S, Obermeyer K, eds), pp 301–308. Cambridge, MA: MIT.
61. Santoro R, Moerel M, De Martino F, Goebel R, Ugurbil K, Yacoub E, Formisano E (2014) Encoding of natural sounds at multiple spectral and temporal resolutions in the human auditory cortex. PLoS Comput Biol 10: e1003412.CrossRef Medline
62. Scott SK, Johnsrude IS (2003) The neuroanatomical and functional organization of speech perception. Trends Neurosci 26:100–107.CrossRef Medline
63. Sen K, Theunissen FE, Doupe AJ (2001) Feature analysis of natural sounds in the songbird auditory forebrain. J Neurophysiol 86:1445–1458. Medline
64. Singh NC, Theunissen FE (2003) Modulation spectra of natural sounds and ethological theories of auditory processing. J Acoust Soc Am 114:3394–3411.CrossRef Medline
65. Slaney M (1998) Lyon’s cochlear model. Interval Research Corproation Technical Report #1998-010
66. Stowe LA, Haverkort M, Zwarts F (2005) Rethinking the neurological basis of language. Lingua 115:997–1042. CrossRef
67. Suga N, O’Neill WE, Manabe T (1978) Cortical neurons sensitive to combinations of information-bearing elements of biosonar signals in the moustache bat. Science 200:778–781.CrossRef Medline
68. Tian B, Reser D, Durham A, Kustov A, Rauschecker JP (2001) Functional specialization in rhesus monkey auditory cortex. Science 292:290–293.CrossRef Medline
69. Turkeltaub PE, Coslett HB (2010) Localization of sublexical speech perception components. Brain Lang 114:1–15.CrossRef Medline
70. Turney PD, Pantel P (2010) From frequency to meaning: vector space models of semantics. J Artif Intell Res 37:141–188. CrossRef
71. Visser M, Jefferies E, Lambon Ralph MA (2010) Semantic processing in the anterior temporal lobes: a meta-analysis of the functional neuroimaging literature. J Cogn Neurosci 22:1083–1094.CrossRef Medline
72. Wehbe L, Murphy B, Talukdar P, Fyshe A, Ramdas A, Mitchell T (2014) Simultaneously uncovering the patterns of brain regions involved in different story reading subprocesses. PLoS One 9:e112575.CrossRef Medline
73. Welch P (1967) The use of fast Fourier transform for the estimation of power spectra: a method based on time averaging over short, modified periodograms. IEEE Trans Audio Electroacoust 15:70–73. CrossRef
74. Wild CJ, Davis MH, Johnsrude IS (2012) Humanauditory cortex is sensitive to the perceived clarity of speech. Neuroimage 60:1490–1502.CrossRef Medline
75. Wilson SM, Saygin AP, Sereno MI, Iacoboni M (2004) Listening to speech activates motor areas involved in speech production. Nat Neurosci 7:701–702.CrossRef Medline
76. Woolley SM, Gill PR, Fremouw T, Theunissen FE (2009) Functional groups in the avian auditory system. J Neurosci 29:2780–2793.CrossRef Medline
77. Zhang L, Shu H, Zhou F, Wang X, Li P (2010) Common and distinct neural substrates for the perception of speech rhythm and intonation. Hum Brain Mapp 31:1106–1116.CrossRef Medline
78. Zion Golumbic EM, Ding N, Bickel S, Lakatos P, Schevon CA, McKhann GM, Goodman RR, Emerson R, Mehta AD, Simon JZ, Poeppel D, Schroeder CE (2013) Mechanisms underlying selective neuronal tracking of attended speech at a “cocktail party”. Neuron 77:980–991.CrossRef Medline