Dynamic encoding of acoustic features in neural responses to continuous speech

Journal of Neuroscience

Volumn 37, Issue 8, 2017, Pages 2176-2185

  Khalighinejad, Bahar ^a   da Silva, Guilherme Cruzatto ^a   Mesgarani, Nima ^a  

^a Department of Earth and Environmental Sciences   (United States)

Author keywords

EEG; Event related potential; Phonemes; Speech

Indexed keywords

ARTICLE; AUDITORY EVOKED POTENTIAL; AUDITORY SYSTEM; COCHLEAR NERVE; ELECTROENCEPHALOGRAPHY; FEMALE; HUMAN; HUMAN EXPERIMENT; LATENT PERIOD; MALE; NERVE POTENTIAL; PHONETICS; PRIORITY JOURNAL; SPEECH ANALYSIS; ACOUSTICS; AUDITORY STIMULATION; BRAIN MAPPING; LANGUAGE; PHYSIOLOGY; REACTION TIME; SPEECH; SPEECH PERCEPTION; STATISTICS; TIME FACTOR;

ACOUSTIC STIMULATION; ACOUSTICS; BRAIN MAPPING; ELECTROENCEPHALOGRAPHY; EVOKED POTENTIALS, AUDITORY; FEMALE; HUMANS; LANGUAGE; MALE; PHONETICS; REACTION TIME; SPEECH; SPEECH PERCEPTION; STATISTICS AS TOPIC; TIME FACTORS;

EID: 85013668908     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.2383-16.2017     Document Type: Article

References (76)

1. Aiken SJ, Picton TW (2008) Human cortical responses to the speech envelope. Ear Hear 29:139–157. CrossRef Medline
2. Allen JB (1994) How do humans process and recognize speech? IEEE Trans Speech Audiol Proc 2:567–577. CrossRef
3. Benjamini Y, Hochberg Y (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc B 85: 289–300.
4. Benjamini Y, Yekutieli D (2001) The control of the false discovery rate in multiple testing under dependency. Ann Stat 29:1165–1188. CrossRef
5. Borg I, Groenen PJF (2005) Modern multidimensional scaling: theory and applications. New York: Springer.
6. Ceponiene R, Rinne T, Näätänen R (2002) Maturation of cortical sound processing as indexed by event-related potentials. Clin Neurophysiol 113: 870–882. CrossRef Medline
7. Chan AM, Dykstra AR, Jayaram V, Leonard MK, Travis KE, Gygi B, Baker JM, Eskandar E, Hochberg LR, Halgren E, Cash SS (2014) Speech-specific tuning of neurons in human superior temporal gyrus. Cereb Cortex 24: 2679–2693. CrossRef Medline
8. Chi T, Ru P, Shamma SA (2005) Multiresolution spectrotemporal analysis of complex sounds. J Acoust Soc Am 118:887–906. CrossRef Medline
9. Cichy RM, Pantazis D, Oliva A (2014) Resolving human object recognition in space and time. Nat Neurosci 17:455–462. CrossRef Medline
10. Crowley KE, Colrain IM (2004) A review of the evidence for P2 being an independent component process: age, sleep and modality. Clin Neurophysiol 115:732–744. CrossRef Medline
11. Dehaene-Lambertz G, Dehaene S, Hertz-Pannier L (2002) Functional neuroimaging of speech perception in infants. Science 298:2013–2015. CrossRef Medline
12. Delorme A, Makeig S (2004) EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods 134:9–21. CrossRef Medline
13. Deza MM, Deza E (2009) Encyclopedia of distances. New York: Springer.
14. Di Liberto GM, O’Sullivan JA, Lalor EC (2015) Low-frequency cortical en-trainment to speech reflects phoneme-level processing. Curr Biol 25: 2457–2465. CrossRef Medline
15. Formisano E, De Martino F, Bonte M, Goebel R (2008) “Who” is saying “what”? Brain-based decoding of human voice and speech. Science 322: 970–973. CrossRef Medline
16. Friederici AD, Pfeifer E, Hahne A (1993) Event-related brain potentials during natural speech processing: effects of semantic, morphological and syntactic violations. Brain Res Cogn Brain Res 1:183–192. CrossRef Medline
17. Halle M, Stevens K (1991) Knowledge of language and the sounds of speech. In: Wenner-Gren Center International Symposium Series, pp 1–19. New York: Springer.
18. Hickok G, Poeppel D (2007) The cortical organization of speech processing. Nat Rev Neurosci 8:393–402. CrossRef Medline
19. Hillyard SA, Squires KC, Bauer JW, Lindsay PH (1971) Evoked potential correlates of auditory signal detection. Science 172:1357–1360. CrossRef Medline
20. Honey CJ, Thesen T, Donner TH, Silbert LJ, Carlson CE, Devinsky O, Doyle WK, Rubin N, Heeger DJ, Hasson U (2012) Slow cortical dynamics and the accumulation of information over long timescales. Neuron 76:423– 434. CrossRef Medline
21. Jain AK, Dubes RC (1988) Algorithms for clustering data. Upper Saddle River, NJ: Prentice-Hall.
22. Johnson K (2008) Speaker Normalization in Speech Perception. In: Handbook for Speech Perception (Pisoni D, Remez R, eds). Hoboken, NJ: Wiley. CrossRef
23. Kaan E (2007) Event-related potentials and language processing: a brief overview. Lang Linguist Compass 1:571–591. CrossRef
24. Kaan E, Harris A, Gibson E, Holcomb P (2000) The P600 as an index of syntactic integration difficulty. Lang Cogn Process 15:159–201. CrossRef
25. Kerlin JR, Shahin AJ, Miller LM (2010) Attentional gain control of ongoing cortical speech representations in a “cocktail party”. J Neurosci 30:620– 628. CrossRef Medline
26. Kolk H (1998) Disorders of syntax in aphasia: linguistic-descriptive and processing approaches. In: Handbook of Neurolinguistics, pp 249–260. London: Academic.
27. Kong YY, Somarowthu A, Ding N (2015) Effects of spectral degradation on attentional modulation of cortical auditory responses to continuous speech. J Assoc Res Otolaryngol 16:783–796. CrossRef Medline
28. Kriegeskorte N, Kievit RA (2013) Representational geometry: integrating cognition, computation, and the brain. Trends Cogn Sci 17:401–412. CrossRef Medline
29. Kutas M, Federmeier KD (2011) Thirty years and counting: finding meaning in the N400 component of the event related brain potential (ERP). Annu Rev Psychol 62:621–647. CrossRef Medline
30. Ladefoged P, Johnson K (2010) A course in phonetics, sixth edition. Boston: Wadsworth.
31. Lalor EC, Power AJ, Reilly RB, Foxe JJ (2009) Resolving precise temporal processing properties of the auditory system using continuous stimuli. J Neurophysiol 102:349–359. CrossRef Medline
32. Liégeois-Chauvel C, Musolino A, Badier JM, Marquis P, Chauvel P (1994) Evoked potentials recorded from the auditory cortex in man: evaluation and topography of the middle latency components. Electroencephalogr Clin Neurophysiol 92:204–214. CrossRef Medline
33. Logan JS, Lively SE, Pisoni DB (1991) Training Japanese listeners to identify English/r/and/l: a first report. J Acoust Soc Am 89:874–886. CrossRef Medline
34. Luck SJ (1995) Multiple mechanisms of visual-spatial attention: recent evidence from human electrophysiology. Behav Brain Res 71:113–123. CrossRef Medline
35. Luck SJ (2014) An introduction to the event-related potential technique. Cambridge, MA: MIT.
36. Luo H, Poeppel D (2007) Phase patterns of neuronal responses reliably dis- criminate speech in human auditory cortex. Neuron 54:1001–1010. CrossRef Medline
37. Martin BA, Tremblay KL, Korczak P (2008) Speech evoked potentials: from the laboratory to the clinic. Ear Hear 29:285–313. CrossRef Medline
38. McCallum WC, Curry SH (1980) The form and distribution of auditory evoked potentials and CNVs when stimuli and responses are lateralized. Prog Brain Res 54:767–775. CrossRef Medline
39. Mesgarani N, Chang EF (2012) Selective cortical representation of attended speaker in multi-talker speech perception. Nature 485:233–236. CrossRef Medline
40. Mesgarani N, David SV, Fritz JB, Shamma SA (2008) Phoneme representation and classification in primary auditory cortex. J Acoust Soc Am 123: 899–909. CrossRef Medline
41. Mesgarani N, Cheung C, Johnson K, Chang EF (2014) Phonetic feature encoding in human superior temporal gyrus. Science 343:1006–1010. CrossRef Medline
42. Miller GA, Nicely PE (1955) An analysis of perceptual confusions among some English consonants. J Acoust Soc Am 27:338–352. CrossRef
43. Näätänen R (2001) The perception of speech sounds by the human brain as reflected by the mismatch negativity (MMN) and its magnetic equivalent (MMNm). Psychophysiology 38:1–21. CrossRef Medline
44. Näätänen R, Picton T (1987) The N1 wave of the human electric and magnetic response to sound: a review and an analysis of the component structure. Psychophysiology 24:375–425. CrossRef Medline
45. Ojima S, Nakata H, Kakigi R (2005) An ERP study of second language learning after childhood: effects of proficiency. J Cogn Neurosci 17:1212–1228. CrossRef Medline
46. Osterhout L, McLaughlin J, Bersick M (1997) Event-related brain potentials and human language. Trends Cogn Sci 1:203–209. CrossRef Medline
47. Overath T, McDermott JH, Zarate JM, Poeppel D (2015) The cortical analysis of speech-specific temporal structure revealed by responses to sound quilts. Nat Neurosci 18:903–911. CrossRef Medline
48. Pang EW, Taylor MJ (2000) Tracking the development of the N1 from age 3 to adulthood: an examination of speech and non-speech stimuli. Clin Neurophysiol 111:388–397. CrossRef Medline
49. Patel AD (2003) Language, music, syntax and the brain. Nat Neurosci 6:674–681.CrossRef Medline
50. Patel JK, Kapadia CH, Owen DB (1976) Handbook of statistical distributions. New York: Marcel Dekker.
51. Phillips C, Pellathy T, Marantz A, Yellin E, Wexler K, Poeppel D, McGinnis M, Roberts T (2000) Auditory cortex accesses phonological categories: an MEG mismatch study. J Cogn Neurosci 12:1038–1055. CrossRef Medline
52. Picton TW, Hillyard SA (1974) Human auditory evoked potentials. II: effects of attention. Electroencephalogr Clin Neurophysiol 36:191–199. CrossRef Medline
53. Picton TW, Woods DL, Baribeau-Braun J, Healey TM (1976) Evoked potential audiometry. J Otolaryngol 6:90–119. Medline
54. Picton TW, Bentin S, Berg P, Donchin E, Hillyard SA, Johnson R Jr, Miller GA, Ritter W, Ruchkin DS, Rugg MD, Taylor MJ (2000) Guidelines for using human event-related potentials to study cognition: recording standards and publication criteria. Psychophysiology 37:127–152. CrossRef Medline
55. Poeppel D (2014) The neuroanatomic and neurophysiological infrastructure for speech and language. Curr Opin Neurobiol 28:142–149. CrossRef Medline
56. Polka L, Werker JF (1994) Developmental changes in perception of nonna-tive vowel contrasts. J Exp Psychol Hum Percept Perform 20:421–435. CrossRef Medline
57. Ponton C, Eggermont JJ, Khosla D, Kwong B, Don M (2002) Maturation of human central auditory system activity: separating auditory evoked potentials by dipole source modeling. Clin Neurophysiol 113:407–420.CrossRef Medline
58. Rahne T, Böckmann M, von Specht H, Sussman ES (2007) Visual cues can modulate integration and segregation of objects in auditory scene analysis. Brain Res 1144:127–135. CrossRef Medline
59. Rifkin R, Yeo G, Poggio T (2003) Regularized least-squares classification. Nato Sci Ser Sub Ser III Comput Syst Sci 190:131–154.
60. Rossi S, Gugler MF, Friederici AD, Hahne A (2006) The impact of proficiency on syntactic second-language processing of German and Italian: evidence from event-related potentials. J Cogn Neurosci 18:2030–2048. CrossRef Medline
61. Rousseeuw PJ (1987) Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math 20:53–65. CrossRef
62. Sanders LD, Neville HJ (2003) An ERP study of continuous speech processing: I. Segmentation, semantics, and syntax in native speakers. Brain Res Cogn Brain Res 15:228–240. CrossRef Medline
63. Schoffelen JM, Gross J (2009) Source connectivity analysis with MEG and EEG. Hum Brain Mapp 30:1857–1865. CrossRef Medline
64. Steinschneider M, Liégeois-Chauvel C, Brugge JF (2011a) Auditory evoked potentials and their utility in the assessment of complex sound processing. In: The auditory cortex, pp 535–559. New York: Springer.
65. Steinschneider M, Nourski KV, Kawasaki H, Oya H, Brugge JF, Howard MA 3rd (2011b) Intracranial study of speech-elicited activity on the human posterolateral superior temporal gyrus. Cereb Cortex 21:2332–2347. CrossRef Medline
66. Stevens KN (2000) Acoustic phonetics. Cambridge, MA: MIT.
67. Swaab TY, Brown C, Hagoort P (1998) Understanding ambiguous words in sentence contexts: electrophysiological evidence for delayed contextual selection in Broca’s aphasia. Neuropsychologia 36:737–761. CrossRef Medline
68. Syrdal AK, Gopal HS (1986) A perceptual model of vowel recognition based on the auditory representation of American English vowels. J Acoust Soc Am 79:1086–1100. CrossRef Medline
69. ter Keurs M, Brown CM, Hagoort P (2002) Lexical processing of vocabulary class in patients with Broca’s aphasia: an event-related brain potential study on agrammatic comprehension. Neuropsychologia 40:1547–1561. CrossRef Medline
70. Thierry G, Martin CD, Downing P, Pegna AJ (2007) Controlling for inter-stimulus perceptual variance abolishes N170 face selectivity. Nat Neurosci 10:505–511. Medline
71. Tremblay KL, Friesen L, Martin BA, Wright R (2003) Test-retest reliability of cortical evoked potentials using naturally produced speech sounds. Ear Hear 24:225–232. Medline
72. Vitevitch MS, Luce PA (1999) Probabilistic phonotactics and neighborhood activation in spoken word recognition. J Mem Lang 40:374–408. CrossRef Medline
73. Wang K, Shamma S (1994) Self-normalization and noise-robustness in early auditory representations. IEEE Trans Speech Audiol Proc 2:421– 435. CrossRef
74. Wunderlich JL, Cone-Wesson BK (2006) Maturation of CAEP in infants and children: a review. Hear Res 212:212–223. CrossRef Medline
75. Yang X, Wang K, Shamma SA (1992) Auditory representations of acoustic signals. IEEE Trans Inf Theory 38:824–839. CrossRef
76. Yuan J, Liberman M (2008) Speaker identification on the SCOTUS corpus. J Acoust Soc Am 123:3878. CrossRef