Reconstructing speech from human auditory cortex

PLoS Biology

Volumn 10, Issue 1, 2012, Pages

  Pasley, Brian N ^a   David, Stephen V ^b   Mesgarani, Nima ^b,c   Flinker, Adeen ^a   Shamma, Shihab A ^b   Crone, Nathan E ^d   Knight, Robert T ^a,c   Chang, Edward F ^c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF MARYLAND   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

^d JOHNS HOPKINS UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACCURACY; ACOUSTIC STRESS; ARTICLE; AUDITORY CORTEX; AUDITORY STIMULATION; BRAIN FUNCTION; BRAIN TUMOR; CLINICAL ARTICLE; CONTROLLED STUDY; ELECTROCORTICOGRAPHY; EPILEPSY; HUMAN; NERVE FUNCTION; NEUROSURGERY; SPEECH ANALYSIS; SPEECH DISCRIMINATION; STATISTICAL MODEL; SUPERIOR TEMPORAL GYRUS; ALGORITHM; BIOLOGICAL MODEL; BRAIN MAPPING; COMPUTER SIMULATION; ELECTRODE; ELECTROENCEPHALOGRAPHY; FEMALE; MALE; PHYSIOLOGY; SPEECH;

ALGORITHMS; AUDITORY CORTEX; BRAIN MAPPING; COMPUTER SIMULATION; ELECTRODES, IMPLANTED; ELECTROENCEPHALOGRAPHY; FEMALE; HUMANS; LINEAR MODELS; MALE; MODELS, BIOLOGICAL; SPEECH ACOUSTICS;

EID: 84856467482     PISSN: 15449173     EISSN: 15457885     Source Type: Journal    
DOI: 10.1371/journal.pbio.1001251     Document Type: Article

References (57)

1. Young E. D, (2008) Neural representation of spectral and temporal information in speech. Philos Trans R Soc Lond B Biol Sci 363: 923-945.
2. Joris P. X, Schreiner C. E, Rees A, (2004) Neural processing of amplitude-modulated sounds. Physiol Rev 84: 541-577.
3. Schreiner C. E, Froemke R. C, Atencio C. A, (2011) Spectral processing in auditory cortex. In: Winer J. A, Schreiner C. E, editors. The auditory cortex Springer US pp. 275-308.
4. Hickok G, Poeppel D, (2007) The cortical organization of speech processing. Nat Rev Neurosci 8: 393-402.
5. Rauschecker J. P, Scott S. K, (2009) Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing. Nat Neurosci 12: 718-724.
6. Eggermont J. J, Wang X, (2011) Temporal coding in auditory cortex. In: Winer J. A, Schreiner C. E, editors. The auditory cortex Springer US pp. 275-308.
7. Shamma S. A, (2003) Physiological foundations of temporal integration in the perception of speech. Journal of Phonetics 31: 279-287.
8. Geschwind N, (1965) Disconnexion syndromes in animals and man: Part I. Brain 88: 237-294.
9. Recanzone G. H, Cohen Y. E, (2010) Serial and parallel processing in the primate auditory cortex revisited. Behavioural Brain Research 206: 1-7.
10. Steinschneider M, (2011) Unlocking the role of the superior temporal gyrus for speech sound categorization. Journal of Neurophysiology 105: 2631-2633.
11. Romanski L. M, Averbeck B. B, (2009) The primate cortical auditory system and neural representation of conspecific vocalizations. Annu Rev Neurosci 32: 315-346.
12. Bialek W, Rieke F, de Ruyter van Steveninck R. R, Warland D, (1991) Reading a neural code. Science 252: 1854-1857.
13. Stanley G. B, Li F. F, Dan Y, (1999) Reconstruction of natural scenes from ensemble responses in the lateral geniculate nucleus. J Neurosci 19: 8036-8042.
14. Mesgarani N, David S. V, Fritz J. B, Shamma S. A, (2009) Influence of context and behavior on stimulus reconstruction from neural activity in primary auditory cortex. J Neurophysiol 102: 3329-3339.
15. Ramirez A. D, Ahmadian Y, Schumacher J, Schneider D, Woolley S. M, et al. (2011) Incorporating naturalistic correlation structure improves spectrogram reconstruction from neuronal activity in the songbird auditory midbrain. J Neurosci 31: 3828-3842.
16. Wu M. C, David S. V, Gallant J. L, (2006) Complete functional characterization of sensory neurons by system identification. Annu Rev Neurosci 29: 477-505.
17. Hastie T, Tibshirani R, Friedman J, (2009) Elements of statistical learning New York, NY Springer Science.
18. Chi T, Ru P, Shamma S. A, (2005) Multiresolution spectrotemporal analysis of complex sounds. J Acoust Soc Am 118: 887-906.
19. Chi T, Gao Y, Guyton M. C, Ru P, Shamma S, (1999) Spectro-temporal modulation transfer functions and speech intelligibility. J Acoust Soc Am 106: 2719-2732.
20. Singh N. C, Theunissen F. E, (2003) Modulation spectra of natural sounds and ethological theories of auditory processing. J Acoust Soc Am 114: 3394-3411.
21. Elliott T. M, Theunissen F. E, (2009) The modulation transfer function for speech intelligibility. PLoS Comput Biol 5: e1000302 doi:10.1371/journal.pcbi.1000302.
22. Dau T, Kollmeier B, Kohlrausch A, (1997) Modeling auditory processing of amplitude modulation. I. Detection and masking with narrow-band carriers. J Acoust Soc Am 102: 2892-2905.
23. Evans A. C, Collins D. L, Mills S. R, Brown E. D, Kelly R. L, (1993) pp. 1813-1817.
24. Crone N. E, Boatman D, Gordon B, Hao L, (2001) Induced electrocorticographic gamma activity during auditory perception. Brazier Award-winning article, 2001. Clin Neurophysiol 112: 565-582.
25. Canolty R. T, Soltani M, Dalal S. S, Edwards E, Dronkers N. F, et al. (2007) Spatiotemporal dynamics of word processing in the human brain. Front Neurosci 1: 185-196.
26. Nourski K. V, Reale R. A, Oya H, Kawasaki H, Kovach C. K, et al. (2009) Temporal envelope of time-compressed speech represented in the human auditory cortex. J Neurosci 29: 15564-15574.
27. Flinker A, Chang E. F, Kirsch H. E, Barbaro N. M, Crone N. E, et al. (2010) Single-trial speech suppression of auditory cortex activity in humans. The Journal of Neuroscience 30: 16643-16650.
28. Edwards E, Nagarajan S. S, Dalal S. S, Canolty R. T, Kirsch H. E, et al. (2010) Spatiotemporal imaging of cortical activation during verb generation and picture naming. Neuroimage 50: 291-301.
29. Pei X, Leuthardt E. C, Gaona C. M, Brunner P, Wolpaw J. R, et al. (2011) Spatiotemporal dynamics of electrocorticographic high gamma activity during overt and covert word repetition. Neuroimage 54: 2960-2972.
30. Logothetis N. K, Wandell B. A, (2004) Interpreting the BOLD signal. Annu Rev Physiol 66: 735-769.
31. Liegeois-Chauvel C, Lorenzi C, Trebuchon A, Regis J, Chauvel P, (2004) Temporal envelope processing in the human left and right auditory cortices. Cereb Cortex 14: 731-740.
32. Adelson E. H, Bergen J. R, (1985) Spatiotemporal energy models for the perception of motion. J Opt Soc Am A 2: 284-299.
33. Kouh M, Poggio T, (2008) A canonical neural circuit for cortical nonlinear operations. Neural Computation 20: 1427-1451.
34. Hubel D. H, Wiesel T. N, (1962) Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J Physiol 160: 106-154.
35. David S. V, Vinje W. E, Gallant J. L, (2004) Natural stimulus statistics alter the receptive field structure of v1 neurons. J Neurosci 24: 6991-7006.
36. Willmore B. D, Prenger R. J, Gallant J. L, (2010) Neural representation of natural images in visual area V2. J Neurosci 30: 2102-2114.
37. Wang X, Lu T, Bendor D, Bartlett E, (2008) Neural coding of temporal information in auditory thalamus and cortex. Neuroscience 154: 294-303.
38. Lu T, Liang L, Wang X, (2001) Temporal and rate representations of time-varying signals in the auditory cortex of awake primates. Nat Neurosci 4: 1131-1138.
39. Bendor D, Wang X, (2007) Differential neural coding of acoustic flutter within primate auditory cortex. Nat Neurosci 10: 763-771.
40. Rabiner L, Juang B. H, (1993) Fundamentals of speech recognition Englewood Cliffs, NJ Prentice-Hall, Inc.
41. Rieke F, Bodnar D. A, Bialek W, (1995) Naturalistic stimuli increase the rate and efficiency of information transmission by primary auditory afferents. Proc Biol Sci 262: 259-265.
42. Nourski K. V, Brugge J. F, (2011) Representation of temporal sound features in the human auditory cortex. Reviews in the Neurosciences 22: 187-203.
43. Greenberg S, (2006) A multi-tier theoretical framework for understanding spoken language. In: Listening to speech. In: Greenberg S, Ainsworth W. A, editors. Listening to speech: an auditory perspective Mahwah, NJ Lawrence Erlbaum Associates pp. 411-433.
44. Russ B. E, Ackelson A. L, Baker A. E, Cohen Y. E, (2008) Coding of auditory-stimulus identity in the auditory non-spatial processing stream. Journal of Neurophysiology 99: 87-95.
45. Formisano E, De Martino F, Bonte M, Goebel R, (2008) "Who" is saying "what"? Brain-based decoding of human voice and speech. Science 322: 970-973.
46. Chang E. F, Rieger J. W, Johnson K, Berger M. S, Barbaro N. M, et al. (2010) Categorical speech representation in human superior temporal gyrus. Nat Neurosci 13: 1428-1432.
47. Tsunada J, Lee J. H, Cohen Y. E, (2011) Representation of speech categories in the primate auditory cortex. Journal of Neurophysiology.
48. Mitchell T. M, Shinkareva S. V, Carlson A, Chang K. M, Malave V. L, et al. (2008) Predicting human brain activity associated with the meanings of nouns. Science 320: 1191-1195.
49. Kay K. N, Naselaris T, Prenger R. J, Gallant J. L, (2008) Identifying natural images from human brain activity. Nature 452: 352-355.
50. Naselaris T, Prenger R. J, Kay K. N, Oliver M, Gallant J. L, (2009) Bayesian reconstruction of natural images from human brain activity. Neuron 63: 902-915.
51. Dalal S. S, Guggisberg A. G, Edwards E, Sekihara K, Findlay A. M, et al. (2007) Spatial localization of cortical time-frequency dynamics. Conf Proc IEEE Eng Med Biol Soc 2007: 4941-4944.
52. Garofolo J. S, Lamel L. F, Fisher W. M, Fiscus J. G, Pallet D. S, et al. (1993) Acoustic-phonetic continuous speech corpus. Linguistic Data Consortium.
53. David S. V, Mesgarani N, Shamma S. A, (2007) Estimating sparse spectro-temporal receptive fields with natural stimuli. Network 18: 191-212.
54. Rokach L, (2010) Pattern classification using ensemble method London World Scientific Pub Co Inc.
55. Boulesteix A. L, Strobl C, Augustin T, Daumer M, (2008) Evaluating microarray-based classifiers: an overview. Cancer Inform 6: 77-97.
56. David S. V, Gallant J. L, (2005) Predicting neuronal responses during natural vision. Network 16: 239-260.
57. Miller L. M, Escabi M. A, Read H. L, Schreiner C. E, (2002) Spectrotemporal receptive fields in the lemniscal auditory thalamus and cortex. J Neurophysiol 87: 516-527.