Decoding speech for understanding and treating aphasia

Progress in Brain Research

Volumn 207, Issue , 2013, Pages 435-456

  Pasley, Brian N ^a   Knight, Robert T ^a,b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Aphasia; Decoding; Language; Neural encoding; Speech

Indexed keywords

ANTERIOR INSULA; APHASIA; ARCUATE FASCICULUS; ARTICLE; ATAXIC APHASIA; AUDITORY CORTEX; COMPREHENSION; CORTICAL SENSORY APHASIA; DIAGONAL BAND OF BROCA; ELECTROCORTICOGRAPHY; ELECTROENCEPHALOGRAM; ELECTROENCEPHALOGRAPHY; FUNCTIONAL MAGNETIC RESONANCE IMAGING; FUNCTIONAL NEUROIMAGING; HEMISPHERIC DOMINANCE; HUMAN; INFERIOR FRONTAL GYRUS; INFORMATION PROCESSING; LANGUAGE ABILITY; LANGUAGE DISABILITY; LANGUAGE PROCESSING; LEFT HANDEDNESS; MIDDLE TEMPORAL GYRUS; MOTOR CORTEX; NERVE CELL NETWORK; NERVE CELL PLASTICITY; NONHUMAN; PHONEME; PHONETICS; PRIORITY JOURNAL; QUANTITATIVE ANALYSIS; RIGHT HANDEDNESS; SPEECH AND LANGUAGE; SPEECH ARTICULATION; SPEECH REHABILITATION; SUBCORTEX; SUPERIOR TEMPORAL GYRUS; TEMPORAL LOBE; WERNICKE AREA;

APHASIA; DECODING; LANGUAGE; NEURAL ENCODING; SPEECH;

APHASIA; BRAIN; HUMANS; LANGUAGE; MODELS, NEUROLOGICAL; NEURONAL PLASTICITY; SPEECH;

EID: 84888882854     PISSN: 00796123     EISSN: 18757855     Source Type: Book Series    
DOI: 10.1016/B978-0-444-63327-9.00018-7     Document Type: Chapter

References (66)

1. Adelson E.H., Bergen J.R. Spatiotemporal energy models for the perception of motion. J. Opt. Soc. Am. A 1985, 2:284-299.
2. Aertsen A.M., Johannesma P.I. The spectro-temporal receptive field. A functional characteristic of auditory neurons. Biol. Cybern. 1981, 42:133-143.
3. Bialek W., Rieke F., De Ruyter Van Steveninck R.R., Warland D. Reading a neural code. Science 1991, 252:1854-1857.
4. Bouchard K.E., Mesgarani N., Johnson K., Chang E.F. Functional organization of human sensorimotor cortex for speech articulation. Nature 2013, 495(7441):327-332. Epub 2013 Feb 20. http://dx.doi.org/10.1038/nature11911.
5. Breiman L. Statistical Modeling: The Two Cultures. Stat. Sci. 2001, 16:199-231.
6. Browman C.P., Goldstein L. Articulatory gestures as phonological units. Phonology 1989, 6:201-251.
7. Brumberg J.S., Nieto-Castanon A., Kennedy P.R., Guenther F. Brain-computer interfaces for speech communication. Speech Commun. 2010, 52:367-379.
8. Canolty R.T., Soltani M., Dalal S.S., Edwards E., Dronkers N.F., Nagarajan S.S., Kirsch H.E., Barbaro N.M., Knight R.T. Spatiotemporal dynamics of word processing in the human brain. Front. Neurosci. 2007, 1:185-196.
9. Carmena J.M. Advances in neuroprosthetic learning and control. PLoS Biol. 2013, 11:e1001561.
10. Carmena J.M., Lebedev M.A., Crist R.E., O'Doherty J.E., Santucci D.M., Dimitrov D.F., Patil P.G., Henriquez C.S., Nicolelis M.A. Learning to control a brain-machine interface for reaching and grasping by primates. PLoS Biol. 2003, 1:E42.
11. Chang E.F., Rieger J.W., Johnson K., Berger M.S., Barbaro N.M., Knight R.T. Categorical speech representation in human superior temporal gyrus. Nat. Neurosci. 2010, 13:1428-1432.
12. Chi T., Gao Y., Guyton M.C., Ru P., Shamma S. Spectro-temporal modulation transfer functions and speech intelligibility. J. Acoust. Soc. Am. 1999, 106:2719-2732.
13. Chi T., Ru P., Shamma S.A. Multiresolution spectrotemporal analysis of complex sounds. J. Acoust. Soc. Am. 2005, 118:887-906.
14. Crone N.E., Boatman D., Gordon B., Hao L. Induced electrocorticographic gamma activity during auditory perception. Brazier Award-winning article, 2001. Clin. Neurophysiol. 2001, 112:565-582.
15. Dau T., Kollmeier B., Kohlrausch A. Modeling auditory processing of amplitude modulation. I. Detection and masking with narrow-band carriers. J. Acoust. Soc. Am. 1997, 102:2892-2905.
16. David S.V., Hayden B.Y., Gallant J.L. Spectral receptive field properties explain shape selectivity in area V4. J. Neurophysiol. 2006, 96:3492-3505.
17. David S.V., Mesgarani N., Shamma S.A. Estimating sparse spectro-temporal receptive fields with natural stimuli. Network 2007, 18:191-212.
18. De Boer E. Correlation studies applied to the frequency resolution of the cochlea. J. Audit. Res. 1967, 7:209-217.
19. Decharms R.C., Blake D.T., Merzenich M.M. Optimizing sound features for cortical neurons. Science 1998, 280:1439-1443.
20. Deng L., O'Shaughnessy D. Speech Processing: A Dynamic and Optimization-Oriented Approach 2003, Marcel Dekker, Inc., New York.
21. Depireux D.A., Simon J.Z., Klein D.J., Shamma S.A. Spectro-temporal response field characterization with dynamic ripples in ferret primary auditory cortex. J. Neurophysiol. 2001, 85:1220-1234.
22. Edwards E., Soltani M., Kim W., Dalal S.S., Nagarajan S.S., Berger M.S., Knight R.T. Comparison of time-frequency responses and the event-related potential to auditory speech stimuli in human cortex. J. Neurophysiol. 2009, 102:377-386.
23. Eggermont J.J. Temporal modulation transfer functions in cat primary auditory cortex: separating stimulus effects from neural mechanisms. J. Neurophysiol. 2002, 87:305-321.
24. Elliott T.M., Theunissen F.E. The modulation transfer function for speech intelligibility. PLoS Comput. Biol. 2009, 5:e1000302.
25. Formisano E., De Martino F., Bonte M., Goebel R. "Who" is saying "what"? Brain-based decoding of human voice and speech. Science 2008, 322:970-973.
26. Friederici A.D. The brain basis of language processing: from structure to function. Physiol. Rev. 2011, 91:1357-1392.
27. Garofolo, J. S., Lamel, L. F., Fisher, W. M., Fiscus, J. G., Pallet, D. S., Dahlgrena, N. L., Zue, V., 1993. Acoustic-Phonetic Continuous Speech Corpus. Linguistic Data Consortium. http://catalog.ldc.upenn.edu/LDC93S1.
28. Georgopoulos A.P., Kalaska J.F., Caminiti R., Massey J.T. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 1982, 2:1527-1537.
29. Goense J.B., Logothetis N.K. Neurophysiology of the BOLD fMRI signal in awake monkeys. Curr. Biol. 2008, 18:631-640.
30. Graziano M.S., Aflalo T.N. Rethinking cortical organization: moving away from discrete areas arranged in hierarchies. Neuroscientist 2007, 13:138-147.
31. Greenberg S. A multi-tier theoretical framework for understanding spoken language. Listening to Speech: An Auditory Perspective 2006, Lawrence Erlbaum Associates, Mahwah, NJ. S. Greenberg, W.A. Ainsworth (Eds.).
32. Greenberg S., Ainsworth W.A. Speech processing in the auditory system: an overview. Speech Processing in the Auditory System 2004, Springer-Verlag, New York. S. Greenberg, W.A. Ainsworth, A.N. Popper, R.R. Fay (Eds.).
33. Guenther F.H., Brumberg J.S., Wright E.J., Nieto-Castanon A., Tourville J.A., Panko M., Law R., Siebert S.A., Bartels J.L., Andreasen D.S., Ehirim P., Mao H., Kennedy P.R. A wireless brain-machine interface for real-time speech synthesis. PLoS One 2009, 4:e8218.
34. Hackett T.A. Information flow in the auditory cortical network. Hear. Res. 2011, 271:133-146.
35. Hajime H. Investigating the physiology of laryngeal structures. The Handbook of Phonetic Sciences 1999, Blackwell Publishing, West Sussex, United Kingdom. W.J. Hardcastle, J. Laver (Eds.).
36. Hickok G., Poeppel D. The cortical organization of speech processing. Nat. Rev. Neurosci. 2007, 8:393-402.
37. Kadia S.C., Wang X. Spectral integration in A1 of awake primates: neurons with single- and multipeaked tuning characteristics. J. Neurophysiol. 2003, 89:1603-1622.
38. Kellis S., Miller K., Thomson K., Brown R., House P., Greger B. Decoding spoken words using local field potentials recorded from the cortical surface. J. Neural Eng. 2010, 7:056007.
39. Liberman A.M., Cooper F.S., Shankweiler D.P., Studdert-Kennedy M. Perception of the speech code. Psychol. Rev. 1967, 74:431-461.
40. Lofqvist A. Theories and models of speech production. The Handbook of Phonetic Sciences 1999, Blackwell Publishing, West Sussex, United Kingdom. W.J. Hardcastle, J. Laver (Eds.).
41. Mesgarani, N., Shamma, S., 2011. Speech processing with a cortical representation of audio. Proc. ICASSP 2011. pp. 5872-5875.
42. Mesgarani N., David S.V., Fritz J.B., Shamma S.A. Phoneme representation and classification in primary auditory cortex. J. Acoust. Soc. Am. 2008, 123:899-909.
43. Mesgarani N., David S.V., Fritz J.B., Shamma S.A. Influence of context and behavior on stimulus reconstruction from neural activity in primary auditory cortex. J. Neurophysiol. 2009, 102:3329-3339.
44. Miller L.M., Escabi M.A., Read H.L., Schreiner C.E. Spectrotemporal receptive fields in the lemniscal auditory thalamus and cortex. J. Neurophysiol. 2002, 87:516-527.
45. Mitchell T.M., Shinkareva S.V., Carlson A., Chang K.M., Malave V.L., Mason R.A., Just M.A. Predicting human brain activity associated with the meanings of nouns. Science 2008, 320:1191-1195.
46. Naselaris T., Prenger R.J., Kay K.N., Oliver M., Gallant J.L. Bayesian reconstruction of natural images from human brain activity. Neuron 2009, 63:902-915.
47. Nishimoto S., Vu A.T., Naselaris T., Benjamini Y., Yu B., Gallant J.L. Reconstructing visual experiences from brain activity evoked by natural movies. Curr. Biol. 2011, 21:1641-1646.
48. Nourski K.V., Reale R.A., Oya H., Kawasaki H., Kovach C.K., Chen H., Howard M.A., Brugge J.F. Temporal envelope of time-compressed speech represented in the human auditory cortex. J. Neurosci. 2009, 29:15564-15574.
49. Pasley B.N., David S.V., Mesgarani N., Flinker A., Shamma S.A., Crone N.E., Knight R.T., Chang E.F. Reconstructing speech from human auditory cortex. PLoS Biol. 2012, 10(1):e1001251. Epub 2012 Jan 31. http://dx.doi.org/10.1371/journal.pbio.1001251.
50. Pei X., et al. Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans. J. Neural Eng. 2011, 8:046028.
51. Pei X., Leuthardt E.C., Gaona C.M., Brunner P., Wolpaw J.R., Schalk G. Spatiotemporal dynamics of electrocorticographic high gamma activity during overt and covert word repetition. Neuroimage 2011, 54:2960-2972.
52. Rauschecker J.P., Scott S.K. Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing. Nat. Neurosci. 2009, 12:718-724.
53. Rauschecker J.P., Tian B., Pons T., Mishkin M. Serial and parallel processing in rhesus monkey auditory cortex. J. Comp. Neurol. 1997, 382:89-103.
54. Robson H., Grube M., Lambon Ralph M.A., Griffiths T.D., Sage K. Fundamental deficits of auditory perception in Wernicke's aphasia. Cortex 2013, 49:1808-1822.
55. Sahin N.T., Pinker S., Cash S.S., Schomer D., Halgren E. Sequential processing of lexical, grammatical, and phonological information within Broca's area. Science 2009, 326:445-449.
56. Schonwiesner M., Zatorre R.J. Spectro-temporal modulation transfer function of single voxels in the human auditory cortex measured with high-resolution fMRI. Proc. Natl. Acad. Sci. U. S. A. 2009, 106:14611-14616.
57. Schreiner C.E., Read H.L., Sutter M.L. Modular organization of frequency integration in primary auditory cortex. Annu. Rev. Neurosci. 2000, 23:501-529.
58. Sutter M.L., Schreiner C.E. Physiology and topography of neurons with multipeaked tuning curves in cat primary auditory cortex. J. Neurophysiol. 1991, 65:1207-1226.
59. Talavage T.M., Sereno M.I., Melcher J.R., Ledden P.J., Rosen B.R., Dale A.M. Tonotopic organization in human auditory cortex revealed by progressions of frequency sensitivity. J. Neurophysiol. 2004, 91:1282-1296.
60. Tankus A., Fried I., Shoham S. Structured neuronal encoding and decoding of human speech features. Nat. Commun. 2012, 3:1015.
61. Tankus A., Fried I., Shoham S. Cognitive-motor brain-machine interfaces. J. Physiol. 2013, Paris pii:S0928-4257(13)00035-1, [Epub ahead of print]. http://dx.doi.org/10.1016/j.jphysparis.2013.05.005.
62. Theunissen F.E., David S.V., Singh N.C., Hsu A., Vinje W.E., Gallant J.L. Estimating spatio-temporal receptive fields of auditory and visual neurons from their responses to natural stimuli. Network 2001, 12:289-316.
63. Todorov E. Optimality principles in sensorimotor control. Nat. Neurosci. 2004, 7:907-915.
64. Viswanathan A., Freeman R.D. Neurometabolic coupling in cerebral cortex reflects synaptic more than spiking activity. Nat. Neurosci. 2007, 10:1308-1312.
65. Wrench, A.A., Hardcastle, W.J., 2000. A multichannel articulatory speech database and its application for automatic speech recognition. In: Proceedings of the Fifth Seminar on Speech Production: Models and Data & CREST Workshop on Models of Speech Production: Motor Planning and Articulator Modelling, Kloster Seeon, Bavaria, Germany. pp. 305-308.
66. Wu M.C., David S.V., Gallant J.L. Complete functional characterization of sensory neurons by system identification. Annu. Rev. Neurosci. 2006, 29:477-505.