Electrocorticographic representations of segmental features in continuous speech

Frontiers in Human Neuroscience

Volumn 9, Issue FEB, 2015, Pages

  Lotte, Fabien ^a   Brumberg, Jonathan S ^b   Brunner, Peter ^c,d   Gunduz, Aysegul ^e   Ritaccio, Anthony L ^d   Guan, Cuntai ^f   Schalk, Gerwin ^c,d  

^a INRIA   (France)

^b UNIVERSITY OF KANSAS   (United States)

^c WADSWORTH CENTER   (United States)

^d ALBANY MEDICAL COLLEGE   (United States)

^e J Crayton Pruitt Family Department of Biomedical Engineering   (United States)

^f INSTITUTE FOR INFOCOMM RESEARCH   (Singapore)

Author keywords

Electrocorticography (ECoG); Manner of articulation; Place of articulation; Speech processing; Voicing

Indexed keywords

ADULT; ARTICLE; AUDITORY FEEDBACK; BRAIN REGION; CLINICAL ARTICLE; ELECTROCORTICOGRAPHY; FEMALE; HUMAN; INTRACTABLE EPILEPSY; LANGUAGE PROCESSING; MALE; MIDDLE AGED; MOTOR CONTROL; PHONETICS; SPEECH; SPEECH ARTICULATION; SPEECH DISCRIMINATION; VOICING; YOUNG ADULT;

EID: 84933674352     PISSN: None     EISSN: 16625161     Source Type: Journal    
DOI: 10.3389/fnhum.2015.00097     Document Type: Article

References (43)

1. Antoniou, A. (1993). Digital Filters, 2nd Edn. New York, NY: McGraw Hill.
2. Blakely, T., Miller, K. J., Rao, R. P. N., Holmes, M. D., and Ojemann, J. G. (2008). Localization and classification of phonemes using high spatial resolution electrocorticography (ECoG) grids. Conf. Proc. IEEE Eng. Med. Biol. Soc. 2008, 4964-4967. doi: 10.1109/IEMBS.2008.4650328.
3. Blankertz, B., Lemm, S., Treder, M., Haufe, S., and Müller, K.-R. (2011). Single-trial analysis and classification of ERP components-a tutorial. Neuroimage 56, 814-825. doi: 10.1016/j.neuroimage.2010.06.048.
4. Bouchard, K. E., Mesgarani, N., Johnson, K., and Chang, E. F. (2013). Functional organization of human sensorimotor cortex for speech articulation. Nature 495, 327-332. doi: 10.1038/nature11911.
5. Brown, S., Ngan, E., and Liotti, M. (2008). A larynx area in the human motor cortex. Cereb. Cortex 18, 837-845. doi: 10.1093/cercor/bhm131.
6. Brumberg, J. S., Wright, E. J., Andreasen, D. S., Guenther, F. H., and Kennedy, P. R. (2011). Classification of intended phoneme production from chronic intracortical microelectrode recordings in speech-motor cortex.Front. Neurosci. 5:65. doi: 10.3389/fnins.2011.00065.
7. Chang, E. F., Rieger, J. W., Johnson, K., Berger, M. S., Barbaro, N. M., and Knight, R. T. (2010). Categorical speech representation in human superior temporal gyrus. Nat. Neurosci. 13, 1428-1432. doi: 10.1038/nn.2641.
8. Crone, N. E., Boatman, D., Gordon, B., and Hao, L. (2001). Induced electrocorticographic gamma activity during auditory perception. Clin. Neurophysiol. 112, 565-582. doi: 10.1016/S1388-2457(00)00545-9.
9. Edwards, E., Nagarajan, S. S., Dalal, S. S., Canolty, R. T., Kirsch, H. E., Barbaro, N. M., et al. (2010). Spatiotemporal imaging of cortical activation during verb generation and picture naming. Neuroimage 50, 291-301. doi: 10.1016/j.neuroimage.2009.12.035.
10. Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognit. Lett. 27, 861-874. doi: 10.1016/j.patrec.2005.10.010.
11. Guenther, F. H., Ghosh, S. S., and Tourville, J. A. (2006). Neural modeling and imaging of the cortical interactions underlying syllable production. Brain Lang. 96, 280-301. doi: 10.1016/j.patrec.2005.10.010.
12. Hall, T. A. (2007). "Segmental features," in The Cambridge Handbook of Phonology, ed P. de Lacy (New York, NY: Cambridge University Press), 324.
13. Hanley, J., and McNeil, B. (1983). A Method of Comparing the Areas under ROC curves derived from same cases.Radiology 148, 839-843. doi: 10.1148/radiology.148.3.6878708.
14. Hardcastle, W. J., and Hewlett, N. (eds.) (1999). Coarticulation: Theory, Data and Techniques. New York, NY: Cambridge University Press.
15. Houde, J. F., Nagarajan, S. S., Sekihara, K., and Merzenich, M. M. (2002). Modulation of the auditory cortex during speech: an MEG study. J. Cogn. Neurosci. 14, 1125-1138. doi: 10.1162/089892902760807140.
16. Indefrey, P., and Levelt, W. J. M. (2004). The spatial and temporal signatures of word production components.Cognition 92, 101-144. doi: 10.1016/j.cognition.2002.06.001.
17. International Phonetic Association (1999). Handbook of the International Phonetic Association: A Guide to the Use of the International Phonetic Alphabet. Cambridge: Cambridge University Press.
18. Kellis, S., Miller, K., Thomson, K., Brown, R., House, P., and Greger, B. (2010). Decoding spoken words using local field potentials recorded from the cortical surface. J. Neural Eng. 7:056007. doi: 10.1088/1741-2560/7/5/056007.
19. Kubanek, J., and Schalk, G. (2014). NeuralAct: a tool to visualize electrocortical (ECoG) activity on a three-dimensional model of the cortex. Neuroinformatics doi: 10.1007/s12021-014-9252-3. [Epub ahead of print].
20. Lancaster, J. L., Woldorff, M. G., Parsons, L. M., Liotti, M., Freitas, C. S., Rainey, L., et al. (2000). Automated Talairach atlas labels for functional brain mapping. Hum. Brain Mapp. 10, 120-131. doi: 10.1002/1097-0193(200007)10:3<120::AID-HBM30>3.0.CO;2-8.
21. Ledoit, O., and Wolf, M. (2004). A well-conditioned estimator for large-dimensional covariance matrices. J. Multivariate Anal. 88, 365-411. doi: 10.1016/S0047-259X(03)00096-4.
22. Leuthardt, E. C., Pei, X.-M., Breshears, J., Gaona, C., Sharma, M., Freudenberg, Z., et al. (2012). Temporal evolution of gamma activity in human cortex during an overt and covert word repetition task. Front. Hum. Neurosci. 6:99. doi: 10.3389/fnhum.2012.00099.
23. Levelt, W. J. M., Roelofs, A., and Meyer, A. S. (1999). A theory of lexical access in speech production. Behav. Brain Sci. 22, 1-75. doi: 10.1017/S0140525X99001776.
24. Lotte, F., Congedo, M., Lécuyer, A., Lamarche, F., and Arnaldi, B. (2007). A review of classification algorithms for EEG-based brain-computer interfaces. J. Neural Eng. 4, R1-R13. doi: 10.1088/1741-2560/4/2/R01.
25. Lotte, F., and Guan, C. (2010). "Learning from other subjects helps reducing brain-computer interface calibration time," in International Conference on Audio, Speech and Signal Processing (ICASSP'2010), 614-617. doi: 10.1109/ICASSP.2010.5495183.
26. Martin, S., Brunner, P., Holdgraf, C., Heinze, H.-J., Crone, N. E., Rieger, J., et al. (2014). Decoding spectrotemporal features of overt and covert speech from the human cortex. Front. Neuroengineering 7:14. doi: 10.3389/fneng.2014.00014.
27. Mellinger, J., and Schalk, G. (2010). "Brain-computer interfaces," in Brain-Computer Interfaces, eds B. Graimann, G. Pfurtscheller, and B. Allison (Berlin;Heidelberg: Springer Berlin Heidelberg), 259-279.
28. Mugler, E. M., Patton, J. L., Flint, R. D., Wright, Z. A., Schuele, S. U., Rosenow, J., et al. (2014). Direct classification of all American english phonemes using signals from functional speech motor cortex. J. Neural Eng. 11:035015. doi: 10.1088/1741-2560/11/3/035015.
29. Noble, W. (2009). How does multiple testing correction work? Nat. Biotechnol. 27, 1135-1137. doi: 10.1038/nbt1209-1135.
30. Pasley, B. N., David, S. V., Mesgarani, N., Flinker, A., Shamma, S. A., Crone, N. E., et al. (2012). Reconstructing speech from human auditory cortex. PLoS Biol. 10:e1001251. doi: 10.1371/journal.pbio.1001251.
31. Pei, X., Barbour, D. L., Leuthardt, E. C., and Schalk, G. (2011a). Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans. J. Neural Eng. 8:046028. doi: 10.1088/1741-2560/8/4/046028.
32. Pei, X., Leuthardt, E. C., Gaona, C. M., Brunner, P., Wolpaw, J. R., and Schalk, G. (2011b). Spatiotemporal dynamics of electrocorticographic high gamma activity during overt and covert word repetition. Neuroimage 54, 2960-2972. doi: 10.1016/j.neuroimage.2010.10.029.
33. Penfield, W., and Roberts, L. (1959). Speech and Brain-Mechanisms. Princeton, NJ: Princeton University Press.
34. Peng, H., Long, F., and Ding, C. (2005). Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27, 1226-1238. doi: 10.1109/TPAMI.2005.159.
35. Potes, C., Gunduz, A., Brunner, P., and Schalk, G. (2012). Dynamics of electrocorticographic (ECoG) activity in human temporal and frontal cortical areas during music listening. Neuroimage 61, 841-848.
36. Rabiner, L. R., and Juang, B.-H. (1993). Fundamentals of Speech Recognition. Upper Saddle River, NJ: Prentice Hall.
37. Recasens, D. (1999). "Lingual coarticulation," in Coarticulation: Theory, Data and Techniques, eds W. J. Hardcastle and N. Hewlett (Cambridge University Press), 80-104.
38. Riès, S., Janssen, N., Burle, B., and Alario, F.-X. (2013). Response-locked brain dynamics of word production. PloS ONE 8:e58197. doi: 10.1371/journal.pone.0058197.
39. Saltzman, E. L., and Munhall, K. G. (1989). A dynamical Approach to gestural patterning in speech production.Ecol. Psychol. 1, 333-382. doi: 10.1207/s15326969eco0104/2.
40. Schalk, G., McFarland, D. J., Hinterberger, T., Birbaumer, N., and Wolpaw, J. R. (2004). BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Trans. Biomed. Eng. 51, 1034-1043. doi: 10.1109/TBME.2004.827072.
41. Simonyan, K., and Horwitz, B. (2011). Laryngeal motor cortex and control of speech in humans. Neuroscientist 17, 197-208. doi: 10.1177/1073858410386727.
42. Towle, V. L., Yoon, H. A., Castelle, M., Edgar, J. C., Biassou, N. M., Frim, D. M., et al. (2008). ECoG gamma activity during a language task: differentiating expressive and receptive speech areas. Brain 131(Pt 8), 2013-2027. doi: 10.1093/brain/awn147.
43. Young, S., Evermann, G., Gales, M., Hain, T., Kershaw, D., Liu, X., et al. (2006). The HTK Book (for HTK Version 3.4). Cambridge University Engineering Department.