Classification of intended phoneme production from chronic intracortical microelectrode recordings in speech-motor cortex

Frontiers in Neuroscience

Volumn , Issue MAY, 2011, Pages 1-12

  Brumberg, Jonathan S ^a   Wright, E Joe ^b   Andreasen, Dinal S ^b,c   Guenther, Frank H ^a,d   Kennedy, Philip R ^b  

^a BOSTON UNIVERSITY   (United States)

^b Neural Signals Inc   (United States)

^c GEORGIA TECH RESEARCH INSTITUTE   (United States)

^d MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Chronic recording; Locked in syndrome; Motor cortex; Neurotrophic electrode; Speech prosthesis

Indexed keywords

ADULT; ARTICLE; CASE REPORT; COGNITION; HUMAN; LOCKED IN SYNDROME; MALE; MICROELECTRODE; MOTOR CORTEX; MOVEMENT (PHYSIOLOGY); NEUROPHYSIOLOGY; PHONEME; QUALITY OF LIFE; RATING SCALE; RECORDING; SENSATION; SPEECH;

EID: 80051905631     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: 10.3389/fnins.2011.00065     Document Type: Article

References (68)

1. Bartels, J. L., Andreasen, D., Ehirim, P., Mao, H., Seibert, S., Wright, E. J., and Kennedy, P. R. (2008). Neurotrophic electrode: method of assembly and implantation into human motor speech cortex. J. Neurosci. Methods 174, 168-176.
2. Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey, B., Kubler, A., Perelmouter, J., Taub, E., and Flor, H. (1999). A spelling device for the paralysed. Nature 398, 297-298.
3. Birbaumer, N., Kubler, A., Ghanayim, N., Hinterberger, T., Perelmouter, J., Kaiser, J., Iversen, I., Kotchoubey, B., Neumann, N., and Flor, H. (2000). The thought translation device (TTD) for completely paralyzed patients. IEEE Trans. Rehabil. Eng. 8, 190-193.
4. Blakely, T., Miller, K. J., Rao, R., Holmes, M., and Ojemann, J. G. (2008). "Localization and classification of phonemes using high spatial resolution electrocorticography (ECoG) grids," in Conference Proceedings of the IEEE Engineering in Medicine and Biology Society, 2008. 30th Annual International Conference of the IEEE EMBS 2008, Vancouver, August 20-24, 2008, 4964-4967.
5. Blankertz, B., Krauledat, M., Dornhege, G., Williamson, J., Murray-Smith, R., and Müller, K.-R. (2007). "A note on brain actuated spelling with the Berlin brain-computer interface," in Universal Access in Human-Computer Interaction. Ambient Interaction, ed. C. Stephanidis (Heidelberg: Springer), 759-768.
6. Brumberg, J. S., Kennedy, P. R., and Guenther, F. H. (2009). "Artificial speech synthesizer control by brain-computer interface," in Proceedings of the 10th Annual Conference of the International Speech Communication Association. (Brighton: International Speech Communication Association).
7. Brumberg, J. S., Nieto-Castanon, A., Kennedy, P. R., and Guenther, F. H. (2010). Brain-computer interfaces for speech communication. Speech Commun. 52, 367-379.
8. Carmena, J. M., Lebedev, M. A., Crist, R. E., O'Doherty, J. E., Santucci, D. M., Dimitrov, D. F., Patil, P. G., Henriquez, C. S., and Nicolelis, M. A. L. (2003). Learning to control a brain-machine interface for reaching and grasping by primates. PLoS Biol. 1, e42. doi: 10.1371/journal.pbio.0000042
9. Chapin, J. K., Moxon, K. A., Markowitz, R. S., and Nicolelis, M. A. L. (1999). Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nat. Neurosci. 2, 664-670.
10. Cheng, M., Gao, X., Gao, S., and Xu, D. (2002). Design and implementation of a brain-computer interface with high transfer rates. IEEE Trans. Biomed. Eng. 49, 1181-1186.
11. Crone, N. E., Boatman, D., Gordon, B., and Hao, L. (2001a). Induced electrocorticographic gamma activity during auditory perception. Clin. Neurophysiol. 112, 565-582.
12. Crone, N. E., Hao, L., Hart, J., Boatman, D., Lesser, R. P., Irizarry, R., and Gordon, B. (2001b). Electrocorticographic gamma activity during word production in spoken and sign language. Neurology 57, 2045-2053.
13. Denby, B., Schultz, T., Honda, K., Hueber, T., Gilbert, J. M., and Brumberg, J. S. (2010). Silent speech interfaces. Speech Commun. 52, 270-287.
14. Dimitriadou, E., Hornik, K., Leisch, F., Meyer, D., and Weingessel, A. (2010). e1071: Misc Functions of the Department of Statistics (e1071), TU Wien. Available at: http://cran.r-project.org/web/packages/e1071/index.html
15. Doble, J. E., Haig, A. J., Anderson, C., and Katz, R. (2003). Impairment, activity, participation, life satisfaction, and survival in persons with locked-in syndrome for over a decade: follow-up on a previously reported cohort. J. Head Trauma Rehabil. 18, 435-444.
16. Donchin, E., Spencer, K., and Wijesinghe, R. (2000). The mental prosthesis: assessing the speed of a P300-based brain-computer interface. IEEE Trans. Rehabil. Eng. 8, 174-179.
17. Donoghue, J. P., Nurmikko, A., Black, M., and Hochberg, L. R. (2007). Assistive technology and robotic control using motor cortex ensemble-based neural interface systems in humans with tetraplegia. J. Physiol. 579, 603-611.
18. Farwell, L., and Donchin, E. (1988). Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr. Clin. Neurophysiol. 70, 510-523.
19. Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Ann. Eugen. 7, 179-188.
20. Friedman, J. H. (1991). Multivariate adaptive regression splines. Ann. Stat. 19, 1-67.
21. Friman, O., Luth, T., Volosyak, I., and Graser, A. (2007). "Spelling with steady-state visual evoked potentials," in 2007 3rd International IEEE/EMBS Conference on Neural Engineering (Kohala Coast, HI: IEEE), 354-357.
22. Guenther, F. H., Brumberg, J. S., Wright, E. J., Nieto-Castanon, A., Tourville, J. A., Panko, M., Law, R., Siebert, S. A., Bartels, J. L., Andreasen, D. S., Ehirim, P., Mao, H., and Kennedy, P. R. (2009). A wireless brain-machine interface for real-time speech synthesis. PLoS ONE 4, e8218. doi: 10.1371/journal.pone.0008218
23. Guenther, F. H., Ghosh, S. S., and Tourville, J. A. (2006). Neural modeling and imaging of the cortical interactions underlying syllable production. Brain Lang. 96, 280-301.
24. Hastie, T., Tibshirani, R., and Buja, A. (1994). Flexible discriminant analysis by optimal scoring. J. Am. Stat. Assoc. 89, 1255-1270.
25. Hastie, T., Tibshirani, R., and Friedman, J. H. (2001). The Elements of Statistical Learning. New York: Springer, 560.
26. Hochberg, L. R., Serruya, M. D., Friehs, G. M., Mukand, J. A., Saleh, M., Caplan, A. H., Branner, A., Chen, D., Penn, R. D., and Donoghue, J. P. (2006). Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442, 164-171.
27. Hoogerwerf, A., and Wise, K. (1994). A three-dimensional microelectrode array for chronic neural recording. IEEE Trans. Biomed. Eng. 41, 1136-1146.
28. Jeannerod, M., and Frak, V. (1999). Mental imaging of motor activity in humans. Curr. Opin. Neurobiol. 9, 735-739.
29. Jones, K., Campbell, P., and Normann, R. (1992). A glass/silicon composite intracortical electrode array. Ann. Biomed. Eng. 20, 423-437.
30. Kellis, S., Miller, K, Thompson, K., Brown, R., House, P., and Greger, B. (2010). Decoding spoken words using local field potentials recorded from the cortical surface. J. Neural Eng. 7, 056007.
31. Kennedy, P. R. (1989). The cone electrode: a long-term electrode that records from neurites grown onto its recording surface. J. Neurosci. Methods 29, 181-193.
32. Kennedy, P. R. (2006). "Comparing electrodes for use as cortical control signals: tiny tines or tiny cones on wires: which is best?" in The Biomedical Engineering Handbook, 3rd Edn, ed. J. D. Brazino (Boca Raton: CRC/Taylor & Francis), 32.1-32.14.
33. Kennedy, P. R., and Bakay, R. A. E. (1998). Restoration of neural output from a paralyzed patient by direct brain connection. Neuroreport 9, 1707-1711.
34. Kennedy, P. R., Bakay, R. A. E., Moore, M. M., Adams, K., and Goldwaithe, J. (2000). Direct control of a computer from the human central nervous system. IEEE Trans. Rehabil. Eng. 8, 198-202.
35. Kennedy, P. R., Bakay, R. A. E., and Sharpe, S. M. (1992a). Behavioral correlates of action potentials recorded chronically inside the cone electrode. Neuroreport 3, 605-608.
36. Kennedy, P. R., Mirra, S. S., and Bakay, R. A. E. (1992b). The cone electrode: ultrastructural studies following long-term recording in rat and monkey cortex. Neurosci. Lett. 142, 89-94.
37. Kennedy, P. R., Kirby, T., Moore, M. M., King, B., and Mallory, A. (2004). Computer control using human intracortical local field potentials. IEEE Trans. Neural Syst. Rehabil. Eng. 12, 339-344.
38. Lacourse, M. G., Cohen, M. J., Lawrence, K. E., and Romero, D. H. (1999). Cortical potentials during imagined movements in individuals with chronic spinal cord injuries. Behav. Brain Res. 104, 73-88.
39. Leisch, F., Hornik, K., and Ripley, B. D. (2009). Mda: Mixture and Flexible Discriminant Analysis. S Original by Trevor Hastie and Robert Tibshirani. Available at: http://cran.r-project.org/web/packages/mda/index.html
40. Leuthardt, E. C., Schalk, G., Wolpaw, J. R., Ojemann, J. G., and Moran, D. W. (2004). A brain-computer interface using electrocorticographic signals in humans. J. Neural Eng. 1, 63-71.
41. Lewicki, M. S. (1998). A review of methods for spike sorting: the detection and classification of neural action potentials. Network 9, R53-R78.
42. Matthews, B., Kim, J., Brumberg, J. S., and Clements, M. (2010). "A probabilistic decoding approach to a neural prosthesis for speech," in Proceedings of the 2010 4th Annual Conference on Bioinformatics and Biomedical Engineering (iCBBE), Chengdu, June 18-20, 2010.
43. Maynard, E. M., Nordhausen, C. T., and Normann, R. A. (1997). The utah intracortical electrode array: a recording structure for potential brain-computer interfaces. Electroencephalogr. Clin. Neurophysiol. 102, 228-239.
44. Neuper, C., Müller, G. R., Kübler, A., Birbaumer, N., and Pfurtscheller, G. (2003). Clinical application of an EEG-based brain-computer interface: a case study in a patient with severe motor impairment. Clin. Neurophysiol. 114, 399-409.
45. Neuper, C., Scherer, R., Reiner, M., and Pfurtscheller, G. (2005). Imagery of motor actions: differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG. Cogn. Brain Res. 25, 668-677.
46. Nicolelis, M. A. L., Dimitrov, D., Carmena, J. M., Crist, R., Lehew, G., Kralik, J. D., and Wise, S. P. (2003). Chronic, multisite, multielectrode recordings in macaque monkeys. Proc. Natl. Acad. Sci. U.S.A. 100, 11041-11046.
47. Obermaier, B., Muller, G., and Pfurtscheller, G. (2003). "Virtual keyboard" controlled by spontaneous EEG activity. IEEE Trans. Neural Syst. Rehabil. Eng. 11, 422-426.
48. Penfield, W., and Roberts, L. (1959). Speech and Brain Mechanisms. Princeton, NJ: Princeton University Press.
49. Plum, F., and Posner, J. B. (1972). The diagnosis of stupor and coma. Contemp. Neurol. Ser. 10, 1-286.
50. R Development Core Team. (2009). R: A Language and Environment f or Statistical Computing, Vienna. Available at: http://www.R-project.org
51. Rabiner, L., and Juang, B. H. (1993). Fundamentals of Speech Recognition. Upper Saddle River, NJ: Prentice-Hall, Inc.
52. Rao, C. R. (1973). Linear Statistical Inference and its Applications. New York: Wiley.
53. Roth, M., Decety, J., Raybaudi, M., Massarelli, R., Delon-Martin, C., Segebarth, C., Morand, S., Gemignani, A., Decorps, M., and Jeannerod, M. (1996). Possible involvement of primary motor cortex in mentally simulated movement: a functional magnetic resonance imaging study. Neuroreport 7, 1280-1284.
54. Sadhir, R., James, W., Yasuda, H., Sharma, A., Nichols, M., and Haln, A. (1981). The adhesion of glow-discharge polymers, Silastic and Parylene to implantable platinum electrodes: results of tensile pull tests after exposure to isotonic sodium chloride. Biomaterials 2, 239-243.
55. Scherer, R., Muller, G., Neuper, C., Graimann, B., and Pfurtscheller, G. (2004). An asynchronously controlled EEG-based virtual keyboard: improvement of the spelling rate. IEEE Trans. Biomed. Eng. 51, 979-984.
56. Schwartz, A. B. (2004). Cortical neural prosthetics. Annu. Rev. Neurosci. 27, 487-507.
57. Serruya, M. D., Hatsopoulos, N. G., Paninski, L., Fellows, M. R., and Donoghue, J. P. (2002). Instant neural control of a movement signal. Nature 416, 141-142.
58. Stevens, K. N. (2000). Acoustic Phonetics. Cambridge, MA: MIT Press. Sutter, E. E. (1992). The brain response interface: communication through visually-induced electrical brain responses. J. Microcomput. Appl. 15, 31-45.
59. Taylor, D. M., Tillery, S. I., and Schwartz, A. B. (2002). Direct cortical control of 3D neuroprosthetic devices. Science 296, 1829-1832.
60. Towle, V. L., Yoon, H., Castelle, M., Edgar, J. C., Biassou, N. M., Frim, D. M., Spire, J., and Kohrman, M. H. (2008). ECoG gamma activity during a language task: differentiating expressive and receptive speech areas. Brain 131, 2013-2027.
61. Vapnik, V. N. (1995). The Nature of Statistical Learning Theory. New York: Springer Verlag.
62. Vaughan, T., McFarland, D., Schalk, G., Sarnacki, W., Krusienski, D., Sellers, E., and Wolpaw, J. (2006). The wadsworth BCI research and development program: at home with BCI. IEEE Trans. Neural Syst. Rehabil. Eng. 14, 229-233.
63. Velliste, M., Perel, S., Spalding, M. C., Whitford, A. S., and Schwartz, A. B. (2008). Cortical control of a prosthetic arm for self-feeding. Nature 453, 1098-1101.
64. Venables, W. N., and Ripley, B. D. (2002). Modern Applied Statistics with S. Fourth Edition. New York: Springer.
65. Ward, M. P., Rajdev, P., Ellison, C., and Irazoqui, P. P. (2009). Toward a comparison of microelectrodes for acute and chronic recordings. Brain Res. 1282, 183-200.
66. Williams, J. C., Rennaker, R. L., and Kipke, D. R. (1999). Long-term neural recording characteristics of wire microelectrode arrays implanted in cerebral cortex. Brain Res. Brain Res. Protoc. 4, 303-313.
67. Wise, K. D., Angell, J. B., and Starr, A. (1970). An integrated-circuit approach to extracellular microelectrodes. IEEE Trans. Biomed. Eng. 17, 238-247.
68. Wolpaw, J. R., and McFarland, D. J. (2004). Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. Proc. Natl. Acad. Sci. U.S.A. 101, 17849-17854.