Cognitive-motor brain-machine interfaces

Journal of Physiology Paris

Volumn 108, Issue 1, 2014, Pages 38-44

  Tankus, Ariel ^a,b   Fried, Itzhak ^b,c,d   Shoham, Shy ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^b TEL AVIV SOURASKY MEDICAL CENTER   (Israel)

^c UNIVERSITY OF CALIFORNIA   (United States)

^d TEL AVIV UNIVERSITY   (Israel)

Author keywords

Brain machine interface; Decision making; Direct object control; Human neurophysiology; Percept decoding; Speech

Indexed keywords

ARCUATE NUCLEUS; AUDITORY DISCRIMINATION; BRAIN FUNCTION; BRAIN MACHINE INTERFACE; COGNITION; DECISION MAKING; ELECTROCORTICOGRAPHY; ELECTROENCEPHALOGRAM; FEEDBACK SYSTEM; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; LIMB MOVEMENT; MOTOR PERFORMANCE; NEUROIMAGING; NEUROMODULATION; NONHUMAN; PRIMARY MOTOR CORTEX; REVIEW; VISUAL DISCRIMINATION; VISUAL INFORMATION; WORKING MEMORY;

AREAS OF THE VISUAL CORTEX; BMI; BRAIN-MACHINE INTERFACE; DECISION MAKING; DIRECT OBJECT CONTROL; ECOG; EEG; ELECTROCORTICOGRAPHY; ELECTROENCEPHALOGRAPHY; FDA; FLEXIBLE DISCRIMINANT ANALYSIS; FMRI; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN NEUROPHYSIOLOGY; LDA; LFP; LINEAR DISCRIMINANT ANALYSIS; LOCAL FIELD POTENTIAL; M1; PARIETAL REACH REGION; PERCEPT DECODING; PRIMARY MOTOR CORTEX; PRR; RAC/MOF; ROSTRAL ANTERIOR CINGULATE AND ADJACENT MEDIAL ORBITOFRONTAL CORTEX; SPEECH; STG; SUPERIOR TEMPORAL GYRUS; SUPPORT VECTOR MACHINE; SVM; V1, V2, V3, V3A, V3B, V4;

ANIMALS; AUDITORY PERCEPTION; BRAIN-COMPUTER INTERFACES; COGNITION; HUMANS; PSYCHOMOTOR PERFORMANCE; SPEECH; VISUAL PERCEPTION;

EID: 84892960017     PISSN: 09284257     EISSN: 17697115     Source Type: Journal    
DOI: 10.1016/j.jphysparis.2013.05.005     Document Type: Review

References (102)

1. Andersen R.A., Hwang E.J., Mulliken G.H. Cognitive neural prosthetics. Annu. Rev. Psychol. 2010, 61:169-190.
2. Birbaumer N. Brain-computer-interface research: coming of age. Clin. Neurophysiol. 2006, 117:479-483.
3. Birbaumer N., et al. A spelling device for the paralysed. Nature 1999, 398(6725):297-298.
4. Blatt G.J., Andersen R.A., Stoner G.R. Visual receptive field organization and cortico-cortical connections of the lateral intraparietal area (area LIP) in the macaque. J. Comp. Neurol. 1990, 299(4):421-445.
5. Brumberg J.S., Guenther F.H. Development of speech prostheses: current status and recent advances. Expert Rev. Med. Devices 2010, 7(5):667-679.
6. Brumberg J.S., et al. Brain-computer interfaces for speech communication. Speech Commun. 2010, 52(4):367-379.
7. Brumberg J.S., et al. Classification of intended phoneme production from chronic intracortical microelectrode recordings in speech-motor cortex. Frontiers Neurosci. 2011, 5:65.
8. Buchsbaum B.R., Hickok G., Humphries C. Role of left posterior superior temporal gyrus in phonological processing for speech perception and production. Cognitive Sci. 2001, 25(5):663-678.
9. Carmena J.M., et al. Learning to control a brain-machine interface for reaching and grasping by primates. PLoS Biol. 2003, 1(2):193-208.
10. Cavada C., Goldman-Raiuc P.S. Posterior parietal cortex in rhesus monkey: I. Parcellation of areas based on distinctive limbic and sensory corticocortical connections. J. Comp. Neurol. 1989, 287:393-421.
11. Cecotti H. Spelling with non-invasive brain-computer interfaces-current and future trends. J. Physiol., Paris 2011, 105(1-3):106-114.
12. Chang S.-E., et al. Common neural substrates support speech and non-speech vocal tract gestures. NeuroImage 2009, 47(1):314-325.
13. Clausen J. Moving minds: ethical aspects of neural motor prostheses. Biotechnol. J. 2008, 3(12):1493-1501.
14. David S.V., Gallant J.L. Predicting neuronal responses during natural vision. Network 2005, 16(2-3):239-260.
15. Fant G. Acoustic Theory of Speech Production 1970, Walter de Gruyter.
16. Fins J.J., et al. Ethical guidance for the management of conflicts of interest for researchers, engineers and clinicians engaged in the development of therapeutic deep brain stimulation. J. Neural. Eng. 2011, 8(3):033001.
17. Fitzsimmons N.A., et al. Extracting kinematic parameters for monkey bipedal walking from cortical neuronal ensemble activity. Front. Integr. Neurosci. 2009, 3:3.
18. Formisano E., et al. "Who" is saying "What"α Brain-based decoding of human voice and speech. Science 2008, 322(5903):970-973.
19. Funatsu E., et al. An artificial retina chip made of a 128×128 pn-np variable-sensitivity photodetector array. IEEE Photonics Technol. Lett. 1995, 7(2):188-190.
20. Gelbard-Sagiv H., et al. Internally generated reactivation of single neurons in human hippocampus during free recall. Science 2008, 322(5898):96-101.
21. Georgopoulos A.P., Schwartz A.B., Kettner R.E. Neuronal population coding of movement direction. Science 1986, 233(4771):1416-1419.
22. Ghosh S.S., Tourville J.A., Guenther F.H. A neuroimaging study of premotor lateralization and cerebellar involvement in the production of phonemes and syllables. J. Speech Lang. Hear. Res. 2008, 51(5):1183-1202.
23. Guenther F.H., et al. A wireless brain-machine interface for real-time speech synthesis. PloS One 2009, 4(12):e8218.
24. Guger C., et al. How many people are able to control a P300-based brain-computer interface (BCI)α. Neurosci. Lett. 2009, 462(1):94-98.
25. Halgren E., et al. Spatiotemporal stages in face and word-processing. 2. Depth-recorded potentials in the human frontal and rolandic cortices. J. Physiol. Paris 1994, 88(1):51-80.
26. Hampton A.N., O'Doherty J.P. Decoding the neural substrates of reward-related decision making with functional MRI. Proc. Nat. Acad. Sci. 2007, 104(4):1377-1382.
27. Hatsopoulos N.G., Donoghue J.P. The science of neural interface systems. Annu. Rev. Neurosci. 2009, 32(1):249-266.
28. Hatsopoulos N., Joshi J., O'Leary J.G. Decoding continuous and discrete motor behaviors using motor and premotor cortical ensembles. J. Neurophysiol. 2004, 92(2):1165-1174.
29. Haynes J.-D. Decoding and predicting intentions. Ann. N.Y. Acad. Sci. 2011, 1224(1):9-21.
30. Hickok G. The functional neuroanatomy of language. Phys. Life Rev. 2009, 6(3):121-143.
31. Hickok G., Poeppel D. The cortical organization of speech processing. Nat. Rev. Neurosci. 2007, 8(5):393-402.
32. Hickok G., Houde J., Rong F. Sensorimotor integration in speech processing: computational basis and neural organization. Neuron 2011, 69(3):407-422.
33. Hochberg L.R., et al. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 2006, 442(13):164-171.
34. Hochberg L.R., et al. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature 2012, 485(7398):372-375.
35. Hwang E.J., Andersen R.A. Brain-control of movement execution onset using LFPs in posterior parietal cortex. J. Neurosci. 2009, 29(45):14363-14370.
36. Hwang E.J., Andersen R.A. Spiking and LFP activity in PRR during symbolically instructed reaches. J. Neurophysiol. 2012, 107:836-849.
37. Jerbi K., et al. Watching brain TV and playing brain ball exploring novel BCI strategies using real-time analysis of human intracranial data. Int. Rev. Neurobiol. 2009, 86:159-168.
38. Kagan I., et al. Space representation for eye movements is more contralateral in monkeys than in humans. Proc. Nat. Acad. Sci. 2010, 107(17):7933-7938.
39. Kay K.N., et al. Identifying natural images from human brain activity. Nature 2008, 452(7185):352-355.
40. Kellis S., et al. Decoding spoken words using local field potentials recorded from the cortical surface. J. Neural Eng. 2010, 7(5):056007.
41. Kennedy P.R., Bakay R.A. Restoration of neural output from a paralyzed patient by a direct brain connection. NeuroReport 1998, 9(8):1707-1711.
42. Kennedy P.R., et al. Direct control of a computer from the human central nervous system. IEEE Trans. Rehab. Eng. 2000, 8(2):198-202.
43. Kim W.-C., et al. Smooth pursuit eye movement system using artificial retina chip and shape memory alloy actuator. IEEE Sens. J. 2005, 5(3):501-509.
44. Kokotilo K.J., Eng J.J., Curt A. Reorganization and preservation of motor control of the brain in spinal cord injury: a systematic review. J. Neurotrauma. 2009, 26(11):2113-2126.
45. Kravitz D.J., et al. A new neural framework for visuospatial processing. Nat. Rev. Neurosci. 2011, 12:217-230.
46. Kreiman G., Koch C., Fried I. Category-specific visual responses of single neurons in the human medial temporal lobe. Nat. Neurosci. 2000, 3(9):946-953.
47. Krusienski D.J., Shih J.J. Control of a visual keyboard using an electrocorticographic brain-computer interface. Neurorehab. Neur. Repair 2011, 25(4):323-331.
48. Kujala J., et al. Phase coupling in a cerebro-cerebellar network at 8-13Hz during reading. Cereb. Cortex 2007, 17(6):1476-1485.
49. Leuthardt E.C., et al. The emerging world of motor neuroprosthesis: a neurosurgical perspective. Neurosurgery 2006, 59(1):1-14.
50. Leuthardt E.C., et al. Using the electrocorticographic speech network to control a brain-computer interface in humans. J. Neural Eng. 2011, 8(3):036004.
51. Libet B., et al. Time of conscious intention to act in relation to onset of cerebral activity (readiness-potential) the unconscious initiation of a freely voluntary act. Brain 1983, 106(3):623-642.
52. Lindner A., et al. Human posterior parietal cortex plans where to reach and what to avoid. J. Neurosci. 2010, 30(35):11715-11725.
53. Lorach H., et al. Artificial retina: the multichannel processing of the mammalian retina achieved with a neuromorphic asynchronous light acquisition device. J. Neural Eng. 2012, 9(6):066004.
54. Mahmoudi B., Sanchez J.C. A Symbiotic brain-machine interface through value-based decision making. PLoS ONE 2011, 6(3).
55. Mahowald M.A., Mead C. A silicon model of early visual processing. Neural Network 1988, 1:91-97.
56. Markowitz D.A., et al. Optimizing the decoding of movement goals from local field potentials in Macaque cortex. J. Neurosci. 2011, 31(50):18412-18422.
57. Mesgarani N., Chang E.F. Selective cortical representation of attended speaker in multi-talker speech perception. Nature 2012, 485(7397):233-236.
58. Mesgarani N., et al. Influence of context and behavior on stimulus reconstruction from neural activity in primary auditory cortex. J. Neurophysiol. 2009, 102(6):3329-3339.
59. Mohr J.P. Broca's area and Broca's aphasia. Studies in Neurolinguistics 1976, 201-235. Academic Press, New York, London. H. Whitaker, H.A. Whitaker (Eds.).
60. Moran D. Evolution of brain-computer interface: action potentials, local field potentials and electrocorticograms. Curr. Opin. Neurobiol. 2010, 20(6):741-745.
61. Mukamel R., Fried I. Human intracranial recordings and cognitive neuroscience. Annu. Rev. Psychol. 2012, 63(1):511-537.
62. Naselaris T., et al. Bayesian reconstruction of natural images from human brain activity. Neuron 2009, 63(6):902-915.
63. Normann R.A., et al. Toward the development of a cortically based visual neuroprosthesis. J. Neural Eng. 2009, 6(3):035001.
64. Ojemann G., Mateer C. Human language cortex: localization of memory, syntax, and sequential motor-phoneme identification systems. Science 1979, 205(4413):1401-1403.
65. Ojemann G.A., et al. Neuronal activity in human lateral temporal cortex related to short-term verbal memory, naming and reading. Brain 1988, 111(Pt 6):1383-1403.
66. Paninski L., et al. Spatiotemporal tuning of motor cortical neurons for hand position and velocity. J. Neurophysiol. 2004, 91(1):515-532.
67. Pasley B.N., et al. Reconstructing speech from human auditory cortex. PLoS Biol. 2012, 10(1):e1001251.
68. Paus T. Primate anterior cingulate cortex: where motor control, drive and cognition interface. Nat. Rev. Neurosci. 2001, 2(6):417-424.
69. Paus T., et al. Role of the human anterior cingulate cortex in the control of oculomotor, manual, and speech responses: a positron emission tomography study. J. Neurophysiol. 1993, 70(2):453-469.
70. Peeva M.G., et al. Distinct representations of phonemes, syllables, and supra-syllabic sequences in the speech production network. NeuroImage 2010, 50(2):626-638.
71. Pei X., et al. Decoding vowels and consonants in spoken and imagined words using electrocorticographic signals in humans. J. Neural Eng. 2011, 8(4):046028.
72. Pei X., Hill J., Schalk G. Silent communication: toward using brain signals. IEEE Pulse 2012, 3(1):43-46.
73. Pereira F. Generating text from functional brain images. Frontiers Human Neurosci. 2011, 5:72.
74. Perez, O., Mukamel, R., Tankus, A., Yeshurun, Y., & Fried, I., submitted for publication. Preconscious Prediction of a Driver's Decision Using Intracranial Recordings. Nature Scientific Reports.
75. Posner J.B., et al. Plum and Posner's Diagnosis of Stupor and Coma 2007, Oxford University Press, USA. fourth ed.
76. Price C.J. The anatomy of language: a review of 100 fMRI studies published in 2009. Ann. N.Y. Acad. Sci. 2010, 1191(1):62-88.
77. Quian-Quiroga R., et al. Invariant visual representation by single neurons in the human brain. Nature 2005, 435(7045):1102-1107.
78. Ramirez A.D., et al. Incorporating naturalistic correlation structure improves spectrogram reconstruction from neuronal activity in the songbird auditory midbrain. J. Neurosci. 2011, 31(10):3828-3842.
79. Reutsky-Gefen I., et al. Holographic optogenetic stimulation of patterned neuronal activity for vision restoration. Nature Commun. 2013, 4(1509).
80. Riva G., Waterworth J.A., Waterworth E.L. The layers of presence: a bio-cultural approach to understanding presence in natural and mediated environments. CyberPsychol. Behav. 2004, 7(4):402-416.
81. Rushworth M.F.S., Behrens T.E.J., Johansen-Berg H. Connection patterns distinguish 3 regions of human parietal cortex. Cereb. Cortex 2006, 16(10):1418-1430.
82. Ryan D.B., et al. Predictive spelling with a P300-based brain-computer interface: Increasing the rate of communication. Int. J. Human-Comput. Inter. 2011, 27(1):69-84.
83. Sabri M., et al. Attentional and linguistic interactions in speech perception. NeuroImage 2008, 39(3):1444-1456.
84. Sahin N.T., et al. Sequential processing of lexical, grammatical, and phonological information within Broca's area. Science 2009, 326(5951):445-449.
85. Schwartz A.B. Cortical neural prosthetics. Annu. Rev. Neurosci. 2004, 27:487-507.
86. Shoham S., et al. Motor-cortical activity in tetraplegics. Nature 2001, 413(6858):793.
87. Shoham S., et al. Statistical encoding model for a primary motor cortical brain-machine interface. IEEE Trans. Biomed. Eng. 2005, 52(7):1312-1322.
88. Snyder L.H., Batista A.P., Andersen R.A. Coding of intention in the posterior parietal cortex. Nature 1997, 386(6621):167-170.
89. Sorger B., et al. A real-time fMRI-based spelling device immediately enabling robust motor-independent communication. Curr. Biol. 2012, 22(14):1333-1338.
90. Sörös P., et al. Clustered functional MRI of overt speech production. NeuroImage 2006, 32(1):376-387.
91. Stanley G.B., Li F.F., Dan Y. Reconstruction of natural scenes from ensemble responses in the lateral geniculate nucleus. J. Neurosci. 1999, 19(18):8036-8042.
92. Stark E., Abeles M. Predicting movement from multiunit activity. J. Neurosci. 2007, 27(31):8387-8394.
93. Tankus A., Fried I. Visuomotor coordination and motor representation by human temporal lobe neurons. J. Cogn. Neurosci. 2012, 24(3):600-610.
94. Tankus A., Fried I., Shoham S. Structured neuronal encoding and decoding of human speech features. Nature Commun. 2012, 3:1015.
95. Tankus A., Fried I., Shoham S. Sparse decoding of multiple spike trains for brain-machine interfaces. J. Neural Eng. 2012, 9(5):054001.
96. Taylor D.M., Tillery S.I.H., Schwartz A.B. Direct cortical control of 3D neuroprosthetic devices. Science 2002, 7(296):1829-1832.
97. Tian X., Poeppel D. Mental imagery of speech and movement implicates the dynamics of internal forward models. Frontiers Psychol. 2010, 1.
98. Velliste M., et al. Cortical control of a prosthetic arm for self-feeding. Nature 2008, 453:1098-1101.
99. Vlek R.J., et al. Sequenced subjective accents for brain-computer interfaces. J. Neural Eng. 2011, 8(3):036002.
100. Wise R.J., et al. Brain regions involved in articulation. Lancet 1999, 353(9158):1057-1061.
101. Yagi, T., et al., 1999. An artificial retina with adaptive mechanisms and its application to retinal prosthesis. In: International Conference on Systems, Man, and Cybernetics, 1999. IEEE SMC '99 Conference Proceedings, vol. 4. pp. 418-423.
102. Zander T.O., Kothe C. Towards passive brain-computer interfaces: applying brain-computer interface technology to human-machine systems in general. J. Neural Eng. 2011, 8(2):025005.