Observation-based learning for brain-machine interfaces

Current Opinion in Neurobiology

Volumn 18, Issue 6, 2008, Pages 589-594

  Tkach, Dennis ^a   Reimer, Jake ^a   Hatsopoulos, Nicholas G ^a,b  

^a UNIVERSITY OF CHICAGO   (United States)

^b UNIVERSITY OF CHICAGO   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BRAIN; BRAIN CORTEX; BRAIN NERVE CELL; COMPUTER INTERFACE; HUMAN; LEARNING ALGORITHM; MACHINE LEARNING; MOTOR CORTEX; MOTOR SYSTEM; OBSERVATIONAL STUDY; PREMOTOR CORTEX; PRIORITY JOURNAL; REVIEW; SENSORY NERVE;

ANIMALS; BRAIN; HUMANS; LEARNING; MOTOR CORTEX; NEURONS;

EID: 67349222518     PISSN: 09594388     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.conb.2008.09.016     Document Type: Review

References (54)

1. di Pellegrino G., Fadiga L., Fogassi L., Gallese V., and Rizzolatti G. Understanding motor events: a neurophysiological study. Exp Brain Res 91 (1992) 176-180. Discovery and the first account of mirror neurons in the monkey PMv. Single unit recordings were made during monkey's reaching and grasping movements toward food or nonfood objects. Responses were accidentally observed in some of the neurons when experimenters were moving the objects around in the experimental setup and the monkey remained motionless. When analyzed, the activity of the same neurons closely resembled the activity when the monkey performed the observed reach and grasp movements.
2. Blakemore S.J., and Decety J. From the perception of action to the understanding of intention. Nat Rev Neurosci 2 (2001) 561-567
3. Umilta M.A., Kohler E., Gallese V., Fogassi L., Fadiga L., Keysers C., and Rizzolatti G. I know what you are doing. A neurophysiological study. Neuron 31 (2001) 155-165. This study provides evidence that mirror neurons encode the overall goal, or intention of the observed movements. Monkeys performed reach and grasp movements and then observed experimenters perform the same actions while the monkeys remained motionless. Single unit recordings were made from neurons in the ventral premotor cortex of the monkeys. The mirror-like response was found to persist in a population of the recorded neurons even when the target object was occluded from view during the observed reach motion.
4. Fogassi L., Ferrari P.F., Gesierich B., Rozzi S., Chersi F., and Rizzolatti G. Parietal lobe: from action organization to intention understanding. Science 308 (2005) 662-667. This study shows that some parietal mirror neurons discriminate between observed actions. Monkeys were trained to reach for and grasp either an object or a piece of food in an identical fashion. If the object was food, then the monkeys were allowed to eat the piece of food, otherwise they placed the nonfood object into a container. The monkeys then observed human experimenters perform the same actions. One-third of the recorded parietal mirror neurons fired similarly for both food-directed and object-directed movements, while the remaining two-thirds discharged differently for food-directed grasps.
5. Gazzola V., Aziz-Zadeh L., and Keysers C. Empathy and the somatotopic auditory mirror system in humans. Curr Biol 16 (2006) 1824-1829
6. Schurmann M., Jarvelainen J., Avikainen S., Cannon T.D., Lonnqvist J., Huttunen M., and Hari R. Manifest disease and motor cortex reactivity in twins discordant for schizophrenia. Br J Psychiatry 191 (2007) 178-179
7. Buccino G., and Amore M. Mirror neurons and the understanding of behavioural symptoms in psychiatric disorders. Curr Opin Psychiatry 21 (2008) 281-285
8. Enticott P.G., Hoy K.E., Herring S.E., Johnston P.J., Daskalakis Z.J., and Fitzgerald P.B. Reduced motor facilitation during action observation in schizophrenia: a mirror neuron deficit?. Schizophr Res 102 (2008) 116-121
9. Keysers C., Kohler E., Umilta M.A., Nanetti L., Fogassi L., and Gallese V. Audiovisual mirror neurons and action recognition. Exp Brain Res 153 (2003) 628-636
10. Rizzolatti G., and Craighero L. The mirror-neuron system. Annu Rev Neurosci 27 (2004) 169-192
11. Iacoboni M., and Dapretto M. The mirror neuron system and the consequences of its dysfunction. Nat Rev Neurosci 7 (2006) 942-951
12. Iacoboni M., and Mazziotta J.C. Mirror neuron system: basic findings and clinical applications. Ann Neurol 62 (2007) 213-218
13. Fabbri-Destro M., and Rizzolatti G. Mirror neurons and mirror systems in monkeys and humans. Physiology (Bethesda) 23 (2008) 171-179. A good, concise and recent review of mirror-neuron studies done in monkeys as well as in humans. This study expands the notion of a mirror neuron as a passive resonator of observed movements to consider the role of the mirror neuron system in speech learning perception of emotion.
14. Ferrari P.F., Gallese V., Rizzolatti G., and Fogassi L. Mirror neurons responding to the observation of ingestive and communicative mouth actions in the monkey ventral premotor cortex. Eur J Neurosci 17 (2003) 1703-1714
15. Gallese V., Fadiga L., Fogassi L., and Rizzolatti G. Action recognition in the premotor cortex. Brain 119 Pt 2 (1996) 593-609
16. Rizzolatti G., Fadiga L., Gallese V., and Fogassi L. Premotor cortex and the recognition of motor actions. Brain Res Cogn Brain Res 3 (1996) 131-141
17. Perrett D.I., Harries M.H., Bevan R., Thomas S., Benson P.J., Mistlin A.J., Chitty A.J., Hietanen J.K., and Ortega J.E. Frameworks of analysis for the neural representation of animate objects and actions. J Exp Biol 146 (1989) 87-113
18. Jellema T., Baker C.I., Wicker B., and Perrett D.I. Neural representation for the perception of the intentionality of actions. Brain Cogn 44 (2000) 280-302
19. Kohler E., Keysers C., Umilta M.A., Fogassi L., Gallese V., and Rizzolatti G. Hearing sounds, understanding actions: action representation in mirror neurons. Science 297 (2002) 846-848
20. Kurata K., and Tanji J. Premotor cortex neurons in macaques: activity before distal and proximal forelimb movements. J Neurosci 6 (1986) 403-411
21. Rizzolatti G., Camarda R., Fogassi L., Gentilucci M., Luppino G., and Matelli M. Functional organization of inferior area 6 in the macaque monkey. II. Area F5 and the control of distal movements. Exp Brain Res 71 (1988) 491-507
22. Jeannerod M., Arbib M.A., Rizzolatti G., and Sakata H. Grasping objects: the cortical mechanisms of visuomotor transformation. Trends Neurosci 18 (1995) 314-320
23. Murata A., Gallese V., Luppino G., Kaseda M., and Sakata H. Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP. J Neurophysiol 83 (2000) 2580-2601
24. Iacoboni M., Molnar-Szakacs I., Gallese V., Buccino G., Mazziotta J.C., and Rizzolatti G. Grasping the intentions of others with one's own mirror neuron system. PLoS Biol 3 (2005) e79
25. Raos V., Umilta M.A., Murata A., Fogassi L., and Gallese V. Functional properties of grasping-related neurons in the ventral premotor area F5 of the macaque monkey. J Neurophysiol 95 (2006) 709-729
26. Umilta M.A., Brochier T., Spinks R.L., and Lemon R.N. Simultaneous recording of macaque premotor and primary motor cortex neuronal populations reveals different functional contributions to visuomotor grasp. J Neurophysiol 98 (2007) 488-501
27. Graziano M.S.A., Yap G.S., and Gross C.G. Coding of visual space by premotor neurons. Science 266 (1994) 1054-1057
28. Graziano M.S., Hu X.T., and Gross C.G. Visuospatial properties of ventral premotor cortex. J Neurophysiol 77 (1997) 2268-2292
29. Graziano M.S. Where is my arm? The relative role of vision and proprioception in the neuronal representation of limb position. Proc Natl Acad Sci U S A 96 (1999) 10418-10421
30. Cisek P., and Kalaska J.F. Neural correlates of mental rehearsal in dorsal premotor cortex. Nature 431 (2004) 993-996. Mirror-like responses are recorded on a single-cell level in PMd. Monkeys perform a visual reach task and then observe experimenters perform the same task. Neural analysis shows that same cells are active in the same way during action execution, action observation, and before the onset of observed action. It is suggested that such PMd neurons are involved in a covert mental-rehearsal.
31. Tkach D., Reimer J., and Hatsopoulos N.G. Congruent activity during action and action observation in motor cortex. J Neurosci 27 (2007) 13241-13250. Mirror-like responses are shown to occur on a single-cell level in MI. Monkeys perform a random target pursuit task and then observe video playback of the same task. Neural analysis shows that same cells are active in the same way during action execution and action observation. Components of the visual scene during playback are varied to determine which elicits the observed mirror response. Consistent with the definition of a mirror neuron, the presence of both the transitive movement and the goal is necessary to invoke the most congruent response.
32. Aziz-Zadeh L., Maeda F., Zaidel E., Mazziotta J., and Iacoboni M. Lateralization in motor facilitation during action observation: a TMS study. Exp Brain Res 144 (2002) 127-131
33. Shmuelof L., and Zohary E. A mirror representation of others' actions in the human anterior parietal cortex. J Neurosci 26 (2006) 9736-9742
34. Touzalin-Chretien P., and Dufour A. Motor cortex activation induced by a mirror: evidence from lateralized readiness potentials. J Neurophysiol 100 (2008) 19-23
35. Wolpert D.M., Ghahramani Z., and Jordan M.I. An internal model for sensorimotor integration. Science 269 (1995) 1880-1882
36. Desmurget M., and Grafton S. Forward modeling allows feedback control for fast reaching movements. Trends Cogn Sci 4 (2000) 423-431
37. Kilner J.M., Friston K.J., and Frith C.D. Predictive coding: an account of the mirror neuron system. Cogn Process 8 (2007) 159-166. Authors present a theoretical model based on a predictive coding framework to describe how the mirror neuron system may facilitate the understanding of observed movements. The model is based on the idea of the inverse utilization of the sensory state prediction model utilized during movement execution. The authors cite some anatomical and experimental evidence for their model.
38. Warland D.K., Rienagel P., and Meister M. Decoding visual information from a population of retinal ganglion cells. J Neurophysiol 78 (1997) 2336-2350
39. Schwartz A.B., Taylor D.M., and Tillery S.I. Extraction algorithms for cortical control of arm prosthetics. Curr Opin Neurobiol 11 (2001) 701-707
40. Taylor D.M., Tillery S.I., and Schwartz A.B. Direct cortical control of 3D neuroprosthetic devices. Science 296 (2002) 1829-1832
41. Paninski L., Fellows M.R., Hatsopoulos N.G., and Donoghue J.P. Spatiotemporal tuning of motor cortical neurons for hand position and velocity. J Neurophysiol 91 (2004) 515-532
42. Roth M., Decety J., Raybaudi M., Massarelli R., Delon-Martin C., Segebarth C., Morand S., Gemignani A., Decorps M., and Jeannerod M. Possible involvement of primary motor cortex in mentally simulated movement: a functional magnetic resonance imaging study. Neuroreport 7 (1996) 1280-1284
43. Carrillo-de-la-Pena M.T., Galdo-Alvarez S., and Lastra-Barreira C. Equivalent is not equal: primary motor cortex (MI) activation during motor imagery and execution of sequential movements. Brain Res 1226 (2008) 134-143
44. Hochberg L.R., Serruya M.D., Friehs G.M., Mukand J.A., Saleh M., Caplan A.H., Branner A., Chen D., Penn R.D., and Donoghue J.P. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442 (2006) 164-171
45. Solodkin A., Hlustik P., Chen E.E., and Small S.L. Fine modulation in network activation during motor execution and motor imagery. Cereb Cortex 14 (2004) 1246-1255
46. Lotze M., and Halsband U. Motor imagery. J Physiol Paris 99 (2006) 386-395
47. Cheng Y., Meltzoff A.N., and Decety J. Motivation modulates the activity of the human mirror-neuron system. Cereb Cortex 17 (2007) 1979-1986
48. Wahnoun R., Tillery S.I., and He J. Neuron selection and visual training for population vector based cortical control. Conf Proc IEEE Eng Med Biol Soc 6 (2004) 4607-4610. First account of utilizing the mirror-neuron system to train a BMI. An adaptive decoding algorithm is used to decode the position of a video cursor in a 3D virtual space. The authors demonstrate that the decoding algorithm can be initialized using neural activity parameters acquired from when the monkeys motionlessly observed computer-generated movements on a monitor in front of them.
49. Wahnoun R., He J., and Helms Tillery S.I. Selection and parameterization of cortical neurons for neuroprosthetic control. J Neural Eng 3 (2006) 162-171
50. Velliste M., Perel S., Spalding M.C., Whitford A.S., and Schwartz A.B. Cortical control of a prosthetic arm for self-feeding. Nature 453 (2008) 1098-1101
51. Wolpaw J.R., Birbaumer N., McFarland D.J., Pfurtscheller G., and Vaughan T.M. Brain-computer interfaces for communication and control. Clin Neurophysiol 113 (2002) 767-791
52. Kilner J.M., Vargas C., Duval S., Blakemore S.J., and Sirigu A. Motor activation prior to observation of a predicted movement. Nat Neurosci 7 (2004) 1299-1301
53. Kemere C., Shenoy K.V., and Meng T.H. Model-based neural decoding of reaching movements: a maximum likelihood approach. IEEE Trans Biomed Eng 51 (2004) 925-932
54. Musallam S., Corneil B.D., Greger B., Scherberger H., and Andersen R.A. Cognitive control signals for neural prosthetics. Science 305 (2004) 258-262