Neural and Computational Mechanisms of Action Processing: Interaction between Visual and Motor Representations

Neuron

Volumn 88, Issue 1, 2015, Pages 167-180

  Giese, Martin A ^a   Rizzolatti, Giacomo ^b,c  

^a UNIVERSITY OF TÜBINGEN   (Germany)

^b ISTITUTO ITALIANO DI TECNOLOGIA   (Italy)

^c UNIVERSITY OF PARMA   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL NEURAL NETWORK; BAYES THEOREM; BIOLOGICAL MODEL; BRAIN FUNCTION; FEASIBILITY STUDY; FUNCTIONAL ANATOMY; HAPLORHINI; HUMAN; MATHEMATICAL COMPUTING; MIRROR NEURON; MOTOR PERFORMANCE; NERVOUS SYSTEM ELECTROPHYSIOLOGY; NEUROANATOMY; NEUROPHYSIOLOGY; NONHUMAN; PARIETAL CORTEX; PREMOTOR CORTEX; PRIORITY JOURNAL; RECOGNITION; REVIEW; THEORETICAL MODEL; VISUAL SYSTEM FUNCTION; ANIMAL; BRAIN; COGNITION; MOVEMENT PERCEPTION; PHYSIOLOGY; VISION;

ANIMALS; BRAIN; COGNITION; HAPLORHINI; MIRROR NEURONS; MOTION PERCEPTION; VISUAL PERCEPTION;

EID: 84943249280     PISSN: 08966273     EISSN: 10974199     Source Type: Journal    
DOI: 10.1016/j.neuron.2015.09.040     Document Type: Review

References (118)

1. Abdollahi R.O., Jastorff J., Orban G.A. Common and segregated processing of observed actions in human SPL. Cereb. Cortex 2013, 23:2734-2753.
2. Ahissar M., Hochstein S. The reverse hierarchy theory of visual perceptual learning. Trends Cogn. Sci. 2004, 8:457-464.
3. Amari S. Dynamics of pattern formation in lateral-inhibition type neural fields. Biol. Cybern. 1977, 27:77-87.
4. Anselmi, F., Rosasco, L., Tan, C., and Poggio, T. (2015). Deep Convolutional Networks are Hierarchical Kernel Machines. arXiv preprint arXiv:1508.01084.
5. Arbib M.A. From monkey-like action recognition to human language: an evolutionary framework for neurolinguistics. Behav. Brain Sci. 2005, 28:105-124. discussion 125-167.
6. Barraclough N.E., Keith R.H., Xiao D., Oram M.W., Perrett D.I. Visual adaptation to goal-directed hand actions. J. Cogn. Neurosci. 2009, 21:1806-1820.
7. Bastos A.M., Usrey W.M., Adams R.A., Mangun G.R., Fries P., Friston K.J. Canonical microcircuits for predictive coding. Neuron 2012, 76:695-711.
8. Bonaiuto J., Arbib M.A. Extending the mirror neuron system model, II: what did I just do? A new role for mirror neurons. Biol. Cybern. 2010, 102:341-359.
9. Bonaiuto J., Rosta E., Arbib M. Extending the mirror neuron system model, I. Audible actions and invisible grasps. Biol. Cybern. 2007, 96:9-38.
10. Bonini L., Maranesi M., Livi A., Fogassi L., Rizzolatti G. Space-dependent representation of objects and other's action in monkey ventral premotor grasping neurons. J. Neurosci. 2014, 34:4108-4119.
11. Buccino G., Lui F., Canessa N., Patteri I., Lagravinese G., Benuzzi F., Porro C.A., Rizzolatti G. Neural circuits involved in the recognition of actions performed by nonconspecifics: an FMRI study. J. Cogn. Neurosci. 2004, 16:114-126.
12. Caggiano V., Fogassi L., Rizzolatti G., Casile A., Giese M.A., Thier P. Mirror neurons encode the subjective value of an observed action. Proc. Natl. Acad. Sci. USA 2012, 109:11848-11853.
13. Caggiano V., Fogassi L., Rizzolatti G., Pomper J.K., Thier P., Giese M.A., Casile A. View-based encoding of actions in mirror neurons of area f5 in macaque premotor cortex. Curr. Biol. 2011, 21:144-148.
14. Caggiano V., Fogassi L., Rizzolatti G., Thier P., Casile A. Mirror neurons differentially encode the peripersonal and extrapersonal space of monkeys. Science 2009, 324:403-406.
15. Caligiore D., Borghi A.M., Parisi D., Baldassarre G. TRoPICALS: a computational embodied neuroscience model of compatibility effects. Psychol. Rev. 2010, 117:1188-1228.
16. Caligiore D., Pezzulo G., Miall R.C., Baldassarre G. The contribution of brain sub-cortical loops in the expression and acquisition of action understanding abilities. Neurosci. Biobehav. Rev. 2013, 37:2504-2515.
17. Chersi F., Ferrari P.F., Fogassi L. Neuronal chains for actions in the parietal lobe: a computational model. PLoS ONE 2011, 6:e27652.
18. Cisek P., Kalaska J.F. Neural mechanisms for interacting with a world full of action choices. Annu. Rev. Neurosci. 2010, 33:269-298.
19. Cook R., Bird G., Catmur C., Press C., Heyes C. Mirror neurons: from origin to function. Behav. Brain Sci. 2014, 37:177-192.
20. Dautenhahn K., Nehaniv C.L. Like Me? - Measures of Correspondence and Imitation' Cybernetics and Systems. Cybern. Syst. 2001, 32:11-51.
21. Dayan P., Abbott L.F. Theoretical Neuroscience 2001, MIT Press, Cambridge, MA.
22. Deco G., Jirsa V.K., McIntosh A.R. Emerging concepts for the dynamical organization of resting-state activity in the brain. Nat. Rev. Neurosci. 2011, 12:43-56.
23. Demiris J., Hayes G.M. Imitation as a dual-route process featuring predictive and learning components: a biologically plausible computational model. Imitation in animals and artifacts 2002, 327-361. MIT Press, Cambridge, MA, USA. K. Dautenhahn, C.L. Nehaniv (Eds.).
24. Demiris Y., Khadhouri B. Hierarchical attentive multiple models for execution and recognition of actions. Robot. Auton. Syst. 2006, 54:361-369.
25. Dum R.P., Strick P.L. Premotor areas: nodal points for parallel efferent systems involved in the central control of movement. Motor Control: Concepts and Issues 1991, 383-397. Wiley, Chichester. D.R. Humphrey, H.J. Freund (Eds.).
26. Erlhagen W., Bastian A., Jancke D., Riehle A., Schöner G. The distribution of neuronal population activation (DPA) as a tool to study interaction and integration in cortical representations. J. Neurosci. Methods 1999, 94:53-66.
27. Erlhagen W., Mukovskiy A., Bicho E. A dynamic model for action understanding and goal-directed imitation. Brain Res. 2006, 1083:174-188.
28. Erlhagen W., Schöner G. Dynamic field theory of movement preparation. Psychol. Rev. 2002, 109:545-572.
29. Escobar M.-J., Kornprobst P. Action Recognition with a Biol.-inspired Feedforward Motion Processing Model: The Richness of Center-Surround Interactions. Computer Vision - Eccv 2008, Pt Iv, Proceedings 2008, 186-199. D. Forsyth, P. Torr, A. Zisserman (Eds.).
30. Fleischer F., Caggiano V., Thier P., Giese M.A. Physiologically inspired model for the visual recognition of transitive hand actions. J. Neurosci. 2013, 33:6563-6580.
31. Fleischer F., Christensen A., Caggiano V., Thier P., Giese M.A. Neural theory for the perception of causal actions. Psychol. Res. 2012, 76:476-493.
32. Fluet M.C., Baumann M.A., Scherberger H. Context-specific grasp movement representation in macaque ventral premotor cortex. J. Neurosci. 2010, 30:15175-15184.
33. Friston K. A theory of cortical responses. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2005, 360:815-836.
34. Friston K. Hierarchical models in the brain. PLoS Comput. Biol. 2008, 4:e1000211.
35. Friston K. The free-energy principle: a unified brain theory?. Nat. Rev. Neurosci. 2010, 11:127-138.
36. Friston K., Mattout J., Kilner J. Action understanding and active inference. Biol. Cybern. 2011, 104:137-160.
37. Friston K.J., Daunizeau J., Kilner J., Kiebel S.J. Action and behavior: a free-energy formulation. Biol. Cybern. 2010, 102:227-260.
38. Gallese V., Fadiga L., Fogassi L., Rizzolatti G. Action recognition in the premotor cortex. Brain 1996, 119:593-609.
39. Gallese V., Keysers C., Rizzolatti G. A unifying view of the basis of social cognition. Trends Cogn. Sci. 2004, 8:396-403.
40. Giese M. Dynamic Neural Field Theory for Motion Perception 1999, Kluwer, Dordrecht.
41. Giese M.A., Poggio T. Neural mechanisms for the recognition of biological movements. Nat. Rev. Neurosci. 2003, 4:179-192.
42. Grafton S.T., Hamilton A.F. Evidence for a distributed hierarchy of action representation in the brain. Hum. Mov. Sci. 2007, 26:590-616.
43. Haruno M., Wolpert D.M., Kawato M. Hierarchical MOSAIC for movement generation. Int. Congr. Ser. 2003, 1250:575-590.
44. Haruno M., Wolpert D.M., Kawato M. Mosaic model for sensorimotor learning and control. Neural Comput. 2001, 13:2201-2220.
45. Jhuang, H., Serre, T., Wolf, L., Poggio, T., and Ieee (2007). A biologically inspired system for action recognition. In 2007 Ieee 11th International Conference on Computer Vision, pp. 1253-1260.
46. Jung M., Hwang J., Tani J. Self-Organization of Spatio-Temporal Hierarchy via Learning of Dynamic Visual Image Patterns on Action Sequences. PLoS ONE 2015, 10:e0131214.
47. Karpathy, A., Johnson, J., and Li, F.-F. (2015). Visualizing and Understanding Recurrent Networks. arXiv arXiv:1506.02078.
48. Karpathy A., Toderici G., Shetty S., Leung T., Sukthankar R., Fei-Fei L. Large-Scale Video Classification with Convolutional Neural Networks. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition 2014, 1725-1732. IEEE Computer Society.
49. Keysers C. The Empathic Brain. How the Discovery of Mirror Neurons Changes Our Understanding of Human Nature 2011, Social Brain Press, Lexington, Ky.
50. Keysers C., Gazzola V. Integrating simulation and theory of mind: from self to social cognition. Trends Cogn. Sci. 2007, 11:194-196.
51. Keysers C., Perrett D.I. Demystifying social cognition: a Hebbian perspective. Trends Cogn. Sci. 2004, 8:501-507.
52. Kiebel S.J., Daunizeau J., Friston K.J. A hierarchy of time-scales and the brain. PLoS Comput. Biol. 2008, 4:e1000209.
53. Kiebel S.J., von Kriegstein K., Daunizeau J., Friston K.J. Recognizing sequences of sequences. PLoS Comput. Biol. 2009, 5:e1000464.
54. Kilner J.M. More than one pathway to action understanding. Trends Cogn. Sci. 2011, 15:352-357.
55. Kilner J.M., Friston K.J., Frith C.D. The mirror-neuron system: a Bayesian perspective. Neuroreport 2007, 18:619-623.
56. Kilner J.M., Lemon R.N. What we know currently about mirror neurons. Curr. Biol. 2013, 23:R1057-R1062.
57. Kohler E., Keysers C., Umiltà M.A., Fogassi L., Gallese V., Rizzolatti G. Hearing sounds, understanding actions: action representation in mirror neurons. Science 2002, 297:846-848.
58. Kraskov A., Dancause N., Quallo M.M., Shepherd S., Lemon R.N. Corticospinal neurons in macaque ventral premotor cortex with mirror properties: a potential mechanism for action suppression?. Neuron 2009, 64:922-930.
59. Kraskov A., Philipp R., Waldert S., Vigneswaran G., Quallo M.M., Lemon R.N. Philos. Trans. R. Soc. Lond. B. Biol. Sci. 2014, 369:20130174.
60. Lange J., Lappe M. A model of biological motion perception from configural form cues. J. Neurosci. 2006, 26:2894-2906.
61. Layher G., Giese M.A., Neumann H. Learning representations of animated motion sequences-a neural model. Top. Cogn. Sci. 2013, 6:170-182.
62. Le Q.V., Zou W.Y., Yeung S.Y., Ng A.Y. Learning hierarchical invariant spatio-temporal features for action recognition with independent subspace analysis. Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition 2011, 3361-3368. IEEE Computer Society.
63. LeCun Y., Bengio Y., Hinton G. Deep learning. Nature 2015, 521:436-444.
64. Maranesi M., Livi A., Fogassi L., Rizzolatti G., Bonini L. Mirror neuron activation prior to action observation in a predictable context. J. Neurosci. 2014, 34:14827-14832.
65. Marr D. Vision. A Computational Investigation into the Human Representation and Processing of Visual Information 1982, W. H. Freeman and Company, San Francisco, CA.
66. Maunsell J.H. Neuronal representations of cognitive state: reward or attention?. Trends Cogn. Sci. 2004, 8:261-265.
67. Metta G., Sandini G., Natale L., Craighero L., Fadiga L. Understanding mirror neurons - A bio-robotic approach. Interact. Stud. 2006, 7:197-232.
68. Miall R.C. Connecting mirror neurons and forward models. Neuroreport 2003, 14:2135-2137.
69. Moeslund T.B., Hilton A., Krüger V. A survey of advances in vision-based human motion capture and analysis. Comput. Vis. Image Underst. 2006, 104:90-126.
70. Mukamel R., Ekstrom A.D., Kaplan J., Iacoboni M., Fried I. Single-neuron responses in humans during execution and observation of actions. Curr. Biol. 2010, 20:750-756.
71. Murata A., Fadiga L., Fogassi L., Gallese V., Raos V., Rizzolatti G. Object representation in the ventral premotor cortex (area F5) of the monkey. J. Neurophysiol. 1997, 78:2226-2230.
72. Nelissen K., Borra E., Gerbella M., Rozzi S., Luppino G., Vanduffel W., Rizzolatti G., Orban G.A. Action observation circuits in the macaque monkey cortex. J. Neurosci. 2011, 31:3743-3756.
73. Oberman L.M., Ramachandran V.S. The simulating social mind: the role of the mirror neuron system and simulation in the social and communicative deficits of autism spectrum disorders. Psychol. Bull. 2007, 133:310-327.
74. Oram M.W., Perrett D.I. Integration of form and motion in the anterior superior temporal polysensory area (STPa) of the macaque monkey. J. Neurophysiol. 1996, 76:109-129.
75. Oztop E., Arbib M.A. Schema design and implementation of the grasp-related mirror neuron system. Biol. Cybern. 2002, 87:116-140.
76. Oztop E., Bradley N.S., Arbib M.A. Infant grasp learning: a computational model. Exp. Brain Res. 2004, 158:480-503.
77. Oztop E., Kawato M., Arbib M. Mirror neurons and imitation: a computationally guided review. Neural Networks 2006, 19:254-271.
78. Perrett D.I., Harries M.H., Bevan R., Thomas S., Benson P.J., Mistlin A.J., Chitty A.J., Hietanen J.K., Ortega J.E. Frameworks of analysis for the neural representation of animate objects and actions. J. Exp. Biol. 1989, 146:87-113.
79. Perrett D.I., Oram M.W. Neurophysiology of shape processing. Image Vis. Comput. 1993, 11:317-333.
80. Perrett D.I., Smith P.A., Potter D.D., Mistlin A.J., Head A.S., Milner A.D., Jeeves M.A. Visual cells in the temporal cortex sensitive to face view and gaze direction. Proc. R. Soc. Lond. B Biol. Sci. 1985, 223:293-317.
81. Petit J.-L. Constitution by movement: Husserl in light of recent neurobiological findings. Naturalizing Phenomenology 1999, 220-244. Stanford University Press, Stanford, CA. J. Petitot, F. Varela, B. Pachoud, J.M. Roy (Eds.).
82. Prevete R., Tessitore G., Santoro M., Catanzariti E. A connectionist architecture for view-independent grip-aperture computation. Brain Res. 2008, 1225:133-145.
83. Puce A., Perrett D. Electrophysiology and brain imaging of biological motion. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2003, 358:435-445.
84. Pulvermüller F. Brain mechanisms linking language and action. Nat. Rev. Neurosci. 2005, 6:576-582.
85. Riesenhuber M., Poggio T. Hierarchical models of object recognition in cortex. Nat. Neurosci. 1999, 2:1019-1025.
86. Rizzolatti G., Arbib M.A. Language within our grasp. Trends Neurosci. 1998, 21:188-194.
87. Rizzolatti G., Cattaneo L., Fabbri-Destro M., Rozzi S. Cortical mechanisms underlying the organization of goal-directed actions and mirror neuron-based action understanding. Physiol. Rev. 2014, 94:655-706.
88. Rizzolatti G., Craighero L. The mirror-neuron system. Annu. Rev. Neurosci. 2004, 27:169-192.
89. Rizzolatti G., Fadiga L., Gallese V., Fogassi L. Premotor cortex and the recognition of motor actions. Brain Res. Cogn. Brain Res. 1996, 3:131-141.
90. Rizzolatti G., Fogassi L. The mirror mechanism: recent findings and perspectives. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2014, 369:20130420.
91. Rizzolatti G., Fogassi L., Gallese V. Neurophysiological mechanisms underlying the understanding and imitation of action. Nat. Rev. Neurosci. 2001, 2:661-670.
92. Rizzolatti G., Sinigaglia C. Mirrors in the brain: how our minds share actions and emotions 2008, Oxford University Press, Oxford, New York.
93. Rizzolatti G., Sinigaglia C. The functional role of the parieto-frontal mirror circuit: interpretations and misinterpretations. Nat. Rev. Neurosci. 2010, 11:264-274.
94. Roesch M.R., Olson C.R. Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. J. Neurophysiol. 2003, 90:1766-1789.
95. Roesch M.R., Olson C.R. Neuronal activity related to anticipated reward in frontal cortex: does it represent value or reflect motivation?. Ann. N Y Acad. Sci. 2007, 1121:431-446.
96. Sauser E.L., Billard A.G. Parallel and distributed neural models of the ideomotor principle: an investigation of imitative cortical pathways. Neural Networks 2006, 19:285-298.
97. Schaal S., Ijspeert A., Billard A. Computational approaches to motor learning by imitation. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2003, 358:537-547.
98. Schindler K., Van Gool L., de Gelder B. Recognizing emotions expressed by body pose: a biologically inspired neural model. Neural Networks 2008, 21:1238-1246.
99. Schrodt F., Layher G., Neumann H., Butz M.V. Modeling Perspective-Taking upon Observation of 3D Biological Motion. Proceedings of the Joint IEEE International Conferences on Development and Learning and Epigenetic Robotics 2014, 305-310. IEEE, Los Alamitos.
100. Schütz-Bosbach S., Prinz W. Perceptual resonance: action-induced modulation of perception. Trends Cogn. Sci. 2007, 11:349-355.
101. Singer J.M., Sheinberg D.L. Temporal cortex neurons encode articulated actions as slow sequences of integrated poses. J. Neurosci. 2010, 30:3133-3145.
102. Sinigaglia C. What type of action understanding is subserved by mirror neurons?. Neurosci. Lett. 2013, 540:59-61.
103. Spaulding S. Mirror Neurons and Social Cognition. Mind Lang. 2013, 28:233-257.
104. Tani J., Ito M., Sugita Y. Self-organization of distributedly represented multiple behavior schemata in a mirror system: reviews of robot experiments using RNNPB. Neural Networks 2004, 17:1273-1289.
105. Tarr M.J., Bülthoff H.H. Image-based object recognition in man, monkey and machine. Cognition 1998, 67:1-20.
106. Tessitore G., Prevete R., Catanzariti E., Tamburrini G. From motor to sensory processing in mirror neuron computational modelling. Biol. Cybern. 2010, 103:471-485.
107. Ullman S. High-Level Vision 1996, MIT Press, Cambridge, MA.
108. Umiltà M.A., Kohler E., Gallese V., Fogassi L., Fadiga L., Keysers C., Rizzolatti G. I know what you are doing. a neurophysiological study. Neuron 2001, 31:155-165.
109. Vangeneugden J., De Mazière P.A., Van Hulle M.M., Jaeggli T., Van Gool L., Vogels R. Distinct mechanisms for coding of visual actions in macaque temporal cortex. J. Neurosci. 2011, 31:385-401.
110. Vangeneugden J., Pollick F., Vogels R. Functional differentiation of macaque visual temporal cortical neurons using a parametric action space. Cereb. Cortex 2009, 19:593-611.
111. Vigneswaran G., Philipp R., Lemon R.N., Kraskov A. M1 corticospinal mirror neurons and their role in movement suppression during action observation. Curr. Biol. 2013, 23:236-243.
112. Wilson H.R., Cowan J.D. Excitatory and inhibitory interactions in localized populations of model neurons. Biophys J. 1972, 12:1-24.
113. Wilson M., Knoblich G. The case for motor involvement in perceiving conspecifics. Psychol. Bull. 2005, 131:460-473.
114. Wolpert D.M., Doya K., Kawato M. A unifying computational framework for motor control and social interaction. Philos. Trans. R. Soc. Lond. B Biol. Sci. 2003, 358:593-602.
115. Yamashita Y., Tani J. Emergence of functional hierarchy in a multiple timescale neural network model: a humanoid robot experiment. PLoS Comput. Biol. 2008, 4:e1000220.
116. Yamins D.L., Hong H., Cadieu C.F., Solomon E.A., Seibert D., DiCarlo J.J. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc. Natl. Acad. Sci. USA 2014, 111:8619-8624.
117. Zeiler M., Fergus R. Visualizing and Understanding Convolutional Networks. Computer Vision - ECCV 2014, D 2014, 818-833. Springer International Publishing, Fleet. T. Pajdla, B. Schiele, T. Tuytelaars (Eds.).
118. Zhang K. Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: a theory. J. Neurosci. 1996, 16:2112-2126.