Cognitive neuroscience: Neural mechanisms for the recognition of biological movements

Nature Reviews Neuroscience

Volumn 4, Issue 3, 2003, Pages 179-192

  Giese, Martin A ^a   Poggio, Tomaso ^b  

^a UNIVERSITY HOSPITAL TÜBINGEN   (Germany)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BODY MOVEMENT; BRAIN CORTEX; COMPUTER MODEL; HUMAN; IMITATION; LEARNING; MOTOR ACTIVITY; MOVEMENT PERCEPTION; NEUROPHYSIOLOGY; NONHUMAN; NONVERBAL COMMUNICATION; NUCLEAR MAGNETIC RESONANCE IMAGING; PRIORITY JOURNAL; REVIEW; VISUAL MEMORY; VISUAL SYSTEM; ANIMAL; BIOLOGICAL MODEL; PATTERN RECOGNITION; PHYSIOLOGY; VISUAL CORTEX;

ANIMALS; HUMANS; MODELS, NEUROLOGICAL; MOTION PERCEPTION; PATTERN RECOGNITION, VISUAL; VISUAL CORTEX;

EID: 0037364522     PISSN: 1471003X     EISSN: 14710048     Source Type: Journal    
DOI: 10.1038/nrn1057     Document Type: Article

References (142)

1. Ewert, J.-P. Neuroethology of releasing mechanisms: Prey- catching in toads. Behav. Brain Sci. 10, 337-405 (1987).
2. Morris, D. The reproductive behaviour of the zebra finch (Poephilla guttata), with special reference to pseudofemale behaviour. Behaviour 6, 271-322 (1954).
3. Darwi n, C. The Expressions of the Emotions in Man and Animals (Appleton, New York, 1872).
4. Andrew, R. J. Evolution of facial expression. Science 142, 1034-1041 (1963).
5. Adolphs, R. The neurobiology of social cognition. Curr. Opin. Neurobioi. 11, 231-239 (2001).
6. Johansson, G. Visual perception of biological motion and a model for its analysis. Percept. Psychophys. 14, 201-211 (1973).
7. Bruce, C., Desimone, R. Gross, C. G. Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. J. Neurophysioi. 46, 369-384 (1981).
8. Perrett, D. I. et al. Visual analysis of body movements by neurones in the temporal cortex in the macaque monkey: a preliminary report. Behav. Brain Res. 16, 153-170 (1985).
9. Perrett, D. I. et al. in AI and the Eye (eds Blake, A. Troscianko, T.) 181-200 (John Wiley, Chichester, 1990).
10. Oram, M. W. Perrett, D. I. Responses of anterior superior temporal polysensory (STPa) neurons to 'biological motion' stimuli. J. Cogn. Neurosci. 6, 99-116 (1994).
11. Jellema, T, Baker, C. I., Wicker, B. Perrett, D. I. Neural representation for the perception of the intentionality of actions. Brain Cogn. 44, 280-302 (2000).
12. Perrett, D. I. et al. Frameworks of analysis for the neural representation of animate objects and actions. Neural representation for the perception of the intentionality of actions. J. Exp. Biol. 146, 87-113 (1989).
13. Rizzolatti, G., Fogassi, L. Gallese, V. Neurophysiological mechanisms underlying the understanding and imitation of action. Nature Rev. Neurosci. 2, 661-670 (2001).
14. Gallese, V, Fadiga, L., Fogassi, L. Rizzolatti, G. Action recognition in the premotor cortex. Brain 119, 593-609 (1996).
15. Rizzolatti, G. Arbib, M. A. Language within our grasp. Trends Neurosci. 21, 188-194 (1998).
16. Kohler, E et al. Hearing sounds, understanding actions: action representation in mirror neurons. Science 297, 846-848 (2002).
17. Decety, J. Grezes, J. Neural mechanisms subserving the perception of human actions. Trends Cogn. Sci. 3, 172-178 (1999).
18. Allison, T., Puce, A. McCarthy, G. Social perception from visual cues: role of the STS region. Trends Cogn. Sci. 4, 267-278 (2000).
19. Grossman, E. D. Blake, R. Brain areas active during visual perception of biological motion. Neuron 35, 1167-1175 (2002).
20. Bonda, E. et al. Specific involvement of human parietal systems and the amygdala in the perception of biological motion J. Neurosci. 16, 3737-3744 (1996).
21. Howard, R. J. et al. A direct demonstration of functional specialization within motion-related visual and auditory cortex of the human brain. Curr. Bioi. 6, 1015-1019 (1996).
22. Grossman, E, et al. Brain areas involved in perception of biological motion. J. Cogn. Neurosci. 12, 711-720 (2000).
23. Vaina, L. M. et al. Functional neuroanatomy of biological motion perception in humans. Proc. NatiAcad. Sci USA 98, 11656-11661 (2001).
24. Iacoboni, M. et al. Cortical mechanisms of human imitation. Science 286, 2526-2528 (1999).
25. Buccino, G. et al. Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study. Eur. J. Neurosci. 13, 400-404 (2001).
26. Servos, P., Osu, R., Santi, A. Kawato, M. The neural substrates of biological motion perception: an fMRI study. Cereb. Cortex 12, 772-782 (2002).
27. Mather, G., Radford, K. West, S. Low-level visual processing of biological motion. Proc. R. Soc. Lond. B 249, 149-155 (1992).
28. Bobick, A. F. Movement, activity, and action: the role of knowledge in the perception of motion. Phii. Trans. R. Soc. Lond. B 352, 1257-1265 (1997).
29. Todd, J. T. Perception of gait. J. Exp. Psychoi. Hum. Percept. Perform. 9, 31-42 (1993).
30. Beintema, J. A. Lappe, M. Perception of biological motion without local image motion. Poc. Nati Acad. Sci. USA 99, 5661-5663 (2002).
31. Downing, P, Jiang, Y., Shuman, M. Kanwisher, N. A cortical area selective for visual processing of the human body. Science 293, 2470-2473 (2001).
32. Pollick, F. E., Fidopiastis, C. Braden, V. Recognizing the style of spatially exaggerated tennis serves. Perception 30, 323-338 (2001).
33. Hill, H. Johnston, A. Categorizing sex and identity from the biological motion of faces. Curr. Bioi. 11, 880-885 (2001).
34. Tarr, M. J. Bulthoff, H. H. Image-based object recognition in man, monkey and machine. Cognition 67, 1-20 (1998).
35. Poggio, T. Edelman, S. A network that learns to recognize three-dimensional objects. Nature 343, 263-266 (1990).
36. Logothetis, N. K., Pauls, J. Poggio, T. Shape representation in the inferior temporal cortex of monkeys. Curr. Bioi. 5, 552-563 (1995).
37. Ungerleider, L. G. Mishkin, M. in Anaiysis of Visuai Behavior (eds Ingle, D. J., Goodale, M. A. Mansfield, R. J. W.) 549-586 (MIT Press, Cambridge, Massachusetts, 1982).
38. Felleman, D. J. van Essen, D. C. Distributed hierarchical processing in the primate visual cortex. Cereb. Cortex 1, 1-47 (1991).
39. Milner, A. D. Goodale, M. A. Visual pathways to perception and action. Prog. Brain Res. 95, 317-337 (1993).
40. Thorpe, S., Fize, D. Marlot, C. Speed of processing in the human visual system. Nature 381, 520-522 (1996).
41. Oram, M. W. Perrett, D. I. Integration of form and motion in the anterior superior temporal polysensory area (STPa) of the macaque monkey. J. Neurophysioi. 76, 109-129 (1996).
42. Johansson, G. Spatio-temporal differentiation and integration in visual motion perception. Psychoi. Res. 38, 379-393 (1976).
43. Riesenhuber, M. Poggio, T. Hierarchical models of object recognition. Nature Neurosci. 2, 1019-1025 (1999).
44. Riesenhuber, M. Poggio, T. Models of object recognition. Nature Neurosci. 3, Suppl. 1199-1204 (2000).
45. Riesenhuber, M. Poggio, T. Neural mechanisms of object recognition. Curr. Opin. Neurobioi. 12, 162-168 (2002).
46. Kourtzi, Z. Kanwisher, N. Activation in human MT/MST by static images with implied motion. J. Cogn. Neurosci. 12, 48-55 (2000).
47. Saleem, K. S., Suzuki, W., Tanaka, K. Hashikawa, T. Connections between anterior inferotemporal cortex and superior temporal sulcus regions in the macaque monkey. J. Neurosci. 20, 5083-5101 (2000).
48. Giese, M. A. Vaina, L. M. Pathways in the analysis of biological motion: computational model and fMRI results. Perception 30, Suppl. 116 (2001).
49. Giese, M. A. Poggio, T. Neural mechanisms for the recognition of biological movements and actions. AI Memo 2002-012/CBCL Paper 219 (Massachusetts Institute of Technology, Cambridge, Massachusetts, 2002).
50. Perrett, D. I. Oram, M. W. Neurophysiology of shape processing. Image Vis. Comput. 11, 317-333 (1993).
51. Rolls, E. T. Milward, T. A model of invariant object recognition in the visual system: learning rules, activation functions, lateral inhibition, and information-based performance measures. Neurai Comput. 12, 2547-2572 (2000).
52. Hubel, D. H. Wiesel, T. N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physioi. (Lond.) 160, 106-154 (1962).
53. Jones, J. P. Palmer, L. A. An evaluation of the two- dimensional Gabor filter model of simple receptive fields in cat striate cortex. J. Neurophys. 58, 1233-1258 (1987).
54. Dow, B. M., Snyder, A. Z., Vautin, R. G. Bauer, R. Magnification factor and receptive field size in foveal striate cortex of the monkey. Exp. Brain Res. 44, 213-228 (1981).
55. Hegde, J. van Essen, D. C. Selectivity for complex shapes in primate visual area V2. J. Neurosci. 20, RC61 (2000).
56. Gallant, J. L., Braun, J. van Essen, D. C. Selectivity for polar, hyperbolic, and Cartesian gratings in macaque visual cortex. Science 259, 100-103 (1993).
57. Pinon, M. C., Gattass, R. Sousa, A. P. Area V4 in Cebus monkey: extent and visuotopic organization. Cereb. Cortex 8, 685-701 (1998).
58. Pasupathy, A. Connor, C. E. Response to contour features in macaque area V4. J. Neurophysioi. 82, 2490-2502 (1999).
59. Hubel, D. Wiesel, T. Ferrier lecture. Functional architecture of the macaque monkey visual cortex. Proc. R. Soc. Lond. B 198, 1-59 (1977).
60. Fukushima, K. Neocognitron: a self-organizing neural network model for a mechanisms of pattern recognition unaffected by shift in position. Bioi. Cybern. 36, 193-202 (1980).
61. Mel, B. W. Fieser, J. Minimizing binding errors using learned conjunctive features. Neurai Comp. 12, 731-762 (2000).
62. Lampl, I., Riesenhuber, M., Poggio, T. Ferster, D. The max operation in cells in the cat visual cortex. Soc. Neurosci. Abstr. 30, 619 (2001).
63. Gawne, T. J. Martin, J. Response of primate visual cortical V4 neurons to two simultaneously presented stimuli. J. Neurophysioi. 88, 1128-1135 (2002).
64. Logothetis, N. K. Sheinberg, D. L. Visual object recognition. Annu. Rev. Neurosci. 19, 577-621 (1996).
65. Tanaka, K. Inferotemporal cortex and object vision. Annu. Rev. Neurosci. 19, 109-139 (1996).
66. Young, M. Yamane, S. Sparse population coding of faces in the inferotemporal cortex. Science 256, 1327-1331 (1992).
67. Rolls, E. T. Tovee, M. J. Sparseness of the neuronal representation of stimuli in the primate temporal visual cortex. Exp. Brain Res. 73, 713-726 (1995).
68. Saito, H. in Brain Mechanisms of Perception and Memory (eds Ono, T., Squire, L. R., Raichle, M. E., Perrett, D. I. Fukuda, M.) 121-140 (Oxford Univ. Press, Oxford, UK, 1993).
69. Smith, A. T. Snowden, R. J. Visuai Detection of Motion (Academic, London, UK, 1994).
70. Rodman, H. R. Albright, T. D. Single-unit analysis of pattern-motion selective properties in the middle temporal visual area (MT). Exp. Brain Res. 75, 53-64 (1989).
71. Poggio, T. Reichardt, W. Visual control of orientation behavior in the fly. Part II: towards the underlying neural interactions. Quart. Rev. Biophys. 9, 377-438 (1976).
72. Heeger, D. J. Optical flow using spatiotemporal filters. Int. J. Comput. Vis. 1, 279-302 (1988).
73. Grossberg, S., Mingolla, E. Viswanathan, L. Neural dynamics of motion integration and segmentation within and across apertures. Vision. Res. 41, 2521-2553 (2001).
74. Sereno, M. E. Neurai Computation of Pattern Motion (MIT Press, Cambridge, Massachusetts, 1993).
75. Nowlan, S. J. Sejnowski, T. J. A selection model for motion processing in area MT of primates. J. Neurosci. 15, 1195-1214 (1995).
76. Simoncelli, E. P. Heeger, D. J. A model of neuronal responses in visual area MT. Vision. Res. 38, 743-761 (1998).
77. Albright, T. D. Desimone, R. Local precision of visuotopic organization in the middle temporal area (MT) of the macaque. Exp. Brain Res. 65, 582-592 (1987).
78. Lagae, L., Raiguel, S. Orban, G. A. Speed and direction selectivity of macaque middle temporal neurons. J. Neurophysioi. 69, 19-39 (1993).
79. Allman, J., Miezin, F., McGuinness, E. Direction- and velocity-specific responses from beyond the classical receptive field in the middle temporal visual area (MT). Perception 14, 105-126 (1985).
80. Xiao, D. K. et al. Spatial heterogeneity of inhibitory surrounds in the middle temporal visual area. Proc. Nati Acad. Sci. USA 92, 11303-11306 (1995).
81. Born, R. T. Center-surround interactions in the middle temporal visual area of the owl monkey. J. Neurophysioi. 84, 2658-2669 (2000).
82. Tanaka, K., Fukuda, Y. Saito, H. Analysis of motion of the visual field by direction, expansion/contraction, and rotation cells clustered in the dorsal part of the medial superior temporal area of the macaque monkey. J. Neurophysioi. 62, 626-641 (1989).
83. Eifuku, S. Wurtz, R. H. Response to motion in extrastiate area MSTl: center-surround interactions. J. Neurophysioi. 80, 282-296 (1998).
84. Orban, G. et al. A motion area in human visual cortex. Proc. Natl Acad. Sci. USA 92, 993-997 (1995).
85. Dupont, P. et al. The kinetic occipital region in human visual cortex. Cereb. Cortex 7, 283-292 (1997).
86. Graziano, M. A., Andersen, R. A. Snowden, R. J. Tuning of MST neurons to spiral motions. J. Neurosci. 14, 54-67 (1994).
87. Geesaman, B. J. Andersen, R. A. The analysis of complex motion patterns by form/cue invariant MSTd neurons. J. Neurosci. 16, 4716-4732 (1996).
88. Xie, X. Giese, M. A. Nonlinear dynamics of direction- selective nonlinear neural media. Phys. Rev. E Stat. Noniin. Soft Matter Phys. 65, 051904 (2002).
89. Markram, H., Lubke, J., Frotscher, M. Sakmann, B. Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275, 213-215 (1997).
90. Sejnowski, T. J. The book of Hebb. Neuron 24, 773-776 (1999).
91. Giese, M. A., Jastorff, J. Kourtzi, Z. Learning of the discrimination of artificial complex biological motion. Perception 31, Suppl. 117 (2002).
92. Mineiro, P. Zipser, D. Analysis of direction selectivity arising from recurrent cortical interactions. Neurai Comput. 10, 353-371 (1998).
93. Miyashita, Y. Chang, H. S. Neural correlate of pictorial short-term memory in the primate temporal cortex. Nature 331, 68-70 (1988).
94. Stryker, M. P. Neurobiology. Temporal associations. Nature 354, 108-109 (1991).
95. Koch, C., Poggio, T. Torre, V. Nonlinear interactions in a dendritic tree: localization, timing, and role in information processing. Proc. Nati Acad. Sci. USA 80, 2799-2802 (1983).
96. Dittrich, W. H. Action categories and the perception of biological motion. Perception 22, 15-22 (1993).
97. Kozlowski, L. T. Cutting, J. E. Recognizing the sex of a walker from a dynamic point light display. Percept. Psychophys. 21, 575-580 (1977).
98. Cutting, J. E. Kozlowski, L. T. Recognizing friends by their walk: gait perception without familiarity cues. Buii. Psychon. Soc. 9, 353-356 (1977).
99. Dittrich, W. H., Troscianko, T., Lea, S. E. Morgan, D. Perception of emotion from dynamic point-light displays represented in dance. Perception 25, 727-738 (1996).
100. Pollick, F. E., Paterson, H. M., Bruderlin, A. Sanford, A. J. Perceiving affect from arm movement. Cognition 82, B51-B61 (2001).
101. Giese, M. A. Lappe, M. Measuring generalization fields for the recognition of biological motion. Vision Res. 42, 1847-1858 (2002).
102. Troje, N. Decomposing biological motion: A framework for analysis and synthesis of human gait patterns. J. Vis. 2, 371-387 (2002).
103. Hill, H. Pollick, F. E. Exaggerating temporal differences enhances recognition of individuals from point light displays. Psychoi. Sci. 11, 223-228 (2000).
104. Giese, M. A. Poggio, T. Morphable models for the analysis and synthesis of complex motion patterns. Int. J. Comp. Vis. 38, 59-73 (2000).
105. Verfaillie, K., de Troy, A. van Rensbergen, J. Transsaccadic integration of biological motion. J. Exp. Psychoi. Learn. Mem. Cogn. 20, 649-670 (1994).
106. Pavlova, M. Biological motion perception under various presentation rates. Perception 24, Suppl. 112 (1995).
107. Sumi, S. Upside-down presentation of the Johansson moving light-spot pattern. Perception 13, 283-302 (1984).
108. Pavlova, M. Sokolov, S. Orientation specificity in biological motion perception. Percept. Psychophys. 62, 889-899 (2000).
109. Bulthoff, I., Bulthoff, H. H. Sinha, P. Top-down influences on stereoscopic depth-perception. Nature Neurosci. 1, 254-257 (1998).
110. Grossman, E. D. Blake, R. Brain activity evoked by inverted and imagined biological motion. Vision Res. 41, 1475-1482 (2001).
111. Marr, D. Nishihara, H. K. Representation and recognition of the spatial organization of three-dimensional shapes. Proc. R. Soc. Lond. B 200, 269-294 (1978).
112. Hogg, D. Model-based vision: a program to see a walking person. Image Vis. Comp. 1, 5-19 (1983).
113. Hoffman, D. D. Flinchbaugh, B. E. The interpretation of biological motion. Bioi. Cybern. 42, 195-204 (1982).
114. Webb, J. A. Aggarwal, J. K. Structure from motion of rigid and jointed objects. Artif. Inteii. 19, 107-130 (1982).
115. Oram, M. W. Perrett, D. I. Neural processing of biological motion in the macaque temporal cortex. In Computationai Vision Based on Neurobioiogy (ed. Lawton, T. B.) SPIE Proc. 2054, 155-165 (1994).
116. Pinto, J. Shiffrar, M. Subconfigurations of the human form in the perception of biological motion displays. Acta Psychoi. 102, 293-318 (1999).
117. Cutting, J. E., Moore, C. Morrison, R. Masking the motions of human gait. Percept. Psychophys. 44, 339-347 (1988).
118. Bertenthal, B. I. J. Pinto, J. Global processing of biological motion. Psychoi. Sci. 5, 221-225 (1994).
119. Thornton, I. M., Rensink, R. A. Shiffrar, M. Active versus passive processing of biological motion. Perception 31, 837-853 (2002).
120. McLeod, P, Dittrich, W., Driver, J., Perrett, D. Zihl, J. Preserved and impaired detection of structure from motion by a 'motion-blind' patient. Vis. Cogn. 3, 363-391 (1996).
121. Vaina, L. M., Lemay, M., Bienfang, D. C., Choi, A. Y. Nakayama, K. Intact 'biological motion' and 'structure from motion' perception in a patient with impaired motion mechanisms: a case study. Vis. Neurosci. 5, 353-369 (1990)
122. Neri, P., Morrone, C. Burr, D. C. Seeing biological motion. Nature 395, 894-896 (1998).
123. Logothetis, N. K., Pauls, J., Augath, M., Trinath, T. Oeltermann, A. Neurophysiological investigation of the basis of the fMRI signal. Nature 412, 150-157 (2001).
124. Grill-Spector, K. Malach, R. fMR-adaptation: a tool for studying the functional properties of human cortical neurons. Acta Psychoi. 107, 293-321 (2001).
125. Poggio, T. A theory of how the brain might work. Proc. Coid Spring Harb. Symp. Quant. Bioi. 55, 899-910 (1990).
126. Wallis, G. Rolls, E. T. Invariant face and object recognition in the visual system. Prog. Neurobioi. 51, 167 (1997).
127. Fox, R. McDaniel, C. The perception of biological motion by human infants. Science 218, 486-487 (1982).
128. Cavanagh, P, Labianca, A. P. Thornton, I. M. Attention- based visual routines: sprites. Cognition 80, 47-60 (2001).
129. Shiffrar, M. Freyd, J. J. Apparent motion of the human body. Psychoi. Sci. 1, 257-264 (1993).
130. Umilta, M. A. et al. I know what you are doing: a neurophysiological study. Neuron 31, 155-165 (2001).
131. Verfaille, K. Daems, A. Representing and anticipating actions in vision. Vis. Cogn. 9, 217-232 (2002).
132. Bassili, J. N. Facial motion in the perception of faces and of emotional expression. J. Exp. Psychoi. Hum. Percept. Perform. 4, 373-379 (1978).
133. Poizner, H., Bellugi, U. Lutes-Driscoll, V. Perception of American Sign Language in dynamic point-light displays. J. Exp. Psychoi. Hum. Percept. Perform. 7, 430-440 (1981).
134. Runeson, S. Frykholm, G. Visual perception of lifted weight. J. Exp. Psychoi. Hum. Percept. Perform. 7, 733-740 (1981).
135. Verfaillie, K. Perceiving human locomotion: priming effects in direction discrimination. Brain Cogn. 44, 192-213 (2000).
136. Goddard, N. H. The Perception of Articuiated Motion: Recognizing Moving Light Dispiays. Thesis, Dept. of Computer Science, Univ. of Rochester (1992).
137. Little, J. J. Boyd, J. Recognizing people by their gait: the shape of motion. J. Videre 1, 1-32 (1998).
138. Rosenblum, M., Yacoob, Y. Davis, L. S. Human expression recognition from motion using a radial basis function network architecture. IEEE Trans. Neurai Networks 7, 1121-1138 (1996).
139. Cedras, C. Shah, M. Motion-based recognition: a survey. Image Vis. Comp. 13, 129-155 (1995).
140. Gavrila, D. M. The visual analysis of human movement: a survey. Comp. Vis. Image Underst. 73, 82-98 (1999).
141. Aggarwal, J. K. Cai, Q. Human motion analysis: a review. Comp. Vis. Image Underst. 73, 428-440 (1999).
142. Song, Y., Goncalves, L., di Bernardo, E. Perona, P. Monocular perception of biological motion in Johansson displays. Comp. Vis. Image Underst. 81, 303-327 (2001).