Critical features for the perception of emotion from gait

Journal of Vision

Volumn 9, Issue 6, 2009, Pages

  Roether, Claire L ^a   Omlor, Lars ^a   Christensen, Andrea ^a   Giese, Martin A ^a  

^a UNIVERSITY OF TÜBINGEN   (Germany)

Author keywords

Biological motion; Emotion; Emotional body expression; Locomotion; Sparse feature learning; Unsupervised learning

Indexed keywords

ACCELERATION; ADULT; ALGORITHM; ANGER; ARTICLE; ARTIFICIAL INTELLIGENCE; BIOLOGICAL MODEL; BIOMECHANICS; BODY POSTURE; COMPARATIVE STUDY; EMOTION; FEAR; FEMALE; GAIT; HAPPINESS; HUMAN; JOINT; MALE; NONLINEAR SYSTEM; PHYSIOLOGY; RECOGNITION; VISION; WALKING;

ACCELERATION; ADULT; ALGORITHMS; ANGER; ARTIFICIAL INTELLIGENCE; BIOMECHANICS; EMOTIONS; FEAR; FEMALE; GAIT; HAPPINESS; HUMANS; JOINTS; MALE; MODELS, BIOLOGICAL; NONLINEAR DYNAMICS; POSTURE; RECOGNITION (PSYCHOLOGY); VISUAL PERCEPTION; WALKING; YOUNG ADULT;

EID: 67650539036     PISSN: None     EISSN: 15347362     Source Type: Journal    
DOI: 10.1167/9.6.15     Document Type: Article

References (118)

1. Aronoff, J., Woike, B. A., & Hyman, L. M. (1992). Which are the stimuli in facial displays of anger and happiness? Configurational bases of emotion recognition. Journal of Personality and Social Psychology, 62, 1050-1066.
2. Atkinson, A. P., Dittrich, W. H., Gemmell, A. J., & Young, A. W. (2004). Emotion perception from dynamic and static body expressions in point-light and full-light displays. Perception, 33, 717-746.
3. Atkinson, A. P., Tunstall, M. L., & Dittrich, W. H. (2007). Evidence for distinct contributions of form and motion information to the recognition of emotions from body gestures. Cognition, 104, 59-72.
4. Bartlett, E. S., & Izard, C. E. (1972). A dimensional and discrete emotions investigation of the subjective experience of emotion. In C. E. Izard (Ed.), Patterns of emotions: A new analysis of anxiety and depression. New York: Academic Press.
5. Bartlett, M. S., Movellan, J. R., & Sejnowski, T. J. (2002). Face recognition by independent component analysis. IEEE Transactions on Neural Networks, 13, 1450-1464.
6. Bassili, J. N. (1978). Facial motion in the perception of faces and of emotional expression. Journal of Experimental Psychology: Human Perception and Performance, 4, 373-379.
7. Bassili, J. N. (1979). Emotion recognition: The role of facial motion and the relative importance of upper and lower areas of the face. Journal of Personality and Social Psychology, 37, 2049-2059.
8. Bertenthal, B. I., & Pinto, J. (1994). Global processing of biologicalmotions. Psychological Science, 5, 221-225.
9. Bortz, J. (1993). Statistik für Sozialwissenschaftler (6 ed.). Heidelberg: Springer-Verlag.
10. Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10, 433-436.
11. Brescoll, V. L., & Uhlmann, E. L. (2008). Can an angry woman get ahead? Status conferral, gender, and expression of emotion in the workplace. Psychological Science, 19, 268-275.
12. Bull, N. (1951). The attitude theory of emotion (vol. 81). New York: Johnson Reprint Corporation.
13. Cacioppo, J. T., Berntson, G. G., Larsen, J. T., Poehlmann, K. M., & Ito, T. A. (2000). The psychophysiology of emotion. In R. Lewis & J. M. Haviland-Jones (Eds.), The handbook of emotion (pp. 173-191). New York: Guilford Press.
14. Calder, A. J., Burton, A. M., Miller, P., Young, A. W., & Akamatsu, S. (2001). A principal component analysis of facial expressions. Vision Research, 41, 1179-1208.
15. Carey, S., & Diamond, R. (1994). Are faces perceived as configurations more by adults than by children? Visual Cognition, 1, 253-274.
16. Chan, Y. T., Hattin, R. V., & Plant, J. B. (1978). The least squares estimation of time delay and its use in signal detection. IEEE Transactions on Acoustics, Speech, and Signal Processing, 26, 217-222.
17. Cichocki, A., & Amari, S.-I. (2002). Adaptive blind signal and image processing: Learning algorithms and applications. Chichester, UK: John Wiley & Sons.
18. Coulson, M. (2004). Attributing emotion to static body postures: Recognition accuracy, confusions, and viewpoint dependence. Journal of Nonverbal Behavior, 28, 117-139.
19. d'Avella, A., & Bizzi, E. (2005). Shared and specific muscle synergies in natural motor behaviors. Proceedings of the National Academy of Sciences of the United States of America, 102, 3076-3081.
20. Davis, B. L., & Vaughan, C. L. (1993). Phasic behavior of EMG signals during gait: Use of multivariate statistics. Journal of Electromyography and Kinesiology, 3, 51-60.
21. de Gelder, B. (2006). Towards the neurobiology of emotional body language. Nature Reviews Neuroscience, 7, 242-249.
22. de Meijer, M. (1989). The contribution of general features of body movement to the attribution of emotions. Journal of Nonverbal Behavior, 13, 247-268.
23. de Meijer,M. (1991). The attribution of aggression and grief to body movements: The effect of sex-stereotypes. European Journal of Social Psychology, 21, 249-259.
24. Dittrich, W. H. (1993). Action categories and the perception of biological motion. Perception, 22, 15-22.
25. Dittrich, W. H., Troscianko, T., Lea, S. E., & Morgan, D. (1996). Perception of emotion from dynamic point-light displays represented in dance. Perception, 25, 727-738.
26. Donker, S. F., Beek, P. J., Wagenaar, R. C., & Mulder, T. (2001). Coordination between arm and leg movements during locomotion. Journal of Motor Behavior, 33, 86-102.
27. Donker, S. F., Mulder, T., Nienhuis, B., & Duysens, J. (2002). Adaptations in arm movements for added mass to wrist or ankle during walking. Experimental Brain Research, 146, 26-31.
28. Eckstein, M. P., & Ahumada, A. J., Jr. (2002). Classification images: A tool to analyze visual strategies. Journal of Vision, 2(1):i, i, http://journalofvision.org/2/1/i/, doi:10.1167/2.1.i.
29. Ekman, P. (1965). Differential communication of affect by head and body cues. Journal of Personality and Social Psychology, 2, 726-735.
30. Ekman, P. (1969). The repertoire of nonverbal behavior: Categories, origins, usage and coding. Semiotica, 1, 49-98.
31. Ekman, P. (1992). Are there basic emotions? Psychological Review, 99, 550-553.
32. Ekman, P., & Friesen, W. V. (1967). Head and body cues in the judgment of emotion: A reformulation. Perceptual and Motor Skills, 24, 711-724.
33. Ekman, P., & Friesen, W. V. (1971). Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 17, 124-129.
34. Ekman, P., & Friesen, W. V. (1978). Facial action coding system. Palo Alto: Consulting Psychologists Press.
35. Ekman, P., Sorenson, E. R., & Friesen, W. V. (1969). Pancultural elements in facial displays of emotion. Science, 164, 86-88.
36. Elfenbein, H. A., Foo, M. D., White, J. B., Tan, H. H., & Aik, V. C. (2007). Reading your counterpart: The benefit of emotion recognition accuracy for effectiveness in negotiation. Journal of Nonverbal Behavior, 31, 205-223.
37. Ellison, J. W., & Massaro, D. W. (1997). Featural evaluation, integration, and judgment of facial affect. Journal of Experimental Psychology: Human Perception and Performance, 23, 213-226.
38. Ernst, M. O., & Banks, M. S. (2002). Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415, 429-433.
39. Etcoff, N. L.,&Magee, J. J. (1992). Categorical perception of facial expressions. Cognition, 44, 227-240.
40. Fridlund, A. J. (1990). Sociality of solitary smiling: Potentiation by an implicit audience. Journal of Personality and Social Psychology, 60, 229-240.
41. Fridlund, A. J., Kenworthy, K. G., & Jaffey, A. K. (1992). Audience effects in affective imagery: Replication and extension to dysphoric memory. Journal of Nonverbal Behavior, 16, 191-212.
42. Frijda, N. H. (1988). The laws of emotion. American Psychologist, 43, 349-358.
43. Fu, W. J. (1998). Penalized regressions: The bridge versus the lasso. Journal of Computational and Graphical Statistics, 7, 397-416.
44. Gellhorn, E. (1964). Motion and emotion: The role of proprioception in the physiology and pathology of the emotions. Psychological Review, 71, 457-472.
45. Giese, M. A., & Lappe, M. (2002). Measurement of generalization fields for the recognition of biological motion. Vision Research, 42, 1847-1858.
46. Giese, M. A., & Poggio, T. (2000). Morphable models for the analysis and synthesis of complex motion patterns. International Journal of Computer Vision, 38, 59-73.
47. Giese, M. A., Thornton, I., & Edelman, S. (2008). Metrics of the perception of body movement. Journal of Vision, 8(9):13, 1-18, http://journalofvision.org/8/9/13/, doi:10.1167/8.9.13.
48. Golubitsky, M., Stewart, I., Buono, P. L., & Collins, J. J. (1999). Symmetry in locomotor central pattern generators and animal gaits. Nature, 401, 693-695.
49. Gosselin, F., & Schyns, P. G. (2001). Bubbles: A technique to reveal the use of information in recognition tasks. Vision Research, 41, 2261-2271.
50. Grèzes, J., Pichon, S., & de Gelder, B. (2007). Perceiving fear in dynamic body expressions. Neuroimage, 35, 959-967.
51. Hancock, P. J., Burton, A. M., & Bruce, V. (1996). Face processing: Human perception and principal components analysis. Memory and Cognition, 24, 21-40.
52. Henley, N. M., & Harmon, S. (1985). The nonverbal semantics of power and gender. In S. L. Ellyson & J. F. Dovidio (Eds.) Power, dominance, and nonverbal behavior. New York: Springer.
53. Hojen-Sorensen, P. A. d. F. R., Winther, O., & Hansen, L. K. (2002). Mean-field approaches to independent component analysis. Neural Computation, 14, 889-918.
54. Ivanenko, Y. P., Cappellini, G., Dominici, N., Poppele, R. E., & Lacquaniti, F. (2005). Coordination of locomotion with voluntary movements in humans. Journal of Neuroscience, 25, 7238-7253.
55. Ivanenko, Y. P., Poppele, R. E., & Lacquaniti, F. (2004). Five basic muscle activation patterns account for muscle activity during human locomotion. Journal of Physiology, 556, 267-282.
56. Izard, C. E. (1977). Human emotions. New York: Plenum Press.
57. Jolliffe, I. T. (2002). Principal component analysis (2 ed.). New York: Springer.
58. Jordan, H., Fallah, M., & Stoner, G. R. (2006). Adaptation of gender derived from biological motion. Nature Neuroscience, 9, 738-739.
59. Kamachi, M., Bruce, V., Mukaida, S., Gyoba, J., Yoshikawa, S., & Akamatsu, S. (2001). Dynamic properties influence the perception of facial expressions. Perception, 30, 875-887.
60. Kirtley, C. (2006). Clinical gait analysis (1 ed.). Oxford: Churchill Livingstone.
61. Kleinsmith, A., De Silva, R., & Bianchi-Berthouze, N. (2006). Cross-cultural differences in recognizing affect from body posture. Interacting with Computers, 18, 1371-1389.
62. Knill, D. C., & Richards, W. (2008). Perception as Bayesian inference (1 ed.). Cambridge: Cambridge University Press.
63. Lee, D. D., & Seung, H. S. (1999). Learning the parts of objects by non-negative matrix factorization. Nature, 401, 788-791.
64. Leopold, D. A., O'Toole, A. J., Vetter, T., & Blanz, V. (2001). Prototype-referenced shape encoding revealed by high-level aftereffects. Nature Neuroscience, 4, 89-94.
65. Lu, H., & Liu, Z. (2006). Computing dynamic classification images from correlation maps. Journal of Vision, 6(4):12, 475-483, http://journalofvision.org/6/4/12/, doi:10.1167/6.4.12.
66. Maloney, L. T. (2002). Illuminant estimation as cue combination. Journal of Vision, 2(6):6, 493-504, http://journalofvision.org/ 2/6/6/, doi:10.1167/2.6.6.
67. Mather, G., Radford, K., & West, S. (1992). Low-level visual processing of biological motion. Proceedings in Biological Sciences, 249, 149-155.
68. Meinshausen, N., Rocha, G., & Yu, B. (2007). A tale of three cousins: Lasso, L2Boosting and Dantzig. Annals of Statistics, 35, 2373-2384.
69. Merkle, L. A., Layne, C. S., Bloomberg, J. J., & Zhang, J. J. (1998). Using factor analysis to identify neuromuscular synergies during treadmill walking. Journal of Neuroscience Methods, 82, 207-214.
70. Mezger, J., Ilg, W., & Giese, M. A. (2005). Trajectory synthesis by hierarchical spatio-temporal correspondence: Comparison of different methods. Paper presented at the ACM SIGGRAPH Symposium on Applied Perception in Graphics and Visualization, A Coruna, Spain.
71. Montepare, J. M., Goldstein, S. B., & Clausen, A. (1987). The identification of emotions from gait information. Journal of Nonverbal Behavior, 11, 33-42.
72. Montepare, J. M., Koff, E., Zaitchik, D., & Albert, M. (1999). The use of body movements and gestures as cues to emotions in younger and older adults. Journal of Nonverbal Behavior, 23, 133-152.
73. Mukovskiy, A., Park, A.-N., Omlor, L., Slotine, J. J., & Giese, M. A. (2008). Selforganization of character behavior by mixing of learned movement primitives. Konstanz, Germany: The 13th International Fall Workshop of Vision, Modeling, and Visualization (VMV).
74. Nocedal, J., & Wright, S. (2006). Numerical optimization (2nd ed.). Berlin: Springer.
75. O'Toole, A. J., Roark, D. A., & Abdi, H. (2002). Recognizing moving faces: A psychological and neural synthesis. Trends in Cognitive Sciences, 6, 261-266.
76. Oja, E., & Plumbley, M. (2003). Blind separation of positive sources using non-negative PCA. Paper presented at the 4th International Symposium on Independent Component Analysis and Blind Signal Separation (ICA), Nara, Japan.
77. Oldfield, R. C. (1971). The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia, 9, 97-113.
78. Olree, K. S., & Vaughan, C. L. (1995). Fundamental patterns of bilateral muscle activity in human locomotion. Biological Cybernetics, 73, 409-414.
79. Omlor, L., & Giese, M. A. (2007a). Blind source separation for over-determined delayed mixtures. In B. Schölkopf, J. Platt, & T. Hoffman (Eds.), Advances in neural information processing systems (vol. 19, pp. 1049-1056). Cambridge, MA: MIT Press.
80. Omlor, L., & Giese, M. A. (2007b). Extraction of spatio-temporal primitives of emotional body expressions. Neurocomputing, 70, 1938-1942.
81. Osgood, C. E. (1966). Dimensionality of the semantic space for communication via facial expressions. Scandinavian Journal of Psychology, 7, 1-30.
82. Padgett, C., & Cottrell, G. (1995). Identifying emotion in static face images. Paper presented at the 2nd Joint Symposium on Neural Computation. La Jolla, CA, University of California, San Diego.
83. Pavlova, M., Sokolov, A., & Sokolov, A. A. (2005). Perceived dynamics of static images enables emotional attribution. Perception, 34, 1107-1116.
84. Pinto, J., & Shiffrar, M. (1999). Subconfigurations of the human form in the perception of biological motion displays. Acta Psychologica (Amsterdam), 102, 293-318.
85. Pollick, F. E., Lestou, V., Ryu, J., & Cho, S. B. (2002). Estimating the efficiency of recognizing gender and affect from biological motion. Vision Research, 42, 2345-2355.
86. Pollick, F. E., Paterson, H. M., Bruderlin, A., & Sanford, A. J. (2001). Perceiving affect from arm movement. Cognition, 82, B51-61.
87. Roether, C. L., Omlor, L., & Giese, M. A. (2008). Lateral asymmetry of bodily emotion expression. Current Biology, 18, R329-330.
88. Roether, C. L., Omlor, L., & Giese, M. A. (in press). Features in the recognition of emotions from dynamic bodily expression. In G. Masson & U. Ilg (Eds.), Dynamics of visual motion processing: Neuronal, behavioral and computational approaches. Berlin, Heidelberg: Springer.
89. Santello, M., Flanders, M., & Soechting, J. F. (2002). Patterns of hand motion during grasping and the influence of sensory guidance. Journal of Neuroscience, 22, 1426-1435.
90. Santello, M., & Soechting, J. F. (1997). Matching object size by controlling finger span and hand shape. Somatosensory and Motor Research, 14, 203-212.
91. Sawada, M., Suda, K., & Ishii, M. (2003). Expression of emotions in dance: Relation between arm movement characteristics and emotion. Perceptual and Motor Skills, 97, 697-708.
92. Scherer, K. R., Johnstone, T., & Klasmeyer, G. (2003). Vocal expression of emotion. In R. J. Davidson (Ed.), Handbook of affective sciences (pp. 433-456). Oxford: Oxford University Press.
93. Schlosberg, H. (1954). Three dimensions of emotion. Psychological Review, 61, 81-88.
94. Schmidt, K. L., & Cohn, J. F. (2001). Human facial expressions as adaptations: Evolutionary questions in facial expression research. Yearbook of Physical Anthropology, 44, 3-24.
95. Schouwstra, S. J., & Hoogstraten, J. (1995). Head position and spinal position as determinants of perceived emotional state. Perceptual and Motor Skills, 81, 673-674.
96. Schyns, P. G., Petro, L. S., & Smith, M. L. (2007). Dynamics of visual information integration in the brain for categorizing facial expressions. Current Biology, 17, 1580-1585.
97. Sogon, S., & Masutani, M. (1989). Identification of emotion from body movements: A cross-cultural study of Americans and Japanese. Psychological Reports, 65, 35-46.
98. Spiegel, J., & Machotka, P. (1974). Messages of the body (1 ed.). New York: Free Press.
99. Tanaka, J. W., & Farah, M. J. (1993). Parts and wholes in face recognition. Quarterly Journal of Experimental Psychology A: Human Perceptual Psychology, 46, 225-245.
100. Thurman, S. M., & Grossman, E. D. (2008). Temporal "Bubbles" reveal key features for point-light biological motion perception. Journal of Vision, 8(3):28, 1-11, http://journalofvision.org/8/3/28/, doi:10.1167/8.3.28.
101. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B, 58, 267-288.
102. Tresch, M. C., Cheung, V. C., & d'Avella, A. (2006). Matrix factorization algorithms for the identification of muscle synergies: Evaluation on simulated and experimental data sets. Journal of Neurophysiology, 95, 2199-2212.
103. Troje, N. F. (2002). Decomposing biological motion: A framework for analysis and synthesis of human gait patterns. Journal of Vision, 2(5), 371-387, http://journalofvision.org/2/5/2/, doi:10.1167/2.5.2.
104. Troje, N. F., Sadr, J., Geyer, H., & Nakayama, K. (2006). Adaptation aftereffects in the perception of gender from biological motion. Journal of Vision, 6(8):7, 850-857, http://journalofvision.org/ 6/8/7/, doi:10.1167/6.8.7.
105. Turk, M., & Pentland, A. (1991). Eigen faces for recognition. Journal of Cognitive Neuroscience, 3, 71-86.
106. Unuma, M., Anjyo, K., & Takeuchi, R. (1995). Fourier principles for emotion-based human figure animation. Motion signal processing. Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (pp. 91-96). New York: ACM Press.
107. Valentin, D., Abdi, H., Edelman, B., & O'Toole, A. J. (1997). Principal component and neural network analyses of face images: What can be generalized in gender classification? Journal of Mathematical Psychology, 41, 398-413.
108. Vangeneugden, J., Pollick, F., & Vogels, R. (2008). Functional differentiation of macaque visual temporal cortical neurons using a parametric action space. Cerebral Cortex.
109. Vapnik, V. N. (1999). An overview of statistical learning theory. IEEE Transactions on Neural Networks, 10, 988-999.
110. Walk, R. D., & Homan, C. P. (1984). Emotion and dance in dynamic light displays. Bulletin of the Psychonomic Society, 22, 437-440.
111. Wallbott, H. G. (1998). Bodily expression of emotion. European Journal of Social Psychology, 28, 879-896.
112. Wallbott, H. G., & Scherer, K. R. (1986). Cues and channels in emotion recognition. Journal of Personality and Social Psychology, 51, 690-699.
113. Webster, M. A., Kaping, D., Mizokami, Y., & Duhamel, P. (2004). Adaptation to natural facial categories. Nature, 428, 557-561.
114. Westermann, R., Spies, K., Stahl, G., & Hesse, F. W. (1996). Relative effectiveness and validity of mood induction procedures: A meta-analysis. European Journal of Social Psychology, 26, 557-580.
115. Wundt, W. (2004). Grundriss der Psychologie (1 ed.). Saarbrücken: Vdm Verlag Dr. Müller.
116. Xu, H., Dayan, P., Lipkin, R. M., & Qian, N. (2008). Adaptation across the cortical hierarchy: Low-level curve adaptation affects high-level facial-expression judgments. Journal of Neuroscience, 28, 3374-3383.
117. Yacoob, Y., & Black, M. J. (1999). Parameterized modeling and recognition of activities. Computer Vision and Image Understanding, 73, 232-247.
118. Zuckerman, M., Lipets, M. S., Koivumaki, J. H., & Rosenthal, R. (1975). Encoding and decoding nonverbal cues of emotion. Journal of Personality and Social Psychology, 32, 1068-1076.