Toward a minimal representation of affective gestures

IEEE Transactions on Affective Computing

Volumn 2, Issue 2, 2011, Pages 106-118

  Glowinski, Donald ^a   Dael, Nele ^b   Camurri, Antonio ^a   Volpe, Gualtiero ^a   Mortillaro, Marcello ^b   Scherer, Klaus ^b  

^a UNIVERSITY OF GENOA   (Italy)

^b UNIVERSITY OF GENEVA   (Switzerland)

Author keywords

automatic features extraction.; Emotion; expressive gesture

Indexed keywords

AFFECTIVE BEHAVIORS; AFFECTIVE GESTURE; COMPUTATIONAL RESOURCES; DATA SETS; DYNAMIC QUALITY; EMOTION; EMOTION EXPRESSION; EXPRESSIVE GESTURE; FEATURE REDUCTION; FEATURES EXTRACTION; GESTURE FEATURES; MINIMAL REPRESENTATION; MULTI-MODAL; NETWORKED MEDIA; PLUG-INS; SENSOR SYSTEMS; SOFTWARE MODULES; USER-CENTRIC; VISUAL INFORMATION; VISUAL TRACKING;

CONVOLUTIONAL CODES;

EID: 84857910982     PISSN: 19493045     EISSN: None     Source Type: Journal    
DOI: 10.1109/T-AFFC.2011.7     Document Type: Article

References (77)

1. http://www.affective-sciences.org/gemep., 2011.
2. A.P. Atkinson, W.H. Dittrich, A.J. Gemmell, and A.W. Young, "Emotion Perception from Dynamic and Static Body Expressions in Point-Light and Full-Light Displays," Perception, vol. 33, pp. 717-746, 2004. (Pubitemid 39130224)
3. T. Balomenos, A. Raouzaiou, S. Ioannou, A. Drosopoulos, K. Karpouzis, and S. Kollias, "Emotion Analysis in Man-Machine Interaction Systems," Proc. Workshop Machine Learning for Multimodal Interaction, pp. 318-328, 2004.
4. T. Banziger and K.R. Scherer, "Using Actor Portrayals to Systematically Study Multimodal Emotion Expression: The GEMEP Corpus," Proc. Second Int'l Conf. Affective Computing and Intelligent Interaction, pp. 467-487, 2007.
5. T. Banziger and K.R. Scherer, "Chapter Blueprint for Affective Computing: A Sourcebook," Introducing the Geneva Multimodal Emotion Portrayal Corpus, pp. 271-294, Oxford Univ. Press, 2010.
6. D. Bernhardt and P. Robinson, "Detecting Affect from Non-Stylised Body Motions," Proc. Second Int'l Conf. Affective Computing and Intelligent Interaction, pp. 59-70, 2007.
7. N. Bianchi-Berthouze, P. Cairns, A. Cox, C. Jennett, and W.W. Kim, "On Posture as a Modality for Expressing and Recognizing Emotions," Proc. Workshop the Role of Emotion in HCI, 2006.
8. N. Bianchi-Berthouze and A. Kleinsmith, "A Categorical Approach to Affective Gesture Recognition," Connection Science, vol. 15, no. 4, pp. 259-269, 2003. (Pubitemid 38474544)
9. R.L. Birdwhistell, Kinesics and Context: Essays on Body Motion Communication. Univ. of Pennsylvania Press, 1970.
10. A.F. Bobick, "Movement, Activity and Action: The Role of Knowledge in the Perception of Motion," Philosophical Trans. Royal Soc. B: Biological Sciences, vol. 352, no. 1358, pp. 1257-1265, 1997. (Pubitemid 27393108)
11. J.C. Borod, C.S. Haywood, and E. Koff, "Neuropsychological Aspects of Facial Asymmetry During Emotional Expression: A Review of the Normal Adult Literature," Neuropsychology Rev., vol. 7, no. 1, pp. 41-60, 1997. (Pubitemid 27363899)
12. E. Braun, B. Geurten, and M. Egelhaaf, "Identifying Prototypical Components in Behaviour Using Clustering Algorithms," PLoS ONE, vol. 5, no. 2, pp. e9361, 2010.
13. P.E. Bull, "Body Movement Scoring System," SSRC End-of-Grant Report, HR 6404/2, The Social Functions of Speech-Related Body Movement, pp. 1-16, 1981.
14. A.Camurri, P.Coletto , C. Drioli, B.Mazzarino,G.Volpe, Audio processing in a multimodal framework.Audio Engineering Society-118th Convention Spring Preprints 2005. 3, 1109-1115 (2005).
15. A. Camurri, I. Lagerlöf, and G. Volpe, "Recognizing Emotion from Dance Movement: Comparison of Spectator Recognition and Automated Techniques," Int'l J. Human-Computer Studies, Elsevier Science, vol. 59, pp. 213-225, July 2003.
16. A. Camurri, B. Mazzarino, and G. Volpe, "Expressive Interfaces," Cognition, Technology, and Work, vol. 6, no. 1, pp. 15-22, 2004. (Pubitemid 38305313)
17. A. Camurri, G. Volpe, G. De Poli, and M. Leman, "Communicating Expressiveness and Affect in Multimodal Interactive Systems," IEEE Multimedia, vol. 12, no. 1, pp. 43-53, Jan.-Mar. 2005. (Pubitemid 40351838)
18. G. Castellano and M. Mancini, "Analysis of Emotional Gestures for the Generation of Expressive Copying Behaviour in an Embodied Agent," Gesture-Based Human-Computer Interaction and Simulation, Springer, pp. 193-198, 2009.
19. G. Castellano, M. Mortillaro, A. Camurri, G. Volpe, and K. Scherer, "Automated Analysis of Body Movement in Emotionally Expressive Piano Performances," J. Music Perception, vol. 26, no. 2, pp. 103-119, 2008.
20. G. Castellano, S.D. Villalba, and A. Camurri, "Recognising Human Emotions from Body Movement and Gesture Dynamics," Proc. Second Int'l Conf. Affective Computing and Intelligent Interaction, 2007.
21. P. Cordier, M. Mendes France, J. Pailhous, and P. Bolon, "Entropy as a Global Variable of the Learning Process," Human Movement Science, vol. 13, no. 6, pp. 745-763, 1994. (Pubitemid 124013741)
22. M. Costa, W. Dinsbach, A.S.R. Manstead, and P.E.R. Bitti, "Social Presence, Embarrassment, and Nonverbal Behavior," J. Nonverbal Behavior, vol. 25, no. 4, pp. 225-240, 2001.
23. S. Dahl and A. Friberg, "Visual Perception of Expressiveness in Musicians' Body Movements," Music Perception, vol. 24, no. 5, pp. 433-454, 2007.
24. A.C. Davison and D.V. Hinkley, Bootstrap Methods and Their Application. Cambridge Univ. Press, 1999.
25. B. De Gelder, "Towards the Neurobiology of Emotional Body Language," Nature Rev. Neuroscience (Print), vol. 7, no. 3, pp. 242-249, 2006. (Pubitemid 43292469)
26. W.T. Dempster and G.R.L. Gaughran, "Properties of Body Segments Based on Size and Weight," Am. J. Anatomy, vol. 120, no. 1, pp. 33-54, 1967.
27. W.H. Dittrich, T. Troscianko, S.E.G. Lea, and D. Morgan, "Perception of Emotion from Dynamic Point-Light Displays Represented in Dance," Perception-London, vol. 25, pp. 727-738, 1996. (Pubitemid 126694959)
28. E. Douglas-Cowie, N. Campbell, R. Cowie, and P. Roach, "Emotional Speech: Towards a New Generation of Databases," Speech Comm., vol. 40, nos. 1/2, pp. 33-60, 2003.
29. B. Efron and R. Tibshirani, An Introduction to the Bootstrap. Chapman & Hall, 1993.
30. P. Ekman, "Differential Communication of Affect by Head and Body Cues," J. Personality and Social Psychology, vol. 2, no. 5, pp. 726-735, 1965.
31. P. Ekman and W.V. Friesen, "Head and Body Cues in the Judgment of Emotion: A Reformulation," Perceptual and Motor Skills, vol. 24, no. (3 PT 1), pp. 711-724, 1967.
32. P. Ekman and W.V. Friesen, The Repertoire of Nonverbal Behavior. Mouton de Gruyter, 1969.
33. P. Ekman and W.V. Friesen, "Constants across Cultures in the Face and Emotion," J. Personality and Social Psychology, vol. 17, pp. 124-129, 1971.
34. R. el Kaliouby and P. Robinson, "Generalization of a Vision-Based Computational Model of Mind-Reading," Proc. First Int'l Conf. Affective Computing and Intelligent Interaction, pp. 582-589, 2005.
35. T. Flash and N. Hogan, "The Coordination of Arm Movements: An Experimentally Confirmed Mathematical Model," J. Neuroscience, vol. 5, no. 7, pp. 1688-1703, 1985. (Pubitemid 15234427)
36. N. Freedman, "Hands, Words and Mind: On the Structuralization of Body Movements during Discourse and the Capacity for Verbal Representation, " Communicative Structures and Psychic Structures: A Psychoanalytic Interpretation of Comm., pp. 109-132, 1977.
37. S. Frey and J. Pool, "A New Approach to the Analysis of Visible Behavior," Forschungsberichte aus dem Psychologischen Institut, Universität Bern, 1976.
38. D. Glowinski, A. Camurri, G. Volpe, N. Dael, and K. Scherer, "Technique for Automatic Emotion Recognition by Body Gesture Analysis," Proc. IEEE CS Workshops Computer Vision and Pattern Recognition, pp. 1-6, 2008.
39. H. Gunes and M. Piccardi, "Bi-Modal Emotion Recognition from Expressive Face and Body Gestures," J. Network and Computer Applications, vol. 30, no. 4, pp. 1334-1345, 2007. (Pubitemid 47263518)
40. H. Gunes and M. Piccardi, "Automatic Temporal Segment Detection and Affect Recognition from Face and Body Display," IEEE Trans. Systems, Man, and Cybernetics, Part B, vol. 39, no. 1, pp. 64-84, Feb. 2009.
41. S. Hashimoto, "KANSEI as the Third Target of Information Processing and Related Topics in Japan," Proc. AIMI Int'l Workshop KANSEI, pp. 101-104, 1997.
42. J.L. Horn, "A Rationale and Test for the Number of Factors in Factor Analysis," Psychometrika, vol. 30, no. 2, pp. 179-185, 1965.
43. A. Hreljac, "The Relationship between Smoothness and Performance during the Practice of a Lower Limb Obstacle Avoidance Task," Biological Cybernetics, vol. 68, no. 4, pp. 375-379, 1993.
44. A. Kapoor, W. Burleson, and R.W. Picard, "Automatic Prediction of Frustration," Int'l J. Human-Computer Studies, vol. 65, no. 8, pp. 724-736, 2007. (Pubitemid 46810294)
45. A. Kapur, A. Kapur, N. Virji-Babul, G. Tzanetakis, and P.F. Driessen, "Gesture-Based Affective Computing on Motion Capture Data," Proc. First Int'l Conf. Affective Computing and Intelligent Interaction, pp. 1-7, 2005. (Pubitemid 43763304)
46. S. Kita, I. Van Gijn, and H. Van der Hulst, "Movement Phases in Signs and Co-Speech Gestures, and Their Transcription by Human Coders," Proc. Int'l Gesture Workshop Gesture and Sign Language in Human-Computer Interaction, pp. 23-36, 1998. (Pubitemid 128067873)
47. R. Laban and L. Ullmann, The Mastery of Movement. Plays, 1971.
48. I. Laso-Ballesteros and P. Daras, "User Centric Future Media Internet," technical report, EU Commission, Sept. 2008.
49. D. McNeill, Hand and Mind: What Gesture Reveals about Thought. Univ. of Chicago Press, 1992.
50. A. Mehrabian, Nonverbal Comm. Aldine, 2007.
51. M. Meijer, "The Contribution of General Features of Body Movement to the Attribution of Emotions," J. Nonverbal Behavior, vol. 13, no. 4, pp. 247-268, 1989.
52. J. Montepare, E. Koff, D. Zaitchik, and M. Albert, "The Use of Body Movements and Gestures as Cues to Emotions in Younger and Older Adults," J. Nonverbal Behavior, vol. 23, no. 2, pp. 133-152, 1999.
53. S. Mota and R.W. Picard, "Automated Posture Analysis for Detecting Learner's Interest Level," Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshop, vol. 5, p. 49, 2003.
54. B.P.O Connor, "SPSS and SAS Programs for Determining the Number of Components Using Parallel Analysis and Velicer's MAP Test," Behavior Research Methods Instruments and Computers, vol. 32, no. 3, pp. 396-402, 2000.
55. M. Pantic, A. Pentland, A. Nijholt, and T.S. Huang, "Human Computing and Machine Understanding of Human Behavior: A Survey," Proc. Int'l Conf. Artificial Intelligence for Human Computing, pp. 47-71, 2007.
56. P.R. Peres-Neto, D.A. Jackson, and K.M. Somers, "How Many Principal Components? Stopping Rules for Determining the Number of Non-Trivial Axes Revisited," Computational Statistics and Data Analysis, vol. 49, no. 4, pp. 974-997, 2005. (Pubitemid 40710156)
57. R.W. Picard, Affective Computing. MIT Press, 1997.
58. J.R. Pijpers, R.R.D. Oudejans, F. Holsheimer, and F.C. Bakker, "Anxiety-Performance Relationships in Climbing: A Process-Oriented Approach," Psychology of Sport and Exercise, vol. 4, no. 3, pp. 283-304, 2003. (Pubitemid 36678187)
59. F.E. Pollick, H.M. Paterson, A. Bruderlin, and A.J. Sanford, "Perceiving Affect from Arm Movement," Cognition, vol. 82, no. 2, pp. 51-61, 2001.
60. C.L. Roether, L. Omlor, and M.A. Giese, "Lateral Asymmetry of Bodily Emotion Expression," Current Biology, vol. 18, no. 8, pp. R329-R330, 2008. (Pubitemid 351522454)
61. J.A. Russell, "A Circumplex Model of Affect," J. Personality and Social Psychology, vol. 39, no. 6, pp. 1161-1178, 1980.
62. J.A. Russell, J.A. Bachorowski, and J.M. Fernandez-Dols, "Facial and Vocal Expressions of Emotion," Ann. Rev. Psychology, vol. 54, no. 1, pp. 329-349, 2003. (Pubitemid 37417497)
63. K.R. Scherer and H. Ellgring, "Multimodal Expression of Emotion: Affect Programs or Componential Appraisal Patterns," Emotion, vol. 7, no. 1, pp. 158-171, 2007. (Pubitemid 46433172)
64. P. Siple, Understanding Language through Sign Language Research. Academic Press Inc., 1978.
65. R. Timmers, M. Marolt, A. Camurri, and G. Volpe, "Listeners' Emotional Engagement with Performances of a Scriabin Etude: An Explorative Case Study," Psychology of Music, vol. 34, no. 4, pp. 481-510, 2006.
66. E. Todorov and M.I. Jordan, "Smoothness Maximization along a Predefined Path Accurately Predicts the Speed Profiles of Complex Arm Movements," J. Neurophysiology, vol. 80, no. 2, pp. 696-714, 1998. (Pubitemid 28387653)
67. J.L. Tracy and R.W. Robins, "The Prototypical Pride Expression: Development of a Nonverbal Behavior Coding System," Emotion, vol. 7, no. 4, pp. 789-801, 2007.
68. J. Van den Stock, R. Righart, and B. de Gelder, "Whole Body Expressions Influence Recognition of Facial Expressions and Emotional Prosody," Emotion, vol. 7, pp. 487-494, 2007. (Pubitemid 47445349)
69. W.F. Velicer, "Determining the Number of Components from the Matrix of Partial Correlations," Psychometrika, vol. 41, no. 3, pp. 321-327, 1976.
70. S. Vieillard and M. Guidetti, "Children's Perception and Understanding of (Dis)similarities among Dynamic Bodily/Facial Expressions of Happiness, Pleasure, Anger, and Irritation," J. Experimental Child Psychology, vol. 102, no. 1, pp. 78-95, 2009.
71. A. Vinciarelli, M. Pantic, and H. Bourlard, "Social Signals, Their Function, and Automatic Analysis: A Survey," Proc. 10th Int'l Conf. Multimodal Interfaces, pp. 61-68, 2008.
72. P. Viviani and C. Terzuolo, "Trajectory Determines Movement Dynamics," Neuroscience, vol. 7, no. 2, pp. 431-437, 1982. (Pubitemid 12168729)
73. H.G. Wallbott, "Movement Quality Changes in Psychopathological Disorders," Normalities and Abnormalities in Human Movement, Medicine and Sport Science, vol. 29, pp. 128-146, 1989.
74. H.G. Wallbott, "Bodily Expression of Emotion," European J. Social Psychology, vol. 28, pp. 879-896, 1998.
75. H.G. Wallbott and K.R. Scherer, "Cues and Channels in Emotion Recognition," J. Personality and Social Psychology, vol. 51, no. 4, pp. 690-699, 1986.
76. A.C.C. Williams, "Facial Expression of Pain: An Evolutionary Account," Behavioral and Brain Sciences, vol. 25, no. 4, pp. 439-455, 2003.
77. J.H. Yan, "Effects of Aging on Linear and Curvilinear Aiming Arm Movements," Experimental Aging Research, vol. 26, no. 4, pp. 393-407, 2000.