Affective body expression perception and recognition: A survey

IEEE Transactions on Affective Computing

Volumn 4, Issue 1, 2013, Pages 15-33

  Kleinsmith, Andrea ^a   Bianchi Berthouze, Nadia ^b  

^a UNIVERSITY OF FLORIDA   (United States)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

affective body movement; Affective body posture; affective recognition systems; cross cultural differences; spatiotemporal affective body features

Indexed keywords

AFFECTIVE BODY POSTURES; AFFECTIVE RECOGNITION; BODY MOVEMENTS; CROSS-CULTURAL DIFFERENCES; SPATIOTEMPORAL AFFECTIVE BODY FEATURES;

SURVEYS;

EID: 84866500519     PISSN: 19493045     EISSN: None     Source Type: Journal    
DOI: 10.1109/T-AFFC.2012.16     Document Type: Article

References (187)

1. N. Fragopanagos and J.G. Taylor, "Emotion Recognition in Human-Computer Interaction," Neural Networks, vol. 18, no. 4, pp. 389-405, 2005. (Pubitemid 40922647)
2. C.E. Izard, P.B. Ackerman, K.M. Schoff, and S.E. Fine, "Self-Organization of Discrete Emotions, Emotion Patterns, and Emotion-Cognition Relations," Emotion, Development, and Self-Organization: Dynamic Systems Approaches to Emotional Development, D.L. Marc and I. Granic, eds., pp. 15-36, Cambridge Univ. Press, 2002.
3. P. Ekman and W. Friesen, Manual for the Facial Action Coding System. Consulting Psychology Press, 1978.
4. R. von Laban, The Mastery of Movement. MacDonald & Evans Ltd., 1971.
5. R.H. Rozensky and L. Feldman-Honor, "Notation Systems for Coding Nonverbal Behavior: A Review," J. Behavioral Assessment, vol. 4, no. 2, pp. 119-132, 1982. (Pubitemid 13209613)
6. A. Mehrabian and J. Friar, "Encoding of Attitude by a Seated Communicator via Posture and Position Cues," J. Consulting and Clinical Psychology, vol. 33, pp. 330-336, 1969.
7. M. Argyle, Bodily Communication. Methuen & Co. Ltd., 1988.
8. P.E. Bull, Posture and Gesture. Pergamon, 1987.
9. P. Ekman and W. Friesen, "Detecting Deception from the Body or Face," J. Personality and Social Psychology, vol. 29, no. 3, pp. 288-298, 1974.
10. L. McClenney and R. Neiss, "Post-Hypnotic Suggestion: A Method for the Study of Nonverbal Communication," J. Nonverbal Behavior, vol. 13, pp. 37-45, 1989.
11. H. Meeren, C. van Heijnsbergen, and B. de Gelder, "Rapid Perceptual Integration of Facial Expression and Emotional Body Language," Proc. Nat'l Academy of Sciences USA, vol. 102, no. 45, pp. 16518-16523, 2005. (Pubitemid 41688946)
12. J. Van den Stock, R. Righart, and B. de Gelder, "Body Expressions Influence Recognition of Emotions in the Face and Voice," Emotion, vol. 7, no. 3, pp. 487-494, 2007. (Pubitemid 47445349)
13. A.N. Antle, G. Corness, and M. Droumeva, "What the Body Knows: Exploring the Benefits of Embodied Metaphorsin Hybrid Physical Digital Environments," Interacting with Computers, vol. 21, pp. 66-75, 2009.
14. N. Bianchi-Berthouze, "Understanding the Role of Body Movement in Player Engagement," Human-Computer Interaction, in press, http://web4.cs.ucl.ac.uk/uclic/people/n.berthouze/BerthouzeHCI12.pdf
15. F. Muller and N. Bianchi-Berthouze, "Evaluating Exertion Games," Evaluating User Experience in Games, HCI Series, Part 4, pp. 187-207, Springer 2010.
16. P.M. Niedenthal, L.W. Barsalou, P. Winkielman, S. Krauth-Gruber, and F. Ric, "Embodiment in Attitudes, Social Perception, and Emotion," Personality and Social Psychology Rev., vol. 9, pp. 184-211, 2005. (Pubitemid 41356682)
17. L. Barsalou, "Grounded Cognition," Ann. Rev. Psychogy, vol. 59, pp. 617-645, 2008.
18. S. Goldin-Meadow, H. Nusbaum, S.D. Kelly, and S. Wagner, "Explaining Math: Gesturing Lightens the Load," Psychological Science, vol. 12, no. 6, pp. 516-522, 2001. (Pubitemid 33653571)
19. J. Chandler and N. Schwarz, "How Extending Your Middle Finger Affects Your Perception of Others: Learned Movements Influence Concept Accessibility," J. Experimental Social Psychology, vol. 45, no. 1, pp. 123-128, 2009.
20. S. Neill and C. Caswell, Body Language for Competent Teachers. Routledge, 1993.
21. D.H. Jonassen and B.L. Grabowski, Handbook of Individual Differences, Learning, and Instruction. Erlbaum, 1993.
22. A. Kapoor, W. Burleson, and R.W. Picard, "Automatic Prediction of Frustration," Int'l J. Human-Computer Studies, vol. 65, no. 8, pp. 724-736, 2007. (Pubitemid 46810294)
23. A. Kvale, A.E. Ljunggren, and T.B. Johnsen, "Examination of Movement in Patients with Long-Lasting Musculoskeletal Pain: Reliability and Validity," Physiotherapy Research Int'l, vol. 8, pp. 36-52, 2003.
24. G.K. Haugstad, T.S. Haugstad, U.M. Kirste, S. Leganger, S. Wojniusz, I. Klemmetsen, and U.F. Malt, "Posture, Movement Patterns, and Body Awareness in Women with Chronic Pelvic Pain," J. Psychosomatic Research, vol. 61, no. 5, pp. 637-644, 2006. (Pubitemid 44647266)
25. B. Bunkan, A.E. Ljunggren, S. Opjordsmoen, O. Moen, and S. Friis, "What Are the Dimensions of Movement?" Nord J. Psychiatry, vol. 55, pp. 33-40, 2001.
26. J.W.S. Vlaeyen and S.J. Linton, "Fear-Avoidance and Its Consequences in Muscleskeleton Pain: A State of the Art," Pain, vol. 85, no. 3, pp. 317-332, 2000. (Pubitemid 30214160)
27. M.M. Gross, E.A. Crane, and B.L. Fredrickson, "Methodology for Assessing Bodily Expression of Emotion," J. Nonverbal Behavior, vol. 34, pp. 223-248, 2010.
28. H. Gunes and M. Pantic, "Automatic, Dimensional and Continuous Emotion Recognition," Int'l J. Synthetic Emotion, vol. 1, no. 1, pp. 68-99, 2010.
29. D. McNeill, Hand and Mind-What Gestures Reveal about Thought. The Univ. of Chicago Press, 1992.
30. B. DeCarolis, C. Pelachaud, I. Poggi, and M. Steedman, "APML, a Mark-Up Language for Believable Behavior Generation," Life-Like Characters: Tools, Affective Functions and Applications, H. Prendinger, M. Ishizuka, eds., pp. 65-85, Springer, 2004.
31. J. Cassell, "Body Language: Lessons from the Near-Human," Genesis Redux: Essays on the History and Philosophy of Artificial Life. J. Riskin, ed., pp. 346-374, Univ. of Chicago Press, 2007.
32. C. Pelachaud, "Studies on Gesture Expressivity for a Virtual Agent," Speech Comm., vol. 51, no. 7, pp. 630-639, 2009.
33. R. Picard, "Toward Agents that Recognize Emotion," Proc. IMAGINA, pp. 153-165, 1998.
34. B. de Gelder, "Why Bodies? Twelve Reasons for Including Bodily Expressions in Affective Neuroscience," Philosophical Trans. Royal Soc., vol. 364, no. 3, pp. 3475-3484, 2009.
35. W. Zhao, R. Chellappa, and A. Rosenfeld, "Face Recognition: A Literature Survey," ACM Computing Surveys, vol. 35, pp. 399-458, 2003.
36. M. Pantic and L.J.M. Rothkrantz, "Automatic Analysis of Facial Expressions: The State of the Art," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1424-1445, Dec. 2000. (Pubitemid 32088469)
37. B. de Gelder, "Towards the Neurobiology of Emotional Body Language," Nature Rev. Neuroscience, vol. 7, no. 3, pp. 242-249, 2006. (Pubitemid 43292469)
38. P. Ekman and W. Friesen, "Felt, False and Miserable Smiles," J. Nonverbal Behavior, vol. 6, no. 4, pp. 238-252, 1982.
39. R. Adolphs, "Neural Systems for Recognizing Emotion," Current Opinion in Neurobiology, vol. 12, no. 2, pp. 169-177, 2002. (Pubitemid 34518336)
40. H.G. Wallbott, "Bodily Expression of Emotion," European J. Social Psychology, vol. 28, pp. 879-896, 1998.
41. P. Ekman and W. Friesen, "Head and Body Cues in the Judgment of Emotion: A Reformulation," Perceptual and Motor Skills, vol. 24, pp. 711-724, 1967.
42. P. Ekman and W. Friesen, "The Repertoire of Non-Verbal Behavioral Categories: Origins, Usage and Coding," Semiotica, vol. 1, pp. 49-98, 1969.
43. C. Darwin, The Expression of the Emotions in Man and Animals. Murray, 1872.
44. P. Ekman and W. Friesen, "Nonverbal Leakage and Clues to Deception," Psychiatry, vol. 32, pp. 88-105, 1969.
45. P. Ekman, "Mistakes When Deceiving," Annals New York Academy of Sciences, vol. 364, pp. 269-278, 1981.
46. J.E. Hocking and D.G. Leathers, "Nonverbal Indicators of Deception: A New Theoretical Perspective," Comm. Monographs, vol. 4, no. 2, pp. 119-131, 1980.
47. R. Kraut, "Verbal and Nonverbal Cues in the Perception of Lying," J. Personality and Social Psychology, vol. 36, no. 4, pp. 380-391, 1978.
48. J.E. Hocking, "Detecting Deceptive Communication from Verbal, Visual, and Paralinguistic Cues: An Exploratory Experiment," unpublished doctoral dissertation, Michigan State Univ., 1976.
49. G.E. Littlepage and M.A. Pineault, "Detection of Deceptive Factual Statements from the Body and the Face," Personality and Social Psychology Bull., vol. 5, pp. 325-328, 1979.
50. R.E. Riggio and H.S. Friedman, "Individual Differences and Cues to Deception," J. Personality and Social Psychology, vol. 45, no. 4, pp. 899-915, 1983.
51. P. Ekman and W. Friesen, Pictures of Facial Affect. Consulting Psychologists Press, 1976.
52. N. Hadjikhani and B. de Gelder, "Seeing Fearful Body Expressions Activates the Fusiform Cortex and Amygdala," Current Biology, vol. 13, pp. 2201-2205, 2003. (Pubitemid 38024947)
53. B. de Gelder, J. Snyder, D. Greve, G. Gerard, and N. Hadjikhani, "Fear Fosters Flight: A Mechanism for Fear Contagion When Perceiving Emotion Expressed by a Whole Body," Proc. Nat'l Academy of Sciences USA, vol. 101, no. 47, pp. 16701-16706, 2004. (Pubitemid 39557776)
54. H.M. Paterson, F.E. Pollick, and E. Jackson, "Movement and Faces in the Perception of Emotion from Motion," Perception, vol. 31, no. 118, pp. 232-232, 2002.
55. F.E. Pollick, H. Paterson, and P. Mamassian, "Combining Faces and Movements to Recognize Affect," J. Vision, vol. 4, no. 8, pp. 232-232, 2004.
56. R. Tuminello and D. Davidson, "What the Face and Body Reveal: In-Group Emotion Effects and Stereotyping of Emotion in African-American and European-American Children," J. Experimental Child Psychology, special issue on rmotion assessment in children and adolescents, vol. 110, pp. 258-274, 2011.
57. M.L. Willis, R. Palermo, D. Burke, "Judging Approachability on the Face of It: The Influence of Face and Body Expressions on the Perception of Approachability," Emotion, vol. 11, no. 3, pp. 514-523, 2011.
58. H.M. Paterson, F.E. Pollick, and A.J. Sanford, "The Role of Velocity in Affect Discrimination," Proc. 23rd Ann. Conf. Cognitive Science Soc., pp. 756-761, 2001.
59. A. Kleinsmith and N. Bianchi-Berthouze, "Grounding Affective Dimensions into Posture Features," Proc. First Int'l Conf. Affective Computing and Intelligent Interaction, pp. 263-270, 2005. (Pubitemid 43763337)
60. A. Kleinsmith, N. Bianchi-Berthouze, and A. Steed, "Automatic Recognition of Non-Acted Affective Postures," IEEE Trans. Systems, Man, and Cybernetics, Part B, vol. 41, no. 4, pp. 1027-1038, Aug. 2011.
61. M. Karg, K. Kuhnlenz, and M. Buss, "Recognition of Affect Based on Gait Patterns," IEEE Trans. Systems, Man, and Cybernetics, Part B, vol. 40, no. 4, pp. 1050-1061, Aug. 2010.
62. J.K. Burgoon, M.L. Jensen, T.O. Meservy, J. Kruse, and J.F. Nunamaker, "Augmenting Human Identification of Emotional States in Video," Proc. Int'l Conf. Intelligent Data Analysis, 2005.
63. D. Matsumoto, "Culture and Nonverbal Behavior," Handbook of Nonverbal Comm., V. Manusov and M. Patterson, eds., Sage, 2005.
64. S. Kopp, L. Gesellensetter, N. Kramer, and I. Wachsmuth, "A Conversational Agent as Museum Guide-Design and Evaluation of a Real-World Application," Proc. Fifth Int'l Workshop Intelligent Virtual Agents, J. Tao, T. Tan, and R. Picard, eds., pp. 329-343, 2005.
65. W. Swartout, D. Traum, R. Artstein, D. Noren, P. Debevec, K. Bronnenkant, J. Williams, A. Leuski, S. Narayanan, D. Piepol, C. Lane, J. Morie, P. Aggarwal, M. Liewer, J-Y. Chiang, J. Gerten, S. Chu, and K. White, "Ada and Grace: Toward Realistic and Engaging Virtual Museum Guides," Proc. Int'l Conf. Intelligent Virtual Agents, pp. 286-300, 2010.
66. P. Dunn and A. Marinetti, Cultural Adaptation: Necessity for eLearning, http://www.linezine.com/7.2/articles/pdamca.htm, Retrieved Jan. 2006.
67. D. Keltner, P. Ekman, G.C. Gonzaga, and J. Beer, "Facial Expression of Emotion," Handbook of Affective Sciences, R. Davidson, K. Scherer, and H. Goldsmith, eds., Oxford Univ. Press, 2003.
68. H.A. Elfenbein, M.K. Mandal, N. Ambady, S. Harizuka, and S. Kumar, "Cross-Cultural Patterns in Emotion Recognition: Highlighting Design and Analytical Techniques," Emotion, vol. 2, no. 1, pp. 75-84, 2002.
69. P. Ekman, "Strong Evidence for Universals in Facial Expressions: A Reply to Russell's Mistaken Critique," Psychological Bull., vol. 115, pp. 268-287, 1994. (Pubitemid 24976747)
70. L. Yin, X. Wei, Y. Sun, J. Wang, and M.J. Rosato, The Binghamton University 3D Facial Expression Data Base, http://www.cs. binghamton.edu/, 2012.
71. J.A. Russell, "Is There Universal Recognition of Emotion from Facial Expressions? A Review of the Cross-Cultural Studies," Psychological Bull., vol. 115, pp. 102-141, 1994. (Pubitemid 24976739)
72. B. Mesquita, "Emotions as Dynamic Cultural Phenomena," Handbook of Affective Sciences, R. Davidson, K. Scherer, and H. Goldsmith, eds., Oxford Univ. Press, 2003.
73. H. Elfenbein and N. Ambady, "On the Universality and Cultural Specificity of Emotion Recognition: A Metaanalysis," Psychological Bull., vol. 128, pp. 205-235, 2002.
74. L. Ducci, L. Arcuri, T. Georgis, and T. Sineshaw, "Emotion Recognition in Ethiopia: The Effects of Familiarity with Western Culture on Accuracy of Recognition," J. Cross-Cultural Psychology, vol. 13, pp. 340-351, 1982.
75. U. Hess, S. Senecal, and G. Kirouac, "Recognizing Emotional Facial Expressions: Does Perceived Sociolinguistic Group Make a Difference?" Int'l J. Psychology, vol. 31, p. 93, 1996.
76. T. Kudoh and D. Matsumoto, "Cross-Cultural Examination of the Semantic Dimensions of Body Postures," J. Personality Social Psychology, vol. 48, no. 6, pp. 1440-1446, 1985.
77. D. Matsumoto and T. Kudoh, "Cultural Similarities and Differences in the Semantic Dimensions of Body Postures," J. Nonverbal Behavior, vol. 11, no. 3, pp. 166-179, 1987.
78. M.H. Bond, H. Nakazato, and D. Shiraishi, "Universality and Distinctiveness in Dimensions of Japanese Person Perception," J. Cross-Cultural Psychology, vol. 6, pp. 346-357, 1975.
79. D. Matsumoto and H. Kishimoto, "Developmental Characteristics in Judgments of Emotion from Nonverbal Vocal Cues," Int'l J. Intercultural Relations, vol. 7, pp. 415-424, 1983.
80. A. Kleinsmith, P.R. de Silva, and N. Bianchi-Berthouze, "Cross-Cultural Differences in Recognizing Affect from Body Posture," Interacting with Computers, vol. 18, pp. 1371-1389, 2006. (Pubitemid 44709886)
81. M. Brandt and J. Boucher, "Concepts of Depression in Emotion Lexicons of Eight Cultures," Int'l J. Intercultural Relations, vol. 10, pp. 321-346, 1986.
82. D. Matsumoto, T. Consolacion, H. Yamada, R. Suzuki, B. Franklin, S. Paul, R. Ray, and H. Uchida, "American-Japanese Cultural Differences in Judgments of Emotional Expressions of Different Intensities," Cognition and Emotion, vol. 16, pp. 721-747, 2002. (Pubitemid 35256182)
83. M.A. Giese and T. Poggio, "Neural Mechanisms for the Recognition of Biological Movements," Neuroscience, vol. 4, pp. 179-191, 2003.
84. L.M. Vania, M. Lemay, D.C. Bienfang, A.Y. Choi, and K. Nakayama, "Intact Biological Motion and Structure from Motion Perception in a Patient with Impaired Motion Mechanisms: A Case Study," Visual Neuroscience, vol. 5, pp. 353-369, 1990.
85. J. Lange and M. Lappe, "The Role of Spatial and Temporal Information in Biological Motion Perception," Advances in Cognitive Psychology, vol. 3, no. 4, pp. 419-428, 2007.
86. M. Hirai and K. Hiraki, "The Relative Importance of Spatial versus Temporal Structure in the Perception of Biological Motion: An Event-Related Potential Study," Cognition, vol. 99, pp. B15-B29, 2006. (Pubitemid 43145924)
87. M.V. Peelen, A.J. Wiggett, and P.E. Downing, "Patterns of fMRI Activity Dissociate Overlapping Functional Brain Areas that Respond to Biological Motion," Neuron, vol. 49, pp. 815-822, 2006. (Pubitemid 43376079)
88. P. McLeod, W. Dittrich, J. Driver, D. Perret, and J. Zihl, "Preserved and Impaired Detection of Structure from Motion by a 'Motion-Blind' Patient," Visual Cognition, vol. 3, pp. 363-391, 1996. (Pubitemid 126683404)
89. A.P. Atkinson, W.H. Dittrich, A.J. Gemmell, and A.W. Young, "Evidence for Distinct Contributions of Form and Motion Information to the Recognition of Emotions from Body Gestures," Cognition, vol. 104, pp. 59-72, 2007. (Pubitemid 46627560)
90. L. Omlor and M.A. Giese, "Extraction of Spatio-Temporal Primitives of Emotional Body Expressions," Neurocomputing, vol. 70, nos. 10-12, pp. 1938-1942, 2007. (Pubitemid 46657011)
91. C. Roether, L. Omlor, A. Christensen, and M.A. Giese, "Critical Features for the Perception of Emotion from Gait," J. Vision, vol. 8, nos. 6-15, pp. 1-32, 2009.
92. J. Aronoff, B.A. Woike, and L.M. Hyman, "Which Are the Stimuli in Facial Displays of Anger and Happiness? Configurational Bases of Emotion Recognition," J. Personality and Social Psychology, vol. 62, pp. 1050-1066, 1992.
93. J.A. Harrigan and R. Rosenthal, "Physicians Head and Body Positions as Determinants of Perceived Rapport," J. Applied Social Psychology, vol. 13, no. 6, pp. 496-509, 1983.
94. A. Mehrabian, "Inference of Attitude from the Posture, Orientation, and Distance of a Communicator," J. Consulting and Clinical Psychology, vol. 32, pp. 296-308, 1968.
95. W.T. James, "A Study of the Expression of Bodily Posture," J. General Psychology, vol. 7, pp. 405-437, 1932.
96. S. Dahl and A. Friberg, "Visual Perception of Expressiveness in Musicians' Body Movements," Music Perception, vol. 24, no. 5, pp. 433-454, 2007.
97. G. Castellano, M. Mortillaro, A. Camurri, G. Volpe, and K. Scherer, "Automated Analysis of Body Movement in Emotionally Expressive Piano Performances," Music Perception, vol. 26, no. 2, pp. 103-120, 2008.
98. D. Glowinski, N. Dael, A. Camurri, G. Volpe, M. Mortillaro, and K. Scherer, "Towards a Minimal Representation of Affective Gestures," IEEE Trans. Affective Computing, vol. 2, no. 2, pp. 106-118, Apr.-June 2011.
99. T. Banziger and K.R. Scherer, "Introducing the Geneva Multimodal Emotion Portrayal Corpus," Blueprint for Affective Computing: A Sourcebook, pp. 271-294, Oxford Univ. Press, 2010.
100. M. Coulson, "Attributing Emotion to Static Body Postures: Recognition Accuracy, Confusions, and Viewpoint Dependence," J. Nonverbal Behavior, vol. 28, pp. 117-139, 2004. (Pubitemid 39015676)
101. M. de Meijer, "The Contribution of General Features of Body Movement to the Attribution of Emotions," J. Nonverbal Behavior, vol. 13, pp. 247-268, 1989.
102. R. De Silva and N. Bianchi-Berthouze, "Modeling Human Affective Postures: An Information Theoretic Characterization of Posture Features," J. Computer Animation and Virtual Worlds, vol. 15, nos. 3/4, pp. 269-276, 2004. (Pubitemid 41108005)
103. A. Kleinsmith and N. Bianchi-Berthouze, "Recognizing Affective Dimensions from Body Posture," Proc. Int'l Conf. Affective Computing and Intelligent Interaction, pp. 48-58, 2007.
104. R.A. Calvo and S. D'Mello, "Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications," IEEE Trans. Affective Computing, vol. 1, no. 1, pp. 18-37, Jan.-June 2010.
105. H. Gunes, M. Piccardi, and M. Pantic, "From the Lab to the Real World: Affect Recognition Using Multiple Cues and Modalities," Affective Computing: Focus on Emotion Expression, Synthesis, and Recognition, J. Or, ed., pp. 185-218, I-Tech Educational Publishing, 2008.
106. M. Pantic and L.J.M. Rothkrantz, "Automatic Analysis of Facial Expressions: The State of the Art," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1424-1445, Dec. 2000. (Pubitemid 32088469)
107. M.S. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan, "Recognizing Facial Expression: Machine Learning and Application to Spontaneous Behavior," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 568-573, 2005.
108. M. Rosenblum, Y. Yacoob, and L.S. Davis, "Human Emotion Recognition from Motion Using a Radial Basis Function Network Architecture," Proc. Workshop Motion of Non-Rigid and Articulated Objects, 1994.
109. P.Y. Oudeyer, "The Production and Recognition of Emotions in Speech: Features and Algorithms," Int'l J. Human-Computer Studies, vol. 59, pp. 157-183, 2003.
110. V. Kostov and S. Fukuda, "Emotion in User Interface, Voice Interaction System," Proc. IEEE Int'l Conf. Systems, Man, and Cybernetics, pp. 798-803, 2007.
111. C. Lee, S. Narayanan, and R. Pieraccini, "Classifying Emotions in Human-Machine Spoken Dialogs," Proc. IEEE Int'l Conf. Multimedia and Expo, 2002.
112. V. Petrushin, "Emotion in Speech: Recognition and Application to Call Centers," Proc. Artificial Neural Networks in Eng. Conf., pp. 7-10, 1999.
113. A. Camurri, B. Mazzarino, M. Ricchetti, R. Timmers, and G. Volpe, "Multimodal Analysis of Expressive Gesture in Music and Dance Performances," Gesture-Based Comm. in HCI, vol. 2915, pp. 20-39, 2004. (Pubitemid 38285426)
114. A. Kapur, A. Kapur, N. Virji-Babul, G. Tzanetakis, and P.F. Driessen, "Gesture-Based Affective Computing on Motion Capture Data," Proc. First Int'l Conf. Affective Computing and Intelligent Interaction, pp. 1-7, 2005. (Pubitemid 43763304)
115. F.E. Pollick, V. Lestou, J. Ryu, and S-B. Cho, "Estimating the Efficiency of Recognizing Gender and Affect from Biological Motion," Vision Research, vol. 42, pp. 2345-2355, 2002. (Pubitemid 35279169)
116. D. Bernhardt and P. Robinson, "Detecting Affect from Non-Stylised Body Motions," Proc. Second Int'l Conf. Affective Computing and Intelligent Interaction, pp. 59-70, 2007.
117. N. Bianchi-Berthouze and A. Kleinsmith, "A Categorical Approach to Affective Gesture Recognition," Connection Science, vol. 15, pp. 259-269, 2003. (Pubitemid 38474544)
118. N. Savva, A. Scarinzi, and N. Bianchi-Berthouze, "Continuous Recognition of Player's Affective Body Expression as Dynamic Quality of Aesthetic Experience," IEEE Trans. Computational Intelligence and AI in Games, vol. 4, no. 3, pp. 199-212, 2011.
119. L. Gong, T. Wang, C. Wang, F. Liu, F. Zhang, and X. Yu, "Recognizing Affect from Non-Stylized Body Motion Using Shape of Gaussian Descriptors," Proc. ACM Symp. Applied Computing, 2010.
120. D. Bernhardt and P. Robinson, "Detecting Emotions from Connected Action Sequences, Visual Informatics: Bridging Research and Practice," Proc. First Int'l Visual Informatics Conf. Visual Informatics: Bridging Research and Practice, pp. 1-11, 2009.
121. A. Kapoor, R.W. Picard, and Y. Ivanov, "Probabilistic Combination of Multiple Modalities to Detect Interest," Proc. 17th Int'l Conf. Pattern Recognition, vol. 3, pp. 969-972, 2004.
122. H. Gunes and M. Piccardi, "Bi-Modal Emotion Recognition from Expressive Face and Body Gestures," J. Network and Computer Applications, vol. 30, pp. 1334-1345, 2007. (Pubitemid 47263518)
123. H. Gunes and M. Piccardi, "Automatic Temporal Segment Detection and Affect Recognition from Face and Body Display," IEEE Trans. Systems, Man, and Cybernetics, Part B, vol. 39, no. 1, pp. 64-84, Feb. 2009.
124. C.F. Shan, S.G. Gong, and P.W. McOwan, "Beyond Facial Expressions: Learning Human Emotion from Body Gestures," Proc. British Machine Vision Conf., 2007.
125. H. Park, J. Park, U. Kim, and W. Woo, "Emotion Recognition from Dance Image Sequences using Contour Approximation," Proc. Int'l Workshop Structural, Syntactic, and Statistical Pattern Recognition, pp. 547-555, 2004.
126. A. Camurri, R. Trocca, and G. Volpe, "Interactive Systems Design: A KANSEI-Based Approach," Proc. Conf. New Interfaces for Musical Expression, pp. 1-8, 2002.
127. S. Kamisato, S. Odo, Y. Ishikawa, and K. Hoshino, "Extraction of Motion Characteristics Corresponding to Sensitivity Information Using Dance Movement," J. Advanced Computational Intelligence and Intelligent Informatics, vol. 8, no. 2, pp. 167-178, 2004.
128. A. Camurri, I. Lagerlof, and G. Volpe, "Recognizing Emotion from Dance Movement: Comparison of Spectator Recognition and Automated Techniques," Int'l J. Human-Computer Studies, vol. 59, nos. 1/2, pp. 213-225, 2003.
129. R. von Laban, Modern Educational Dance. MacDonald & Evans, Ltd., 1963.
130. Y. Ma, H.M. Paterson, and F.E. Pollick, "A Motion Capture Library for the Study of Identity, Gender, and Emotion Perception from Biological Motion," Behavior Research Methods, vol. 38, no. 1, pp. 134-141, 2006. (Pubitemid 44945608)
131. F.E. Pollick, H.M. Paterson, A. Bruderlin, and A.J. Sanford, "Perceiving Affect from Arm Movement," Cognition, vol. 82, pp. 51-61, 2001.
132. J. Sanghvi, G. Castellano, I. Leite, A. Pereira, P.W. McOwan, and A. Paiva, "Automatic Analysis of Affective Postures and Body Motion to Detect Engagement with a Game Companion," Proc. Int'l Conf. Human-Robot Interaction, 2011.
133. A. Kapoor, S. Mota, and R.W. Picard, "Towards a Learning Companion that Recognizes Affect," Technical Report 543, MIT Media Lab, 2001.
134. R. Kaliouby and P. Robinson, "Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures," Proc. IEEE Int'l Workshop Real Time Computer Vision for Human Computer Interaction at CVPR, 2004.
135. G. Varni, G. Volpe, and A. Camurri, "A System for Real-Time Multimodal Analysis of Nonverbal Affective Social Interaction in User-Centric Media," IEEE Trans. Multimedia, vol. 12, no. 6, pp. 576-590, Oct. 2010.
136. W.H. Dittrich, "Action Categories and the Perception of Biological Motion," Perception, vol. 22, no. 1, pp. 15-22, 1993.
137. A.P. Atkinson, "Impaired Recognition of Emotions from Body Movements is Associated with Elevated Motion Coherence Thresholds in Autism Spectrum Disorders," Neuropsychologia, vol. 47, no. 13, pp. 3023-3029, 2009.
138. H.L. Gallagher and C.D. Frith, "Dissociable Neural Pathways for the Perception and Recognition of Expressive and Instrumental Gestures," Neuropsychologia, vol. 42, no. 13, pp. 1725-1736, 2004. (Pubitemid 39194634)
139. A.J. Calder, A.M. Burton, P. Miller, A.W. Young, and S. Akamatsu, "A Principal Component Analysis of Facial Expressions," Vision Research, vol. 41, no. 9, pp. 1179-1208, 2001. (Pubitemid 32268615)
140. U. Martens, H. Leuthold, and S.R. Schweinberger, "On the Temporal Organization of Facial Identity and Expression Analysis: Inferences from Event-Related Brain Potentials," Cognitive, Affective, and Behavioral Neuroscience, vol. 10, no. 4, pp. 505-522, 2010.
141. P. Ekman and W.V. Friesen, Pictures of Facial Affect. Consulting Psychologists Press, ht tp://www.paulekman.com/researchproducts.phpt, Oct. 2008.
142. J. Fürnkranz and E. HÄullermeier, "Preference Learning," Kunstliche Intelligenz, vol. 19, no. 1, pp. 60-61, 2005.
143. J. Doyle, "Prospects for Preferences," Computational Intelligence, vol. 20, no. 2, pp. 111-136, 2004.
144. G.N. Yannakakis, "Preference Learning for Affective Modelling," Proc. Third Int'l Conf. Affective Computing and Intelligent Interaction, pp. 126-131, 2009.
145. H. Meng, A. Kleinsmith, and N. Bianchi-Berthouze, "Multi-Score Learning for Affect Recognition: the Case of Body Postures," Proc. Fourth Int'l Conf. Affective Computing and Intelligent Interaction, 2011.
146. A. Mehrabian and N. Epstein, "A Measure of Emotional Empathy," J. Personality, vol. 40, pp. 525-543, 1972.
147. V.S. Sheng, F. Provost, and P.G. Ipeirotis, "Get Another Label? Improving Data Quality and Data Mining Using Multiple, Noisy Labellers," Proc. 14th ACM SIGKDD Int'l Conf. Knowledge Disc and Data Mining, pp. 614-622, 2008.
148. V. Raykar, S. Yu, L. Zhao, G. Valadez, C. Florin, L. Bogoni, and L. Moy, "Learning from Crowds," J. Machine Learning Research, vol. 11, no. 7, pp. 1297-1322, 2010.
149. J. Howe, Crowd Sourcing: Why the Power of the Crowd Is Driving the Future of Business. Crown Publishing Group 2009.
150. A. Tarasov, C. Cullen, and S.J. Delany, "Using Crowdsourcing for Labelling Emotional Speech Assets," Proc. W3C Workshop Emotion Markup Language, 2010.
151. P. Juslin and P. Laukka, "Communication of Emotions in Vocal Expression and Music Performance," Psychological Bull., vol. 129, no. 5, pp. 770-814, 2002.
152. M. Kienast and W.F. Sendlmeier, "Acoustical Analysis of Spectral and Temporal Changes in Emotional Speech," Proc. ISCA Workshop Speech and Emotion, pp. 92-97, 2000.
153. L. Leinonen and T. Hiltunen, "Expression of Emotional-Motivational Connotations with a One-Word Utterance," J. Acoustical Soc. of Am., vol. 102, no. 3, pp. 1853-1863, 1997. (Pubitemid 27391368)
154. S. Yacoub, S. Simske, X. Lin, and J. Burns, "Recognition of Emotions in Interactive Voice Response Systems," Eurospeech: Proc. Eighth European Conf. Speech Comm. and Technology, 2003.
155. H.G. Wallbott and K.R. Scherer, "Cues and Channels in Emotion Recognition," J. Personality and Social Psychological, vol. 51, no. 4, pp. 690-699, 1986.
156. N. Sebe, Y. Sun, E. Bakker, M.S. Lew, I. Cohen, and T.S. Huang, "Towards Authentic Emotion Recognition," Proc. IEEE Int'l Conf. Systems, Man, and Cybernetics, pp. 623-628, 2004. (Pubitemid 40415928)
157. T. Bänziger and K.R. Scherer, "Using Actor Portrayals to Systematically Study Multimodal Emotion Expression: The GEMEP Corpus," Proc. Int'l Conf. Affective Computing and Intelligence Interaction, pp. 476-487, 2007.
158. G. Castellano, I. Leite, A. Pereira, C. Martinho, A. Paiva, and P. McOwan, "Affect Recognition for Interactive Companions: Challenges and Design in Real-World Scenarios," J. Multimodal User Interfaces, vol. 3, nos. 1/2, pp. 89-98, 2010.
159. A. Ashraf, S. Lucey, J. Cohn, T. Chen, K. Prkachin, and P. Solomon, "The Painful Face: Pain Expression Recognition Using Active Appearance Models," Image and Vision Computing, vol. 27, pp. 1788-1796, 2009.
160. S. Afzal and P. Robinson, "Natural Affect Data-Collection and Annotation in a Learning Context," Proc. Third Int'l Conf. Affective Computing and Intelligent Interaction, pp. 22-28, 2009.
161. N. Dael, M. Mortillaro, and K.R. Scherer, "Introducing the Body Action and Posture Coding System (BAP): Development and Reliability," manuscript dubmitted for publication.
162. H. Gunes and M. Piccardi, "A Bimodal Face and Body Gesture Database for Automatic Analysis of Human Nonverbal Affective Behavior," Proc. 18th Int'l Conf. Pattern Recognition, pp. 1148-1153, 2006. (Pubitemid 46532102)
163. M. Karg, R. Jenke, W. Seiberl, K. Kuhnlenz, A. Schwirtz, and M. Buss, "A Comparison of PCA, KPCA, and LDA for Feature Extraction to Recognize Affect in Gait Patterns," Proc. Third In'l Conf. Affective Computing and Intelligent Interaction, pp. 195-200, 2009.
164. P.J. Watson, C.K. Booker, C.J. Main, and A.C. Chen, "Surface Electromyography in the Identification of Chronic Low Back Pain Patients: The Development of the Flexion Relaxation Ratio," Clinical Biomechanics, vol. 12, no. 3, pp. 165-71, 1997. (Pubitemid 27247342)
165. M. Pluess, A. Conrad, and F.H. Wilhelm, "Muscle Tension in Generalized Anxiety Disorder: A Critical Review of the Literature," J. Anxiety Disorders, vol. 23, no. 1, pp. 1-11, 2009.
166. M.E. Geisser, A.J. Haig, A.S. Wallbom, and E.A. Wiggert, "Pain-Related Fear, Lumbar Flexion, and Dynamic EMG among Persons with Chronic Musculoskeletal Low Back Pain," Clinical J. Pain, vol. 20, no. 2, pp. 61-69, 2004. (Pubitemid 38252750)
167. M. Kamachi, M. Lyons, and J. Gyoba, The Japanese Female Facial Expression Database, http://www.kasrl.org/jaffe.html, retrieved Oct. 2008.
168. M. Pantic, M. Valstar, R. Rademaker, and L. Maat, "Web-Based Database for Facial Expression Analysis," Proc. IEEE Int'l Conf. Multimedia and Expo, 2005.
169. M. Pantic, M. Valstar, R. Rademaker, and L. Maat, The MMI Face Database, http://www.mmifacedb.com/, retrieved, Oct. 2008.
170. S. Baron-Cohen, O. Golan, S. Wheelright, and J. Hill, The Mindreading DVD, http://www.jkp.com/mindreading/, retrieved Oct. 2008.
171. The Psychological Image Collection at Stirling. http://pics.psych. stir.ac.uk/, retrieved Oct. 2008.
172. Center for the Study of Emotion and Attention. The International Affective Picture System. http://csea.phhp.udeg;.edu/media/iaps message.html, Retrieved, Oct. 2008.
173. T. Kanade, J.F. Cohn, and Y. Tian, "Comprehensive Database for Facial Expression Analysis," Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 46-53, 2000.
174. J.K. Burgoon, M.L. Jensen, T.O. Meservy, J. Kruse, and J.F. Nunamaker Jr., "Augmenting Human Identification of Emotional States in Video," Proc. Int'l Conf. Intelligent Data Analysis, 2005.
175. P. Ekman, "Lying and Deception," Memory for Everyday and Emotional Events, N.L. Stein, P.A. Ornstein, B. Tversky, and C. Brainerd, Lawrence Erlbaum Assoc., 1997.
176. B.M. DePaulo, J.J. Lindsay, B.E. Malone, L. Muhlenbruck, K. Charlton, and H. Cooper, "Cues to Deception," Psychological Bull., vol. 129, pp.74-112, 2003.
177. H. Gunes and M. Piccardi, "Observer Annotation of Affective Display and Evaluation of Expressivity: Face versus Face-and-Body," Proc. HCSNet Workshop Use of Vision in HCI, pp. 35-42, 2006.
178. C. Clavel, J. Plessier, J-C. Martin, L. Ach, and B. Morel, "Combining Facial and Postural Expressions of Emotions in a Virtual Character," Proc. Ninth Int'l Conf. Intelligent Virtual Agents, pp. 387-300, 2009.
179. D.A. Leopold, A.J. O'Toole, T. Vetter, and V. Blanz, "Prototype- Referenced Shape Encoding Revealed by High-Level Aftereffects," Nature Neuroscience, vol. 4, pp. 89-94, 2001. (Pubitemid 32095531)
180. J.A. Russell, "Core Affect and the Psychological Construction of Emotion," Psychological Rev., vol. 110, pp. 145-172, 2003. (Pubitemid 37170695)
181. M. Pasch, N. Bianchi-Berthouze, B. van Dijk, and A. Nijholt, "Movement-Based Sports Video Games: Investigating Motivation and Gaming Experience," Entertainment Computing, vol. 9, no. 2, pp. 169-180, 2009.
182. J. Nijhar, N. Bianchi-Berthouze, and G. Boguslawski, "Does Movement Recognition Precision Affect the Player Experience in Exertion Games?" Proc. Int'l Conf. Intelligent Technologies for Interactive Entertainment, pp. 73-82, 2012.
183. M. Gendron, K.A. Lindquist, L. Barsalou, and L.F. Barrett, "Emotion Words Shape Emotion Percepts," Emotion, vol. 12, pp. 314-325, 2012.
184. K.A. Lindquist, L.F. Barrett, E. Bliss-Moreau, and J.A. Russell, "Language and the Perception of Emotion," Emotion, vol. 6, pp. 125-138, 2006. (Pubitemid 43829902)
185. L.W. Barsalou, "Perceptual Symbol Systems," Behavioral and Brain Sciences, vol. 22, pp. 577-660, 1999.
186. P.M. Niedenthal, L.W. Barsalou, P. Winkielman, S. Krauth-Gruber, and F. Ric, "Embodiment in Attitudes, Social Perception, and Emotion," Personality and Social Psychological Rev., vol. 9, no. 3, pp. 184-211, 2005. (Pubitemid 41356682)
187. J. Chandler and N. Schwarz, "How Extending Your Middle Finger Affects Your Perception of Others: Learned Movements Influence Concept Accessibility," J. Experimental Social Psychology, vol. 45, no. 1, pp. 123-128, 2009.