Human and Virtual Agent Expressive Gesture Quality Analysis and Synthesis

Coverbal Synchrony in Human-Machine Interaction

Volumn , Issue , 2013, Pages 269-292

  Niewiadomski, Radoslaw ^a   Mancini, Maurizio ^b   Piana, Stefano ^b  

^a TELECOM PARISTECH   (France)

^b UNIVERSITY OF GENOA   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84919379650     PISSN: None     EISSN: None     Source Type: Book    
DOI: None     Document Type: Chapter

References (64)

1. Allbeck, J. and N. Badler. 2003. Representing and Parameterizing Agent Behaviors. In H. Prendinger and M. Ishizuka (eds.), Life-like Characters: Tools, Affective Functions and Applications, Springer, Germany, pp. 19-38.
2. Argyle, M. 1998. Bodily Communication. Methuen and Co., London, 2nd edition.
3. Ball, G. and J. Breese. 2000. Emotion and Personality in a Conversational Agent. In J. Cassell, J. Sullivan, S. Prevost, and E. Churchill (eds.), Embodied Conversational Characters. MIT Press, Cambridge.
4. Balomenos, T., A. Raouzaiou, S. Ioannou, A. Drosopoulos, K. Karpouzis and S. Kollias. 2005. Emotion Analysis in Man-Machine Interaction Systems. In S. Bengio and H. Bourlard (eds.), Machine Learning for Multimodal Interaction, LNCS 3361, Springer Verlag, pp. 318-328.
5. Bernhardt, D. and P. Robinson. 2007. Detecting Affect from Non-stylised Body Motions. In A.C. Paiva, R. Prada, and R.W. Picard (eds.), Proceedings of the 2nd International Conference on Affective Computing and Intelligent Interaction (ACII ‘07), Springer Verlag, Berlin, Heidelberg, pp. 59-70.
6. Bianchi-Berthouze, N. 2012. Understanding the role of body movement in player engagement. Paper accepted to Human-Computer Interaction.
7. Bishko, L. 1991. The use of Laban-based analysis for the discussion of computer animation. In The 3rd Annual Conference of the Society for Animation Studies.
8. Bousmalis, K., M. Mehu and M. Pantic. 2009. Spotting Agreement and Disagreement: A Survey of Nonverbal Audiovisual Cues and Tools. In Proceedings of IEEE International Conference Affective Computing and Intelligent Interfaces, pp. 1-9.
9. Camurri, A., I. Lagerlöf and G. Volpe. 2003. Recognizing Emotion from Dance Movement: Comparison of Spectator Recognition and Automated Techniques. International Journal of Human-Computer Studies, 59(1-2):213-225.
10. Camurri, A., B. Mazzarino and G. Volpe. 2004. Analysis of Expressive Gesture: The EyesWeb Expressive Gesture Processing Library. In A. Camurri and G. Volpe (eds.), Gesture-based Communication in Human-Computer Interaction, LNAI 2915, Springer Verlag, pp. 460-467.
11. Camurri, A., B. Mazzarino, M. Ricchetti, R. Timmers and G. Volpe. 2004. Multimodal Analysis of Expressive Gesture in Music and Dance Performances. In A. Camurri and G. Volpe (eds.), Gesture-based Communication in Human-Computer Interaction, LNAI 2915, Springer Verlag, pp. 20-39.
12. Camurri, A., G. Castellano, M. Ricchetti and G. Volpe. 2006. Subject Interfaces: Measuring Bodily Activation during an Emotional Experience of Music. In S. Gibet, N. Courty, and J.F. Kamp (eds.), Gesture in Human-Computer Interaction and Simulation, volume 3881, Springer Verlag, pp. 268-279.
13. Camurri, A., Coletta, P., Varni, G. and Ghisio, S. (2007). Developing Multimodal Interactive Systems with EyesWeb XMI. In Proceedings of the 2007 Conference on New Interfaces for Musical Expression (NIME07), ACM, pp. 302-305.
14. Caridakis, G., A. Raouzaiou, E. Bevacqua, M. Mancini, K. Karpouzis, L. Malatesta and C. Pelachaud. 2007. Virtual Agent Multimodal Mimicry of Humans, In J.-C. Martin, P. Paggio, P. Kühnlein, R. Stiefelhagen, and F. Pianesi (eds.), Language Resources and Evaluation, special issue of Multimodal Corpora For Modelling Human Multimodal Behavior, Springer, Netherlands, pp. 367-388.
15. Cassell, J., J. Sullivan, S. Prevost and E.F. Churchill. 2000. Embodied Conversational Agents. The MIT Press, Cambridge, Massachusetts.
16. Castellano, G. 2006. Human full-body movement and gesture analysis for emotion recognition: A dynamic approach. Paper presented at HUMAINE Crosscurrents meeting, Athens.
17. Castellano, G., L. Kessous and G. Caridakis. 2007. Multimodal Emotion Recognition from Expressive Faces, Body Gestures and Speech. In F. de Rosis (ed.), Proceedings of the Doctoral Consortium of 2nd International Conference on Affective Computing and Intelligent Interaction.
18. Chi, D., M. Costa, L. Zhao and N. Badler. 2000. The EMOTE Model for Effort and Shape. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH 2000). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, pp. 173-182.
19. Drosopoulos, A., T. Balomenos, S. Ioannou, K. Karpouzis and S. Kollias. 2003. Emotionally-Rich Man-Machine Interaction Based on Gesture Analysis. Human-Computer Interaction International, in the procedings of the HCI International conference, Greece 4:1372-1376.
20. Fitts, P.M. 1954. The Information Capacity of the Human Motor System in Controlling the Amplitude of Movement. Journal of Experimental Psychology, 47(6):381-391.
21. Gallaher, P.E. 1992. Individual Differences in Nonverbal Behavior: Dimensions of Style. Journal of Personality and Social Psychology, 63(1):133-145.
22. Glowinski, D., N. Dael, A. Camurri, G. Volpe, M. Mortillaro and K. Scherer. 2011. Toward a Minimal Representation of Affective Gestures, IEEE Transcations on Affective Computing, 2(2):106-118.
23. Goldberg, L.R. 1993. The Structure of Phenotypic Personality Traits. American Psychologist, 48(1):26-34.
24. Grammer, K., K. Kruck, A. Juette and B. Fink. 2000. Non-Verbal Behavior as Courtship Signals: The Role of Control and Choice in Selecting Partners. Evolution and Human Behavior, 21(6):371-390.
25. Gunes, H. and M. Piccardi, 2005. Fusing Face and Body Display for Bi-modal Emotion Recognition: Single Frame Analysis and Multi-frame Post Integration. In Proceedings of the First international conference on Affective Computing and Intelligent Interaction, pp. 102-111.
26. Hartmann, B., M. Mancini, S. Buisine and C. Pelachaud. 2005. Design and Evaluation of Expressive Gesture Synthesis for Embodied Conversational Agents. In Third International Joint Conference on Autonomous Agents and Multi-Agent Systems, Utrecht, Holland, pp. 1095-1096.
27. Hsieh, C.-M. and A. Luciani. 2005. Generating Dance Verbs and Assisting Computer Choreography. ACM Multimedia, Singapore, 12:774-782.
28. Hsieh, C.-M. and A. Luciani 2006. Minimal Dynamic Modeling for Dance Verbs. Journal of Visualization and Computer Animation, 17:359-369.
29. Hung, H. and D. Gatica-Perez. 2010. Estimating Cohesion in Small Groups Using Audio-Visual Nonverbal Behavior. IEEE Transactions on Multimedia, 12(6):563-575.
30. Jayagopi, D., H. Hung, C. Yeo and D. Gatica-Perez. 2009. Modeling Dominance in Group Conversations Using Non-verbal Activity Cues. IEEE Trans. on Audio, Speech, and Language Processing, Special Issue on Multimodal Processing for Speech-based Interactions, 17(3):501-513.
31. Johansson, G. 1973. Visual Perception of Biological Motion and a Model for Its Analysis. Perception and Psychophysics, 14:201-211.
32. Kleinsmith, A. and N. Bianchi-Berthouze. 2011. Form as a Cue in the Automatic Recognition of Non-acted Affective Body Expressions. Proceedings of Affective Computing and Intelligent Interaction 2011, LNCS 6974, Springer Berlin Heidelberg, pp. 155-164.
33. Kleinsmith, A., N. Bianchi-Berthouze and A. Steed 2011. Automatic Recognition of Non-acted Affective Postures. IEEE Transactions on Systems, Man, and Cybernetics, 41(4):1027-1038.
34. Kopp, S., T. Sowa and I. Wachsmuth. 2003. Imitation Games with an Artificial Agent: From Mimicking to Understanding Shape-related Iconic Gestures. In A. Camurri and G. Volpe (eds.), Gesture-based Communication in Human-Computer Interaction, LNAI 2915, Springer Verlag, pp. 436-447.
35. Kostek, B. and Szczuko, P. (2006). Rough Set-Based Application to Recognition of Emotionally-Charged Animated Character’s Gestures, Transactions on Rough Sets V, LNCS 4100, Springer Berlin Heidelberg, pp. 146-166. http://link.springer.com/chapter/10.1007%2F11847465_7
36. Laban, R. and Lawrence, F.C. (1947). Effort. Macdonald and Evans, USA.
37. Lakens, D. and M. Stel. 2011. If They Move in Sync, They Must Feel in Sync: Movement Synchrony Leads to Attributions of Rapport and Entitativity. Social Cognition, 29(1):1-14.
38. Le, Q.A., S. Hanoune and C. Pelachaud. 2011. Design and Implementation of an Expressive Gesture Model for a Humanoid Robot. In 11th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2011), Bled, Slovenia, pp. 134-140.
39. Mancini, M. and G. Castellano. 2007. Real-time Analysis and Synthesis of Emotional Gesture Expressivity. In Proceedings of the Doctoral Consortium of 2nd International Conference on Affective Computing and Intelligent Interaction.
40. Mazzarino, B., M. Peinado, R. Boulic, G. Volpe and M.M. Wanderley. 2007. Improving the Believability of Virtual Characters Using Qualitative Gesture Analysis. In S. Gibet, N. Courty, and J.F. Kamp (eds.), Gesture in Human-Computer Interaction and Simulation, volume 3881, Springer Verlag, pp. 48-56.
41. Mazzarino, B. and M. Mancini. 2009. The Need for Impulsivity and Smoothness-Improving HCI by Qualitatively Measuring New High-Level Human Motion Features. Proceedings of the International Conference on Signal Processing and Multimedia Applications SIGMAP, Milan, Italy, pp. 62-67.
42. McNeill, D. 1996. Hand and Mind: What Gestures Reveal about Thought. University of Chicago Press, Chicago, Illinois.
43. Meijer, M. 1989. The Contribution of General Features of Body Movement to the Attribution of Emotions. Journal of Nonverbal Behavior, 13(4):247-268.
44. Neff, M. and E. Fiume. 2000. Modeling tension and relaxation for computer animation. In Proceedings of the 2002 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA ‘02), pp. 81-88.
45. Neff, M. and E. Fiume. 2003. Aesthetic edits for Character Animation. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA ‘03), pp. 239-244.
46. Niewiadomski, R., E. Bevacqua, Q.A. Le, M. Obaid, J. Looser and C. Pelachaud. 2011. Cross-media Agent Platform. In Proceedings of 2011 Web3D ACM Conference, Paris, France, pp. 11-19.
47. Ortony, A., G.L. Clore and A. Collins. 1988. The Cognitive Structure of Emotions. Cambridge University Press, Cambridge.
48. Paiva, A., R. Chaves, M. Piedade, A. Bullock, G. Andersson and K. Höök. 2003. Sentoy: A Tangible Interface to Control the Emotions of a Synthetic Character. In AAMAS ‘03: Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems, New York, NY, ACM Press, pp. 1088-1089.
49. Piana, S., M. Mancini and A. Camurri. 2012. Automated Analysis of Non-verbal Expressive Gesture. Sixth International Workshop on Human Aspects in Ambient Intelligence, International Joint Conference on Ambient Intelligence, Pisa, Italy.
50. Pollick, F.E. 2004. The Features People Use to Recognize Human Movement Style. In A. Camurri and G. Volpe (eds.), Gesture-based Communication in Human-Computer Interaction, LNAI 2915, Springer Verlag, pp. 10-19.
51. Pugliese, R. and K. Lehtonen. 2011. A Framework for Motion Based Bodily Enaction with Virtual Characters, In H. Vilhjälmsson, S. Kopp, S. Marsella, and K. Thórisson (eds.), Proceedings of Intelligent Virtual Agents 2011, LNCS 6895, Springer, Berlin/Heidelberg, pp. 162-168.
52. Rehm, M. 2010. Non-symbolic Gestural Interaction for AmI. In H. Aghajan, R.L.-C. Delgado, and J.C. Augusto (eds.), Human-Centric Interfaces for Ambient Intelligence, ACM Press, Burlington: Elsevier Science, pp. 327-345.
53. Reidsma, D., A. Nijholt, R. Poppe, R. Rienks and H. Hondorp. 2006. Virtual Rap Dancer: Invitation to Dance. In Conference on Human Factors in Computing Systems, Montréal, Québec, Canada. ACM Press, pp. 263-266.
54. Sanghvi, J., G. Castellano, I. Leite, A. Pereira, P.W. McOwan and A. Paiva. 2011. Automatic Analysis of Affective Postures and Body Motion to Detect Engagement with a Game Companion. In Proceedings of the 6th International Conference on Human-Robot Interaction, New York, NY. ACM Press, pp. 305-312.
55. Schröder, M., E. Bevacqua, R. Cowie, F. Eyben, H. Gunes, D. Heylen, M. ter Maat, G. McKeown, S. Pammi, M. Pantic, C. Pelachaud, B. Schuller, E. de Sevin, M. Valstar and M. Wollmer. 2011. Building Autonomous Sensitive Artificial Listeners. IEEE Transactions on Affective Computing, 3(2):165-183.
56. Szczuko, P., B. Kostek and A. Czyzewski. 2009. New Method for Personalization of Avatar Animation. In K. Cyran, S. Kozielski, J. Peters, U. Stanczyk and A. Wakulicz-Deja (eds.), Man-Machine Interactions, Advances in Intelligent and Soft Computing, Springer Berlin Heidelberg, pp. 435-443.
57. Taylor, R., D. Torres and P. Boulanger. 2005. Using Music to Interact with a Virtual Character. In The 2005 International Conference on New Interfaces for Musical Expression, pp. 220-223.
58. Tsuruta, S., W. Choi and K. Hachimura. 2010. Generation of Emotional Dance Motion for Virtual Dance Collaboration System. Digital Humanities, In Procedings of DH Humanities 2010, London, UK, pp. 368-371.
59. Urbain, J., R. Niewiadomski, E. Bevacqua, T. Dutoit, A. Moinet, C. Pelachaud, B. Picart, J. Tilmanne and J. Wagner. 2010. AVLaughterCycle. Enabling a Virtual Agent to Join in Laughing with a Conversational Partner Using a Similarity-driven Audiovisual Laughter Animation. Journal of Multimodal User Interfaces, 4(1):47-58.
60. Varni, G., A. Camurri, P. Coletta and G. Volpe. 2009. Toward a Real-Time Automated Measure of Empathy and Dominance. CSE, 4:843-848.
61. Wallbott, H.G. 1998. Bodily Expression of Emotion. European Journal of Social Psychology, 28:879-896.
62. Wallbott, H.G. and K.R. Scherer. 1986. Cues and Channels in Emotion Recognition. Journal of Personality and Social Psychology, 51(4):690-699.
63. Wilson, A., A. Bobick and J. Cassell. 1996. Recovering the Temporal Structure of Natural Gesture. In Proceedings of the Second International Conference on Automatic Face and Gesture Recognition, Killington, pp. 66-71.
64. Winter, D. 1990. Biomechanics and Motor Control of Human Movement. John Wiley and Sons, Toronto