Expressive copying behavior for social agents: A perceptual analysis

IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans

Volumn 42, Issue 3, 2012, Pages 776-783

  Castellano, Ginevra ^a   Mancini, Maurizio ^b   Peters, Christopher ^c   McOwan, Peter W ^d  

^a UNIVERSITY OF BIRMINGHAM   (United Kingdom)

^b UNIVERSITY OF GENOA   (Italy)

^c Faculty of Engineering and Computing   (United Kingdom)

^d QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

Author keywords

Copying behavior; movement expressivity; social agents

Indexed keywords

EMBODIED VIRTUAL AGENTS; HUMAN INTERACTIONS; INTERACTORS; MOVEMENT EXPRESSIVITY; PERCEPTUAL ANALYSIS; SOCIAL AGENTS;

VIDEO RECORDING;

EID: 84860129067     PISSN: 10834427     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCA.2011.2172415     Document Type: Article

References (54)

1. R. W. Picard, Affective Computing. Cambridge, MA: MIT Press, Sep. 1997.
2. C. Breazeal, "Role of expressive behaviour for robots that learn from people," Philos. Trans. Roy. Soc. B, vol. 364, no. 1535, pp. 3527-3538, Dec. 2009.
3. EU I-SEARCH Project. [Online]. Available: Www.isearch-project.eu
4. C. Conati, S. Marsella, and A. Paiva, "Affective interactions: The computer in the affective loop," in Proc. 10th Int. Conf. IUI, New York, 2005, p. 7-7.
5. K. Höök, "Affective loop experiences: Designing for interactional embodiment," Philos. Trans. Roy. Soc. B, vol. 364, no. 1535, pp. 3585-3595, Dec. 2009.
6. D. Robinson and L. Smith-Lovin, "Emotion display as a strategy for identity negotiation," Motiv. Emotion, vol. 23, no. 2, pp. 73-104, 1999.
7. M. Álvarez, R. Galn, F.Matía, D. Rodríguez-Losada, and A. Jiménez, "An emotional model for a guide robot," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 40, no. 5, pp. 982-992, Sep. 2010.
8. R. Van Baaren, R. Holland, K. Kawakami, and A. van Knippenberg, "Mimicry and prosocial behavior," Psychol. Sci., vol. 15, no. 1, pp. 71-74, Jan. 2004.
9. S. Baron-Cohen, "The empathizing system: A revision of the 1994 model of the mindreading system," in Origins of the Social Mind, B. Ellis and D. Bjorklund, Eds. London, U.K.: Guilford Publ. Inc., 2005.
10. C. Peters, "A perceptually-based theory of mind model for agent interaction initiation," International Journal of Humanoid Robotics (IJHR), Special Issue: Achieving Human-Like Qualities in Interactive Virtual and Physical Humanoids, vol. 3, no. 3, pp. 321-340, 2006.
11. Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang, "A survey of affect recognition methods: Audio, visual, and spontaneous expressions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 1, pp. 39-58, Jan. 2009.
12. R. el Kaliouby and P. Robinson, "Real-time inference of complex mental states from facial expressions and head gestures," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit.-Workshop Real Time Comput. Vis. Human Comput. Interact., 2004, p. 154.
13. A. Chakraborty, A. Konar, U. Chakraborty, and A. Chatterjee, "Emotion recognition from facial expressions and its control using fuzzy logic," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 39, no. 4, pp. 726-743, Jul. 2009.
14. G. Castellano, G. Caridakis, A. Camurri, K. Karpouzis, G. Volpe, and S. Kollias, "Body gesture and facial expression analysis for automatic affect recognition," in Blueprint for Affective Computing: A Sourcebook, K. R. Scherer, T. Bänziger, and E. B. Roesch, Eds. Oxford, U.K.: Oxford Univ. Press, 2010.
15. B. Schuller, R. J. Villar, G. Rigoll, and M. Lang, "Meta-classifiers in acoustic and linguistic feature fusion-based affect recognition," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Philadelphia, PA, 2005, pp. 325-328.
16. H. Gunes and M. Piccardi, "Automatic temporal segment detection and affect recognition from face and body display," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 39, no. 1, pp. 64-84, Feb. 2009.
17. G. Castellano, A. Pereira, I. Leite, A. Paiva, and P. W. McOwan, "Detecting user engagement with a robot companion using task and social interaction-based features," in Proc. ICMI-MLMI, Cambridge,MA, 2009, pp. 119-126.
18. C. Breazeal, "Regulation and entrainment in human-robot interaction," Int. J. Robot. Res., vol. 21, no. 10/11, pp. 883-902, Oct. 2002.
19. K. Dautenhahn, "Trying to imitate-A step towards releasing robots from social isolation," in Proc. Perception Actions Conf., 1994, pp. 290-301.
20. J. Gratch, A. Okhmatovskaia, F. Lamothe, S. Marsella, M. Morales, R. J. van der Werf, and L.-P. Morency, "Virtual rapport," in Proc. 6th Int. Conf. IVA, Marina del Rey, CA, 2006, pp. 14-27.
21. R. Byrne and A. Russon, "Learning by imitation: A hierarchical approach," Behav. Brain Sci., vol. 21, no. 5, pp. 667-684, Oct. 1998.
22. K. R. Scherer, "On the nature and function of emotion: A component process approach," in Approaches to Emotion, K. R. Scherer and P. Ekman, Eds. Hillsdale, NJ: Erlbaum, 1984, pp. 293-317.
23. A. Camurri, B. Mazzarino, M. Ricchetti, R. Timmers, and G. Volpe, "Multimodal analysis of expressive gesture in music and dance performances," in Gesture-Based Communication in Human-Computer Interaction, vol. 2915, A. Camurri and G. Volpe, Eds. Berlin, Germany: Springer Verlag, 2004, ser. LNAI, pp. 20-39.
24. R. T. Boone and J. G. Cunningham, "Children's decoding of emotion in expressive body movement: The development of cue attunement," Dev. Psychol., vol. 34, no. 5, pp. 1007-1016, Sep. 1998.
25. H. G. Wallbott, "Bodily expression of emotion," Eur. J. Soc. Psychol., vol. 28, no. 6, pp. 879-896, Dec. 1998.
26. P. Ekman andW. Friesen, "Detecting deception from the body or face," J. Person. Social Psychol., vol. 29, no. 3, pp. 288-298, 1974.
27. M. De Meijer, "The contribution of general features of body movement to the attribution of emotions," J. Nonverbal Behav., vol. 13, no. 4, pp. 247-268, Dec. 1989.
28. B. de Gelder, "Why bodies? Twelve reasons for including bodily expressions in affective neuroscience," Philos. Trans. Roy. Soc. B, vol. 364, no. 1535, pp. 3575-3584, Dec. 2009.
29. D. Bernhardt and P. Robinson, "Detecting affect from non-stylised body motions," in Proc. 2nd Int. Conf. ACII, vol. 4738, ser. LNCS, A. Paiva, R. Prada, and R. W. Picard, Eds., Lisbon, Portugal, Sep. 12-14, 2007, pp. 59-70, Springer-Verlag.
30. G. Castellano, S. D. Villalba, and A. Camurri, "Recognising human emotions from body movement and gesture dynamics," in Proc. 2nd Int. Conf. ACII, vol. 4738, ser. LNCS, A. Paiva, R. Prada, and R. W. Picard, Eds., Lisbon, Portugal, Sep. 12-14, 2007, pp. 71-82, Springer-Verlag.
31. G. Castellano, M. Mortillaro, A. Camurri, G. Volpe, and K. Scherer, "Automated analysis of body movement in emotionally expressive piano performances," Music Perception, vol. 26, no. 2, pp. 103-119, Dec. 2008.
32. J. Sanghvi, G. Castellano, I. Leite, A. Pereira, P. W. McOwan, and A. Paiva, "Automatic analysis of affective postures and body motion to detect engagement with a game companion," in Proc. ACM/ IEEE Int. Conf. Human-Robot Interact., Lausanne, Switzerland, 2011, pp. 305-312.
33. A. Kleinsmith, N. Bianchi-Berthouze, and A. Steed, "Automatic recognition of non-acted affective postures," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 41, no. 4, pp. 1027-1038, Aug. 2011.
34. J. Cassell, "More than just another pretty face: Embodied conversational interface agents," Commun. ACM, vol. 43, no. 4, pp. 70-78, 2000.
35. C. Peters, G. Castellano, M. Rehm, E. Andre, G. Volpe, A. Camurri, A. Raouzaiou, K. Rapantzikos, K. Karpouzis, and A. Vasalou, "Fundamentals of agent perception and attention modelling," in Emotion-Oriented Systems: The Humaine Handbook, P. Petta, C. Pelachaud, and R. Cowie, Eds. New York: Springer-Verlag, 2011.
36. R. M. Maatman, J. Gratch, and S. Marsella, "Natural behavior of a listening agent," in Proc. 5th Int. Conf. IVA, Kos, Greece, 2005, pp. 25-36.
37. S. Kopp, T. Sowa, and I. Wachsmuth, "Imitation games with an artificial agent: From mimicking to understanding shape-related iconic gestures," in Gesture-Based Communication in Human-Computer Interaction, vol. 2915, A. Camurri and G. Volpe, Eds. Berlin, Germany: Springer Verlag, 2004, ser. LNAI, pp. 436-447.
38. D. Reidsma, A. Nijholt, R. Poppe, R. Rienks, and H. Hondorp, "Virtual rap dancer: Invitation to dance," in Proc. CHI Extended Abstracts Human Factors Comput. Syst., 2006, pp. 263-266.
39. L. Riek and P. Robinson, "Real-time empathy: Facial mimicry on a robot," in Proc. Workshop AFFINE ICMI, Chania, Crete, Greece, 2009.
40. J. C. Lester, S. G. Towns, and P. J. FitzGerald, "Achieving affective impact: Visual emotive communication in lifelike pedagogical agents," Proc. Int. J. Artif. Intell. Educ., vol. 10, pp. 278-291, 1999.
41. T. Bickmore and R. W. Picard, "Establishing and maintaining long-term human-computer relationships," ACM Trans. Comput.-Human Interact., vol. 12, no. 2, pp. 293-327, Jun. 2004.
42. A. Camurri, P. Coletta, G. Varni, and S. Ghisio, "Developing multimodal interactive systems with EyesWeb XMI," in Proc. Conf. New Interfaces Musical Exp., 2007, pp. 305-308.
43. C. Pelachaud, "Multimodal expressive embodied conversational agents," in Proc. 13th Annu. ACM Int. Conf. Multimedia, 2005, pp. 683-689.
44. G. Castellano and M. Mancini, "Analysis of emotional gestures for the generation of expressive copying behaviour in an embodied agent," in Proc. Adv. Gesture-Based Human-Comput. Interact. Simul.: 7th Int. GW, Lisbon, Portugal, May 2007, Revised Selected Papers, vol. 5085, ser. LNAI, M. S. Dias, S. Gibet, M. Wanderley, and R. Bastos, Eds., Berlin, Germany, 2009, pp. 193-198, Springer-Verlag.
45. K. R. Thórisson, T. List, C. Pennock, and J. DiPirro, "Whiteboards: Scheduling blackboards for interactive robots," in Proc. AAAI Workshop Modular Construct. Human-Like Intell., Pittsburgh, PA, Jul. 9-13, 2005.
46. T. Bänziger and K. Scherer, "Using actor portrayals to systematically study multimodal emotion expression: The GEMEP Corpus," in Affective Computing and Intelligent Interaction 2007, vol. 4738, R. W. Picard, Ed. Berlin, Germany: Springer-Verlag, 2007, ser. LNCS, pp. 476-487.
47. T. Banziger and K. Scherer, "Introducing the Geneva Multimodal Emotion Portrayal (GEMEP) Corpus," in Blueprint for Affective Computing: A Sourcebook, K. R. Scherer, T. Bänziger, and E. B. Roesch, Eds. Oxford, U.K.: Oxford University Press, 2010, pp. 271-294.
48. A. Camurri, B. Mazzarino, and G. Volpe, "Analysis of expressive gesture: The Eyesweb Expressive Gesture Processing Library," in Gesture-Based Communication in Human-Computer Interaction, vol. 2915, A. Camurri and G. Volpe, Eds. Berlin, Germany: Springer Verlag, 2004, ser. LNAI, pp. 460-467.
49. H. G. Wallbott and K. R. Scherer, "Cues and channels in emotion recognition," J. Person. Soc. Psychol., vol. 51, no. 4, pp. 690-699, 1986.
50. G. Volpe, "Computational models of expressive gesture in multimedia systems," Ph.D. Dissertation, Faculty of Eng., Univ. Genova, Genoa, Italy, Apr. 2003.
51. B. Hartmann, M. Mancini, S. Buisine, and C. Pelachaud, "Design and evaluation of expressive gesture synthesis for embodied conversational agents," in Proc. 3rd Int. Joint Conf. AAMAS, Utretch, Jul. 2005, pp. 1095-1096.
52. D. Tolani, "Inverse kinematics methods for human modeling and simulation," Ph.D. dissertation, Univ. Pennsylvania, Philadelphia, PA, 1998.
53. B. Hartmann, M. Mancini, and C. Pelachaud, "Implementing expressive gesture synthesis for embodied conversational agents," in Proc. Gesture Workshop, 2005, pp. 188-199.
54. D. McNeill, Hand and Mind: What Gestures Reveal About Thought. Chicago, IL: Univ. Chicago Press, 1992.