AVLaughterCycle: Enabling a virtual agent to join in laughing with a conversational partner using a similarity-driven audiovisual laughter animation

Journal on Multimodal User Interfaces

Volumn 4, Issue 1, 2010, Pages 47-58

  Urbain, Jérôme ^a   Niewiadomski, Radoslaw ^b   Bevacqua, Elisabetta ^b   Dutoit, Thierry ^a   Moinet, Alexis ^a   Pelachaud, Catherine ^b   Picart, Benjamin ^a   Tilmanne, Joëlle ^a   Wagner, Johannes ^c  

^a UNIVERSITY OF MONS   (Belgium)

^b INSTITUT TELECOM   (France)

^c UNIVERSITY OF AUGSBURG   (Germany)

Author keywords

Acoustic similarity; Embodied Conversational Agent; Facial motion tracking; Laughter

Indexed keywords

APPLICATION ARCHITECTURE; CORE COMPONENTS; EMBODIED CONVERSATIONAL AGENT; FACIAL MOTIONS; FACIAL MOVEMENTS; LAUGHTER; REAL TIME; SIMILARITY SEARCH; SUBJECTIVE EVALUATIONS; UNDERLYING COMPONENTS; VIRTUAL AGENT; VISUAL CUES;

ANIMATION; DATABASE SYSTEMS; HUMAN COMPUTER INTERACTION; KNOWLEDGE MANAGEMENT; SEARCH ENGINES;

MAN MACHINE SYSTEMS;

EID: 78649634828     PISSN: 17837677     EISSN: 17838738     Source Type: Journal    
DOI: 10.1007/s12193-010-0053-1     Document Type: Article

References (41)

1. Bartlett MS, Littlewort GC, Frank MG, Lainscsek C, Fasel IR, Movellan JR (2006) Automatic recognition of facial actions in spontaneous expression. J Multimed 1(6): 22-35.
2. Becker-Asano C, Ishiguro H (2009) Laughter in social robotics-no laughing matter. In: Intl workshop on social intelligence design (SID2009), pp 287-300.
3. Berk L, Tan S, Napier B, Evy W (1989) Eustress of mirthful laughter modifies natural killer cell activity. Clin Res 37(1): 115A.
4. Cantoche (2010) http://www. cantoche. com/.
5. Cassell J, Bickmore T, Billinghurst M, Campbell L, Chang K, Vilhjálmsson H, Yan H (1999) Embodiment in conversational interfaces: Rea. In: Proceedings of the CHI'99 conference. ACM, New York, pp 520-527.
6. Curio C, Breidt M, Kleiner M, Vuong QC, Giese MA, Bülthoff HH (2006) Semantic 3D motion retargeting for facial animation. In: APGV'06: Proceedings of the 3rd symposium on applied perception in graphics and visualization. ACM, New York, pp 77-84.
7. DiLorenzo PC, Zordan VB, Sanders BL (2008) Laughing out loud. In: SIGGRAPH'08: ACM SIGGRAPH 2008. ACM, New York.
8. Dupont S, Dubuisson T, Urbain J, Frisson C, Sebbe R, D'Alessandro N (2009) Audiocycle: browsing musical loop libraries. In: Proc of IEEE content based multimedia indexing conference (CBMI09).
9. Haptek (2010) http://www. haptek. com/.
10. Heylen D, Kopp S, Marsella S, Pelachaud C, Vilhjálmsson H (2008) Why conversational agents do what they do? Functional representations for generating conversational agent behavior. In: The first functional markup language workshop, Estoril, Portugal.
11. Janin A, Baron D, Edwards J, Ellis D, Gelbart D, Morgan N, Peskin B, Pfau T, Shriberg E, Stolcke A, Wooters C (2003) The ICSI meeting corpus. In: 2003 IEEE international conference on acoustics, speech, and signal processing (ICASSP), Hong-Kong.
12. Knox MT, Mirghafori N (2007) Automatic laughter detection using neural networks. In: Proceedings of interspeech 2007, Antwerp, Belgium, pp 2973-2976.
13. Kopp S, Jung B, Leßmann N, Wachsmuth I (2003) Max-a multimodal assistant in virtual reality construction. Künstl Intell 17(4): 11-18.
14. Lasarcyk E, Trouvain J (2007) Imitating conversational laughter with an articulatory speech synthesis. In: Proceedings of the interdisciplinary workshop on the phonetics of laughter, Saarbrucken, Germany, pp 43-48.
15. Natural Point, Inc (2009) Optitrack-optical motion tracking solutions. http://www. naturalpoint. com/optitrack/.
16. Niewiadomski R, Bevacqua E, Mancini M, Pelachaud C (2009) Greta: an interactive expressive ECA system. In: Sierra C, Castelfranchi C, Decker KS, Sichman JS (eds) 8th international joint conference on autonomous agents and multiagent systems (AAMAS 2009), IFAAMAS, Budapest, Hungary, 10-15 May 2009, vol 2, pp 1399-1400.
17. Nijholt A (2002) Embodied agents: a new impetus to humor research. In: Proc Twente workshop on language technology 20 (TWLT 20), pp 101-111.
18. Ostermann J (2002) Face animation in MPEG-4. In: Pandzic IS, Forchheimer R (eds) MPEG-4 facial animation-the standard implementation and applications. Wiley, New York, pp 17-55.
19. Pantic M, Bartlett MS (2007) Machine analysis of facial expressions. In: Delac K, Grgic M (eds) Face recognition. I-Tech Education and Publishing, Vienna, pp 377-416.
20. Parke FI (1982) Parameterized models for facial animation. IEEE Comput Graph Appl 2(9): 61-68.
21. Peeters G (2004) A large set of audio features for sound description (similarity and classification) in the CUIDADO project. Tech rep, Institut de Recherche et Coordination Acoustique/Musique (IRCAM).
22. Petridis S, Pantic M (2009) Is this joke really funny? Judging the mirth by audiovisual laughter analysis. In: Proceedings of the IEEE international conference on multimedia and expo, New York, USA, pp 1444-1447.
23. Pighin F, Hecker J, Lischinski D, Szeliski R, Salesin DH (1998) Synthesizing realistic facial expressions from photographs. In: SIGGRAPH'98. ACM, New York, pp 75-84.
24. Ruch W, Ekman P (2001) The expressive pattern of laughter. In: Kaszniak A (ed) Emotion, qualia and consciousness. World Scientific, Singapore, pp 426-443.
25. Savran A, Sankur B (2009) Automatic detection of facial actions from 3D data. In: Proceedings of the IEEE 12th international conference on computer vision workshops (ICCV workshops), Kyoto, Japan, pp 1993-2000.
26. Schröder M (2003) Experimental study of affect bursts. Speech Commun 40(1-2): 99-116.
27. Siebert X, Dupont S, Fortemps P, Tardieu D (2009) MediaCycle: browsing and performing with sound and image libraries. In: Dutoit T, Macq B (eds) QPSR of the numediart research program, Numediart research program on digital art technologies, vol 2, pp 19-22.
28. Skype Communications S à rl (2009) The skype laughter chain. http://www. skypelaughterchain. com/.
29. Stoiber N, Seguier R, Breton G (2010) Facial animation retargeting and control based on a human appearance space. J Vis Comput Animat 21(1): 39-54.
30. Strommen E, Alexander K (1999) Emotional interfaces for interactive aardvarks: designing affect into social interfaces for children. In: Proceedings of ACM CHI'99, pp 528-535.
31. Sundaram S, Narayanan S (2007) Automatic acoustic synthesis of human-like laughter. J Acoust Soc Am 121(1): 527-535.
32. Thórisson KR, List T, Pennock C, DiPirro J (2005) Whiteboards: scheduling blackboards for semantic routing of messages & streams. In: Thórisson KR, Vilhjálmsson H, Marsella S (eds) Proc of AAAI-05 workshop on modular construction of human-like intelligence, Pittsburgh, Pennsylvania, pp 8-15.
33. Trouvain J (2003) Segmenting phonetic units in laughter. In: Proceedings of the 15th international congress of phonetic sciences, Barcelona, Spain, pp 2793-2796.
34. Truong KP, van Leeuwen DA (2007) Automatic discrimination between laughter and speech. Speech Commun 49(2): 144-158.
35. Truong KP, van Leeuwen DA (2007) Evaluating automatic laughter segmentation in meetings using acoustic and acoustic-phonetic features. In: Proceedings of the interdisciplinary workshop on the phonetics of laughter, Saarbrucken, Germany, pp 49-53.
36. Urbain J, Bevacqua E, Dutoit T, Moinet A, Niewiadomski R, Pelachaud C, Picart B, Tilmanne J, Wagner J (2010) AVLaughterCycle: an audiovisual laughing machine. In: Camurri A, Mancini M, Volpe G (eds) Proceedings of the 5th international summer workshop on multimodal interfaces (eNTERFACE'09). DIST-University of Genova, Genova.
37. Urbain J, Bevacqua E, Dutoit T, Moinet A, Niewiadomski R, Pelachaud C, Picart B, Tilmanne J, Wagner J (2010) The AVLaughterCycle database. In: Proceedings of the seventh conference on international language resources and evaluation (LREC'10), European Language Resources Association (ELRA), Valletta, Malta.
38. Vilhjálmsson H, Cantelmo N, Cassell J, Chafai NE, Kipp M, Kopp S, Mancini M, Marsella S, Marshall AN, Pelachaud C, Ruttkay Z, Thórisson KR, van Welbergen H, van der Werf R (2007) The behavior markup language: recent developments and challenges. In: 7th international conference on intelligent virtual agents, Paris, France, pp 99-111.
39. Wagner J, André E, Jung F (2009) Smart sensor integration: a framework for multimodal emotion recognition in real-time. In: Affective computing and intelligent interaction (ACII 2009), Amsterdam, The Netherlands, pp 1-8.
40. Zhang W, Wang Q, Tang X (2009) Performance driven face animation via non-rigid 3D tracking. In: MM '09: proceedings of the seventeen ACM international conference on multimedia. ACM, New York, pp 1027-1028.
41. Zign Creations: Zign Track (2009) http://www. zigncreations. com/zigntrack. html.