Building autonomous sensitive artificial listeners (Extended abstract)

2015 International Conference on Affective Computing and Intelligent Interaction, ACII 2015

Volumn , Issue , 2015, Pages 456-462

  Schroder, Marc ^h   Bevacqua, Elisabetta ^e,i   Cowie, Roddy ^f   Eyben, Florian ^a,j   Gunes, Hatice ^b,d   Heylen, DIrk ^g   Ter Maat, Mark ^g   McKeown, Gary ^f   Pammi, Sathish ^h   Pantic, Maja ^b   Pelachaud, Catherine ⁱ   Schuller, Bjorn ^a,b,j   De Sevin, Etienne ⁱ   Valstar, Michel ^b,c   Wollmer, Martin ^a,j  

^a AUDEERING GMBH   (Germany)

^b IMPERIAL COLLEGE LONDON   (United Kingdom)

^c UNIVERSITY OF NOTTINGHAM   (United Kingdom)

^d QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

^e National Engineering School of Brest   (France)

^f QUEEN'S UNIVERSITY BELFAST   (United Kingdom)

^g UNIVERSITY OF TWENTE   (Netherlands)

^h GERMAN RESEARCH CENTER FOR ARTIFICIAL INTELLIGENCE DFKI   (Germany)

ⁱ TELECOM PARISTECH   (France)

^j TECHNICAL UNIVERSITY OF MUNICH   (Germany)

more..

Author keywords

[No Author keywords available]

Indexed keywords

INTELLIGENT COMPUTING; INTERACTIVE COMPUTER SYSTEMS; MODELING LANGUAGES; REAL TIME SYSTEMS; SPEECH; SPEECH PROCESSING; SPEECH RECOGNITION;

AFFECTIVE COMPUTING; DIALOGUE MANAGEMENT; EXTENDED ABSTRACTS; INCREMENTAL ANALYSIS; MULTIMODAL DIALOGUE SYSTEMS; NON-VERBAL BEHAVIOURS; SPOKEN LANGUAGE UNDERSTANDING; VERBAL INTERACTION;

BEHAVIORAL RESEARCH;

EID: 84964036733     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ACII.2015.7344610     Document Type: Conference Paper

References (44)

1. R. Cowie, "Describing the forms of emotional colouring that pervade everyday life," in The Oxford Handbook of Philosophy of Emotion, P. Goldie, Ed. Oxford University Press, 2010, pp. 63-94, doi:10.1093/oxfordhb/9780199235018.003.0004.
2. V. H. Yngve, "On getting a word in edgewise," in Chicago Linguistic Society. Papers from the 6th regional meeting, vol. 6, 1970, pp. 567-577.
3. J. Allwood, J. Nivre, and E. Ahlsén, "On the semantics and pragmatics of linguistic feedback," Journal of Semantics, vol. 9, no. 1, pp. 1-26, 1992. [Online]. Available
4. M. Schroder, E. Bevacqua, R. Cowie, F. Eyben, H. Gunes, D. Heylen, M. ter Maat, G. McKeown, S. Pammi, M. Pantic, C. Pelachaud, B. Schuller, E. de Sevin, M. Valstar, and M. Wollmer, "Building autonomous sensitive artificial listeners," IEEE Transactions on Affective Computing, vol. 3, no. 2, pp. 165-183, April-June 2012.
5. G. McKeown, M. Valstar, R. Cowie, M. Pantic, and M. Schroder, "The SEMAINE database: Annotated multimodal records of emotionally coloured conversations between a person and a limited agent," IEEE Transactions on Affective Computing, 2011.
6. M. Schroder, "The SEMAINE API: towards a standards-based framework for building emotion-oriented systems," Advances in Human-Computer Interaction, vol. 2010, no. 319406, 2010. [Online]. Available
7. M. Wollmer, B. Schuller, F. Eyben, and G. Rigoll, "Combining long short-term memory and dynamic bayesian networks for incremental emotion-sensitive artificial listening," IEEE Journal of Selected Topics in Signal Processing, vol. 4, no. 5, pp. 867-881, 2010.
8. F. Eyben, M. Wollmer, M. F. Valstar, H. Gunes, B. Schuller, and M. Pantic, "String-based audiovisual fusion of behavioural events for the assessment of dimensional affect," in Proc. IEEE Intl. Conf. Automatic Face and Gesture Recognition, Santa Barbara, CA, USA, 2011, pp. 322-329.
9. H. Gunes and M. Pantic, "Dimensional emotion prediction from spontaneous head gestures for interaction with sensitive artificial listeners," in Proc. Intelligent Virtual Agents, Philadelphia, USA, 2010, p. 371-377.
10. B. Jiang, M. F. Valstar, and M. Pantic, "Action unit detection using sparse appearance descriptors in space-time video volumes," in Proc. Face and Gesture recognition, Santa Barbara, CA, USA, 2011, pp. 314-321.
11. M. ter Maat, K. P. Truong, and D. Heylen, "How Turn-Taking strategies influence users' impressions of an agent," in Proc. Intelligent Virtual Agents, Philadelphia, USA, 2010, pp. 441-453. [Online]. Available
12. E. Bevacqua, E. de Sevin, C. Pelachaud, M. McRorie, and I. Sneddon, "Building credible agents: Behaviour influenced by personality and emotional traits," in Proc. Kansei Engineering and Emotion Research, Paris, France, 2010.
13. J. Cassell, T. Bickmore, L. Campbell, H. Vilhjalmsson, and H. Yan, "Human conversation as a system framework: designing embodied conversational agents," in Embodied conversational agents. MIT Press, 2000, pp. 29-63. [Online]. Available
14. S. Hyniewska, R. Niewiadomski, M. Mancini, and C. Pelachaud, "Expression of affects in embodied conversational agents," in A blueprint for Affective Computing, K. R. Scherer, T. Banziger, and E. B. Roesch, Eds. Oxford University Press, 2010.
15. M. Schroder, "Expressive speech synthesis: Past, present, and possible futures," in Affective Information Processing, J. Tao and T. Tan, Eds. London: Springer, 2009, pp. 111-126. [Online]. Available
16. K. van Deemter, B. Krenn, P. Piwek, M. Klesen, M. Schroder, and S. Baumann, "Fully generated scripted dialogue for embodied agents," Artificial Intelligence, vol. 172, no. 10, pp. 1219-1244, Jun. 2008.
17. B. Kempe, N. Pfleger, and M. Lockelt, "Generating verbal and nonverbal utterances for virtual characters," in Virtual Storytelling. Springer, 2005, pp. 73-76. [Online]. Available
18. J. A. Russell, "A circumplex model of affect," Journal of Personality and Social Psychology, vol. 39, pp. 1161-1178, 1980.
19. R. Cowie, E. Douglas-Cowie, S. Savvidou, E. McMahon, M. Sawey, and M. Schroder, "'FEELTRACE': an instrument for recording perceived emotion in real time," in Proc. ISCA Workshop on Speech and Emotion, Northern Ireland, 2000, pp. 19-24.
20. R. Cowie, C. Cox, J.-C. Martin, A. Batliner, D. Heylen, and K. Karpouzis, "Issues in Data Labelling," in Emotion-Oriented Systems: The Humaine Handbook, R. Cowie, C. Pelachaud, and P. Petta, Eds. Berlin, Heidelberg: Springer-Verlag, oct 2010, pp. 213-241.
21. R. Cowie and R. R. Cornelius, "Describing the emotional states that are expressed in speech," Speech Communication, vol. 40, no. 1-2, pp. 5-32, 2003.
22. L. Devillers, R. Cowie, J. C. Martin, E. Douglas-Cowie, S. Abrilian, and M. McRorie, "Real life emotions in french and english TV video clips: an integrated annotation protocol combining continuous and discrete approaches," in Proc. LREC, Genoa, Italy, 2006.
23. M. Johnston, P. Baggia, D. C. Burnett, J. Carter, D. A. Dahl, G. McCobb, and D. Raggett, "EMMA: Extensible MultiModal Annotation markup language," World Wide Web Consortium, W3C Recommendation, Feb. 2009. [Online]. Available
24. S. Kopp, B. Krenn, S. Marsella, A. Marshall, C. Pelachaud, H. Pirker, K. Thórisson, and H. Vilhjalmsson, "Towards a common framework for multimodal generation: The behavior markup language," in Proc. Intelligent Virtual Agents, Marina del Rey, CA, USA, 2006, pp. 205-217. [Online]. Available
25. M. Wollmer, F. Eyben, J. Keshet, A. Graves, B. Schuller, and G. Rigoll, "Robust discriminative keyword spotting for emotionally colored spontaneous speech using bidirectional LSTM networks," in Proc. of ICASSP, Taipei, Taiwan, 2009.
26. M. Wollmer, F. Eyben, B. Schuller, and G. Rigoll, "Recognition of spontaneous conversational speech using long short-term memory phoneme predictions," in Proc. of Interspeech, Makuhari, Japan, 2010, pp. 1946-1949.
27. M. Wollmer, F. Eyben, A. Graves, B. Schuller, and G. Rigoll, "Bidirectional LSTM networks for context-sensitive keyword detection in a cognitive virtual agent framework," Cognitive Computation, vol. 2, no. 3, pp. 180-190, 2010.
28. A. Stupakov, E. Hanusa, J. Bilmes, and D. Fox, "COSINE-A corpus of multi-party conversational speech in noisy environments," in Proc. of ICASSP, Taipei, Taiwan, 2009.
29. P. Viola and M. Jones, "Robust real-time object detection," Int J Comput Vis, vol. 57, no. 2, pp. 137-154, 2002.
30. T. Ojala, M. Pietikainen, and T. Maenpaa, "Multiresolution grey-scale and rotation invariant texture classification with local binary patterns," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 7, pp. 971-987, 2002.
31. F. Eyben, H. Gunes, M. Pantic, M. Schroeder, B. Schuller, M. F. Valstar, and M. Wollmer, "User-profiled human behaviour interpreter," in SEMAINE Report D3c, 2010. [Online]. Available
32. H. Gunes and M. Pantic, "Automatic, dimensional and continuous emotion recognition," International Journal of Synthetic Emotions, vol. 1, no. 1, pp. 68-99, 2010.
33. Gunes H.,Pantic M. Automatic measurement of affect in dimensional and continuous spaces: Why, what, and how, Proc. of Measuring Behavior pp. 122-126, 2010.
34. B. Schuller, R. Muller, F. Eyben, J. Gast, B. Hornler, M. Wollmer, G. Rigoll, A. Hothker, and H. Konosu, "Being bored? recognising natural interest by extensive audiovisual integration for real-life application," Image and Vision Computing Journal, vol. 27, no. 12, pp. 1760-1774, 2009.
35. M. ter Maat and D. Heylen, "Selecting appropriate agent responses based on non-content features," in Proc. Affective Interaction in Natural Environments, ACM Multimedia, Firenze, Italy, 2010.
36. R. M. Maatman, J. Gratch, and S. Marsella, "Natural behavior of a listening agent," in 5th International Conference on Interactive Virtual Agents, Kos, Greece, 2005.
37. N. Ward and W. Tsukahara, "Prosodic features which cue back-channel responses in english and japanese," Journal of Pragmatics, vol. 23, pp. 1177-1207, 2000.
38. I. Poggi, Mind, hands, face and body. A goal and belief view of multimodal communication. Berlin: Weidler, 2007.
39. S. H. E. de Sevin and C. Pelachaud, "Influence of personality traits on backchannel selection," in Proc. Intelligent Virtual Agents, Philadelphia, USA, 2010.
40. M. McRorie, I. Sneddon, E. de Sevin, E. Bevacqua, and C. Pelachaud, "A model of personality and emotional traits," in Proc. Intelligent Virtual Agents, Amsterdam, 2009.
41. M. Mancini and C. Pelachaud, "Distinctiveness in multimodal behaviors," in Conference on Autonomous Agents and MultiAgent System, 2008.
42. M. Schroder, S. Pammi, and O. Turk, "Multilingual MARY TTS participation in the blizzard challenge 2009," in Blizzard Challenge 2009, Edinburgh, UK, 2009.
43. S. Pammi, M. Charfuelan, and M. Schroder, "Multilingual voice creation toolkit for the MARY TTS platform," in Proc. LREC. Valettea, Malta: ELRA, 2010. [Online]. Available
44. R. Niewiadomski, E. Bevacqua, M. Mancini, and C. Pelachaud, "Greta: an interactive expressive eca system," in AAMAS'09-Autonomous Agents and MultiAgent Systems, Budapest, Hungary, 2009.