Social Signal Interpretation (SSI): A Framework for Real-time Sensing of Affective and Social Signals

KI - Kunstliche Intelligenz

Volumn 25, Issue 3, 2011, Pages 251-256

  Wagner, Johannes ^a   Lingenfelser, Florian ^a   Bee, Nikolaus ^a   André, Elisabeth ^a  

^a UNIVERSITY OF AUGSBURG   (Germany)

Author keywords

Affective computing; Human centred computing; Machine learning; Multimodal fusion; Real time recognition; Social signal processing

Indexed keywords

HUMAN COMPUTER INTERACTION; LEARNING SYSTEMS; ONLINE SYSTEMS; SIGNAL PROCESSING; SPEECH RECOGNITION;

AFFECTIVE COMPUTING; BUILDING SYSTEMS; HUMAN-CENTERED COMPUTING; KEY ISSUES; MULTI-MODAL FUSION; REAL TIME RECOGNITION; REAL TIME SENSING; SIGNAL INTERPRETATION; SOCIAL SIGNAL PROCESSING; SOCIAL SIGNALS;

USER INTERFACES;

EID: 84867521250     PISSN: 09331875     EISSN: 16101987     Source Type: Journal    
DOI: 10.1007/s13218-011-0115-x     Document Type: Article

References (30)

1. Bee N, Falk B, André E (2009) Simplified facial animation control utilizing novel input devices: a comparative study. In: International conference on intelligent user interfaces (IUI’09), pp 197–206
2. Bee N, André E, Vogt T, Gebhard P (2010) The use of affective and attentive cues in an empathic computer-based companion. In: Wilks Y (ed) Close engagements with artificial companions: key social, psychological, ethical and design issues. Benjamins, Amsterdam, pp 131–142
3. Burkhardt F, Paeschke A, Rolfes M, Sendlmeier W, Weiss B (2005) A database of german emotional speech. In: Proceedings of Interspeech, Lisbon, pp 1517–1520
4. Caridakis G, Raouzaiou A, Karpouzis K, Kollias S (2006) Synthesizing gesture expressivity based on real sequences. In: Workshop on multimodal corpora: from multimodal behaviour theories to usable models, LREC 2006 conference, Genoa, Italy, 24–26 May
5. Caridakis G, Wagner J, Raouzaiou A, Curto Z, André E, Karpouzis K (2010) A multimodal corpus for gesture expressivity analysis. In: Multimodal corpora: advances in capturing, coding and analyzing multimodality, LREC, Malta, 17–23 May 2010
6. Charles F, Pizzi D, Cavazza M, Vogt T, André E (2009) Emotional input for character-based interactive storytelling. In: The 8th international conference on autonomous agents and multiagent systems (AAMAS) Budapest, Hungary
7. Conati C, Chabbal R, Maclaren H (2003) A study on using biometric sensors for detecting user emotions in educational games. In: Proceedings of the workshop “Assessing and adapting to user attitude and affects: why, when and how?” In conjunction with UM’03, 9th international conference on user modeling
8. Ekman P (1992) An argument for basic emotions. Cogn Emot 6(3):169–200 DOI: 10.1080/02699939208411068
9. Gebhard P (2005) ALMA: a layered model of affect. In: AAMAS ’05: Proceedings of the fourth international joint conference on autonomous agents and multiagent systems. ACM, New York, pp 29–36 DOI: 10.1145/1082473.1082478
10. Gilroy S, Cavazza M, Vervondel V (2011) Evaluating multimodal affective fusion with physiological signals. In: Proceedings of the international conference on intelligent user interfaces. Stanford University, Palo Alto
11. Gilroy SW, Cavazza M, Niiranen M, André E, Vogt T, Urbain J, Seichter H, Benayoun M, Billinghurst M (2009) Pad-based multimodal affective fusion. In: Affective computing and intelligent interaction (ACII), Amsterdam
12. Gratch J, Marsella S (2004) A domain-independent framework for modeling emotion. Cogn Syst Res 5(4):296–306
13. Hartmann B, Mancini M, Pelachaud C (2006) Implementing expressive gesture synthesis for embodied conversational agents, vol 3881, pp 188–199
14. Hönig F, Wagner J, Batliner A, Nöth E (2009) Classification of user states with physiological signals: on-line generic features vs. specialized. In: Stewart B, Weiss S (eds) Proceedings of the 17th European signal processing conference (EUSIPCO), Glasgow, Scotland, pp 2357–2361
15. Jacucci G, Spagnolli A, Chalambalakis A, Morrison A, Liikkanen L, Roveda S, Bertoncini M (2009) Bodily explorations in space: social experience of a multimodal art installation. In: Proceedings of the 12th IFIP TC 13 international conference on human-computer interaction: part II, INTERACT ’09. Springer, Berlin, pp 62–75
16. Kim J, André E (2008) Emotion recognition based on physiological changes in music listening. IEEE Trans Pattern Anal Mach Intell 30:2067–2083 DOI: 10.1109/TPAMI.2008.26
17. Kim J, Lingenfelser F (2010) Ensemble approaches to parametric decision fusion for bimodal emotion recognition. In: Int conf on bio-inspired systems and signal processing (Biosignals 2010)
18. Kim J, Ragnoni A, Biancat J (2010) In-vehicle monitoring of affective symptoms for diabetic drivers. In: Fred JFA, Gamboa H (eds) Int conf on health informatics (HEALTHINF 2010), BIOSTEC. INSTICC Press, Valencia, pp 367–372
19. Küblbeck C, Ernst A (2006) Face detection and tracking in video sequences using the modifiedcensus transformation. Image Vis Comput 24:564–572 DOI: 10.1016/j.imavis.2005.08.005
20. Lingenfelser F, Wagner J, TVJK, André E (2010) Age and gender classification from speech using decision level fusion and ensemble based techniques. In: INTERSPEECH 2010
21. Lingenfelser F, Wagner J, Vogt T, Kim J, André E (2010) Age and gender classification from speech using decision level fusion and ensemble based techniques. In: INTERSPEECH 2010
22. Mehrabian A (1995) Framework for a comprehensive description and measurement of emotional states. Genet Soc Gen Psychol Monogr 121(3):339–361
23. Pantic M, Nijholt A, Pentland A, Huang TS (2008) Human-centred intelligent human-computer interaction (hci): how far are we from attaining it? Int J Auton Adapt Commun Syst 1(2):168–187 DOI: 10.1504/IJAACS.2008.019799
24. Schuller B, Steidl S, Batliner A (2009) The INTERSPEECH 2009 emotion challenge. In: ISCA (ed) Proceedings of Interspeech 2009, pp 312–315
25. Vogt T, André E (2009) Exploring the benefits of discretization of acoustic features for speech emotion recognition. In: Proceedings of 10th conference of the International Speech Communication Association (INTERSPEECH), ISCA, Brighton, UK, pp 328–331
26. Vogt T, André E (2011) An evaluation of emotion units and feature types for real-time speech emotion recognition. This volume
27. Wagner J, Kim J, André E (2005) From physiological signals to emotions: implementing and comparing selected methods for feature extraction and classification. In: IEEE international conference on multimedia and Expo, ICME 2005, pp 940–943 DOI: 10.1109/ICME.2005.1521579
28. Wagner J, André E, Jung F (2009) Smart sensor integration: a framework for multimodal emotion recognition in real-time. In: Affective computing and intelligent interaction (ACII 2009), IEEE
29. Wagner J, Jung F, Kim J, André E, Vogt T (2010) The smart sensor integration framework and its application in EU projects. In: Kim J, Karjalainen P (eds) Workshop on bio-inspired human-machine interfaces and healthcare applications (B-Interface 2010), Biostec 2010. INSTICC Press, Valencia, pp 13–21
30. Wobbrock JO, Wilson AD, Li Y (2007) Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In: Proceedings of the 20th annual ACM symposium on user interface software and technology, UIST ’07. ACM, New York, pp 159–168 DOI: 10.1145/1294211.1294238