An event driven fusion approach for enjoyment recognition in real-time

MM 2014 - Proceedings of the 2014 ACM Conference on Multimedia

Volumn , Issue , 2014, Pages 377-386

  Lingenfelser, Florian ^a   Wagner, Johannes ^a   André, Elisabeth ^a   McKeown, Gary ^b   Curran, Will ^b  

^a UNIVERSITY OF AUGSBURG   (Germany)

^b QUEEN'S UNIVERSITY BELFAST   (United Kingdom)

Author keywords

Affect recognition; Event driven fusion; Multi modal fusion; Social signal processing

Indexed keywords

HUMAN COMPUTER INTERACTION; VECTOR SPACES;

AFFECT RECOGNITION; ASYNCHRONOUS EVENT; AUTOMATIC CLASSIFICATION; EVENT-DRIVEN; MULTI-MODAL FUSION; REAL-TIME APPLICATION; SOCIAL SIGNAL PROCESSING; SOCIAL SIGNALS;

SIGNAL PROCESSING;

EID: 84913530421     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2647868.2654924     Document Type: Conference Paper

References (31)

1. C.-C. Chang and C.-J. Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1-27:27, 2011.
2. S. K. D'Mello and J. Kory. Consistent but modest: a meta-analysis on unimodal and multimodal affect detection accuracies from 30 studies. In L.-P. Morency, D. Bohus, H. K. Aghajan, J. Cassell, A. Nijholt, and J. Epps, editors, ICMI, pages 31-38. ACM, 2012.
3. R. Duin and D. Tax. Experiments with classifier combining rules. In Lecture Notes in Computer Science, volume 1857, pages 16-29. Springer, 2000.
4. S. Dupont and J. Luettin. Audio-visual speech modeling for continuous speech recognition. IEEE Transactions on Multimedia, 2(3):141-151, 2000.
5. F. Eyben, M. Wóllmer, M. F. Valstar, H. Gunes, B. Schuller, and M. Pantic. String-based audiovisual fusion of behavioural events for the assessment of dimensional affect. In IEEE International Conference on Automatic Face & Gesture Recognition and Workshops, FG 2011, pages 322-329, USA, March 2011. IEEE Computer Society.
6. G. Fumera and F. Roli. A theoretical and experimental analysis of linear combiners for multiple classifier systems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(6):942-956, 2005.
7. S. W. Gilroy, M. Cavazza, R. Chaignon, S.-M. Múkelú, M. Niranen, E. André, T. Vogt, J. Urbain, M. Billinghurst, H. Seichter, and M. Benayoun. E-tree: Emotionally driven augmented reality art. In Proceedings of the 16th ACM International Conference on Multimedia, MM'08, pages 945-948, New York, NY, USA, 2008. ACM.
8. S. W. Gilroy, M. Cavazza, M. Niranen, E. Andre, T. Vogt, J. Urbain, M. Benayoun, H. Seichter, and M. Billinghurst. Pad-based multimodal affective fusion. In Affective Computing and Intelligent Interaction and Workshops, 2009. ACII 2009. 3rd International Conference on, 2009.
9. J. Hofmann, F. Stoffel, A. Weber, and T. Platt. The 16 enjoyable emotions induction task (16-eeit). 2012.
10. L. I. Kuncheva. Switching between selection and fusion in combining classifiers: An experiment. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 32(2):146-156, 2002.
11. L. I. Kuncheva. A theoretical study on six classifier fusion strategies. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(2):281-286, 2002.
12. F. Lingenfelser, J. Wagner, and E. André. A systematic discussion of fusion techniques for multi-modal affect recognition tasks. In Proceedings of the 13th International Conference on Multimodal Interfaces, ICMI'11, pages 19-26, New York, NY, USA, 2011. ACM.
13. M. Mancini, L. Ach, E. Bantegnie, T. Baur, N. Berthouze, D. Datta, Y. Ding, S. Dupont, H. Griffin, F. Lingenfelser, R. Niewiadomski, C. Pelachaud, O. Pietquin, B. Piot, J. Urbain, G. Volpe, and J. Wagner. Laugh when you're winning. In C. T. R. J. C.-M. Rybarczyk, Y., editor, Innovative and Creative Developments in Multimodal Interaction Systems, Proceedings of 9th IFIP WG 5.5 International Summer Workshop on Multimodal Interfaces, eNTERFACE 2013, volume 425, Lisbon, Portugal, 2014. Springer.
14. G. U. Mehlmann and E. André. Modeling multimodal integration with event logic charts. In L.-P. Morency, D. Bohus, H. K. Aghajan, J. Cassell, A. Nijholt, and J. Epps, editors, ICMI, pages 125-132. ACM, 2012.
15. A. V. Nefian, L. Liang, X. Pi, L. Xiaoxiang, C. Mao, and K. Murphy. A coupled hmm for audio-visual speech recognition. In in International Conference on Acoustics, Speech and Signal Processing (CASSP'02), pages 2013-2016, 2002.
16. M. A. Nicolaou, H. Gunes, and M. Pantic. Audio-visual classification and fusion of spontaneous affective data in likelihood space. In ICPR, pages 3695-3699. IEEE, 2010.
17. M. A. Nicolaou, H. Gunes, and M. Pantic. Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space. T. Affective Computing, 2(2):92-105, 2011.
18. M. Pantic, A. Pentland, A. Nijholt, and T. Huang. Human Computing and Machine Understanding of Human Behavior: A Survey, volume 4451, pages 47-71. 2007.
19. S. Petridis, M. Pantic, and J. F. Cohn. Prediction-based classification for audiovisual discrimination between laughter and speech. In Automatic Face and Gesture Recognition and Workshops (FG 2011), 2011 IEEE International Conference, pages 619-626, 2011.
20. R. Polikar. Ensemble based systems in decision making. IEEE Circuits and Systems Magazine, (3): 21-45, 2006.
21. H. Sloetjes, A. Russel, and A. Klassmann. Elan: a free and open-source multimedia annotation tool. In INTERSPEECH, pages 4015-4016. ISCA, 2007.
22. M. Song, J. Bu, C. Chen, and N. Li. Audio-visual based emotion recognition - a new approach. In CVPR (2), pages 1020-1025, 2004.
23. K. M. Ting and I. H. Witten. Issues in stacked generalization. Journal of Artificial Intelligence Research, 10:115-124, 1999.
24. A. Vinciarelli, M. Pantic, D. Heylen, C. Pelachaud, I. Poggi, F. D'ericco, and M. Schroeder. Bridging the gap between social animal and unsocial machine: A survey of social signal processing. IEEE Transactions on Affective Computing, 3:69-87, April 2012. Issue 1.
25. T. Vogt, E. André, and N. Bee. Emovoice - a framework for online recognition of emotions from voice. In Perception in Multimodal Dialogue Systems, 4th IEEE Tutorial and Research Workshop on Perception and Interactive Technologies for Speech-Based Systems, Kloster Irsee, Germany, LNCS, pages 188-199. Springer, 2008.
26. J. Wagner, F. Lingenfelser, E. Andre, J. Kim, and T. Vogt. Exploring fusion methods for multimodal emotion recognition with missing data. IEEE Transactions on Affective Computing, 2(4):206-218, 2011.
27. J. Wagner, F. Lingenfelser, T. Baur, I. Damian, F. Kistler, and E. André. The social signal interpretation (ssi) framework: multimodal signal processing and recognition in real-time. In A. Jaimes, N. Sebe, N. Boujemaa, D. Gatica-Perez, D. A. Shamma, M. Worring, and R. Zimmermann, editors, ACM Multimedia, pages 831-834. ACM, 2013.
28. M. Wóllmer, B. Schuller, F. Eyben, and G. Rigoll. Combining long short-term memory and dynamic bayesian networks for incremental emotion-sensitive artificial listening. J. Sel. Topics Signal Processing, 4(5):867-881, 2010.
29. M. Wóllmer, M. Hmer, M. Al-Hames, F. Eyben, B. Schuller, and G. Rigoll. A multidimensional dynamic time warping algorithm for efficient multimodal fusion of asynchronous data streams. Neurocomput., 73(1-3):366-380, Dec. 2009.
30. Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang. A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE transactions on pattern analysis and machine intelligence, 31(1):39-58, January 2009.
31. Z. Zeng, J. Tu, B. M. Pianfetti, and T. S. Huang. Audio-visual affective expression recognition through multistream fused hmm. Trans. Multi., 10(4):570-577, June 2008.