Audiovisual classification of vocal outbursts in human conversation using long-short-term memory networks

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2011, Pages 5844-5847

  Eyben, Florian ^a   Petridis, Stavros ^b   Schuller, Björn ^a   Tzimiropoulos, George ^b   Zafeiriou, Stefanos ^b   Pantic, Maja ^b,c  

^a TECHNICAL UNIVERSITY OF MUNICH   (Germany)

^b IMPERIAL COLLEGE LONDON   (United Kingdom)

^c UNIVERSITY OF TWENTE   (Netherlands)

Author keywords

Audio visual Processing; Laughter; Long Short Term Memory; Non linguistic Vocalisations

Indexed keywords

APPEARANCE BASED; AUDIO-VISUAL PROCESSING; DESCRIPTORS; DYNAMIC SEQUENCES; LAUGHTER; MEMORY NETWORK; NON-LINGUISTIC VOCALISATIONS; SHAPE FEATURES; SHORT TERM MEMORY; STATIC APPROACH;

BRAIN; CLASSIFICATION (OF INFORMATION); LINGUISTICS; SIGNAL PROCESSING; SPEECH COMMUNICATION;

RECURRENT NEURAL NETWORKS;

EID: 80051631670     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2011.5947690     Document Type: Conference Paper

References (22)

1. R.R. Provine, "Laughter punctuates speech: linguistic, social and gender contexts of laughter," Ethology, vol. 15, pp. 291-298, 1993.
2. K. Laskowski, "Contrasting Emotion-Bearing Laughter Types in Multiparticipant Vocal Activity Detection for Meetings," in Proc. ICASSP, Taipei, Taiwan, 2009, pp. 4765-4768, IEEE.
3. K.P. Truong and D.A. van Leeuwen, "Automatic discrimination between laughter and speech," Speech Communication, vol. 49, pp. 144-158, 2007.
4. N. Campbell, "Whom we laugh with affects how we laugh," in Proceedings of the Interdisciplinary Workshop on The Phonetics of Laughter, Jürgen Trouvain and Nick Campbell, Eds., Saarbrücken, 2007, pp. 61-65.
5. J.-A. Bacharowski and M.J. Smoski, "The acoustic features of human laughter," J. Ac. Soc. Am., vol. 110, no. 3, pp. 1581-1597, 2001.
6. J. Trouvain, "Segmenting Phonetic Units in Laughter," in Proc. ICPhS, Barcelona, 2003, pp. 2793-2796.
7. S. Petridis and M. Pantic, "Audiovisual Laughter Detection based on Temporal Features," in Proc. ICMI, 2008, pp. 37-44.
8. S. Petridis, A. Asghar, and M. Pantic, "Classifying laughter and speech using audio-visual feature prediction," in IEEE ICASSP, 2010, pp. 5254-5257.
9. A. Ito, Wang Xinyue, M. Suzuki, and S. Makino, "Smile and laughter recognition using speech processing and face recognition from conversation video," in Intern. Conf. on Cyberworlds, 2005, 2005, pp. 8-15.
10. P. Pal, A.N. Iyer, and R.E. Yantorno, "Emotion detection from infant facial expressions and cries," may. 2006, vol. 2.
11. B. Reuderink, M. Poel, K. Truong, R. Poppe, and M. Pantic, "Decision-level fusion for audio-visual laughter detection," LNCS, vol. 5237, pp. 137-148, 2008.
12. S. Escalera, E. Puertas, P. Radeva, and O. Pujol, "Multi-modal laughter recognition in video conversations," pp. 110-115, 2009.
13. S. Scherer, F. Schwenker, N. Campbell, and G. Palm, "Multimodal laughter detection in natural discourses," Human Centered Robot Systems, pp. 111-120, 2009.
14. I. Patras and M. Pantic, "Particle filtering with factorized likelihoods for tracking facial features," in Int'l Conf. on Automatic Face and Gesture Recognition, 2004, pp. 97-104.
15. D. Gonzalez-Jimenez and J. L. Alba-Castro, "Toward pose-invariant 2-d face recognition through point distribution models and facial symmetry," IEEE Trans. Inform. Forensics and Security, vol. 2, no. 3, pp. 413-429, 2007.
16. G. Tzimiropoulos, S. Zafeiriou, and M. Pantic, "Principal Component Analysis of Image Gradient Orientations for Face Recognition," in IEEE FG, to appear, 2011.
17. W. Ruch and P. Ekman, "The expressive pattern of laughter," Emotion, qualia, and consciousness, pp. 426-443, 2001.
18. F. Eyben, M. Wöllmer, and B. Schuller, "openSMILE - the Munich versatile and fast open-source audio feature extractor," in Proc. of ACM Multimedia, Florence, Italy, 2010, ACM.
19. A. Graves, Supervised sequence labelling with recurrent neural networks, Ph.D. thesis, TUM, 2008.
20. B. Schuller, R. Müller, F. Eyben, J. Gast, B. Hörnler, M. Wöllmer, G. Rigoll, A. Höthker, and H. Konosu, "Being Bored? Recognising Natural Interest by Extensive Audiovisual Integration for Real-Life Application," Image and Vision Computing Journal, vol. 27, pp. 1760-1774, 2009.
21. B. Schuller, S. Steidl, A. Batliner, F. Burkhardt, L. Devillers, C. Müller, and S. Narayanan, "The INTERSPEECH 2010 Paralinguistic Challenge," in Proc. INTERSPEECH 2010, Makuhari, Japan, 2010, pp. 2794-2797.
22. B. Schuller, F. Eyben, and G. Rigoll, "Static and dynamic modelling for the recognition of non-verbal vocalisations in conversational speech," in Perception and Interactive Technologies for Speech-based Systems. 2008, vol. LNCS 5078, pp. 99-110, Springer.