Prediction-based classification for audiovisual discrimination between laughter and speech

2011 IEEE International Conference on Automatic Face and Gesture Recognition and Workshops, FG 2011

Volumn , Issue , 2011, Pages 619-626

  Petridis, Stavros ^a   Pantic, Maja ^b   Cohn, Jeffrey F ^c  

^a IMPERIAL COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF TWENTE   (Netherlands)

^c UNIVERSITY OF PITTSBURGH SCHOOL OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AUDIO-VISUAL FEATURES; DATA SETS; FEATURE LEVEL; FEATURE MAPPING; HUMAN ACTIONS; PREDICTION ERRORS; TEMPORAL PATTERN; TEMPORAL RELATIONSHIPS; TEST CASE; TIME EVOLUTIONS; VISUAL FEATURE;

FACE RECOGNITION; GESTURE RECOGNITION; NEURAL NETWORKS; SPEECH RECOGNITION;

FORECASTING;

EID: 79958698495     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/FG.2011.5771468     Document Type: Conference Paper

References (35)

1. M. Bar, "The proactive brain: using analogies and associations to generate predictions," Trends in Cognitive Sciences, vol. 11, no. 7, pp. 280-289, 2007. (Pubitemid 46962332)
2. D.M. Wolpert, Z. Ghahramani, and M.I. Jordan, "An internal model for sensorimotor integration," Science, vol. 269, no. 5232, pp. 1880, 1995.
3. J.M. Zacks and K.M. Swallow, "Event segmentation," Current Directions in Psychological Science, vol. 16, no. 2, pp. 80, 2007.
4. J.M. Zacks and J.Q. Sargent, "Event perception: A theory and its application to clinical neuroscience," Psychology of Learning and Motivation, vol. 53, pp. 253-299, 2010.
5. M.S. Kraus, R.S.E. Keefe, and R.K.R. Krishnan, "Memory-prediction errors and their consequences in schizophrenia," Neuropsychology review, vol. 19, no. 3, pp. 336-352, 2009.
6. K. Friston, "The free-energy principle: a unified brain theory?," Nature Reviews Neuroscience, vol. 11, no. 2, pp. 127-138, 2010.
7. M. Haruno, D.M. Wolpert, and M. Kawato, "Mosaic model for sensorimotor learning and control," Neural Computation, vol. 13, no. 10, pp. 2201-2220, 2001. (Pubitemid 33676881)
8. J. Hawkins and S. Blakeslee, On intelligence, Owl Books, 2005.
9. A. Kehagias and V. Petridis, "Predictive modular neural networks for time series classification," Neural Networks, vol. 10, no. 1, pp. 31-49, 1997. (Pubitemid 27086444)
10. D. Coyle, G. Prasad, and T.M. McGinnity, "A time-series prediction approach for feature extraction in a brain-computer interface," IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 13, no. 4, pp. 461-467, 2005. (Pubitemid 43015552)
11. R. Gutierrez-Osuna, PK Kakumanu, A. Esposito, ON Garcia, A. Bojorquez, JL Castillo, and I. Rudomin, "Speech-driven facial animation with realistic dynamics," IEEE Trans. on Multimedia, vol. 7, no. 1, pp. 33-42, 2005. (Pubitemid 40178367)
12. Laurent Girin, Jean-Luc Schwartz, and Gang Feng, "Audio-visual enhancement of speech in noise," The Journal of the Acoustical Society of America, vol. 109, no. 6, pp. 3007-3020, 2001. (Pubitemid 32545145)
13. A. Vinciarelli, M. Pantic, and H. Bourlard, "Social signal processing: Survey of an emerging domain," Image and Vision Computing, vol. 27, no. 12, pp. 1743-1759, 2009.
14. S. Petridis and M. Pantic, "Audiovisual laughter detection based on temporal features," in Proc. ACM Int'l Conf. on Multimodal interfaces, 2008, pp. 37-44.
15. S. Scherer, F. Schwenker, N. Campbell, and G. Palm, "Multimodal laughter detection in natural discourses," Human Centered Robot Systems, pp. 111-120, 2009.
16. B. Reuderink, M. Poel, K. Truong, R. Poppe, and M. Pantic, "Decisionlevel fusion for audio-visual laughter detection," Lecture Notes in Computer Science, vol. 5237, pp. 137-148, 2008.
17. S. Petridis, H. Gunes, S. Kaltwang, and M. Pantic, "Static vs. Dynamic Modelling of Human Nonverbal Behavior from Multiple Cues and Modalities," in Proc. ACM ICMI, 2009, pp. 23-30.
18. B. Schuller, F. Eyben, and G. Rigoll, "Static and Dynamic Modelling for the Recognition of Non-verbal Vocalisations in Conversational Speech," Lecture Notes in Computer Science, vol. 5078, pp. 99-110, 2008.
19. H.C. Yehia, T. Kuratate, and E. Vatikiotis-Bateson, "Linking facial animation, head motion and speech acoustics," Journal of Phonetics, vol. 30, no. 3, pp. 555-568, 2002.
20. C. Busso and S. S. Narayanan, "Interrelation between speech and facial gestures in emotional utterances: A single subject study," IEEE Transactions on Audio, Speech and Language Processing, vol. 15, no. 8, pp. 2331-2347, 2007, 1558-7916.
21. M. S. Craig, P. Lieshout, and W. Wong, "A linear model of acousticto-facial mapping: Model parameters, data set size, and generalization across speakers," J. Acoustical Soc. America, vol. 124, no. 5, pp. 3183-3190, 2008.
22. K. Kumar, J. Navratil, E. Marcheret, V. Libal, G. Ramaswamy, and G. Potamianos, "Audio-visual speech synchronization detection using a bimodal linear prediction model," in IEEE CVPR Workshops, 2009, pp. 53-59.
23. S. Petridis, A. Asghar, and M. Pantic, "Classifying laughter and speech using audio-visual feature prediction," in IEEE ICASSP, 2010, pp. 5254-5257.
24. I. McCowan, J. Carletta, W. Kraaij, S. Ashby, S. Bourban, M. Flynn, M. Guillemot, T. Hain, J. Kadlec, and V. Karaiskos, "The AMI meeting corpus," in Int'l. Conf. on Methods and Techniques in Behavioral Research, 2005, pp. 137-140.
25. E. Douglas-Cowie, R. Cowie, C. Cox, N. Amir, and D. Heylen, "The Sensitive Artificial Listener: an induction technique for generating emotionally coloured conversation," in Programme of the Workshop on Corpora for Research on Emotion and Affect, pp. 1-4.
26. J.F. Cohn, T.S. Kruez, I. Matthews, Y. Yang, M.H. Nguyen, M.T. Padilla, F. Zhou, and F. De la Torre, "Detecting depression from facial actions and vocal prosody," in Intern. Conf. on ACII 2009. IEEE, 2009, pp. 1-7.
27. J. Urbain, E. Bevacqua, T. Dutoit, A. Moinet, R. Niewiadomski, C. Pelachaud, B. Picart, J. Tilmanne, and J. Wagner, "The AVLaughterCycle Database," Proc. LREC, 2010.
28. J. A. Bachorowski, M. J. Smoski, and M. J. Owren, "The acoustic features of human laughter," Journal of the Acoustical Society of America, vol. 110, no. 1, pp. 1581-1597, 2001.
29. L. Kennedy and D. Ellis, "Laughter detection in meetings," in NIST Meeting Recognition Workshop, 2004.
30. S. Dupont and J. Luettin, "Audio-visual speech modeling for continuous speech recognition," IEEE Trans. on Multimedia, vol. 2, no. 3, pp. 141-151, 2000.
31. G. Potamianos, C. Neti, G. Gravier, A. Garg, and A. W. Senior, "Recent advances in the automatic recognition of audiovisual speech," Proc. of the IEEE, vol. 91, no. 9, pp. 1306-1326, 2003.
32. P.S. Aleksic and A.K. Katsaggelos, "Audio-visual biometrics," Proceedings of the IEEE, vol. 94, no. 11, pp. 2025-2044, 2007. (Pubitemid 46445568)
33. Z. Zeng, J. Tu, B. Pianfetti, M. Liu, T. Zhang, Z. Zhang, T. Huang, and S. Levinson, "Audio-visual affect recognition through multi-stream fused HMM for HCI," in CVPR, 2005, pp. 967-972.
34. I. Patras and M. Pantic, "Particle filtering with factorized likelihoods for tracking facial features," in Int'l Conf. on Automatic Face and Gesture Recognition, 2004, pp. 97-104.
35. D. Gonzalez-Jimenez and J. L. Alba-Castro, "Toward pose-invariant 2-d face recognition through point distribution models and facial symmetry," IEEE Trans. Inform. Forensics and Security, vol. 2, no. 3, pp. 413-429, 2007. (Pubitemid 47290576)