Joint-processing of audio-visual signals in human perception of conflicting synthetic character emotions

2008 IEEE International Conference on Multimedia and Expo, ICME 2008 - Proceedings

Volumn , Issue , 2008, Pages 961-964

  Emily, Mower ^a   Sungbok, Lee ^a   Maja, J Matarić ^a   Shrikanth, Narayanan ^a  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

Audio visual emotion perception; Facial emotion expression; McGurk effect

Indexed keywords

AMOUNT OF INTERACTIONS; AUDIO CHANNELS; AUDIO-VISUAL; AUDIO-VISUAL EMOTION PERCEPTION; COMMERCIAL APPLICATIONS; CONTINUOUS; EMOTIONAL SPACES; EVALUATION FRAMEWORKS; FACIAL EMOTION EXPRESSION; HUMAN PERCEPTIONS; JOINT PROCESSING; MCGURK EFFECT; MULTI-MODAL; SYNTHETIC CHARACTERS; TWO CHANNELS; UNCERTAIN; VIDEO CHANNELS; VIDEO INFORMATIONS;

EXHIBITIONS;

EID: 54049113162     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICME.2008.4607596     Document Type: Conference Paper

References (11)

1. H. McGurk and J. MacDonald, "Hearing lips and seeing voices," Nature, vol. 264, pp. 746-748, 1976.
2. D.W. Massaro and D.G. Stork, "Speech recognition and sensory integration: a 240-year-old theorem helps explain how people and machines can integrate auditory and visual information to understand speech," American Scientist, vol. 86, no. 3. pp. 236-242, May - June 1998.
3. B. de Gelder, "The perception of emotions by ear and by eye," Cognition & Emotion, vol. 14, no. 3, pp. 289-311, 2000.
4. D.W. Massaro, "Fuzzy logical model of bimodal emotion perception: Comment on" The perception of emotions by ear and by eye" by de Gelder and Vroomen," Cognition & Emotion, vol. 14, no. 3, pp. 313-320, 2000.
5. B. de Gelder, K.B.E. Böcker, J. Tuomainen, M. Hensen, and J. Vroomen, "The combined perception of emotion from voice and face: early interaction revealed by human electric brain responses," Neuroscience Letters, vol. 260, no. 2, pp. 133-136, 1999.
6. J.K. Hietanen, J.M. Leppiinen, M. Mi, and V. Surakka, "Evidence for the integration of audio-visual emotional information at the perceptual level of processing," European Journal of Cognitive Psychology, vol. 16, no. 6, pp. 769-790, 2004.
7. Rob Parke, Elaine Chew, and Chris Kyriakakis, "Multiple regression modeling of the emotional content of film and music," Audio Engineering Society, 2007.
8. Emily Mower, Sungbok Lee, Maja J Matarić, and Shrikanth Narayanan, "Human perception of synthetic character emotions in the presence of conflicting and congruent vocal and facial expressions," ICASSP, April 2008.
9. S. Lee, S. Yildirim, A. Kazemzadeh, and S. Narayanan, "An articulatory study of emotional speech production," Proc. Eurospeech, Lisbon, Portugal, pp. 497-500, 2005.
10. S. Sutton, R. Cole, J. de Villiers, J. Schalkwyk, P. Vermeulen, M. Macon, Y. Yan, E. Kaiser, B. Rundle, K. Shobaki, et al., "Universal Speech Tools: the CSLU Toolkit," Proceedings of the International Conference on Spoken Language Processing (ICSLP), pp. 3221-3224, 1998.
11. Carlos Busso, Zhigang Deng, Serdar Yildirim, Murtaza Bulut, Chul Min Lee, Abe Kazemzadeh, Sungbok Lee, Ulrich Neumann, and Shrikanth Narayanan, "Analysis of emotion recognition using facial expressions, speech and multimodal information," Proceedings of the 6th international conference on Multimodal interfaces, October 2004.