Speech emotion analysis: Exploring the role of context

IEEE Transactions on Multimedia

Volumn 12, Issue 6, 2010, Pages 502-509

  Tawari, Ashish ^a   Trivedi, Mohan Manubhai ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Affect analysis; affective computing; context analysis; emotion intelligence; emotion recognition; emotional speech; vocal expression

Indexed keywords

AFFECT ANALYSIS; AFFECTIVE COMPUTING; CONTEXT ANALYSIS; EMOTION INTELLIGENCE; EMOTION RECOGNITION; EMOTIONAL SPEECH; VOCAL EXPRESSION;

DATABASE SYSTEMS; SPEECH ANALYSIS;

SPEECH RECOGNITION;

EID: 77956722861     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2010.2058095     Document Type: Article

References (36)

1. G. Roisman, J. Tsai, and K. Chiang, "The emotional integration of childhood experience: Physiological, facial expressive, and self-reported emotional response during the adult attachment interview," Devel. Psychol., vol.40, no.5, pp. 776-789, 2004.
2. J. F. Cohn and E. Z. Tronick, "Mother-infant face-to-face interaction: Influence is bidirectional and unrelated to periodic cycles in either partner's behavior," Devel. Psychol., vol.23, pp. 68-77, 1998.
3. P. Ekman, D. Matsumoto, and W. Friesen, "Facial expression in affective disorders," What the Face Reveals, pp. 429-439, 2005.
4. M. Pantic and M. Bartlett, "Machine analysis of facial expressions," in Face Recognition, K. Delac and M. Grgic, Eds. Vienna, Austria: I-Tech Education, 2007, pp. 377-416.
5. C. L. Lisetti and F. Nasoz, "Maui: A multimodal affective user interface," in Proc. 10th ACM Int. Conf. Multimedia, New York, 2002, pp. 161-170.
6. Z. Duric, W. Gray, R. Heishman, F. Li, A. Rosenfeld, M. Schoelles, C. Schunn, and H. Wechsler, "Integrating perceptual and cognitive modeling for adaptive and intelligent human-computer interaction," Proc. IEEE, vol.90, no.7, pp. 1272-1289, Jul. 2002.
7. L. Maat and M. Pantic, "Gaze-x: Adaptive affective multimodal interface for single-user office scenarios," in Proc. ACM Int. Conf. Multimodal Interfaces, 2006, pp. 171-178.
8. A. Kapoor, W. Burleson, and R. W. Picard, "Automatic prediction of frustration," Int. J. Human-Comput. Stud., vol.65, no.8, pp. 724-736, 2007.
9. T. Zhang, H. M. Johnson, and S. E. Levinson, "Children's emotion recognition in an intelligent tutoring scenario," in Proc. ICSLP, 2004.
10. O.-W.Kwon, K.-L. Chan, J. Hao, and T.-W. Lee, "Emotion recognition by speech signals," in Proc. 8th Eur. Conf. Speech Commun. Technol. (EUROSPEECH), 2004, pp. 172-187.
11. J. Ang, R. Dhillon, A. Krupski, E. Shriberg, and A. Stolcke, "Prosodybased automatic detection of annoyance and frustration in human-computer dialog," in Proc. ICSLP, 2002, pp. 2037-2040.
12. M. Grimm, K. Kroschel, H. Harris, C. Nass, B. Schuller, G. Rigoll, and T. Moosmayr, "On the necessity and feasibility of detecting a driver's emotional state while driving," in Proc. ACII, 2007, pp. 126-138.
13. A. Austermann, N. Esau, L. Kleinjohann, and B. Kleinjohann, "Prosody based emotion recognition for mexi," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Aug. 2005, pp. 1138-1144.
14. Z. Zeng, M. Pantic, G. Roisman, and T. Huang, "A survey of affect recognition methods: Audio, visual, and spontaneous expressions," IEEE Trans. Pattern Anal. Mach. Intell., vol.31, no.1, pp. 39-58, Jan. 2009.
15. A. Batliner, K. Fischer, R. Huber, J. Spilker, and E. Nöth, "How to find trouble in communication," Speech Commun., vol.40, no.1-2, pp. 117-143, 2003.
16. L. Devillers and L. Vidrascu, "Real-life emotions detection with lexical and paralinguistic cues on human-human call center dialogs," in Proc 9th Int. Conf. Spoken Language Processing (ICSLP), 2006, pp. 801-804.
17. H. J. M. Steeneken and J. H. L. Hansen, "Speech under stress conditions: Overview of the effect on speech production and on system performance," in Proc. ICASSP, Washington, DC, 1999, pp. 2079-2082.
18. R. Banse and K. R. Scherer, "Acoustic profiles in vocal emotion expression," J Pers. Soc. Psychol. vol.70, no.3, pp. 614-636, Mar. 1996 [Online]. Available: http://view.ncbi.nlm.nih.gov/pubmed/8851745
19. T. Scherer, K. R. Johnstone, and T. Bänziger, "Automatic verification of emotionally stressed speakers: The problem of individual differences," in Proc. Int. Workshop Speech Comput., St. Petersburg, Russia, 1998.
20. Z. Callejas and R. López-Cózar, "Influence of contextual information in emotion annotation for spoken dialogue systems," Speech Commun., vol.50, no.5, pp. 416-433, 2008.
21. D. J. Litman and K. Forbes-Riley, "Predicting student emotions in computer-human tutoring dialogues," in Proc. ACL,Morristown, NJ, 2004, p. 351.
22. K. Forbes-Riley and D. Litman, "Predicting emotion in spoken dialogue from multiple knowledge sources," in Proc. HLT-NAACL, D. M. S. Dumais and S. Roukos, Eds., Boston, MA, May 2-7, 2004, pp. 201-208.
23. T. Vogt and E. André, "Improving automatic emotion recognition from speech via gender differentiation," in Proc. LREC, Genoa, Italy, 2006, pp. 1123-1126.
24. S. Casale, A. Russo, G. Scebba, and S. Serrano, "Speech emotion classification using machine learning algorithms," in Proc. Int. Conf. Semantic Computing, Aug. 2008, pp. 158-165.
25. L. Devillers, L. Vidrascu, and L. Lamel, "Challenges in real-life emotion annotation and machine learning based detection," Neural Network, vol.18, no.4, pp. 407-422, 2005.
26. F. Burkhardt, A. Paeschke, M. Rolfes, W. Sendlmeier, and B. Weiss, "A database of german emotional speech," in Proc. Interspeech, D. M. S. Dumais and S. Roukos, Eds., 2005, pp. 1517-1520.
27. M. M. Trivedi, T. Gandhi, and J. McCall, "Looking-in and looking-out of a vehicle: Computer-vision-based enhanced vehicle safety," IEEE Trans. Intell. Transportation Syst., pp. 108-120, Mar. 2007.
28. M. M. Trivedi and S. Y. Cheng, "Holistic sensing and active displays for intelligent driver support systems," IEEE Computer, Special Issue on Human-Centered Computing, vol.40, no.5, pp. -68, May 2007.
29. S. T. Shivappa, B. D. Rao, and M. Trivedi, "Role of head pose estimation in speech acquisition from distant microphones," in Proc. IEEE ICASSP, Apr. 2009, no.1, pp. 3557-3560.
30. M. M. Trivedi, K. S. Huang, and I. Mikic, "Dynamic context capture and distributed video arrays for intelligent spaces," IEEE Trans. Syst., Man, Cybern. A, vol.35, pp. 145-163, 2005.
31. A. Tawari and M. M. Trivedi, "Speech based emotion classification framework for driver assistance system," in Proc. IEEE Intell. Vehicles Symp, Jun. 2010, pp. 174-178.
32. A. Tawari and M. M. Trivedi, "Speech emotion analysis in noisy real world environment," in Proc. 20th ICPR, 2010.
33. B. Paul, Accurate Short-Term Analysis of the Fundamental Frequency and the Harmonics-to-Noise Ratio of a Sampled Sound," 1993 [Online]. Available: citeseer.ist.psu.edu/boersma93accurate.html
34. T. Vogt and E. André, "Exploring the benefits of discretization of acoustic features for speech emotion recognition," in Proc. 10th Conf. INTERSPEECH, Brighton, U.K., Sep. 2009, pp. 328-331.
35. E. Bozkurt, C. Erdem, E. Erzin, T. Erdem, M. Ozkan, and A. Tekalp, "Speech-driven automatic facial expression synthesis," in Proc. 3DTV Conference: The True Vision-Capture, Transmission and Display of 3-D Video, May 2008, pp. 273-276.
36. S. Shivappa, M. M. Trivedi, and B. Rao, "Audio-visual information fusion in human computer interfaces and intelligent environments: A survey," Proc. IEEE, to be published.