Enhancement of emotion detection in spoken dialogue systems by combining several information sources

Speech Communication

Volumn 53, Issue 9-10, 2011, Pages 1210-1228

  López Cózar, Ramón ^a   Silovsky, Jan ^b   Kroul, Martin ^b  

^a UNIVERSITY OF GRANADA   (Spain)

^b TECHNICAL UNIVERSITY OF LIBEREC   (Czech Republic)

Author keywords

Adaptive spoken dialogue systems; Combination of classifiers; Emotion detection; Human computer interaction; Information fusion

Indexed keywords

FORECASTING; HUMAN COMPUTER INTERACTION; INFORMATION FUSION; SPEECH PROCESSING; SPEECH RECOGNITION;

CLASSIFICATION RATES; COMBINATION OF CLASSIFIERS; EMOTION DETECTION; EMOTION PREDICTIONS; INFORMATION SOURCES; MODULAR ARCHITECTURES; PROCESSING STAGE; SPOKEN DIALOGUE SYSTEM;

CLASSIFICATION (OF INFORMATION);

EID: 79960839416     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2011.01.006     Document Type: Article

References (47)

1. Ai, H.; Litman, D.J.; Forbes-Riley, K.; Rotaru, M.; Tetreault, J.; Purandare, A.; 2006. Using system and user performance features to improve emotion detection in spoken tutoring systems. In: Proc. of Interspeech, pp. 797-800.
2. Ang, J.; Dhillon, R.; Krupski, A.; Shriberg, E.; Stolcke, A.; 2002. Prosody-based automatic detection of annoyance and frustration in human-computer dialog. In: Proc. of ICSLP, pp. 2037-2039.
3. Barra-Chicote, R.; Fernández, F.; Lufti, S.; Lucas-Cuesta, J.M.; Macías-Guarasa, J.; Montero, J.M.; San-Segundo, R.; Pardo, J.M.; 2009. Acoustic emotion recognition using dynamic Bayesian networks and multi-space distributions. In: Proc. of Interspeech, pp. 336-339.
4. A. Batliner, K. Fischer, R. Huber, J. Spilker, and E. Nöth How to find trouble in communication Speech Comm. 40 2003 117 143
5. Bozkurt, E.; Erzin, E.; Erdem, C.E.; Erdem, A.T.; 2009. Improving automatic emotion recognition from speech signals. In: Proc. of Interspeech, pp. 324-327.
6. J. Carletta Assessing agreement on classification tasks: the kappa statistic Comput. Linguist. 22 2 1996 249 254
7. J. Cohen A coefficient of agreement for nominal scales Educat. Psychol. Meas. 20 1960 37 46
8. T. Cover, and J. Thomas Elements of Information Theory 1991 Wiley New York
9. Dellaert, F.; Polzin, T.; Waibel, A.; 1996. Recognizing emotion in speech. In: Proc. of ICSLP, pp. 1970-1973.
10. Devillers, L.; Vidrascu, L.; 2006. Real-life emotions detection with lexical and paralinguistic cues on human-human call center dialogs. In: Proc. of Interspeech, pp. 801-804.
11. Hastie, H.W.; Prasad, R.; Walker, M.A.; 2002. What's the trouble: automatically identifying problematic dialogs in DARPA communicator dialog system. In: Proc. of ACL, pp. 384-391.
12. Huber, R.; Batliner, A.; Buckow, J.; Nöth, E.; Warnke, V.; Niemann, H.; 2000. Recognition of emotion in a realistic dialogue scenario. In: Proc. of ICSLP, pp. 665-668.
13. T. Kanda, T. Hirano, D. Eaton, and H. Ishiguro Interactive robots as social partners and peer tutors for children: a field trial Human-Comput. Interact. 19 1 2004 61 84
14. J. Kittler, M. Hatef, R.P.W. Duin, and J. Matas On combining classifiers IEEE Trans. Pattern Anal. Machine Intell. 20 3 1998 226 240
15. J. Klein, Y. Moon, and R.W. Picard This computer responds to user frustration: theory, design and results Interact. Comput. 14 2 2002 119 140 (Pubitemid 34199155)
16. L. Kuncheva, J. Bezdek, and R. Duin Decision templates for multiple classifier fusion: an experimental comparison Pattern Recognit. 34 2 2001 299 314 (Pubitemid 32871876)
17. J.R. Landis, and G.G. Koch The measurement of observer agreement for categorical data Biometrics 33 1977 159 174 (Pubitemid 8054849)
18. C.A. Le, V.-N. Huynh, and A. Shimazu Combining classifiers with multi-representation of context in word sense disambiguation Lect. Notes Artif. Intell. 3518 2005 262 268
19. Lee, C.M.; Narayanan, S.S.; Pieraccini, R.; 2001. Recognition of negative emotions from the speech signal. In: Proc. of ASRU, pp. 240-243.
20. Lee, C.M.; Narayanan, S.S.; Pieraccini, R.; 2002. Combining acoustic and language information for emotion recognition. In: Proc. of ICSLP, pp. 873-876.
21. Lee, C.M.; Narayanan, S.; 2003. Emotion recognition using a data-driven fuzzy inference system. In: Proc. of Eurospeech, pp. 157-160.
22. C.M. Lee, and S.S. Narayanan Toward detecting emotions in spoken dialogs IEEE Trans. Speech Audio Processing 13 2 2005 293 303 (Pubitemid 40320247)
23. Lee, C.-C.; Mower, E.; Busso, C.; Lee, S.; Narayanan, S.; 2009. Emotion recognition using a hierarchical binary decision tree approach. In: Proc. of Interspeech, pp. 320-323.
24. Liscombe, J.; Riccardi, G.; Hakkani-Tür, D.; 2005. Using context to improve emotion detection in spoken dialogue systems. In: Proc. of Interspeech, pp. 1845-1848. (Pubitemid 43908444)
25. D.J. Litman, and K. Forbes-Riley Recognizing student emotions and attitudes on the basis of utterances in spoken tutoring dialogues with both human and computer tutors Speech Comm. 48 2006 559 590
26. R. López-Cózar, and M. Araki Spoken, Multilingual and Multimodal Dialogue Systems, Development and Assessment 2005 John Wiley & Sons Publishers
27. R. López-Cózar, and Z. Callejas Combining language models in the input interface of a spoken dialogue system Comput. Speech Lang. 20 2005 420 440
28. Luengo, I.; Navas, E.; Hernáez, I, Sanchez, J.; 2005. Automatic emotion recognition using prosodic parameters. In: Proc. of Interspeech, pp. 493-496. (Pubitemid 43908107)
29. Luengo, I.; Navas, E.; Hernáez, I.; 2009. Combining spectral and prosodic information for emotion recognition in the Interspeech 2009 Emotion Challenge. In: Proc. of Interspeech, pp. 332-335.
30. Lugger, M.; Yang, B.; 2009. On the relevance of high-level features for speaker independent emotion recognition of spontaneous speech. In: Proc. of Interspeech, pp. 1995-1998.
31. D. Morrison, R. Wang, and L.C. De Silva Ensemble methods for spoken emotion recognition in call-centres Speech Comm 49 2 2007 98 112
32. S. Möller Quality of Telephone-Based Spoken Dialogue Systems 2004 Springer
33. Nakatsu, R.; Hicholson, J.; Tosa, N.; 1999. Emotion recognition and its application to computer agents with spontaneous interactive capabilities. In: Proc. of the International Conference on Multimedia Computing and Systems, Florence, Italy.
34. Neiberg, D.; Elenius, K.; Laskowski, K.; 2006. Emotion recognition in spontaneous speech using GMMs. In: Proc. of Interspeech, pp. 809-812.
35. T.L. Nwe, S.V. Foo, and L.C. De Silva Speech emotion recognition using hidden Markov models Speech Comm. 41 4 2003 603 623
36. A. Ortony, G. Clore, and A. Collins The Cognitive Structure of Emotions 1990 Cambridge University Press Cambridge, UK
37. Petrushin, V.; 2000. Emotion recognition in speech signal, experimental study, development, and application. In: Proc. of ICSLP, Beijing, China.
38. R. Plutchik The Psychology and Biology of Emotions 1994 HaperCollins College New York
39. Polzehl, T.; Sundaram, S.; Ketabdar, H.; Wagner, M.; Metze, F.; 2009. Emotion classification in children's speech using fusion of acoustic and linguistic features. In: Proc. of Interspeech, pp. 340-343.
40. Roli, F.; Kittler, J.; Windeatt, T. (Eds.), 2004. Multiple classifier systems. In: Proc. of 5th Internat. Workshop MSC 2004, LNCS, Vol. 3077. Springer.
41. J. Scheirer, R. Fernandez, J. Klein, and R.W. Picard Frustrating the user on purpose: a step toward building an affective computer Interact. Comput. 14 2 2002 93 118 (Pubitemid 34199154)
42. Schuller, B.; Steidl, S.; Batliner, A.; 2009. The INTERSPEECH 2009 Emotion Challenge. In: Proc. of Interspeech, pp. 312-315.
43. S. Steidl Automatic Classification of Emotion-Related User States in Spontaneous Children's Speech 2009 Logos Verlag Berlin
44. A. Stolcke, K. Ries, H. Coccaro, E. Shriberg, R. Bates, D. Jurafsky, P. Taylor, R. Martin, M. Meteer, and C. Van Ess-Dykema Dialogue act modeling for automatic tagging and recognition of conversational speech Comput. Linguist. 26 3 2000 339 373
45. P. Taylor, S. King, S. Isard, and H. Wright Intonation and dialog context as constraints for speech recognition Lang. Speech 41 3-4 1998 489 508
46. Xu, L.; Xu, M.; Yang, D.; 2009. ANN based decision fusion for speech emotion recognition. In: Proc. of Interspeech, pp. 2035-2038.
47. Yacoub, S.; Simske, S.; Lin, X.; Burns, J.; 2003. Recognition of emotions in interactive voice response systems. In: Proc. of Interspeech, pp. 729-732.