Feature analysis and evaluation for automatic emotion identification in speech

IEEE Transactions on Multimedia

Volumn 12, Issue 6, 2010, Pages 490-501

  Luengo, Iker ^a   Navas, Eva ^a   Hernaez, Inmaculada ^a  

^a UNIVERSITY OF THE BASQUE COUNTRY UPV EHU   (Spain)

Author keywords

Emotion identification; information fusion; parametrization

Indexed keywords

EARLY FUSION; EMOTION IDENTIFICATION; EMOTIONAL INFORMATION; FEATURE ANALYSIS; INFORMATION SOURCES; LATE FUSION; PARAMETRIZATIONS; PROSODIC FEATURES; SPECTRAL ENVELOPE PARAMETERS; SPECTRAL ENVELOPES; SPECTRAL STATISTICS; SYSTEMATIC STUDY; VOICE QUALITY;

FACE RECOGNITION; INFORMATION FUSION;

IDENTIFICATION (CONTROL SYSTEMS);

EID: 77956733663     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2010.2051872     Document Type: Article

References (47)

1. K. R. Scherer, "Vocal communication of emotion: A review of research paradigms," Speech Commun., vol.40, pp. 227-256, Apr. 2003.
2. K. R. Scherer, "Psychological models of emotion," in The Neuropsychology of Emotion, J. Borod, Ed. Oxford, U.K.: Oxford Univ. Press, 2000, ch. 6, pp. 137-166.
3. P. Ekman, "An argument for basic emotions," Cognit. Emotion, vol.6, pp. 169-200, 1992.
4. C. Darwin, The Expression of the Emotions in Man and Animals, 3rd ed. Oxford, U.K.: Oxford Univ. Press, 1998.
5. F. Burkhardt and W. F. Sendlmeier, "Verification of acoustical correlates of emotional speech using formant-synthesis," in Proc. ISCA Tutorial and Research Workshop Speech and Emotion, Belfast, Ireland, Sep. 2000, pp. 151-156.
6. C. F. Huang and M. Akagi, "A three-layered model for expressive speech perception," Speech Commun., vol.50, pp. 810-828, Oct. 2008.
7. K. R. Scherer, R. Banse, H. G.Wallbott, and T. Goldbeck, "Vocal cues in emotion encoding and decoding," Motiv. Emotion, vol.15, no.2, pp. 123-148, 1991.
8. M. Schröder, "Speech and emotion research," Ph.D. disservation, Universität des Saarlandes, Saarbrücken, Germany, 2003.
9. E. Navas, I. Hernáez, A. Castelruiz, J. Sánchez, and I. Luengo, "Acoustic analysis of emotional speech in standard Basque for emotion recognition," in Progress in Pattern Recognition, Image Analysis and Applications, ser. Lecture Notes in Computer Science. Berlin, Germany: Springer, Oct. 2004, vol.3287, pp. 386-393.
10. D. Erickson, "Expressive speech: Production, perception and application to speech synthesis," Acoust. Sci. Tech., vol.26, pp. 317-325, 2005.
11. T. L. Nwe, S. W. Foo, and L. C. de Silva, "Speech emotion recognition using hidden Markov models," Speech Commun., vol.41, pp. 603-623, Jun. 2003.
12. S. Kim, P. G. Georgiou, S. Lee, and S. Narayanan, "Real-time emotion detection system using speech: Multi-modal fusion of different timescale features," in Proc. Int. Workshop Multimedia Signal Processing, Crete, Greece, Oct. 2007, pp. 48-51.
13. B. Vlasenko, B. Schuller, A. Wendemuth, and G. Rigoll, "Frame vs. turn-level: Emotion recognition from speech considering static and dynamic processing," in Affective Computing and Intelligent Interaction, ser. Lecture Notes in Computer Science. Berlin, Germany: Springer, 2007, vol.4738, pp. 139-147.
14. J. Nicholson, K. Takahashi, and R. Nakatsu, "Emotion recognition in speech using neural networks," Neural Comput. Appl., vol.9, pp. 290-296, Dec. 2000.
15. O. W. Kwon, K. Chan, J. Hao, and T. W. Lee, "Emotion recognition by speech signals," in Proc. Eurospeech, Geneva, Switzerland, 2003, pp. 125-128.
16. T. Vogt and E. André, "Improving automatic emotion recognition from speech via gender differentiation," in Proc. LREC, Genoa, Italy, May 2006.
17. B. Schuller, R. Müller, M. Lang, and G. Rigoll, "Speaker independent emotion recognition by early fusion of acoustic and linguistic features within ensembles," in Proc. Interspeech, Lisbon, Portugal, Sep. 2005, pp. 805-808.
18. R. López-Cozar, Z. Callejas, M. Kroul, J. Nouza, and J. Silovský, "Two-level fusion to improve emotion classification in spoken dialogue systems," in Graphics Recognition. Recent Advances and New Opportunities, ser. Lecture Notes in Computer Science. Berlin, Germany: Springer, 2008, vol.5246, pp. 617-624.
19. M. Lugger and B. Yang, "The relevance of voice quality features in speaker independent emotion recognition," in Proc. ICCASP, Honolulu, HI, Apr. 2007, vol.4, pp. 17-20.
20. C. Gobl and A. N. Chasaide, "The role of voice quality in communicating emotion, mood and attitude," Speech Commun., vol.40, pp. 189-212, Apr. 2003.
21. R. Tato, R. Santos, R. Kompe, and J. Pardo, "Emotional space improves emotion recognition," in Proc. ICSLP, Sep. 2002, pp. 2029-2032.
22. R. Müller, B. Schuller, and G. Rigoll, "Enhanced robustness in speech emotion recognition combining acoustic and semantic analyses," in Proc. From Signals to Signs of Emotion and Vice Versa, Santorino, Greece, Sep. 2004.
23. A. Nogueiras, A. Moreno, A. Bonafonte, and J. B. Mariño, "Speech emotion recognition using hidden Markov models," in Proc. Eurospeech, Aalborg, Denmark, Sep. 2001, pp. 2679-2682.
24. J. Kittler, M. Hatef, R. P. W. Duin, and J. Matas, "On combining classifiers," IEEE Trans. Pattern Anal. Mach. Intell., vol.20, no.3, pp. 226-239, Mar. 1998.
25. D. Ruta and B. Gabrys, "An overview of classifier fusion methods," Comput. Inf. Syst., vol.7, no.1, pp. 1-10, Feb. 2000.
26. F. Burkhardt, A. Paeschke, M. Rolfes, W. F. Sendlmeier, and B. Weiss, "A database of German emotional speech," in Proc. Interspeech, Lisbon, Portugal, Sep. 2005, pp. 1517-1520.
27. J. Ramirez, J. C. Segura, C. Benitez, A. de la Torre, and A. Rubio, "Efficient voice activity detection algorithms using long-term speech information," Speech Commun., vol.42, pp. 271-287, Apr. 2004.
28. P. Alku, H. Tiitinen, and R. Näätänen, "A method for generating natural-sounding speech stimuli for cognitive brain research," Clin. Neurophysiol., vol.110, pp. 1329-1333, Aug. 1999.
29. I. Luengo, I. Saratxaga, E. Navas, I. Hernáez, J. Sánchez, and I. n. Sainz, "Evaluation of pitch detection algorithms under real conditions," in Proc. ICASSP, Honolulu, HI, Apr. 2007, pp. 1057-1060.
30. I. Luengo, E. Navas, J. Sánchez, and I. Hernáez, "Detección de vocales mediante modelado de clusters de fonemas," Procesado Del Lenguaje Natural, vol.43, pp. 121-128, Sep. 2009.
31. F. Ringeval and M. Chetouani, "Exploiting a vowel based approach for acted emotion recognition," in Verbal and Nonverbal Features of Human-Human and Human-Machine Interaction, ser. Lecture Notes in Computer Science. Berlin, Germany: Springer, Oct. 2008, vol.5042, pp. 243-254.
32. T. Bäckström, P. Alku, and E. Vilkman, "Time-domain parameterization of the closing phase of glottal airflow waveform from voices over a large intensity range," IEEE Trans. Speech Audio Process., vol.10, no.3, pp. 186-192, Mar. 2002.
33. R. van Son and L. Pols, "An acoustic description of consonant reduction," Speech Commun., vol.28, pp. 125-140, Jun. 1999.
34. K. Fukunaga, Introduction to Statistical Pattern Recognition. New York: Academic, 1990.
35. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. New York: Wiley, 2001.
36. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection," J. Mach. Learn. Res., vol.3, pp. 1157-1182, Mar. 2003.
37. H. Peng, F. Long, and C. Ding, "Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy, " IEEE Trans. Pattern Anal. Mach. Intell., vol.27, no.8, pp. 1226-1238, Aug. 2005.
38. C. J. Burges, "A tutorial on support vector machines for pattern recognition," Data Min. Knowl. Discov., vol.2, pp. 121-167, 1998.
39. C. W. Hsu and C. J. Lin, "A comparison of methods for multi-class support vector machines," IEEE Trans. Neural Netw., vol.13, no.2, pp. 415-425, Mar. 2002.
40. J. Fierrez-Aguilar, D. Garcia-Romero, J. Ortega-Garcia, and J. Gonzalez-Rodriguez, "Adapted user-dependent multimodal biometric authentication exploiting general information," Pattern Recognit. Lett., vol.26, no.16, pp. 2628-2639, Dec. 2005.
41. B. Gutschoven and P. Verlinde, "Multi-modal identity verification using support vector machines (SVM)," in Proc. Int. Conf. Information Fusion, Paris, France, Jul. 2000, vol.2, pp. 3-8.
42. R. Banse and K. R. Scherer, "Acoustic profiles in vocal emotion expression," J. Personal. Social Pathol., vol.70, no.3, pp. 614-636, 1996.
43. o and pause features analysis for anger and fear detection in real-life spoken dialogs," in Proc. Speech Prosody, Nara, Japan, Mar. 2004, pp. 205-208.
44. A. Batliner, S. Steidl, B. Schuller, D. Seppi, K. Laskowski, T. Vogt, L. Devillers, L. Vidrascu, N. Amir, L. Kessous, and V. Aharonson, "Combining efforts for improving automatic classification of emotional user states," in Proc. Information Society-Language Technologies Conf. (IS-LTC), Ljubljana, Slovenia, Oct. 2006, pp. 240-245.
45. I. Luengo, E. Navas, and I. Hernáez, "Combining spectral and prosodic information for emotion recognition in the interspeech 2009 emotion challenge," in Proc. Interspeech, Brighton, U.K., Sep. 2009, pp. 332-335.
46. I. Luengo, E. Navas, I. Hernáez, and J. Sanchez, "Automatic emotion recognition using prosodic parameters," in Proc. Interspeech, Lisbon, Portugal, Sep. 2005, pp. 493-496.
47. E. Navas, I. Hernáez, and I. Luengo, "An objective and subjective study of the role of semantics in building corpora for TTS," IEEE Trans. Speech Audio Process., vol.14, no.4, pp. 1117-1127, Jul. 2006.