Emotion recognition from speech signals using new harmony features

Signal Processing

Volumn 90, Issue 5, 2010, Pages 1415-1423

  Yang, B ^a   Lugger, M ^a  

^a UNIVERSITY OF STUTTGART   (Germany)

Author keywords

Emotion recognition; Feature extraction; Harmony features; Pitch interval

Indexed keywords

EMOTION RECOGNITION; MUSIC THEORY; PITCH CONTOURS; RECOGNITION PERFORMANCE; SPEECH SIGNALS; STATE OF THE ART;

SPEECH RECOGNITION;

FEATURE EXTRACTION;

EID: 75249100219     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2009.09.009     Document Type: Article

References (54)

1. Cowie R., et al. Emotion recognition in human-computer interaction. IEEE Signal Processing Magazine 18 (2001) 32-81
2. Cowie R., and Douglas-Cowie E. Speakers and hearers are people: reflections on speech deterioration as a consequence of acquired deafness. In: Spens K.E., and Plant G. (Eds). Profound Deafness and Speech Communication (1995), Wiley, New York
3. Traunmüller H. Paralinguistic phenomena. In: Ammon U., Dittmar N., et al. (Eds). Sociolinguistics: An International Handbook of the Science of Language and Society (2005), Walter de Gruyter, Berlin 653-665
4. Laver J. The Phonetic Description of Voice Quality (1980), Cambridge University Press, Cambridge
5. F. Burkhardt, M. van Ballegooy, et al., An emotion-aware voice portal, in: Electronic Speech Signal Processing Conference, 2005.
6. F. Burkhardt, T. Polzehl, et al., Detecting real life anger, in: Proceedings of the IEEE ICASSP, 2009, pp. 4761-4764.
7. L. Devillers, L. Vidrascu, Real-life emotions detection with lexical and paralinguistic cues on human-human call center dialogs, in: INTERSPEECH, 2006.
8. Morrison D., Wang R., et al. Ensemble methods for spoken emotion recognition in call-centres. Speech communication 49 (2007) 98-112
9. A. Batliner, et al., You stupid tin box-children interacting with the AIBO robot: a cross-linguistic emotional speech corpus, in: Proceedings of the Fourth International Conference of Language Resources and Evaluation, 2004, pp. 171-174.
10. J. Hirschberg, C. Avesani, The role of prosody in disambiguating potentially ambiguous utterances in English and Italian, in: ESCA Workshop on Intonation, 1997.
11. R. Gretter, D. Seppi, Using prosodic information for disambiguation purposes, in: INTERSPEECH, 2005, pp. 1821-1824.
12. M. Cernak, C. Wellekens, Emotional aspects of intrinsic speech variabilities in automatic speech recognition, in: International Conference on Speech and Computer, 2006, pp. 405-408.
13. M. Schröder, Emotional speech synthesis: a review, in: EUROSPEECH, 2001, pp. 561-564.
14. M. Schröder, R. Cowie, et al., Acoustic correlates of emotion dimensions in view of speech synthesis, in: EUROSPEECH, 2001, pp. 87-90.
15. Lewis M., Haviland-Jones J., and Barrett L.F. Handbook of Emotions (2008), The Guilford Press
16. Ekman P. An argument for basic emotions. Cognition and Emotion 6 (1992) 169-200
17. Schlosberg H. Three dimensions of emotions. Psychological Review 61 (1954) 81-88
18. Plutchik R. Emotion: A Psychoevolutionary Synthesis (1980), Harper & Row, New York
19. Hanjalic A. Extracting moods from pictures and sounds: towards truly personalized TV. IEEE Signal Processing Magazine 23 (2006) 90-100
20. Duda R.O., Hart P.E., and Stork D.G. Pattern Classification. second ed. (2001), Wiley, New York
21. M. Lugger, B. Yang, Psychological motivated multi-stage emotion classification exploiting voice quality features, in: F. Mihelic, J. Zibert (Eds.), Speech Recognition, In-Tech, 2008 (Chapter 22).
22. C. Pereira, Dimensions of emotional meaning in speech, in: ITRW on Speech and Emotion, 2000, pp. 25-28.
23. Y. Xu, S. Chuenwattanapranithi, Perceiving anger and joy in speech through the size code, in: Proceedings of the International Conference on Phonetic Sciences, 2007, pp. 2105-2108.
24. I. Fonagy, Emotions voice and music, Royal Swedish Academy of Music, 1981, pp. 51-79.
25. B. Schuller, D. Arsic, et al., Emotion recognition in the noise applying large acoustic feature sets, in: Speech Prosody, Dresden, 2006.
26. Schölkopf B., and Smola A. Learning with Kernels (2002), MIT Press, Cambridge, MA
27. Vapnik V.N. The Nature of Statistical Learning Theory (2000), Springer, Berlin
28. B. Schuller, G. Rigoll, M. Lang, Hidden Markov model-based speech emotion recognition, in: Proceedings of the IEEE ICASSP, 2003.
29. M. Lugger, B. Yang, Extracting voice quality contours using discrete hidden Markov models, in: Proceedings of the Speech Prosody, 2008.
30. Kay S.M. Fundamentals of Statistical Signal Processing. Detection Theory Vol. 2 (1998), Prentice-Hall, Englewood Cliffs, NJ
31. Akaike H. A new look at the statistical model identification. IEEE Transactions on Automatic Control 19 (1974) 716-723
32. Kuncheva L.I. Combining Pattern Classifiers Methods and Algorithms (2004), Wiley, New York
33. D. Ververidis, C. Kotropoulos, Emotional speech classification using Gaussian mixture models, in: IEEE International Symposium on Circuits and Systems, 2005, pp. 2871-2874.
34. M. Lugger, B. Yang, Combining classifiers with diverse feature sets for robust speaker independent emotion recognition, in: Proceedings of the EUSIPCO, 2009.
35. Reunanen J. Overfitting in making comparisons between variable selection methods. Journal of Machine Learning Research 3 (2003) 1371-1382
36. Pudil P., Ferri F.J., et al. Floating search methods for feature selection with nonmonotonic criterion. Pattern Recognition-Conference B: Computer Vision 2 (1994) 279-283
37. Nwe T., Foo S., and Silva L.D. Speech emotion recognition using hidden Markov models. Speech communication 41 (2003) 603-623
38. M. Lugger, B. Yang, The relevance of voice quality features in speaker independent emotion recognition, in: Proceedings of the IEEE ICASSP, vol. 4, 2007, pp. 17-20.
39. M. Lugger, B. Yang, Cascaded emotion classification via psychological emotion dimensions using a large set of voice quality parameters, in: Proceedings of the IEEE ICASSP, 2008, pp. 4945-4948.
40. D. Talkin, W. Kleijn, K. Paliwal, A robust algorithm for pitch tracking (RAPT), Speech Coding and Synthesis (1995) 495-518.
41. Lee C.M., and Narayanan S.S. Toward detecting emotions in spoken dialogs. IEEE Transactions on Speech and Audio Processing 13 (2005) 293-303
42. Banse R., and Scherer K.R. Acoustic profiles in vocal emotion expression. Journal of Personality and Social Psychology 70 (1996) 614-636
43. G. Fant, Acoustic Theory of Speech Production, The Hague, Mouton, 1970.
44. Stevens K., and Hanson H. Classification of glottal vibration from acoustic measurements. In: Fujimura O., and Hirano M. (Eds). Vocal Fold Physiology (1994), Hiltop University Press 147-170
45. Helmholtz H.L.F. On the Sensations of Tone as a Physiological Basis for the Theory of Music. second ed. (1877), Dover Publications, New York
46. Plomp R., and Levelt J.M. Tonal consonance and critical bandwidth. Journal of the Acoustical Society of America 38 (1965) 548-560
47. G. Schellenberg, S. Trehub, Frequency ratios and the perception of done patterns, Psychonomic Bulletin & Review (1994) 191-201.
48. D. Schwartz, C. Howe, D. Purves, The statistical structure of human speech sounds predicts musical universals, The Journal of Neuroscience (2003) 7160-7168.
49. T. Fujishima, Real-time chord recognition of musical sound: a system using common lisp music, in: ICMC, 1999, pp. 464-467.
50. G.H. Wakefield, Mathematical representation of joint time-chroma distribution, in: SPIE, vol. 3807, 1999.
51. Cook N.D., and Fujidawa T.X. The psychophysics of harmony perception: harmony is a three-tone phenomenon. Empirical Musicology Review 1 2 (2006) 106-126
52. D. Ververidis, C. Kotropoulos, A state of the art review of emotional speech databases, in: Proceedings of the First Richmedia Conference, 2003.
53. F. Burkhardt, A. Paeschke, et al., A database of German emotional speech, in: Proceedings of the Interspeech, 2005, pp. 1517-1520.
54. M. Lugger, B. Yang, An incremental analysis of different feature groups in speaker independent emotion recognition, in: Proceedings of the International Conference on Phonetic Sciences, 2007, pp. 2149-2152.