Continuous emotion recognition with phonetic syllables

Speech Communication

Volumn 57, Issue , 2014, Pages 155-169

  Origlia, A ^a   Cutugno, F ^a   Galata V ^b  

^a UNIVERSITY OF NAPLES FEDERICO II   (Italy)

^b INSTITUTE OF COGNITIVE SCIENCES AND TECHNOLOGIES   (Italy)

Author keywords

Affective computing; Feature extraction; Phonetic syllables; Valence Activation Dominance space

Indexed keywords

AFFECTIVE COMPUTING; AMOUNT OF INFORMATION; FEATURE EXTRACTION METHODS; PHONETIC SYLLABLES; REALTIME PROCESSING; STATE-OF-THE-ART PERFORMANCE; THREE-DIMENSIONAL MODEL; VALENCE-ACTIVATION-DOMINANCE SPACE;

ACOUSTIC PROPERTIES; FEATURE EXTRACTION; RESEARCH;

LINGUISTICS;

EID: 84887045699     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2013.09.012     Document Type: Article

References (74)

1. A. Arvaniti Rhythm, timing and the timing of rhythm Phonetica 66 2009 46 63
2. Avanzi, M.; Lacheret-Dujour, A.; Victorri, B.; 2010. A corpus based learning method for prominence detection in spontaneous speech. In: Proc. of Speech Prosody (Online).
3. Baltrušaitis, T.; Banda, N.; Robinson, P.; 2013. Dimensional affect recognition using continuous conditional random fields. In: Proc. of IEEE International Conference of Automatic Face and Gesture Recognition.
4. Barry, W.J.; Andreeva, B.; Russo, M.; Dimitrova, S.; Kostadinova, T.; 2003. Do rhythm measures tell us anything about language type? In: Proc. of ICPhS, pp. 2693-2696.
5. A. Batliner, D. Seppi, S. Steidl, and B. Schuller Segmenting into adequate units for automatic recognition of emotion-related episodes: a speech-based approach Advances in Human-Computer Interaction 2010 2010 1 15
6. Boersma, P.; 1993. Accurate short-term analysis of the fundamental frequency and the harmonics-to-noise ratio of a sampled sound. In: Proc. of IFA, pp. 97-110.
7. Boersma, P.; Weenink, D.; 2011. Praat: doing phonetics by computer [computer program]. version 5.2.40.
8. C. Breitenstein, D. Van Lancker, and I. Daum The contribution of speech rate and pitch variation to the perception of vocal emotions in a German and an American sample Cognition and Emotion 15 2001 57 79 (Pubitemid 32095593)
9. Burkhardt, F.; Paeschke, A.; Rolfes, M.; Sendlmeier, W.F.; Weiss, B.; 2005. A database of German emotional speech. In: Proc. of Interspeech, pp. 1517-1520. (Pubitemid 43908362)
10. C. Chang, and C. Lin LIBSVM: a library for support vector machines ACM Transactions on Intelligent Systems and Technology 2 2011 1 27
11. Cowie, R.; Douglas-Cowie, E.; Savvidou, S.; McMahon, E.; Sawey, M.; Schröder, M.; 2000. Feeltrace: an instrument for recording perceived emotion in real time. In: Proc. of ISCA Workshop on Speech & Emotion, pp. 19-24.
12. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J.G. Taylor Emotion recognition in human-computer interaction IEEE Signal Processing Magazine 18 2001 33 80
13. F. Cummins, and R. Port Rhythmic constraints on stress timing in English Journal of Phonetics 26 1998 145 171 (Pubitemid 128179514)
14. C. D'Alessandro, and P. Mertens Automatic pitch contour stylization using a model of tonal perception Computer Speech and Language 9 1995 257 288
15. Dellwo, V.; 2006. Rhythm and speech rate: a variation coefficient for ΔC. In: Karnowski, P. Szigeti, I. (Eds.), Language and Language Processing, Peter Lang, Frankfurt am Main, pp. 231-241.
16. Dellwo, V.; Wagner, P.; 2003. Relationships between speech rate and rhythm. In: Proc. of the ICPhS, pp. 471-474.
17. Drioli, C.; Tisato, G.; Cosi, P.; Tesser, F.; 2003. Emotions and voice quality: experiments with sinusoidal modeling. In: Proc. of VOQUAL, pp. 127-132.
18. P. Ekman An argument for basic emotions Cognition and Emotion 6 1992 169 200
19. Espinosa, H.P.; Garcia, C.A.R.; Pineda, L.V.; 2010. Features selection for primitives estimation on emotional speech. In: Proc. of IEEE ICASSP, pp. 5138-5141.
20. R. Fernandez, and R. Picard Recognizing affect from speech prosody using hierarchical graphical models Speech Communication 53 2011 1088 1103
21. L.L. Feth Combinations of amplitude and frequency differences in auditory discrimination Acustica 26 1972 67 77
22. N. Fragopanagos, and J.G. Taylor Special issue: emotion recognition in human-computer interaction Neural Networks 18 2005 389 405 (Pubitemid 40922647)
23. Galatà, V.; 2010. Production and perception of vocal emotions: a cross-linguistic and cross-cultural study. Ph.D. Thesis. University of Calabria, Italy.
24. D. Gharavian, M. Sheikhan, and F. Ashoftedel Emotion recognition improvement using normalized formant supplementary features by hybrid of DTW-MLP-GMM model Neural Computing and Applications 22 2012 1 11
25. Goudbeek, M.; Goldman, J.P.; Scherer, K.R.; 2009. Emotion dimensions and formant position. In: Proc. of Interspeech, pp. 1575-1578.
26. Grimm, M.; Kroschel, K.; 2005. Emotion estimation in speech using a 3D emotion space concept. In: Proc. of IEEE Automatic Speech Recognition & Understanding Workshop, pp. 381-385.
27. M. Grimm, E. Mower, K. Kroschel, and S. Narayanan Primitives-based evaluation and estimation of emotions in speech Speech Communication 49 2007 787 800 (Pubitemid 47268568)
28. Grimm, M.; Kroschel, K.; Narayanan, S.; 2008. The Vera am Mittag German audio-visual emotional speech database. In: Proc. of ICME, pp. 865-868.
29. Gunes, H.; Schuller, B.; Pantic, M.; Cowie, R.; 2011. Emotion representation, analysis and synthesis in continuous space: a survey. In: Proc. of FG, pp. 827-834.
30. Hall, M.A.; 1998. Correlation-based feature subset selection for machine learning. Ph.D. Thesis, Hamilton, New Zealand.
31. D. House Tonal Perception in Speech 1990 Lund University Press Lund
32. House, D.; 1995. Perception of prepausal tonal contours: implications for automatic stylization of intonation. In: Proc. of Eurospeech, pp. 949-952.
33. House, D.; 1996. Differential perception of tonal contours through the syllable. In: Proc. of ICSLP, pp. 2048-2051.
34. O. Jespersen Lehrbuch der Phonetic 1920 B.G. Teubner, Leipzig e Berlin
35. J. Jia, S. Zhang, F. Meng, Y. Wang, and L. Cai Emotional audio-visual speech synthesis based on PAD IEEE Transactions on Audio, Speech & Language Processing 19 2011 570 582
36. Jittiwarangkul, N.; Jitapunkul, S.; Luksaneeyanavin, S.; Ahkuputra, V.; Wutiwiwatchai, C.; 1998. Thai syllable segmentation for connected speech based on energy. In: Proceedings of the IEEE Asia Pacific Conference on Circuits and Systems, pp. 169-172.
37. Kaiser, J.; 1990. On a simple algorithm to calculate the 'energy' of a signal. In: Proc. of IEEE ICASSP, pp. 381-384.
38. Kao, Y.; Lee, L.; 2006. Feature analysis for emotion recognition from mandarin speech considering the special characteristics of Chinese language. In: Proc. of Interspeech, pp. 1814-1817.
39. D.H. Klatt Discrimination of fundamental frequency contours in synthetic speech: implications for models of pitch perception Journal of the Acoustical Society of America 53 1973 8 16
40. Ludusan, B.; Origlia, A.; Cutugno, C.; 2011. On the use of the rhythmogram for automatic syllabic prominence annotation. In: Proc. of Interspeech, pp. 2413-2416.
41. D. Maiwald Ein Funktionsschema des Gehörs zur Beschreibung der Erkennbarkeit kleiner Frequenz-und-Amplitudenänderungen Acustica 18 1967 81 93
42. Martin, P.; 2010. Prominence detection without syllabic segmentation. In: Proc. of Speech Prosody (Online).
43. L. Mary, and B. Yegnanarayana Extraction and representation of prosodic features for language and speaker recognition Speech Communication 50 2008 782 796
44. McKeown, G.; Valstar, M.F.; Cowie, R.; Pantic, M.; 2010. The semaine corpus of emotionally coloured character interactions. In: Proc. of ICME, pp. 1079-1084.
45. A. Mehrabian Pleasure-arousal-dominance: a general framework for describing and measuring individual differences in temperament Current Psychology: Developmental, Learning, Personality, Social 14 1996 261 292
46. D. Mermelstein Automatic segmentation of speech into syllabic units Journal of Acoustical Society of America 54 1975 880 883
47. Mertens, P.; 2004. The prosogram: semi-automatic transcription of prosody based on a tonal perception model. In: Proc. of Speech Prosody.
48. Møller, E.; 1974. Dynamic properties of cochlear nucleus units in response to excitatory and inhibitory tones. In: Proc. of Facts and models in hearing, pp. 227-40.
49. M.A. Nicolaou, H. Gunes, and M. Pantic Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space IEEE Transactions on Affective Computing 2 2011 92 105
50. A. Origlia, G. Abete, and C. Cutugno A dynamic tonal perception model for optimal pitch stylization Computer Speech and Language 27 2013 190 208
51. S. Patel, K.R. Scherer, E. Bjorkner, and J. Sundberg Mapping emotions into acoustic space: the role of voice production Biological Psychology 87 2011 93 98
52. Petrillo, M.; Cutugno, F.; 2003. A syllable segmentation algorithm for English and Italian. In: Proc. of Eurospeech, pp. 2913-2916.
53. Pfitzinger, H.; Burger, S.; Heid, S.; 1996. Syllable detection in read and spontaneous speech. In: Proceedings of the Fourth International Conference on Spoken Language, pp. 1261-1264.
54. I. Pollack Detection of rate of change of auditory frequency Journal of Experimental Psychology 77 1968 535 541
55. F. Ramus, M. Nespor, and J. Mehler Correlates of linguistic rhythm in the speech signal Cognition 73 1999 265 292
56. P. Roach English Phonetics and Phonology A Practical Course 2000 Cambridge University Press
57. M. Rossi Le seuil de glissando ou seuil de perception des variations tonales pour les sons de la parole Phonetica 23 1971 1 33
58. M. Rossi Interactions of intensity glides and frequency glissandos Language and Speech 21 1978 384 394
59. J.A. Russell A circumplex model of affect Journal of Personality & Social Psychology 39 1980 1161 1178
60. K.R. Scherer, T. Johnstone, and G. Klasmeyer Vocal expression of emotions R.J. Davidson, K.R. Scherer, H.H. Goldsmith, Handbook of Affective Sciences 2003 Oxford University Press 433 456
61. H.E.M. Schouten Identification and discrimination of sweep tones Perception and Psychophysics 37 1985 369 376
62. Schuller, B.; Vlasenko, B.; Arsic, D.; Rigoll, G.; Wendemuth, A.; 2008. Combining speech recognition and acoustic word emotion models for robust text independent emotion recognition. In: Proc. of IEEE ICME, pp. 1333-1336.
63. Schuller, B.; Steidl, S.; Batliner, A.; 2009. The interspeech 2009 emotion challenge. In: Proc. of Interspeech, pp. 312-315.
64. B. Schuller, A. Batliner, S. Steidl, and D. Seppi Recognising realistic emotions and affect in speech: state of the art and lessons learnt from the first challenge Speech Communication 53 2011 1062 1087
65. Seppi, D.; Batliner, A.; Steidl, S.; Schuller, B.; Nöth, E.; 2010. Word accent and emotion. In: Proc. of Speech Prosody [Online].
66. R.L. Sergeant, and J.D. Harris Sensitivity to unidirectional frequency modulation Journal of the Acoustical Society of America 34 1962 1625 1628
67. Silipo, R.; Greenberg, S.; 1999. Automatic transcription of prosodic stress for spontaneous English discourse. In: Proc. of ICPhS, pp. 2351-2354.
68. Tamburini, F.; Wagner, P.; 2007. On automatic prominence detection for German. In: Proc. of Interspeech, pp. 1809-1812.
69. J. Terken Fundamental frequency and perceived prominence Journal of the Acoustical Society of America 89 1991 1768 1776
70. J. t'Hart, R. Collier, and A. Cohen A Perceptual Study of Intonation: An Experimental-Phonetic Approach 1990 Cambridge University Press Cambridge
71. B. Vlasenko, B. Schuller, A. Wendemuth, and G. Rigoll Frame vs. Turn-Level: Emotion Recognition from Speech Considering Static and Dynamic Processing 2007 Springer Berlin pp. 139-147
72. Vlasenko, B.; Philippou-Hübner, D.; Prylipko, D.; Böck, R.; Siegert, I.; Wendemuth, A.; 2011. Vowel formants analysis allows straightforward detection of high arousal emotions. In: Proc. of ICME, pp. 4230-4235.
73. S. Wu, T.H. Falk, and W. Chan Automatic speech emotion recognition using modulation spectral features Speech Communication 53 2011 768 785
74. E. Zwicker Direct comparisons between the sensations produced by frequency modulation and amplitude modulation Journal of the Acoustical Society of America 34 1962 1425 1430