Speech emotion recognition using amplitude modulation parameters and a combined feature selection procedure

Knowledge-Based Systems

Volumn 63, Issue , 2014, Pages 68-81

  Mencattini, Arianna ^a   Martinelli, Eugenio ^a   Costantini, Giovanni ^a,d   Todisco, Massimiliano ^a   Basile, Barbara ^b,c   Bozzali, Marco ^b   Di Natale, Corrado ^a,d  

^a UNIVERSITY OF ROME TOR VERGATA   (Italy)

^b IRCCS FONDAZIONE SANTA LUCIA   (Italy)

^c Center of Cognitive Psychotherapy and Experimental Psychopathology of Roma Cognitive Psychotherapy School   (Italy)

^d Institute of Acoustics and Sensors Orso Mario Corbino   (Italy)

Author keywords

Audio signal modulation; Circumplex model of emotions; Partial least square (PLS) regression; Pearson correlation coefficient; Pitch contour characterization; Speech emotion recognition (SER)

Indexed keywords

AMPLITUDE MODULATION; CORRELATION METHODS; MATHEMATICAL MODELS; REGRESSION ANALYSIS;

AUDIO SIGNAL; CIRCUMPLEX MODELS; PARTIAL LEAST-SQUARE REGRESSION; PEARSON CORRELATION COEFFICIENTS; PITCH CONTOURS; SPEECH EMOTION RECOGNITION;

CONTINUOUS SPEECH RECOGNITION;

EID: 84899981373     PISSN: 09507051     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.knosys.2014.03.019     Document Type: Article

References (78)

1. I.-O. Stathopoulou, E. Alepis, G. Tsihrintzis, and M. Virvou On assisting a visual-facial affect recognition system with keyboard-stroke pattern information Knowl.-Based Syst. 23 4 2010 350 356
2. L. Yu, J. Wu, P. Chang, and H. Chu Using a contextual entropy model to expand emotion words and their intensity for the sentiment classification of stock market news Knowl.-Based Syst. 41 2013 89 97
3. I. Mauss, and M. Robinson Measures of emotion: a review Cogn. Emotion 23 2 2009 209 237
4. R. Nakatsu, J. Nicholson, and N. Tosa Emotion recognition and its application to computer agents with spontaneous interactive capabilities Knowl.-Based Syst. 13 7-8 2000 497 504
5. K. Umapathy, and S. Krishnan Feature analysis of pathological speech signals using local discriminant bases technique Med. Biol. Eng. Comput. 43 4 2005 457 464
6. B. Ghoraani, K. Umapathy, L. Sugavaneswaran, and S. Krishnan Pathological speech signal analysis using time-frequency approaches Crit. Rev. Biomed. Eng. 40 1 2012 63 95
7. L. Anolli, R. Ciceri, La voce delle emozioni, Franco Angeli, 1992.
8. E.A. Moataz, K. Mohamed, and F. Karray Survey on speech emotion recognition: features, classification schemes, and databases Pattern Recogn. 44 3 2011 572 587
9. P. Ekman, W. Friesen, and P. Ellsworth Emotion in the Human Face: Guidelines for Research and An Integration of Findings 1972 Pergamon Press
10. P. Ekman, and W. Friesen Measuring facial movement Environ. Psychol. Nonver. Behav. 1 1 1976 56 75
11. R. Abelson, and V. Sermat Multidimensional scaling of facial expressions J. Exp. Psychol. 63 6 1962 546
12. N. Cliff, and F. Young On the relation between unidimensional judgments and multidimensional scaling Organ. Behav. Hum. Perform. 3 1968 269 285
13. H. Schlosberg The description of facial expressions in terms of two dimensions J. Exp. Psychol. 44 4 1952 229 237
14. A. Kring, L. Barrett, and D. Gard On the broad applicability of the affective circumplex: representations of affective knowledge among schizophrenia patients Psychol. Sci. 14 3 2003 207 214
15. J. Russell A circumplex model of affect J. Person. Soc. Psychol. 39 6 1980 1161 1178
16. J. Posner, J. Russell, and B. Peterson The circumplex model of affect: an integrative approach to affective neuroscience, cognitive development, and psychopathology Develop. Psychopathol. 17 3 2005 715 734
17. T. Giannakopoulos, A. Pikrakis, S. Theodoridis, A dimensional approach to emotion recognition of speech from movies, in: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP09, 2009, pp. 65-68.
18. C. Lee, and S. Narayanan Toward detecting emotions in spoken dialogs IEEE Trans. Speech Audio Process. 13 2 2005 293 303
19. T. Vogt, E. André, and N. Bee EmoVoice - a framework for online recognition of emotions from voice Perception in Multimodal Dialogue Systems 2008 Springer 188 199
20. M. Wöllmer, F. Eyben, S. Reiter, B. Schuller, C. Cox, E. Douglas-Cowie, R. Cowie, Abandoning emotion classes-towards continuous emotion recognition with modelling of long-range dependencies, in: INTERSPEECH 2008, 2008, pp. 597-600.
21. C. Oflazoglu, and S. Yildirim Recognizing emotion from Turkish speech using acoustic features EURASIP J. Audio, Speech, Music Process. 2013 1 2013 1 11
22. L. Cen, Z. Yu, and M. Dong Speech emotion recognition system based on L1 regularized linear regression and decision fusion Affective Computing and Intelligent Interaction 2011 Springer 332 340
23. J. Yan, X. Wang, W. Gu, and L. MA Speech emotion recognition based on sparse representation Arch. Acoust. 38 4 2013 465 470
24. M. Alam, Y. Attabi, P. Dumouchel, P. Kenny, D. O'Shaughnessy, Amplitude modulation features for emotion recognition from speech 14th Annual Conference of the International Speech Communication Association (INTERSPEECH 2013), 2013.
25. M. Sheikhan, M. Bejani, and D. Gharavian Modular neural-SVM scheme for speech emotion recognition using ANOVA feature selection method Neural Comput. Appl. 2012 1 13
26. D. Gharavian, M. Sheikhan, A. Nazerieh, and S. Garoucy Speech emotion recognition using FCBF feature selection method and GA-optimized fuzzy ARTMAP neural network Neural Comput. Appl. 21 8 2012 2115 2126
27. B. Schuller, S. Steidl, A. Batliner, A. Vinciarelli, K. Scherer, F. Ringeval, M. Chetouani, F. Weninger, F. Eyben, E. Marchi, et al., The INTERSPEECH 2013 computational paralinguistics challenge: social signals, conflict, emotion, autism, in: Proceedings INTERSPEECH 2013, 14th Annual Conference of the International Speech Communication Association, 2013.
28. F. Weninger, J. Geiger, M. Wöllmer, B. Schuller, and G. Rigoll Feature enhancement by deep LSTM networks for ASR in reverberant multisource environments Comput. Speech Lang. 2014 10.1016/j.csl.2014.01.001
29. A. Origlia, F. Cutugno, and V. Galatà Continuous emotion recognition with phonetic syllables Speech Commun. 57 2014 155 169
30. F. Murphy, I. Nimmo-Smith, and A. Lawrence Functional neuroanatomy of emotions: a meta-analysis Cogn., Affect., Behav. Neurosci. 3 3 2003 207 233
31. L. Anolli, L. Wang, F. Mantovani, and A. De Toni The voice of emotion in Chinese and Italian young adults J. Cross-Cult. Psychol. 39 5 2008 565 598
32. H. Gunes, B. Schuller, M. Pantic, R. Cowie, Emotion representation, analysis and synthesis in continuous space: a survey, in: 2011 IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), IEEE, 2011, pp. 827-834.
33. F. Ringeval, A. Sonderegger, J. Sauer, D. Lalanne, Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions, in: EmoSPACE, Proc. of IEEE Face & Gestures, Shanghai, China, 2013.
34. C. Giovannella, D. Conflitti, R. Santoboni, A. Paoloni, Transmission of vocal emotion: do we have to care about the listener? The case of the Italian speech corpus EMOVO, 2009, pp. 494-499.
35. K. Scherer What are emotions? and how can they be measured? Soc. Sci. Inform. 44 4 2005 695 729
36. X. Huang, A. Acero, and H.-W. Hon Spoken Language Processing: A guide to Theory, Algorithm, and System Development 2001 Prentice Hall
37. R. Rangayyan Biomedical Signal Analysis 2002 IEEE press New York
38. P. Boersma Praat, a system for doing phonetics by computer Glot Int. 5 9/10 2002 341 345
39. C. De Boor A Practical Guide to Splines vol. 27 1978 Springer-Verlag New York
40. H. Teager, and S. Teager Evidence for nonlinear production mechanisms in the vocal tract Speech Prod. Speech Model. Nato Adv. Inst. 55 1990 241 261
41. I. Daubechies Ten Lectures on Wavelets 1992 Society for Industrial and Applied Mathematics Philadelphia, PA, USA
42. R. Coifman, and D. Donoho Translation-Invariant De-noising 1995 Springer
43. D. Reynolds, and R. Rose Robust text-independent speaker identification using Gaussian mixture speaker models IEEE Trans. Speech Audio Process. 3 1 1995 72 83
44. M. Abramowitz, and I. Stegun Handbook of Mathematical Functions: With Formulas, Graphs, and Mathematical Tables vol. 55 1964 Dover Publications. Com
45. G. Costantini, M. Todisco, R. Perfetti, A. Paoloni, G. Saggio, Single-sided objective speech intelligibility assessment based on sparse signal representation, in: IEEE International Workshop on Machine Learning for Signal Processing, MLSP12, 2012.
46. H. Dudley Remaking speech J. Acoust. Soc. Am. 11 1939 169
47. K. Payton, and L. Braida A method to determine the speech transmission index from speech waveforms J. Acoust. Soc. Am. 106 1999 3637
48. L. Atlas, and S. Shamma Joint acoustic and modulation frequency EURASIP J. Appl. Signal Process. 2003 2003 668 675
49. S. Schimmel, L. Atlas, Coherent envelope detection for modulation filtering of speech, in: Proc. of ICASSP, vol. 1, 2005, pp. 221-224.
50. S. Schimmel, L. Atlas, K. Nie, Feasibility of single channel speaker separation based on modulation frequency analysis, in: IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP07, vol. 4, 2007, pp. IV-605.
51. P. Clark, and L. Atlas Time-frequency coherent modulation filtering of nonstationary signals IEEE Trans. Signal Process. 57 11 2009 4323 4332
52. J. Brown Calculation of a constant Q spectral transform J. Acoust. Soc. Am. 89 1991 425
53. G. Costantini, A. Paoloni, M. Todisco, Objective speech intelligibility measures based on speech transmission index for forensic applications, in: Audio Engineering Society Conference: 39th International Conference: Audio Forensics: Practices and Challenges, Audio Engineering Society, 2010.
54. A. Oppenheim, R. Schafer, and J. Buck et al. Discrete-Time Signal Processing vol. 5 1999 Prentice Hall Upper Saddle River
55. B. Efron The Jackknife, The Bootstrap and Other Resampling Plans vol. 38 1982 SIAM
56. M. Tenenhaus, V. Vinzi, Y.-M. Chatelin, and C. Lauro PLS path modeling Comput. Stat. Data Anal. 48 1 2005 159 205
57. I. Murray, and J. Arnott Toward the simulation of emotion in synthetic speech: a review of the literature on human vocal emotion J. Acoust. Soc. Am. 93 2 1993 1097 1108
58. A.-M. Öster, A. Risberg, The Identification of the Mood of a Speaker by Hearing Impaired Listeners, Quarterly Progress and Status Report, Dept. for Speech, Music and Hearing, 1986.
59. R. Duda, P. Hart, and D. Stork Pattern Classification 2012 John Wiley & Sons
60. R. Kohavi, and G. John Wrappers for feature subset selection Artif. Intell. 97 1-2 1997 273 324
61. K. Kira, and L. Rendell A practical approach to feature selection Proceedings of the Ninth International Workshop on Machine Learning 1992 Morgan Kaufmann Publishers Inc. 249 256
62. H. Peng, F. Long, and C. Ding Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy IEEE Trans. Pattern Anal. Mach. Intell. 27 8 2005 1226 1238
63. S. Wold, M. Sjöström, and L. Eriksson PLS-regression: a basic tool of chemometrics Chemometr. Intell. Lab. Syst. 58 2 2001 109 130
64. V. Vinzi, L. Trinchera, and S. Amato PLS path modeling: from foundations to recent developments and open issues for model assessment and improvement Handbook of Partial Least Squares 2010 Springer 47 82
65. X.-S. Gan, J.-S. Duanmu, J.-F. Wang, and W. Cong Anomaly intrusion detection based on PLS feature extraction and core vector machine Knowl.-Based Syst. 40 2013 1 6
66. S.-S. Chen, Y.-W. Chuang, and P.-Y. Chen Behavioral intention formation in knowledge sharing: examining the roles of KMS quality, KMS self-efficacy, and organizational climate Knowl.-Based Syst. 31 2012 106 118
67. F. Lindgren, and P.G.S. Wold The kernel algorithm for PLS J. Chemometr. 7 1 1993 45 59
68. S. de Jong SIMPLS: an alternative approach to partial least squares regression Chemometr. Intell. Lab. Syst. 18 3 1993 251 263
69. J. Eng Receiver operating characteristic analysis: a primer Acad. Radiol. 12 2005 909 916
70. D. Belsley, E. Kuh, and R. Welsh Regression Diagnostics 1980 John Wiley & Sons
71. A. Sen, and M. Srivastava Regression Analysis: Theory, Methods, and Applications 1990 Springer
72. I. Olliffe Note on the use of principal components in regression Appl. Statist. 31 3 1982 300 303
73. V. Vapnik The Nature of Statistical Learning Theory 2000 Springer
74. M. Robnik-Šikonja, I. Kononenko, An adaptation of relief for attribute estimation in regression, in: Machine Learning: Proceedings of the Fourteenth International Conference (ICML'97), 1997, pp. 296-304.
75. D. Mattera, F. Palmieri, and S. Haykin An explicit algorithm for training support vector machines IEEE Signal Process. Lett. 6 9 1999 243 245
76. G. Humphreys, N. Donnelly, and M. Riddoch Expression is computed separately from facial identity, and it is computed separately for moving and static faces: neuropsychological evidence Neuropsychologia 31 2 1993 173 181
77. J. Kurucz, and G. Feldmar Prosopo-affective agnosia as a symptom of cerebral organic disease J. Am. Geriatr. Soc. 27 5 1979 225 230
78. A. Young, F. Newcombe, E. de Haan, and D. Hay et al. Face perception after brain injury Selective impairments affecting identity and expression Brain 116 4 1993 941 959