Iterative feature normalization scheme for automatic emotion detection from speech

IEEE Transactions on Affective Computing

Volumn 4, Issue 4, 2013, Pages 386-397

  Busso, Carlos ^a   Mariooryad, Soroosh ^a   Metallinou, Angeliki ^b   Narayanan, Shrikanth ^c  

^a The University of Texas at Dallas   (United States)

^b PEARSON   (United States)

^c Viterbi School of Engineering University of Southern California ^*  (United States)

Author keywords

emotion; Emotion recognition; features normalization; speaker normalization

Indexed keywords

COMPUTER PROGRAMMING; COMPUTER SCIENCE;

AFFINE TRANSFORMATIONS; EMOTION; EMOTION RECOGNITION; FEATURE NORMALIZATION; FEATURES NORMALIZATION; GLOBAL NORMALIZATION; NATURAL APPROACHES; SPEAKER NORMALIZATION;

SPEECH RECOGNITION;

EID: 84897096415     PISSN: 19493045     EISSN: None     Source Type: Journal    
DOI: 10.1109/T-AFFC.2013.26     Document Type: Article

References (47)

1. O. Kustner, R. Tato, T. Kemp, and B. Meffert,"Towards Real Life Applications in Emotion Recognition," Affective Dialogue Systems (ADS 05), E. Andre, L. Dybkaer, W. Minker, P. Heisterkamp, eds., pp. 25-35, Springer Verlag, May 2004
2. R. Cowie and R. Cornelius,"Describing the Emotional States That Are Expressed in Speech," Speech Comm., vol. 40, no. 1-2, pp. 5-32, April 2003
3. C. Lee and S. Narayanan,"Toward Detecting Emotions in Spoken Dialogs," IEEE Trans. Speech and Audio Processing, vol. 13, no. 2, pp. 293-303, Mar. 2005 (Pubitemid 40320247)
4. C. Clavel, I. Vasilescu, L. Devillers, G. Richard, and T. Ehrette,"Fear-Type Emotion Recognition for Future Audio-Based Surveillance Systems," Speech Comm., vol. 50, no. 6, pp. 487-503, June 2008
5. A. Batliner, A. Buckow, H. Niemann, E. Noth, and V. Warnke,"The Prosody Module," VERBMOBIL: Foundations of Speech-to-Speech Translations, M. Maybury, O. Stock, W. Wahlster, eds., pp. 106-121, Springer Verlag, 2000
6. C. Busso, S. Lee, and S. Narayanan,"Analysis of Emotionally Salient Aspects of Fundamental Frequency for Emotion Detection," IEEE Trans. Audio, Speech and Language Processing, vol. 17, no. 4, pp. 582-596, May 2009
7. C.-C. Lee, E. Mower, C. Busso, S. Lee, and S. Narayanan,"Emotion Recognition Using a Hierarchical Binary Decision Tree Approach," Proc. Interspeech, pp. 320-323, Sept. 2009
8. C. Busso, A. Metallinou, and S. Narayanan,"Iterative Feature Normalization for Emotional Speech Detection," Proc. Intl Conf. Acoustics, Speech, and Signal Processing (ICASSP 11), pp. 5692-5695, May 2011
9. T. Rahman and C. Busso,"A Personalized Emotion Recognition System Using an Unsupervised Feature Adaptation Scheme," Proc. Intl Conf. Acoustics, Speech, and Signal Processing (ICASSP 12), pp. 5117-5120, Mar. 2012
10. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J. Taylor,"Emotion Recognition in Human-Computer Interaction," IEEE Signal Processing Magazine, vol. 18, no. 1, pp. 32-80, Jan. 2001 (Pubitemid 32287669)
11. B. Schuller, A. Batliner, D. Seppi, S. Steidl, T. Vogt, J. Wagner, L. Devillers, L. Vidrascu, N. Amir, L. Kessous, and V. Aharonson,"The Relevance of Feature Type for the Automatic Classification of Emotional User States: Low Level Descriptors and Functionals," Proc. Interspeech, pp. 2253-2256, Aug. 2007
12. J. Deller, J. Hansen, and J. Proakis, Discrete-Time Processing of Speech Signals. IEEE Press, 2000
13. S. Yildirim, M. Bulut, C. Lee, A. Kazemzadeh, C. Busso, Z. Deng, S. Lee, and S. Narayanan,"An Acoustic Study of Emotions Expressed in Speech," Proc. Eighth Intl Conf. Spoken Language Processing (ICSLP 04), pp. 2193-2196, Oct. 2003
14. B. Schuller, G. Rigoll, and M. Lang,"Hidden Markov Model-Based Speech Emotion Recognition," Proc. Intl Conf. Acoustics, Speech, and Signal Processing (ICASSP 03), vol. 2, pp. 1-4, Apr. 2003
15. C.-C. Lee, E. Mower, C. Busso, S. Lee, and S. Narayanan,"Emotion Recognition Using a Hierarchical Binary Decision Tree Approach," Speech Comm., vol. 53, no. 9-10, pp. 1162-1171, Nov.-Dec. 2011
16. M. Wollmer, F. Eyben, S. Reiter, B. Schuller, C. Cox, E. Douglas-Cowie, and R. Cowie,"Abandoning Emotion Classes-Towards Continuous Emotion Recognition with Modelling of Long-Range Dependencies," Proc. Interspeech, pp. 597-600, September 2008
17. A. Metallinou, A. Katsamanis, and S. Narayanan,"A Hierarchical Framework for Modeling Multimodality and Emotional Evolution in Affective Dialogs," Proc. Intl Conf. Acoustics, Speech, and Signal Processing (ICASSP 12), pp. 2401-2404, Mar. 2012
18. T.-L. Pao, J.-H. Yeh, Y.-T. Chen, Y.-M. Cheng, and Y.-Y. Lin,"A Comparative Study of Different Weighting Schemes on KNNBased Emotion Recognition in Mandarin Speech," Advanced Intelligent Computing Theories and Applications. With Aspects of Theoretical and Methodological Issues, D.-S. Huang, L. Heutte, M. Loog, eds., pp. 997-1005, Springer Verlag, July 2007
19. T.-L. Pao, C. Chien, Y.-T. Chen, J.-H. Yeh, Y.-M. Cheng, and W.-Y. Liao,"Combination of Multiple Classifiers for Improving Emotion Recognition in Mandarin Speech," Proc. Third Intl Conf. Intelligent Information Hiding and Multimedia Signal Processing (IIHMSP 07), vol. 1, pp. 35-38, Nov. 2007
20. Z. Yan, Z. Li, Z. Cairong, and Y. Yinhua,"Speech Emotion Recognition Using Modified Quadratic Discrimination Function," J. Electronics, vol. 25, no. 6, pp. 840-844, Nov. 2008
21. B. Vlasenko, B. Schuller, A. Wendemuth, and G. Rigoll,"Frame vs. Turn-Level: Emotion Recognition from Speech Considering Static and Dynamic Processing," Affective Computing and Intelligent Interaction, A. Paiva, R. Prada, R. Picard, eds., pp. 139-147, Springer, Sept. 2007
22. B. Schuller, B. Vlasenko, R. Minguez, G. Rigoll, and A. Wendemuth,"Comparing One and Two-Stage Acoustic Modeling in the Recognition of Emotion in Speech," Proc. IEEE Workshop Automatic Speech Recognition & Understanding (ASRU 07), pp. 596-600, Dec. 2007
23. D. Bitouk, R. Verma, and A. Nenkova,"Class-Level Spectral Features for Emotion Recognition," Speech Comm., vol. 52, pp. 613-625, July-Aug. 2010
24. B. Schuller, B. Vlasenko, F. Eyben, M. Wollmer, A. Stuhlsatz, A. Wendemuth, and G. Rigoll,"Cross-Corpus Acoustic Emotion Recognition: Variances and Strategies," IEEE Trans. Affective Computing, vol. 1, no. 2, pp. 119-131, July-Dec. 2010
25. X. Le, G. Quenot, and E. Castelli,"Recognizing Emotions for the Audio-Visual Document Indexing," Proc. Ninth Intl Symp. Computers and Comm. (ISCC 04), vol. 2, pp. 580-584, June-July 2004 (Pubitemid 40006232)
26. O. W. Kwon, K. Chan, J. Hao, and T. W. Lee,"Emotion Recognition by Speech Signals," Proc. Eighth European Conf. Speech Comm. and Technology (EUROSPEECH 2003), pp. 125-128, Sept. 2003
27. T. Zhang, M. Hasegawa-Johnson, and S. Levinson,"Mental State Detection of Dialogue System Users via Spoken Language," Proc. ISCA & IEEE Workshop on Spontaneous Speech Processing and Recognition (SSPR 03), Apr. 2003
28. V. Sethu, E. Ambikairajah, and J. Epps,"Speaker Normalisation for Speech Based Emotion Detection," Proc. 15th Intl Conf. Digital Signal Processing (DSP 07), pp. 611-614, July 2007
29. S. Mariooryad and C. Busso,"Compensating for Speaker or Lexical Variabilities in Speech for Emotion Recognition," In Press, Speech Comm., vol. 57, pp. 1-12, 2014
30. Z. Zeng, J. Tu, B. Pianfetti, M. Liu, T. Zhang, Z. Zhang, T. Huang, and S. Levinson,"Audio-Visual Affect Recognition through Multi-Stream Fused HMM for HCI," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR 05), vol. 2, pp. 967-972, June 2005
31. L. Fu, X. Mao, and L. Chen,"Relative Speech Emotion Recognition Based Artificial Neural Network," Proc. Pacific-Asia Workshop on Computational Intelligence and Industrial Application (PACIIA 08), vol. 2, pp. 140-144, Dec. 2008
32. A. Blum and T. Mitchell,"Combining Labeled and Unlabeled Data with Co-Training," Proc. 11th Ann. Conf. Computational Learning Theory (COLT 98), pp. 92-100, July 1998
33. C. Busso, P. Georgiou, and S. Narayanan,"Real-Time Monitoring of Participants Interaction in a Meeting Using Audio-Visual Sensors," Proc. Intl Conf. Acoustics, Speech, and Signal Processing (ICASSP 07), vol. 2, pp. 685-688, Apr. 2007
34. C. Busso, Z. Deng, S. Yildirim, M. Bulut, C. Lee, A. Kazemzadeh, S. Lee, U. Neumann, and S. Narayanan,"Analysis of Emotion Recognition Using Facial Expressions, Speech and Multimodal Information," Proc. Sixth Intl Conf. Multimodal Interfaces (ICMI 04), pp. 205-211, Oct. 2004 (Pubitemid 40371911)
35. T. Rahman, S. Mariooryad, S. Keshavamurthy, G. Liu, J. Hansen, and C. Busso,"Detecting Sleepiness by Fusing Classifiers Trained with Novel Acoustic Features," Proc. 12th Ann. Conf. Intl Speech Comm. Association (Interspeech 11), pp. 3285-3288, Aug. 2011
36. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. Witten,"The WEKA Data Mining Software: An Update," ACM SIGKDD Explorations Newsletter, vol. 11, no. 1, pp. 10-18, June 2009
37. C. Busso, M. Bulut, C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. Chang, S. Lee, and S. Narayanan,"IEMOCAP: Interactive Emotional Dyadic Motion Capture Database," J. Language Resources and Evaluation, vol. 42, no. 4, pp. 335-359, Dec. 2008
38. C. Busso and S. Narayanan,"The Expression and Perception of Emotions: Comparing Assessments of Self versus Others," Proc. Interspeech, pp. 257-260, Sept. 2008
39. B. Schuller, S. Steidl, A. Batliner, F. Schiel, and J. Krajewski,"The INTERSPEECH 2011 Speaker State Challenge," Proc. 12th Ann. Conf. Intl Speech Comm. Assoc. (Interspeech 11), pp. 3201-3204, Aug. 2011
40. F. Eyben, M.Wollmer, and B. Schuller,"OpenSMILE: The Munich Versatile and Fast Open-Source Audio Feature Extractor," Proc. ACM Intl Conf. Multimedia (MM 10), pp. 1459-1462, Oct. 2010
41. M.A. Hall,"Correlation Based Feature-Selection for Machine Learning," PhD dissertation, The Univ. of Waikato, Apr. 1999
42. D. Bone, M.P. Black, M. Li, A. Metallinou, S. Lee, and S. Narayanan,"Intoxicated Speech Detection by Fusion of Speaker Normalized Hierarchical Features and GMM Supervectors," Proc. 12th Ann. Conf. Intl Speech Comm. Assoc. (Interspeech 11), pp. 3217-3220, Aug. 2011
43. D. Bone, M. Li, M. Black, and S. Narayanan,"Intoxicated Speech Detection: A Fusion Framework with Speaker-Normalized Hierarchical Functionals and GMM Supervectors," Computer, Speech, and Language, Oct. 2012
44. R. Cowie, E. Douglas-Cowie, S. Savvidou, E. McMahon, M. Sawey, and M. Schroder,"FEELTRACE: An Instrument for Recording Perceived Emotion in Real Time," Proc. ISCA Tutorial and Research Workshop (ITRW 00) on Speech and Emotion, pp. 19-24, Sept. 2000
45. M. Grimm, K. Kroschel, E. Mower, and S. Narayanan,"Primitives-Based Evaluation and Estimation of Emotions in Speech," Speech Comm., vol. 49, no. 10-11, pp. 787-800, Oct.-Nov. 2007 (Pubitemid 47268568)
46. B. Schuller, A. Batliner, S. Steidl, and D. Seppi,"Recognising Realistic Emotions and Affect in Speech: State of the Art and Lessons Learnt from the First Challenge," Speech Comm., vol. 53, no. 1, pp. 1062-1087, Dec. 2011
47. J. Arias, C. Busso, and N. Yoma,"Shape-Based Modeling of the Fundamental Frequency Contour for Emotion Detection in Speech," Computer Speech and Language, vol. 28, pp. 278-294, 2014.