On acoustic emotion recognition: Compensating for covariate shift

IEEE Transactions on Audio, Speech and Language Processing

Volumn 21, Issue 7, 2013, Pages 1458-1468

  Hassan, Ali ^a   Damper, Robert ^a   Niranjan, Mahesan ^a  

^a UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

Author keywords

covariate shift; Emotion recognition; speaker and environment differences; transfer learning

Indexed keywords

ACOUSTIC EMOTION RECOGNITION; COVARIATE SHIFTS; EMOTION RECOGNITION; MAXIMUM LIKELIHOOD LINEAR REGRESSION; SPEAKER AND ENVIRONMENT DIFFERENCES; SUPPORT VECTOR MACHINE CLASSIFIERS; TRANSFER LEARNING; VOCAL TRACT LENGTH NORMALIZATION;

ACOUSTICS; ELECTRICAL ENGINEERING;

SPEECH RECOGNITION;

EID: 84876262647     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2013.2255278     Document Type: Article

References (39)

1. B. Schuller, A. Batliner, S. Steidl, and D. Seppi, "Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge," Speech Commun., vol. 53, no. 9-10, 2011.
2. M. El Ayadi, M. S. Kamel, and F. Karray, "Survey on speech emotion recognition: Features, classification schemes, and databases," Pattern Recogn., vol. 44, no. 3, pp. 572-587, 2011.
3. A. Hassan andR. I.Damper, "Classification of emotional speech using 3DEC hierarchical classifier," Speech Commun., vol. 54, no. 7, pp. 903-916, 2012.
4. M. Shami andW. Verhelst, "An evaluation of the robustness of existing supervised machine learning approaches to the classification of emotions in speech," Speech Commun., vol. 49, no. 3, pp. 201-212, 2007. (Pubitemid 46413361)
5. F. Eyben, A. Batliner, B. Schuller, D. Seppi, and S. Steidl, "Crosscorpus classification of realistic emotions: Some pilot experiments," in Proc. 7th Int. Conf. Lang. Resources Eval. (LREC), Valletta, Malta, 2010, pp. 77-82.
6. B. Schuller,B.Vlasenko, F. Eyben,M.Wöllmer,A. Stuhlsatz,A.Wendemuth, and G. Rigoll, "Cross-corpus acoustic emotion recognition: Variances and strategies," IEEE Trans. Affective Comput., vol. 1, no. 2, pp. 119-131, Jul.-Dec. 2010.
7. I. Lefter, L. Rothkrantz, P. Wiggers, and D. van Leeuwen, "Emotion recognition from speech by combining databases and fusion of classifiers," in Proc. 13th Int. Conf. Text, Speech, Dialogue, Berlin, Heidelberg, Germany, 2010, pp. 353-360, Springer-Verlag.
8. B. Schuller, Z. Zhang, F. Weninger, and G. Rigoll, "Using multiple databases for training in emotion recognition: To unite or to vote?," in Proc. 12th Annu. Conf. Int. Speech Commun. Assoc. (Interspeech'11), Florence, Italy, 2011, pp. 1553-1556.
9. S. Steidl, "Automatic classification of emotion-related user states in spontaneous children's speech," Ph.D. dissertation, Dept. of Comput. Sci., Univ. of Erlangen-Nuremberg, Erlangen, Germany, 2009.
10. I. R. Murray and J. L. Arnott, "Toward the simulation of emotion in synthetic speech: A review of the literature on human vocal emotion," J. Acoust. Soc. Amer., vol. 93, no. 2, pp. 1097-1108, 1993. (Pubitemid 23059837)
11. T. Vogt and E. André, "Comparing feature sets for acted and spontaneous speech in view of automatic emotion recognition," in Proc. IEEE Int. Conf. Multimedia and Expo, ICME'05, Amsterdam, The Netherlands, 2005, pp. 474-477. (Pubitemid 44668907)
12. J. Wilting, E. Krahmer, and M. Swerts, "Real vs. acted emotional speech," in Proc. Int. Conf. Spoken Lang. Process. (Interspeech'06), Pittsburgh, PA, USA, 2006, pp. 1093-1097.
13. S. Shahid, E. Krahmer, and M. Swerts, "Real vs. acted emotional speech: Comparing Caucasian and South Asian speakers and observers," in Proc. 4th Int. Conf. Speech Prosody, Campinas, Brazil, 2008, pp. 669-672.
14. B. Schuller, S. Steidl, and A. Batliner, "The Interspeech 2009 emotion challenge," in Proc. 10th Annu. Conf. Int. Speech Commun. Assoc. (Interspeech' 09), Brighton, U.K., 2009, pp. 312-315.
15. C. Lee, E. Mower, C. Busso, S. Lee, and S. Narayanan, "Emotion recognition using a hierarchical binary decision tree approach," in Proc. 10th Annu. Conf. Int. Speech Commun. Assoc. (Interspeech'09), Brighton, U.K., 2009, pp. 320-323.
16. C. Lee, E. Mower, C. Busso, S. Lee, and S. Narayanan, "Emotion recognition using a hierarchical binary decision tree approach," Speech Commun., vol. 53, no. 9-10, pp. 1162-1171, 2011.
17. S. Planet, I. Iriondo, J. Socoró, C. Monzo, and J. Adell, "GTM-URL contribution to the interspeech 2009 emotion challenge," in Proc. 10th Annu. Conf. Int. Speech Commun. Assoc. (Interspeech'09), Brighton, U.K., 2009, pp. 316-319.
18. I. Luengo, E. Navas, and I.Hernáez, "Combining spectral and prosodic information for emotion recognition in the interspeech 2009 emotion challenge," in Proc. 10th Annu. Conf. Int. Speech Commun. Assoc. (Interspeech' 09), Brighton, U.K., 2009, pp. 332-335.
19. I. Engberg and A. Hansen, "Documentation of the Danish emotional speech database DES,". Aalborg, Denmark, Alborg Univ., 1996, Center for PersonKommunikation, Inst. of Electron. Syst.
20. F. Burkhardt, A. Paeschke, M. Rolfes, W. Sendlmeier, and B. Weiss, "A database of German emotional speech," in Proc. 9th Eur. Conf. Speech Commun. Technol. (Interspeech'09), Lisbon, Portugal, 2005, pp. 1517-1520. (Pubitemid 43908362)
21. M. Slaney and G.McRoberts, "BabyEars: A recognition system for affective vocalizations," Speech Commun., vol. 39, no. 3-4, pp. 367-384, 2003. (Pubitemid 35432920)
22. C. Breazeal and L. Aryananda, "Recognition of affective communicative intent in robot-directed speech," Autonomous Robots, vol. 12, no. 1, pp. 83-104, 2002. (Pubitemid 34156624)
23. SmartKom: Foundations of Multimodal Dialogue Systems, W. Wahlster, Ed. Berlin, Heidelberg, Germany: Springer, 2006.
24. M. Grimm, K. Kroschel, and S. Narayan, "The Vera amMittag German audio-visual emotional speech database," in Proc. IEEE Int. Conf. Multimedia and Expo, Hannover, Germany, 2008, pp. 865-868.
25. Dataset Shift in Mach. Learn., J. Quioñero-Candela, M. Sugiyama, A. Schwaighofer, and N. Lawrence, Eds. Cambridge, MA, USA: MIT Press, 2009.
26. B. Farran, C. Saunders, and M. Niranjan, "Machine learning for intrusion detection: Modeling the distribution shift," in Proc. IEEE Int. Workshop Mach. Learn. for Signal Process. (MLSP), Kittilä, Finland, 2010, pp. 232-237.
27. R. Bellman, Dynamic Programming. Princeton, NJ, USA: Princeton Univ. Press, 1957.
28. A. Gretton, A. Smola, J. Huang, M. Schmittfull, K. Borgwardt, and B. Schölkopf, , J. Quioñero-Candela,M. Sugiyama, A. Schwaighofer, and N. Lawrence, Eds., "Covariate shift and local learning by distribution matching," in Dataset Shift in Machine Learning. Cambridge, MA, USA: MIT Press, 2009, ch. 8, pp. 131-160.
29. T. Kanamori, S. Hido, and M. Sugiyama, "A least-squares approach to direct importance estimation," J. Mach. Learn. Res., vol. 10, pp. 1391-1445, 2009.
30. Y. Tsuboi, H. Kashima, S. Hido, S. Bickel, and M. Sugiyama, "Direct density ratio estimation for large-scale covariate shift adaptation," in Proc. SIAM Int. Conf. Data Mining, Atlanta, Georgia, USA, 2008, pp. 443-454.
31. M. Sugiyama, B.Blankertz,M.Krauledat,G.Dornhege, and K. Muller, "Importance-weighted cross-validation for covariate shift," in Proc. 28th Annu. Symp. German Assoc. for Pattern Recognit., Berlin, Germany, 2006, pp. 354-363. (Pubitemid 44608881)
32. M. Pitz and H. Ney, "Vocal tract normalization equals linear transformation in cepstral space," IEEE Trans. Speech Audio Process., vol. 13, no. 5, pp. 930-944, Sep. 2005. (Pubitemid 41558907)
33. A. Kabir, J. Barker, and M. Giurgiu, "An approach to vocal tract length normalization by robust formant[sic]," in Proc. Int. Conf. Circuits, Syst., Signals (CSS), Malta, 2010, pp. 345-348.
34. W. M. Campbell, D. E. Sturim, and D. A. Reynolds, "Support vector machines using GMM supervectors for speaker verification," IEEE Signal Process. Lett., vol. 13, no. 5, pp. 308-311, May 2006.
35. C. J. Leggetter and P. C. Woodland, "Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov models," Comput. Speech Lang., vol. 9, no. 2, pp. 171-185, 1995.
36. M. Gales and P. Woodland, "Mean and variance adaptation within the MLLR framework," Comput. Speech Lang., vol. 10, no. 4, pp. 249-264, 1996. (Pubitemid 126374488)
37. F. Eyben, M. Wöllmer, and B. Schuller, "OpenEAR-Introducing the Munich open-source emotion and affect recognition toolkit," in Proc. Int. Conf. Affective Comput. Intell. Interact. (ACII '09), Amsterdam, The Netherlands, 2009, pp. 1-6.
38. P. Dumouchel,N.Dehak, Y. Attabi, R.Dehak, and N. Boufaden, "Cepstral and long-term features for emotion recognition," in Proc. 10th Annu. Conf. Int. Speech Commun. Assoc. (Interspeech'09), Brighton, U.K., 2009, pp. 344-347.
39. M. Kockman, L. Burget, and J. ?ernocký, "Brno university of technology system for interspeech 2009 emotion challenge," in Proc. 10th Annu. Conf. Int. Speech Commun. Assoc. (Interspeech'09), Brighton, UK, 2009, pp. 348-351.