Compensating for speaker or lexical variabilities in speech for emotion recognition

Speech Communication

Volumn 57, Issue , 2014, Pages 1-12

  Mariooryad, Soroosh ^a   Busso, Carlos ^a  

^a The University of Texas at Dallas   (United States)

Author keywords

Emotion recognition; Factor analysis; Feature normalization; Speaker variability

Indexed keywords

EMOTION RECOGNITION; FACTORIZATION TECHNIQUES; FEATURE NORMALIZATION; HUMAN MACHINE INTERFACE; SPEAKER CHARACTERISTICS; SPEAKER VARIABILITY; SPEECH EMOTION RECOGNITION SYSTEMS; WHITENING TRANSFORMATION;

COMMUNICATION; FACTOR ANALYSIS; MULTIVARIANT ANALYSIS;

UNCERTAINTY ANALYSIS;

EID: 84884611357     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2013.07.011     Document Type: Article

References (52)

1. A. Batliner, D. Seppi, S. Steidl, and B. Schuller Segmenting into adequate units for automatic recognition of emotion-related episodes: a speech-based approach Advances in Human-Computer Interaction 2010 2010 1 15
2. C. Busso, M. Bulut, C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. Chang, S. Lee, and S. Narayanan IEMOCAP: interactive emotional dyadic motion capture database Journal of Language Resources and Evaluation 42 2008 335 359
3. Busso, C., Lee, S., Narayanan, S., 2007. Using neutral speech models for emotional speech analysis. In: Interspeech 2007 - Eurospeech, Antwerp, Belgium, pp. 2225-2228.
4. C. Busso, S. Lee, and S. Narayanan Analysis of emotionally salient aspects of fundamental frequency for emotion detection IEEE Transactions on Audio, Speech and Language Processing 17 2009 582 596
5. Busso, C., Metallinou, A., Narayanan, S., 2011. Iterative feature normalization for emotional speech detection. In: International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2011), Prague, Czech Republic, pp. 5692-5695.
6. Busso, C., Narayanan, S., 2006. Interplay between linguistic and affective goals in facial expression during emotional utterances. In: Seventh International Seminar on Speech Production (ISSP 2006), Ubatuba-SP, Brazil, pp. 549-556.
7. Busso, C., Narayanan, S., 2007. Joint analysis of the emotional fingerprint in the face and speech: a single subject study. In: International Workshop on Multimedia Signal Processing (MMSP 2007), Chania, Crete, Greece, pp. 43-47.
8. Chappell, D., Hansen, J., 1998. Speaker-specific pitch contour modeling and modification. In: IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP 1998), Seattle, WA, USA, pp. 885-888.
9. R. Chauhan, J. Yadav, S. Koolagudi, and K. Rao Text independent emotion recognition using spectral features S. Aluru, S. Bandyopadhyay, U. Catalyurek, D. Dubhashi, P. Jones, M. Parashar, B. Schmidt, Contemporary Computing Communications in Computer and Information Science vol. 168 2011 Springer-Verlag Berlin, Heidelberg 359 370
10. T. Cover, and J. Thomas Elements of Information Theory 2006 Wiley-Interscience New York, NY, USA
11. Dehak, N., Dehak, R., Kenny, P., Brümmer, N., Ouellet, P., Dumouchel, P., 2009. Support vector machines versus fast scoring in the low-dimensional total variability space for speaker verification. In: Interspeech 2009, Brighton, UK, pp. 1559-1562.
12. Dehak, N., Torres-Carrasquillo, P., Reynolds, D., Dehak, R., 2011. Language recognition via i-vectors and dimensionality reduction. In: Interspeech 2011, Florence, Italy, pp. 857-860.
13. Eyben, F., Wöllmer, M., Schuller, B., 2010. OpenSMILE: the Munich versatile and fast open-source audio feature extractor. In: ACM International conference on Multimedia (MM 2010), Florence, Italy, pp. 1459-1462.
14. Fu, L., Mao, X., Chen, L., 2008. Relative speech emotion recognition based artificial neural network. In: Pacific-Asia Workshop on Computational Intelligence and Industrial Application (PACIIA 2008), Wuhan, China, pp. 140-144.
15. Gish, H., Ng, K., 1993. A segmental speech model with applications to word spotting. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 1993), Minneapolis, MN, USA, pp. 447-450.
16. Gish, H., Ng, K., 1996. Parametric trajectory models for speech recognition. In: Proceedings of the Fourth International Conference on Spoken Language (ICSLP 1996), Philadelphia, PA, USA, pp. 466-469.
17. Y. Gong Stochastic trajectory modeling and sentence searching for continuous speech recognition IEEE Transactions on Speech and Audio Processing 5 1997 33 44 (Pubitemid 127746033)
18. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. Witten The WEKA data mining software: an update ACM SIGKDD Explorations Newsletter 11 2009 10 18
19. J. Hansen, and B. Womack Feature analysis and neural network-based classification of speech under stress IEEE Transactions on Speech and Audio Processing 4 1996 307 313 (Pubitemid 126753019)
20. R. Hastie, and R.M. Dawes Rational Choice in an Uncertain World: The Psychology of Judgement and Decision Making 2001 Sage Publications, Inc. Thousand Oaks, CA, USA
21. H. Hermansky Perceptual linear predictive (PLP) analysis of speech Journal of the Acoustical Society of America 87 1990 1738 1752 (Pubitemid 20256470)
22. P. Kenny, G. Boulianne, P. Ouellet, and P. Dumouchel Speaker and session variability in GMM-based speaker verification IEEE Transactions on Audio, Speech, and Language Processing 15 2007 1448 1460
23. Lee, C., Yildirim, S., Bulut, M., Kazemzadeh, A., Busso, C., Deng, Z., Lee, S., Narayanan, S., 2004. Emotion recognition based on phoneme classes. In: Eighth International Conference on Spoken Language Processing (ICSLP 04), Jeju Island, Korea, pp. 889-892.
24. Lee, S., Yildirim, S., Kazemzadeh, A., Narayanan, S., 2005. An articulatory study of emotional speech production. In: Ninth European Conference on Speech Communication and Technology (Interspeech'2005 - Eurospeech), Lisbon, Portugal, pp. 497-500. (Pubitemid 43908108)
25. Lei, X., Siu, M., Hwang, M., Ostendorf, M., Lee, T., 2006. Improved tone modeling for mandarin broadcast news speech recognition. In: International Conference on Spoken Language (ICSLP 2006), Pittsburgh, PA, USA, pp. 1237-1240.
26. Li, M., Metallinou, A., Bone, D., Narayanan, S., 2012. Speaker states recognition using latent factor analysis based eigenchannel factor vector modeling. In: International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2012), Kyoto, Japan, pp. 1937-1940.
27. Mariooryad, S., Busso, C., 2012. Factorizing speaker, lexical and emotional variabilities observed in facial expressions. In: IEEE International Conference on Image Processing (ICIP 2012), Orlando, FL, USA, pp. 2605-2608.
28. Mariooryad, S., Busso, C., 2013a. Exploring Cross-Modality Affective Reactions for Audiovisual Emotion Recognition. IEEE Transactions on Affective Computing 4(2), 183-196.
29. Mariooryad, S., Busso, C., 2013b. Feature and model level compensation of lexical content for facial emotion recognition. In: IEEE International Conference on Automatic Face and Gesture Recognition (FG 2013), Shanghai, China.
30. Metallinou, A., Busso, C., Lee, S., Narayanan, S., 2010. Visual emotion recognition using compact facial representations and viseme information. In: International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2010), Dallas, TX, USA, pp. 2474-2477.
31. Metallinou, A., Katsamanis, A., Narayanan, S., 2012. A hierarchical framework for modeling multimodality and emotional evolution in affective dialogs. In: International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2012), Kyoto, Japan, pp. 2401-2404.
32. Metallinou, A., Lee, S., Narayanan, S., 2010. Decision level combination of multiple modalities for recognition and analysis of emotional expression. In: International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2010), Dallas, TX, USA, pp. 2462-2465.
33. Mower, E., Metallinou, A., Lee, C.C., Kazemzadeh, A., Busso, C., Lee, S., Narayanan, S., 2009. Interpreting ambiguous emotional expressions. In: International Conference on Affective Computing and Intelligent Interaction (ACII 2009), Amsterdam, The Netherlands.
34. S. Park, H. Choi, N. Roy, and J. How Learning the covariance dynamics of a large-scale environment for informative path planning of unmanned aerial vehicle sensors International Journal of Aeronautical and Space Sciences 11 2010 327 337
35. S. Prince, J. Elder, J. Warrell, and F. Felisberti Tied factor analysis for face recognition across large pose differences IEEE Transactions on Pattern Analysis and Machine Intelligence 30 2008 970 984 (Pubitemid 351650102)
36. L. Rabiner, and M. Sambur An algorithm for detecting the endpoints of isolated utterances Bell System Technical Journal 54 1975 297 315
37. Rahman, T., Busso, C., 2012. A personalized emotion recognition system using an unsupervised feature adaptation scheme. In: International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2012), Kyoto, Japan, pp. 5117-5120.
38. B. Schuller, A. Batliner, S. Steidl, and D. Seppi Recognising realistic emotions and affect in speech: state of the art and lessons learnt from the first challenge Speech Communication 53 2011 1062 1087
39. Schuller, B., Stadermann, J., Rigoll, G., 2006. Affect-robust speech recognition by dynamic emotional adaptation. In: Third International Conference on Speech Prosody 2006. ISCA, Dresden, Germany.
40. Schuller, B., Steidl, S., Batliner, A., Schiel, F., Krajewski, J., 2011. The INTERSPEECH 2011 speaker state challenge. In: 12th Annual Conference of the International Speech Communication Association (Interspeech 2011), Florence, Italy, pp. 3201-3204.
41. B. Schuller, B. Vlasenko, F. Eyben, M. Wöllmer, A. Stuhlsatz, A. Wendemuth, and G. Rigoll Cross-corpus acoustic emotion recognition: variances and strategies IEEE Transactions on Affective Computing 1 2010 119 131
42. Sethu, V., Ambikairajah, E., Epps, J., 2007. Speaker normalisation for speech based emotion detection. In: 15th International Conference on Digital Signal Processing (DSP 2007), Cardiff, Wales, UK, pp. 611-614.
43. Shriberg, E., Graciarena, M., Bratt, H., Kathol, A., Kajarekar, S., Jameel, H., Richey, C., Goodman, F., 2008. Eects of vocal eort and speaking style on text-independent speaker verification. In: Interspeech 2008, Brisbane, Australia, pp. 609-612.
44. J. Tenenbaum, and W. Freeman Separating style and content with bilinear models Journal of Neural Computation 12 2000 1247 1283
45. Vlasenko, B., Philippou-Hübner, D., Prylipko, D., Böck, R., Siegert, I., Wendemuth, A., 2011. Vowels formants analysis allows straightforward detection of high arousal emotions. In: IEEE International Conference on Multimedia and Expo (ICME 2011), Barcelona, Spain.
46. Vlasenko, B., Prylipko, D., Philippou-Hübner, D., Wendemuth, A., 2011. Vowels formants analysis allows straightforward detection of high arousal acted and spontaneous emotions. In: 12th Annual Conference of the International Speech Communication Association (Interspeech'2011), Florence, Italy, pp. 1577-1580.
47. B. Vlasenko, B. Schuller, A. Wendemuth, and G. Rigoll Frame vs. turn-level: emotion recognition from speech considering static and dynamic processing A. Paiva, R. Prada, R. Picard, Affective Computing and Intelligent Interaction 2007 Springer Berlin/Heidelberg, Berlin, Germany 139 147
48. Wöllmer, M., Eyben, F., Reiter, S., Schuller, B., Cox, C., Douglas-Cowie, E., Cowie, R., 2008. Abandoning emotion classes - towards continuous emotion recognition with modelling of long-range dependencies. In: Interspeech 2008 - Eurospeech, Brisbane, Australia, pp. 597-600.
49. Wu, W., Zheng, T., Xu, M., Bao, H., 2006. Study on speaker verification on emotional speech. In: International Conference on Spoken Language (ICSLP 2006), Pittsburgh, PA, USA, pp. 2102-2105.
50. Xia, R., Liu, Y., 2012. Using i-vector space model for emotion recognition. In: Interspeech 2012, Portland, Oregon, USA, pp. 2230-2233.
51. Yildirim, S., Bulut, M., Lee, C., Kazemzadeh, A., Busso, C., Deng, Z., Lee, S., Narayanan, S., 2004. An acoustic study of emotions expressed in speech. In: Eighth International Conference on Spoken Language Processing (ICSLP 04), Jeju Island, Korea, pp. 2193-2196.
52. Z. Zeng, J. Tu, B. Pianfetti, and T. Huang Audiovisual affective expression recognition through multistream fused HMM IEEE Transactions on Multimedia 10 2008 570 577 (Pubitemid 351711233)