Loss-scaled large-margin Gaussian mixture models for speech emotion classification

IEEE Transactions on Audio, Speech and Language Processing

Volumn 20, Issue 2, 2012, Pages 585-598

  Yun, Sungrack ^a   Yoo, Chang D ^a  

^a Korea Advanced Institute of Science and Technology (KAIST)   (South Korea)

Author keywords

Gaussian mixture models (GMMs); margin scaling; speech emotion classification; Watson and Tellegen's model

Indexed keywords

CLASSIFICATION ACCURACY; CONSTRAINED OPTIMIZATION PROBLEMS; DISCRIMINANT FUNCTIONS; DISTANCE METRICS; EMOTION CLASSIFICATION; EMOTION MODELING; EMOTION MODELS; GAUSSIAN MIXTURE MODEL; GAUSSIAN MIXTURE MODELS; GENERALIZATION ABILITY; LEARNING FRAMEWORKS; LOSS FUNCTIONS; MARGIN SCALING; MAXIMUM MUTUAL INFORMATION; PARAMETER SET; SEMI-DEFINITE PROGRAMMING; SPEECH EMOTION CLASSIFICATION; TELLEGEN; TESTING DATA; TRAINING DATA SETS;

ABILITY TESTING; CONSTRAINED OPTIMIZATION; EXPERIMENTS; GAUSSIAN DISTRIBUTION; MAXIMUM LIKELIHOOD; RISK PERCEPTION; STATISTICAL TESTS; STEREOPHONIC BROADCASTING;

CLASSIFICATION (OF INFORMATION);

EID: 83655164697     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2011.2162405     Document Type: Article

References (55)

1. M. Lew, E. M. Bakker, N. Sebe, and T. S. Huang, "Human-computer intelligent interaction:A survey" Lecture Note Comput. Sci., vol. 4796, pp. 1-5, 2007.
2. R. W. Picard, Affective Computing. Cambridge, MA: MIT Press, 1997.
3. J. Tao and T. Tan, "Affective computing: A review" Lecture Note Comput. Sci., vol. 3784, pp. 981-995, 2005. (Pubitemid 43763427)
4. I. Cohen, A. Garg, and T. S. Huang, "Emotion recognition from facial expressions using multilevelHMM" in Proc. Neural Inf. Process. Syst., Workshop Affective Comput., 2000.
5. S. Casale, A. Russo, and G. Scebba, "Speech emotion classification using machine learning algorithms" in Proc. IEEE Int. Conf. Semantic Comput., 2008, pp. 158-165.
6. M. Song, J. Bu, C. Chen, and N. Li, "Audio-visual based emotion recognition-A new approach" in Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recognit., 2004, vol. 2, pp. 1020-1025.
7. H. Gunes and M. Piccardi, "Bi-modal emotion recognition from expressive face and body gestures" J. Netw. Comput. Applicat., vol. 30, no. 4, pp. 1334-1345, 2007. (Pubitemid 47263518)
8. N. Garay, I. Cearreta, J. M. López, and I. Fajardo, "Assistive technology and affective mediation" Assistive Technol., vol. 2, no. 1, pp. 55-83, 2006.
9. A. Haag, S. Goronzy, P. Schaich, and J. Williams, "Emotion recognition using bio-sensors: First steps towards an automatic system" Lecture Note Comput. Sci., vol. 3068, pp. 36-48, 2004. (Pubitemid 38851666)
10. D. Ververidis and C. Kotropoulos, "Fast and accurate sequential floating forward feature selection with the Bayes classifier applied to speech emotion recognition" Signal Process., vol. 88, no. 12, pp. 2869-3014, 2008.
11. P. Pudil, F. Ferri, J. Novovicova, and J. Kittler, "Floating search method for feature selection with nonmonotonic criterion functions" Pattern Recognit., vol. 2, pp. 279-283, 1994.
12. S. Casale, A. Russo, and S. Serrano, "Multistyle classification of speech under stress using feature subset selection based on genetic algorithms" Speech Commun., vol. 49, pp. 801-810, 2007. (Pubitemid 47268572)
13. X. Li, J. Tao, M. T. Johnson, J. Soltis, A. Savage, and K. M. Leong, "Stress and emotion classification using jitter and shimmer features" in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2007, vol. 4, pp. 1081-1084.
14. D. Ververidis and C. Kotropoulos, "Automatic speech classification to five emotional states based on gender information" in Proc. Eur. Signal Process. Conf., 2004, pp. 341-344.
15. M. Lugger and B. Yang, "An incremental analysis of different feature groups in speaker independent emotion recognition" in Proc. Int. Congr. Phonet. Sci., 2007, pp. 2149-2152.
16. O. Kwon, K. Chan, J. Hao, and T. Lee, "Emotion recognition by speech signals" in Proc. Eur. Conf. Speech Commun. Technol., 2003, pp. 125-128.
17. Y. Lin and G. Wei, "Speech emotion recognition based on HMM and SVM" in Proc. IEEE Int. Conf. Mach. Learn. Cybern., 2005, vol. 8, pp. 18-21.
18. J.-L. Gauvain and C.-H. Lee, "Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains" IEEE Trans. Speech Audio Process., vol. 2, no. 2, pp. 291-298, Apr. 1994.
19. L. Fu, X. Mao, and L. Chen, "Speaker independent emotion recognition based on SVM/HMMs fusion system" in Proc. IEEE Int. Conf. Audio, Lang. Image Process., 2008, pp. 61-65.
20. B. Schuller, S. Reiter, R.Muller,M. Al-Hamas, M. Lang, and G. Riqoll, "Speaker independent speech emotion recognition by ensemble classification" in Proc. IEEE-ICME, 2005, pp. 864-867. (Pubitemid 44669004)
21. B. Schuller, D. Seppi, A. Batliner, A. Maier, and S. Steidl, "Towards more reality in the recognition of emotional speech" in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2007, pp. 941-944.
22. B. Taskar, C. Guestrin, and D.Koller, "Max-margin Markov networks" in Proc. Adv. Neural Inf. Process. Syst., 2003, vol. 16.
23. I. Tsochantaridis, T. Joachims, and T. Hofmann, "Large margin methods for structured and interdependent output variables" J. Mach. Learn. Res., vol. 6, pp. 1453-1484, 2005.
24. V. Vapnik, The Nature of Statistical Learning Theory. New York: Springer, 2000.
25. G. Heigold, T. Deselaers, R. Schlüter, and H. Ney, "Modified MMI/ MPE: A direct evaluation of the margin in speech recognition" in Proc. Int. Conf. Mach. Learn., 2008, pp. 384-391.
26. F. Sha and L. K. Saul, "Large margin hidden Markov models for automatic speech recognition" Neural Inf. Process. Syst., vol. 19, pp. 1249-1256, 2007.
27. S. Sarawagi and R. Gupta, "Accurate max-margin training for structured output spaces" in Proc. Int. Conf. Mach. Learn., 2008, pp. 888-895.
28. A. Tellegen, D. Watson, and L. Clark, "On the dimensional and hierarchical structure of affect" Psychol. Sci., vol. 10, no. 4, pp. 297-303, 1999.
29. D. Watson and A. Tellegen, "Toward a consensual structure of mood" Psychol. Bull., vol. 98, no. 2, pp. 219-35, 1985.
30. D.Watson, L. A. Clark, and A. Tellegen, "Development and validation of brief measures of positive and negative affect: The PANAS scales" J. Personal. Soc. Psychol., vol. 54, no. 2, pp. 1063-1070, 1988.
31. S. Yun and C. D. Yoo, "Speech emotion recognition via a max-margin framework incorporating a loss function based on the Watson and Tellegen's emotion model" in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2009, pp. 4169-4172.
32. L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition" Proc. IEEE, vol. 77, no. 2, pp. 257-286, Feb. 1989.
33. L. R. Bahl, P. F. Brown, P. V. de Souza, and R. L. Mercer, "Maximum mutual information estimation of hidden Markov model parameters for speech recognition" in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 1986, vol. 1, pp. 49-52.
34. A. B. Yishai and D. Burshtein, "A discriminative training algorithm for hidden Markov models" IEEE Trans. Speech Audio Process., vol. 12, no. 3, pp. 204-217, May 2004.
35. Y. Normandin, R. Cardin, and R. D. Mori, "High-performance connected digit recognition using maximum mutual information estimation" IEEE Trans. Speech Audio Process., vol. 2, no. 2, pp. 299-311, Apr. 1994.
36. A. Nadas, "A decision theoretic formulation of a training problem in speech recognition and a comparison of training by unconditional vesus conditional maximum likelihood" IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-31, no. 4, pp. 814-817, Aug. 1983. (Pubitemid 14455162)
37. K. Crammer and Y. Singer, "On the algorithmic implementation of multiclass kernel-based vector machines" J. Mach. Learn. Res., vol. 2, pp. 265-292, 2001.
38. N. D. Ratliff, J. A. Bagnell, and M. A. Zinkevich, "(Online) Subgradient methods for structured prediction" in Proc. Int. Conf. Artif. Intell. Statist., 2007.
39. Y. Yin and H. Jiang, "A compact semidefinite programming (SDP) formulation for large margin estimation of HMMs in speech recognition" in Proc. IEEE Workshop Autom. Speech Recognit. Understand., 2007, pp. 312-317.
40. X. Li and H. Jiang, "Solving large-margin hidden Markov model estimation via semidefinite programming" IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 8, pp. 2383-2392, Nov. 2007.
41. S. J. Benson and Y. Ye, "Algorithm 875: DSDP5-Software for semidefinite programming" Res. J. Assoc. for Comput. Machinery Math. Software, vol. 34, no. 3, pp. 16:1-16:20, 2008.
42. F. Burkhardt, A. Paeschke, M. Rolfes, W. Sendlmeier, and B. Weiss, "A database of German emotional speech" in Proc. Int. Conf. Spoken Lang. Process., 2005, pp. 1517-1520. (Pubitemid 43908362)
43. J. Hansen and S. Bou-Ghazale, "Getting started with susas: A speech under simulated and actual stress database" in Proc. Eur. Conf. Speech Commun. Technol., 1997, vol. 4, pp. 1743-1746.
44. I. S. Engberg, A. V. Hansen, O. Andersen, and P. Dalsgaard, "Design, recording and verification of a Danish emotional speech database" in Proc. Eur. Conf. Speech Commun. Technol., 1997, pp. 1695-1698.
45. M. Grimm, K. Kroschel, and S. Narayanan, "The Vera am Mittang German audio-visual emotional speech database" in Proc. IEEE Int. Conf. Multimedia Expo, 2008.
46. Y. Qiao, M. Suzuki, and N. Minematsu, "Affine invariant features and their application to speech recognition" in Proc. IEEE Int. Conf. Acousti., Speech, Signal Process., 2009, pp. 4629-4632.
47. O. Kwon and T. Lee, "Phoneme recognition using ica-based feature extraction and transformation" Signal Process., vol. 84, no. 6, pp. 1005-1019, 2004.
48. D. Jurafsky and J. H. Martin, Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Upper Saddle River, NJ: Prentice-Hall, 2008.
49. S. Young, G. Evermann, D. Kershaw, G. Moore, J. Odell, D. Ollason, D. Povey, V. Valtchev, and P. Woodland, The HTK Boook (for HTK Version 3.2). Cambridge, U.K.: Univ. of Cambridge, 2002.
50. B. Schuller, B. Vlasenko, F. Eyben, M.Wollmer, A. Stuhlsatz, A.Wendemuth, and G. Rigoll, "Cross-corpus acoustic emotion recognition: Variances and strategies" IEEE Trans. Affective Comput., vol. 1, no. 2, pp. 119-132, Jul.-Dec. 2010.
51. S. Casale, A. Russo, and S. Serrano, "Analysis of robustness of attributes selection applied to speech emotion recognition" in Proc. Eur. Signal Process. Conf., 2010, pp. 1174-1178.
52. M. Grimm and K. Kroschel, "Evaluation of natural emotions using self assessment manikins" in Proc. IEEE Workshop Autom. Speech Recognit. Understand., 2005, pp. 381-385.
53. M. Grimm, K. Kroschel, E. Mower, and S. Narayanan, "Primitives- based evaluation and estimation of emotions in speech" Speech Commun., vol. 49, no. 10-11, pp. 787-800, 2007. (Pubitemid 47268568)
54. K. Yao, K. K. Paliwal, and S. Nakamura, "Noise adaptive speech recognition based on sequential noise parameter estimation" Speech Commun., vol. 42, no. 1, pp. 5-23, 2004.
55. S. Boyd, L. E. Ghaoui, E. Feron, and V. Balakrishnan, Linear Matrix Inequalities in System and Control Theory. Philadelphia, PA: Society for Industrial and Applied Mathematics, 1994, vol. 15.