Speech emotion recognition using hidden Markov models

Speech Communication

Volumn 41, Issue 4, 2003, Pages 603-623

  Nwe, Tin Lay ^a   Foo, Say Wei ^b   De Silva, Liyanage C ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^b NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

Emotional speech; Hidden Markov model; Human communication; Log frequency power coefficients; Recognition of emotion

Indexed keywords

ALGORITHMS; DATABASE SYSTEMS; MARKOV PROCESSES; SPEECH RECOGNITION;

EMOTIONAL SPEECH;

SPEECH COMMUNICATION;

EID: 0242721417     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0167-6393(03)00099-2     Document Type: Article

References (62)

1. Arnold, M.B., 1960. Emotion and Personality. Physiological Aspects, Vol. 2. Columbia University Press, New York.
2. Atal B.S. Effectiveness of linear prediction characteristics of the speech wave for automatic speaker identification and verification. J. Acoust. Soc. Amer. 55(6):1974;1304-1312.
3. Becchetti C., Ricotti L.P. Speech Recognition Theory and C++ Implementation. 1998;John Wiley & Sons, New York.
4. Cahn J.E. The generation of affect in synthesized speech. J. Amer. Voice I/O Soc. 8:1990;1-19.
5. Cairns D.A., Hansen J.H.L. Nonlinear analysis and classification of speech under stressed conditions. J. Acoust. Soc. Amer. 96(6):1994;3392-3400.
6. Coleman, R., Williams, R., 1979. Identification of emotional states using perceptual and acoustic analyses. In: Lawrence, V., Weinberg, B. (Eds.), Care of the Professional Voice, Vol. 1. The Voice Foundation, New York.
7. Cornelius R. The Science of Emotion. 1996;Prentice-Hall, Englewood Cliffs, NJ.
8. Cowan, M., 1936. Pitch and Intensity Characteristics of Stage of Speech. Arch. Speech, suppl. to Dec. issue.
9. Cowie R., Douglas-Cowie E., Tsapatsoulis N., Votsis G., Kollias S., Fellenz W., Taylor J.G. Emotion recognition in human-computer interaction. IEEE Sig. Proc. Mag. 18(1):2001;32-80.
10. Crystal D. Prosodic Systems and Intonation in English. 1969;Cambridge University Press, London, UK.
11. Crystal D. The English Tone of Voice. 1975;Edward Arnold, London, UK.
12. Darwin, C., 1965. The Expression of Emotions in Man and Animals. John Murray, Ed., 1872. Reprinted by University Press.
13. Davis M., College H. Recognition of Facial Expressions. 1975;Arno Press, New York.
14. Davis S.B., Mermelstein P. Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Trans. Acoust. Speech Signal Process. 28:1980;357-366.
15. Davitz J.R. The Communication of Emotional Meaning. 1964;McGraw-Hill, New York.
16. De Silva L.C., Miyasato T., Nakatsu R. Use of multimodal information in facial emotion recognition. IEICE Trans. Inf. Syst. E81-D(1):1998;105-114.
17. Dellaert, F., Polzin, T., Waibel, A., 1996. Recognizing Emotion in Speech. Fourth International Conference on Spoken Language Processing 3, 1970-1973.
18. Deller J.R., Proakis J.G., Hansen J.H.L. Discrete-Time Processing of Speech Signals. 1993;Macmillan Pub. Co. Toronto.
19. Ekman P. Darwin and Facial Expressions. 1973;Academic, New York.
20. Ekman P., Friesen W. Unmasking the Face. 1975;Prentice-Hall, Englewood Cliffs, NJ.
21. Elias, N.J., 1975. New Statistical Methods for Assigning Device Tolerances. Proc. IEEE Int. SYmp. Ccts. Sys., 1975, Newton, MA, USA, pp. 329-332.
22. Equitz W.H. A new vector quantization clustering algorithm. IEEE Trans. Acoust. Speech Signal Process. 37(10):1989;1568-1575.
23. Fairbanks G., Pronovost W. An experimental study of the pitch characteristics of the voice during the expression of emotion. Speech Monograph. 6:1939;87-104.
24. Fonagy I. A new method of investigating the perception of prosodic features. Language and Speech. 21:1978;34-49.
25. Fonagy, I., 1978b. In: Sundberg, J. (Ed.), Emotions, Voice and Music in Language and Speech, Vol. 21, pp. 34-49.
26. Fonagy I., Magdics K. Emotional patterns in intonation and music. Z. Phonet. Sprachwiss. Kommunikationsforsch. 16:1963;293-326.
27. Fox N.A. If it's not left it's right. Amer. Psychol. 46:1992;863-872.
28. Frick R. Communicating Emotion: The Role of Prosodic Features. Psychol. Bull. 97(3):1985;412-429.
29. Furui S. Digital Speech Processing, Synthesis and Recognition. 1989;Marcel Dekker, New York.
30. Havrdova Z., Moravek M. Changes of the voice expression during suggestively influenced states of experiencing. Activitas Nervosa Superior. 21:1979;33-35.
31. Huttar G.L. Relations between prosodic variables and emotions in normal american english utterances. J. Speech Hearing Res. 11:1968;481-487.
32. Johnson W., Emde R., Scherer R., Klinnert M. Recognition of emotion from vocal cues. Arch. Gen. Psych. 43:1986;280-283.
33. Kaiser L. Communication of affects by single vowels. Synthese. 14:1962;300-319.
34. Kotlyar G., Mozorov V. Acoustic correlates of the emotional content of vocalized speech. J. Acoust. Acad. Sci. USSR. 22:1976;208-211.
35. Lazarus R.S. Emotion Adaptation. 1991;Oxford University Press, New York.
36. Lee K.F., Hon H.W. Speaker-independent phone recognition using Hidden Markov Models. IEEE Trans. Acoust. Speech Signal Process. 37(11):1989;1641-1648.
37. Lynch G.E. A phonophotographic study of trained and untrained voices reading factual and dramatic material. Arch. Speech. 1:1934;9-25.
38. McGilloway, S., Cowie, R., Douglas-Cowie, E., 1995. Prosodic signs of emotion in speech: preliminary results from a new technique for automatic statistical analysis. In: Proc. XIIIth Int. Congr. Phonetic Sciences, Vol. 1. Stockholm, Sweden, pp. 250-253.
39. McGilloway, S., Cowie, R., Douglas-Cowie, E., Gielen, S., Westerdijk, M., Stroeve, S., 2000. Approaching Automatic Recognition of Emotion from Voice: A Rough Benchmark. ISCA Workshop on Speech and Emotion, Belfast.
40. Muller, A., 1960. Experimentelle Untersuchungen zur stimmlichen Darstellung von Gefuehlen [Experimental Studies on Vocal Portrayal of Emotion], Ph.D. dissertation, Univ. Gottingen, Germany.
41. Murray I.R., Arnott J.L. Toward the simulation of emotion in synthetic speech: A review of the literature on human vocal emotion. J. Acoust. Soc. Amer. 93(2):1993;1097-1108.
42. Nicholson, J., Takahashi, K., Nakatsu, R., 1999. Emotion recognition in speech using neural networks. 6th International Conference on Neural Information Processing, ICONIP '99, Vol. 2, pp. 495-501.
43. Oatley K., Johnson-Laird P. Communicative theory of emotions: Empirical test, mental models and implications for social interaction. Martin L., Tessler A. Goals and Affect. 1995;Erlbaum, Hillsdale, NJ.
44. O'Connor J.D., Arnold G.F. Intonation of Colloquial English. second ed. 1973;Longman, London, UK.
45. Oster, A., Risberg, A., 1986. The Identification of the Mood of A Speaker by Hearing Impaired Listeners. Speech Transmission Lab. Quarterly Progress Status Report 4, Stockholm, pp. 79-90.
46. Otaley K., Jenkis J.M. Understanding Emotions. 1996;Blackwell, Oxford, UK.
47. Plutchik R. The Psychology and Biology of Emotion. 1994;Harper Collins, New York. p. 58.
48. Rabiner L.R., Juang B.H. Fundamentals of Speech Recognition. 1993;Prentice Hall, Englewood Cliffs, NJ.
49. Rabiner L.R., Schafer R.W. Digital Processing of Speech Signals. 1978;Prentice Hall, Englewood Cliffs, NJ.
50. Scherer K.R. On the nature and function of emotion: A component process approach. Scherer K.R., Ekman P. Approaches to Emotion. 1984;Erlbaum, Hillsdale, NJ.
51. Scherer K.R. Vocal effect expression: A review and a model for future research. Psychol. Bull. 99:1986;143-165.
52. Scherer K.R. Vocal affect expression: A review and a model for future research. Psychol. Bull. 99:1986;143-165.
53. Scherer K.R., Banse R., Wallbott H.G. Emotion inferences from vocal expression correlate across languages and cultures. J. Cross-Cultural Psychol. 32(1):2001;76-92.
54. Scherer K., Ekman P. Approaches to Emotion. 1984;Lawrence Erlbaum Associates, Mahwah, NJ.
55. Schubiger M. English Intonation. Its Form and Function. 1958;Niemeyer, Tubingen, Germany.
56. Sulc J. Emotional changes in human voice. Activitas Nervosa Superior. 19:1977;215-216.
57. Trojan F. Der Ausdruck der Sprechstimme. 1952;W. Maudrich, Wien-Dusseldorf, Germany.
58. Utsuki, N., Okamura, N., 1976. Relationship Between Emotional State and Fundamental Frequency of Speech. Rep. Aeromedical Laboratory. Japan Air Self-Defense Force 16, 179-188.
59. Van Bezooijen R. Characteristics and Recognizability of Vocal Expressions of Emotions. 1984;Foris, Dordrecht, The Netherlands.
60. Williams C.E., Stevens K.N. On determining the emotional state of pilots during flight: An exploratory study. Aerospace Med. 40:1969;1369-1372.
61. Williams C.E., Stevens K.N. Vocal correlates of emotional states. Darby J.K. Speech Evaluation in Psychiatry. 1981;189-220 Grune and Stratton, Inc.
62. Yamada, T., Hashimoto, H., Tosa, N., 1995. Pattern recognition of emotion with neural network. In: Proc. 1995 IEEE IECON 21st Internat. Conf. Industrial Electronics, Control, and Instrumentation, Vol. 1, pp. 183-187.