Expressive speech: Production, perception and application to speech synthesis

Acoustical Science and Technology

Volumn 26, Issue 4, 2005, Pages 317-325

  Erickson, Donna ^a  

^a Gifu City Women's College   (Japan)

Author keywords

Data collection; Expressive speech; Labeling; Modeling; Perception; Production; Synthesis; Voice quality

Indexed keywords

DATA REDUCTION; DATABASE SYSTEMS; IMAGE PROCESSING; SENSORY PERCEPTION; SYNCHRONIZATION;

DATA COLLECTION; EXPRESSIVE SPEECH; NON-PROFESSIONAL SPEAKERS; VOICE QUALITY;

SPEECH SYNTHESIS;

EID: 23144458652     PISSN: 13463969     EISSN: None     Source Type: Journal    
DOI: 10.1250/ast.26.317     Document Type: Review

References (85)

1. H. Fujisaki, "Prosody, information, and modeling - With emphasis on tonal features of speech," Proc. Speech Prosody 2004 Nara, pp. 1-10 (2004).
2. K. R. Scherer, "Vocal communication of emotion: A review of research paradigms," Speech Commun., 40, 227-256 (2003).
3. M. Schroeder, "Speech and emotion research: An overview of research frameworks and a dimensional approach to emotional speech synthesis," Doctoral thesis, Phonus 7, Res. Rep. Inst. Phonetics. Saarland Univ. (2004) (http://www.dfki.de/~schroed/publications.htm).
4. C. E. Williams and K. N. Stevens, "On determining the emotional states of pilots during flight: An exploratory study," Aerosp. Med., 40, 1369-1372 (1969).
5. P. Roach, R. Stibbard, J. Osborne, S. Arnfield and J. Setter, "Transcription of prosodic and paralinguistic features of emotional speech," J. Int. Phonet. Assoc., 28, 83-94 (1998).
6. S.-J. Chung, "L'expression et la perception de l'émotion extradite de la parole spontanée: evidence du coréen et de l'anglais. (Expression and perception of emotion extracted from the spontaneous speech in Korean and in English)," Doctoral Dissertation from ILPGA, Sorbonne Nouvelle University, Paris, France (Also available Septentrion Press Universitaires) (2000).
7. J.-A. Bachorowski, M. J. Smoski and M. J. Owren, "The acoustic features of human laughter," J. Acoust. Soc. Am., 110, 1581-1597 (2001).
8. D. Erickson, O. Fujimura and B. Pardo, "Articulatory correlates of prosodic control: Emotion versus emphasis," in Language and Speech: Special Issue on Prosody and Conversation, M. Swerts and J. Hirschberg, Eds. (1998), pp. 399-417.
9. D. Erickson, P. Mokhtari, C. Menezes and A. Fujino, "Voice quality and other acoustic changes in sad speech (grief)," Tech. Rep. Inst. Electron. Inf. Commun. Eng., pp. 43-48 (2003).
10. D. Erickson, K. Yoshida, T. Mochida and Y. Shibuya, "Acoustic and articulatory analysis of sad Japanese speech," Phonet. Soc: Jpn. Fall Meet., pp. 113-118 (2004).
11. N. Campbell, "Developments in corpus-based speech synthesis: Approaching natural conversational speech," IEICE Trans., 87, 497-500 (2004).
12. K. Maekawa, H. Kikuchi and W. Tsukahara, "Corpus of spontaneous Japanese: Design, annotation, and XML representation," Proc. Int. Symp. Large-scale Knowledge Resource (LKR2004), Tokyo Institute of Technology, pp. 19-24 (2004) (downloadable from http://www2.kokken.go.jp/~kikuo/public/LKR04. pdf).
13. E. Douglas-Cowie, N. Campbell, R. Cowie and P. Roach, "Emotional speech: Towards a new generation of databases," Speech Commun. Special Issue Speech and Emotion, 40, 33-60 (2003).
14. K. Maekawa and N. Kitagawa, "How does speech transmit paralinguistic information?," Congnit. Stud., 9, 46-66 (2002).
15. N. Campbell and D. Erickson, "What do people hear? A study of the perception of non-verbal affective information in conversational speech," J. Phonet. Soc. Jpn., 8, 9-28 (2004).
16. T. Ehrette, N. Chateau, C. d'Alessandro and V. Maffiolo, "Prosodic parameters of perceived emotions in vocal server voices," Proc. Speech Prosody 2002 Aix-en-Provence, pp. 259-262 (2002).
17. N. Campbell and P. Mokhtari, "Voice quality. The 4th prosodic parameter," Proc. 15th Int. Congr. Phonetic Sciences, pp. 2417-2420 (2003).
18. ANSI, "Psychoacoustical terminology," Technical Report, S.3.30, American National Standard Report (1973).
19. P. Mokhtari, "Parameterization and control of laryngeal voice quality by principal components of glottal waveforms," J. Phonet. Soc. Jpn., 7, 40-54 (2003).
20. N. Campbell, "Listening between the lines: A study of paralinguistic information carried by tone-of-voice," Proc. Int. Symp. Tonal Aspects of Language - With Emphasis on Tone Languages, Beijing (TAL), pp. 13-16 (2004).
21. K. Sakakibara and H. Imagawa, "Acoustical interpretation of certain laryngeal settings using a physical model," Proc. Speech Prosody 2004 Nara, pp. 637-640 (2004).
22. J. Laver, The Phonetic Description of Voice Quality (Cambridge University Press, Cambridge, 1980).
23. J. Esling, "The IPA categories 'pharyngeal' and 'epiglottal': Laryngoscopic observations of pharyngeal articulations and larynx height," Lang. Speech, 42, 349-372 (1999).
24. M. Teshigawara, "Vocally expressed emotions and stereotypes in Japanese animation: Voice qualities of the bad guys compared to those of the good guys," J. Phonet. Soc. Jpn., 8, 60-76 (2004).
25. T. Ananthapadmanabha and J. Estill, "Analysis and synthesis of six voice qualities," J. Acoust. Soc. Am., Suppl. 1, 86, S 36 (1989).
26. C. Gobi and A. Ni Chasaide, "The role of voice quality in communicating emotion, mood, and attitude," Speech Commun., 40, 189-212 (2003).
27. G. Fant, "The voice source in connected speech," Speech Commun., 22, 125-139 (1997).
28. M. Gordon and P. Ladfoged, "Phonation types: A cross-linguistic overview," J. Phonet., 29, 383-406 (2001).
29. D. G. Childers and C. K. Lee, "Vocal quality factors: Analysis, synthesis and perception," J. Acoust. Soc. Am., 90, 2394-2410 (1991).
30. H. M. Hanson, K. N. Stevens, H.-K. J. Kuo, M. Y. Chen and J. Slifka, "Towards models of phonation," J. Phonet., 29, 451-480 (2001).
31. C. T. Ishii, "Analysis of autocorrelation-based parameters for creaky voice detection," Proc. Speech Prosody 2004 Nara, pp. 643-646 (2004).
32. G. Fant, J. Liljencrants and Q. Lin, "A four-parameter model of glottal flow," STL-QPSR, 4, Speech, Music and Hearing, Royal Institute of Technology, Stockholm, 1-13 (1985).
33. P. Alku and E. Vilkman, "Amplitude domain quotient for characterization of the glottal volume velocity waveform estimated by inverse filtering," Speech Commun., 18, 131-138 (1996).
34. P. Alku, T. Bäcksröm and E. Vilkman, "Normalized amplitude quotient for parameterization of the glottal flow," J. Acoust. Soc. Am., 112, 701-710 (2002).
35. A. Ní Chasaide and C. Gobi, "Voice source variation," in The Handbook of Phonetic Sciences, W. J. Hardcastle and J. Laver, Eds. (Blackwell, Oxford, 1997), pp. 427-461.
36. M. Mazo, D. Erickson and T. Harvey, "Emotion and expression: Temporal data on voice quality in Russian lament," The 8th Vocal Fold Physiology Conf., Kurume, Japan, pp. 173-187 (1995).
37. R. D. Patterson, K. Robinson, J. Holdsworth, D. McKeown, C. Zhang and M. Allerhand, "Complex sounds and auditory images," in Auditory Physiology and Perception, 9th International Symposium on Hearing, Y. Cazals, K. Honern and L. Demany, Eds. (Pergamon Press, Oxford, 1992), pp. 42-446.
38. N. Campbell, "Synthesis units for conversational speech - Using phrasal segments," Proc. Autumn Meet. Acoust. Soc. Jpn., pp. 337-338 (2004).
39. D. H. Kaltt and L. C. Klatt, "Analysis, synthesis, and perception of voice quality variations among female and male talkers," J. Acoust. Soc. Am., 87, 820-857 (1990).
40. H. Kasuya, M. Kawamata and H. Kido, "Multiregressional analysis of autoregressive with exogenous input speech synthesis parameters and voice qualities," J. Acoust. Soc. Am., 116, 2545 (2004).
41. 0 contours using generation process model," Proc. Speech Prosody 2004 Nara, pp. 417-420 (2004).
42. Y. Sagisaka, T. Yamashita and Y. Kokenawa, "Speech synthesis with attitude," Proc. Speech Prosody 2004 Nara, pp. 401-404 (2004).
43. M. Tachibana, J. Yamagishi, K. Onishi, T. Masuko and T. Kobayashi, "HMM-based speech synthesis with various speaking styles using model interpolation," Proc. Speech Prosody 2004 Nara, pp. 413-416 (2004).
44. A. Iida, N. Campbell, F. Higuchi and M. Yasumura, "A corpus-based speech synthesis system with emotion," Speech Commun., 40, 161-187 (2003).
45. H. Mizuno, M. Abe and S. Nakajima, "Development of speech design tool 'SESIGN99' to enhance synthesized speech," Proc. Eurospeech 99, pp. 2083-2086 (1999).
46. Y. Sogabe, K. Kakehi and H. Kawahara, "Psychological evaluation of emotional speech using a new morphing method," 4th ICCS Int. Conf. Cognitive Science, Sydney, Australia July 13-17 (2003).
47. H. Kawahara and H. Matsui, "Auditory morphing based on an elastic perceptual distance metric in an interference-free time-frequency representation," ICASSP '2003, pp. 256-259 (2003).
48. K. Maekawa and T. Kagomiya, "Influence of paralinguistic information on segmental articulation," Proc. Int. Conf. Spoken Language Processing, Vol. 2, pp. 349-352 (2000).
49. D. Erickson, A. Abramson, K. Maekawa and T. Kaburagi, "Articulatory characteristics of emotional utterances in spoken English," Proc. Int. Conf. Spoken Language Processing, Vol. 2, pp. 365-368 (2000).
50. C. Menezes, "Changes in phrasing in semi-spontaneous emotional speech: Articulatory evidences," J. Phonet. Soc. Jpn., 8, 45-59 (2004).
51. K. R. Scherer, R. Banse and H. G. Wallbott, "Emotion inferences from vocal expression correlate across languages and cultures," J. Cross-Cultural Psychol., 32, 76-92 (2001).
52. T. Sadanobu, "A natural history of Japanese pressed voice," J. Phonet. Soc. Jpn., 8, 29-44 (2004).
53. K. Maekawa, "Phonetic and phonological characteristics of paralinguistic information in spoken Japanese," Proc. Int. Conf. Spoken Language Processing, Vol. 2, pp. 635-638 (1998).
54. O. Fujimura and D. Erickson, "The C/D Model for prosodic representation of expressive speech in English," Proc. Autumn Meet. Acoust. Soc. Jpn., pp. 271-272 (2004).
55. H. Fujisaki and K. Hirose, "Analysis of voice fundamental frequency contours for declarative sentences in Japanese," J. Acoust. Soc. Jpn. (E), 5, 233-242 (1984).
56. S. Mozziconacci, "Prosody and emotions," Proc. Speech Prosody 2002 Aix-en-Provence, pp. 1-9 (2002).
57. J. t'Hart, R. Collier and A. Cohen, A Perceptual Study of Intonation: An Experimental Phonetic Approach to Speech Melody (Cambridge University Press, Cambridge, 1990).
58. S. Nooteboom, "The prosody of Speech: Melody and rhythm," in The Handbook of Phonetic Sciences, W. J. Hardcastle and J. Laver, Eds. (Blackwell, Oxford, 1997), pp. 640-673.
59. O. Fujimura, "Phonology and phonetics - A syllable-based model of articulatory organization," J. Acoust. Soc. Jpn. (E), 13, 39-48 (1992).
60. C. E. Williams and K. N. Stevens, "Emotions and speech: Some acoustical correlates," J. Acoust. Soc. Am., 52, 1238-1250 (1972).
61. A. Paeschke, "Global trend of fundamental frequency in emotional speech," Proc. Speech Prosody 2004 Nara, 671-674 (2004).
62. K. R. Scherer, R. Banse, H. G. Wallbott and T. Goldbeck, "Vocal cues in emotion encoding and decoding," Motiv. Emotion, 15, 123-148 (1991).
63. Y. Hashizawa, S. Takeda, M. D. Hamzah and G. Ohyama, "On the differences in prosodic features of emotional expressions in Japanese speech according to the degree of emotion," Proc. Speech Prosody 2004 Nara, pp. 655-658 (2004).
64. C.-F. Huang and M. Akagi, "A perceptual model of emotional speech built by fuzzy logic," Proc. Autumn Meet. Acoust. Soc. Jpn., pp. 287-288 (2004).
65. M. Yanagida, "Discriminating ironies from praising - Acoustic parameters vs. prosodic parameters," Proc. Symp. Prosody and Speech Processing, University of Tokyo, pp. 143-146 (2002).
66. L. Leinonen, T. Hiltunen, I. Linnanakoski and M.-L. Laakso, "Expression of emotional-motivational connotations with a one-word utterance," J. Acoust. Soc. Am., 102, 1853-1863 (1997).
67. Y. Hayashi, "Recognition of vocal expression of mental attitudes in Japanese: Using the interjection "eh"," Proc. Int. Congr. Phonetic Sciences, San Francisco, pp. 2355-2358 (1999).
68. S. Takeda, G. Ohyama, A. Tochitani and Y. Nishizawa, "Analysis of prosodic features of "anger" expressions in Japanese speech," J. Acoust. Soc. Jpn. (J), 58, 561-568 (2002).
69. G. L. Huttar, "Relations between prosodic variables and emotions in normal American English utterances," J. Speech Hear. Res., 11, 467-480 (1968).
70. T. Johnstone and K. R. Scherer, "The effects of emotions on voice quality," Proc. Int. Congr. Phonetic Sciences, San Francisco, pp. 2029-2031 (1999).
71. R. Kehrein, "The prosody of authentic emotions," Proc. Speech Prosody 2002 Aix-en-Provence, pp. 423-426 (2002).
72. C. T. Ishii and N. Campbell, "Acoustic-prosodic analysis of phrase finals in expressive speech," Proc. 1st JST/CREST Int. Workshop Expressive Speech Processing, Kobe, Feb. 21, 22, 2003, pp. 85-88 (2003).
73. K. Sakuraba, "Emotional expression in 'Pikachuu'," J. Phonet. Soc. Jpn., 8, 77-84 (2004).
74. M. D. Pell, "Influence of emotion and focus location on prosody in matched statements and questions," J. Acoust. Soc. Am., 109, 1668-1680 (2001).
75. N. M. Markel, M. F. Bein and J. A. Phillis, "The relationship between words and tone-of-voice," Lang. Speech, 16, 15-21 (1973).
76. L. Devillers, I. Vasilescu and L. Vidrascu, "F0 and pause features analysis for anger and fear detection in real-life spoken dialogs," Proc. Speech Prosody 2004 Nara, pp. 205-208 (2004).
77. D. Erickson, S. Ohashi, Y. Makita, N. Kajimoto and P. Mokhtari, "Perception of naturally-spoken expressive speech by American English and Japanese listeners," Proc. 1st JST/ CREST Int. Workshop Expressive Speech Processing, Kobe, Feb. 21, 22, 2003, pp. 31-36 (2003).
78. P. B. de Mareüil, P. Célérier and J. Toen, "Generation of emotions by a morphing technique in English, French and Spanish," Proc. Speech Prosody 2002 Aix-en-Provence, pp. 187-190 (2002).
79. A. Abelin, "Cross-cultural multimodal interpretation of emotional expressions - An experimental study of Spanish and Swedish," Proc. Speech Prosody 2004 Nara, pp. 647-650 (2004).
80. I. Ibrakhim, "Universal and linguistic features of expressing emotional information: differentiation in the perception level," Proc. Speech Prosody 2004 Nara, pp. 659-662 (2004).
81. D. Erickson and K. Maekawa, "Perception of American English emotion by Japanese listeners," Proc. Spring Meet. Acoust. Soc. Jpn., pp. 333-334 (2001).
82. A. Nakamichi, A. Jogan, M. Usami and D. Erickson, "Perception by native and non-native listeners of vocal emotion in a bilingual movie," Gifu City Women's Coll. Res. Bull., 52, 87-91 (2002).
83. N. Amir, B.-C. Almogi and R. Gal, "Perceiving prominence and emotion in speech - A cross lingual study," Proc. Speech Prosody 2004 Nara, pp. 375-378 (2004).
84. K. Maekawa, "Production and perception of 'paralinguistic' information," Proc. Speech Prosody 2004 Nara, pp. 367-374 (2004).
85. D. Erickson, S. Hayashi, Y. Hosoe, M. Suzuki, Y. Ueno and K. Maekawa, "Perception of American English sarcasm by Japanese listeners," Proc. Spring Meet. Acoust. Soc. Jpn., Vol. 1, pp. 277-278 (2002).