Evaluation of expressive speech synthesis with voice conversion and copy resynthesis techniques

IEEE Transactions on Audio, Speech and Language Processing

Volumn 18, Issue 5, 2010, Pages 965-973

  Türk, Oytun ^a   Schröder, Marc ^b  

^a Sensory Inc   (United States)

^b GERMAN RESEARCH CENTER FOR ARTIFICIAL INTELLIGENCE DFKI   (Germany)

Author keywords

Expressive speech synthesis; Prosody; Voice conversion; Voice quality transformation

Indexed keywords

APPROXIMATE MODEL; COMBINED MODELING; EXPRESSIVE SPEECH; EXPRESSIVE SPEECH SYNTHESIS; FACTORIAL DESIGN; LISTENING TESTS; OPEN SOURCES; RELATIVE CONTRIBUTION; SIGNAL MANIPULATION; SYNTHETIC SPEECH; TEXT TO SPEECH; TRANSFORMATION ALGORITHM; UNIT SELECTION; VOCAL-TRACTS; VOICE CONVERSION; VOICE QUALITY;

WAVELET TRANSFORMS;

SPEECH SYNTHESIS;

EID: 77953699443     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2010.2041113     Document Type: Article

References (47)

1. P. Gebhard, M. Schröder, M. Charfuelan, C. Endres, M. Kipp, S. Pammi, M. Rumpler, and O. Türk, "IDEAS4Games: Building expressive virtual characters for computer games," in Proc. IVA 2008, Tokyo, Japan, pp. 426-440.
2. W. L. Johnson, S. S. Narayanan, R. Whitney, R. Das, M. L. Bulut, and C. LaBore, "Limited domain synthesis of expressive military speech for animated characters," in Proc. ICSLP 2002, Denver, CO.
3. A. W. Black and N. Campbell, "Optimising selection of units from speech databases for concatenative synthesis," in Proc. Eurospeech, Madrid, Spain, 1995, pp. 581-584.
4. A. W. Black, "Unit selection and emotional speech," in Proc. Eurospeech, Geneva, Switzerland, 2003.
5. A. Iida and N. Campbell, "Speech database design for aconcatenative text-to-speech synthesis system for individuals with communication disorders," Int. J. Speech Technol., vol.6, pp. 379-392, 2003.
6. T. Yoshimura, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura, "Simultaneous modeling of spectrum, pitch and duration in HMM-based speech synthesis," in Proc. Eurospeech, Budapest, Hungary, 1999.
7. R. Tsuzuki, H. Zen, K. Tokuda, T. Kitamura, M. Bulut, and S. S. Narayanan, "Constructing emotional speech synthesizers with limited speech database," in Proc. ICSLP, Jeju, Korea, 2004.
8. J. Yamagishi, T. Kobayashi, M. Tachibana, K. Ogata, and Y. Nakano, "Model adaptation approach to speech synthesis with diverse voices and styles," in Proc. ICASSP, Honolulu, Hawaii, 2007, pp. 1233-1236.
9. M. Tachibana, S. Izawa, T. Nose, and T. Kobayashi, "Speaker and style adaptation using average voice model for style control in HMM-based speech synthesis," in Proc. ICASSP, Las Vegas, NV, pp. 4633-4636.
10. K. Miyanaga, T. Masuko, and T. Kobayashi, "A style control technique for HMM-based speech synthesis," in Proc. ICSLP, Jeju, Korea, 2004.
11. T. Nose, J. Yamagishi, and T. Kobayashi, "A style control technique for speech synthesis using multiple regression HSMM," in Proc. INTERSPEECH 2006, Pittsburgh, PA, USA.
12. V. Karaiskos, S. King, R. A. J. Clark, and C. Mayo, "The Blizzard Challenge 2008," in Proc. Blizzard Challenge 2008, Brisbane, Australia.
13. M. Abe, S. Nakamura, K. Shikano, and H. Kuwabara, "Voice conversion through vector quantization," in Proc. IEEE ICASSP, 1988, pp. 565-568.
14. L. M. Arslan, "Speaker transformation algorithm using segmental codebooks," Speech Commun., vol.28, pp. 211-226, 1999.
15. Y. Stylianou, O. Cappe, and E. Moulines, "Continuous probabilistic transform for voice conversion," IEEE Trans. Speech Audio Process., vol.6, no.2, pp. 131-142, Mar. 1998.
16. A. B. Kain, "High resolution voice transformation," Ph.D. dissertation, OGI School of Sci. and Eng., Oregon Health and Sci. Univ., Beaverton, 2001.
17. O. Türk, "Cross-lingual voice conversion," Ph.D. dissertation, Bo?gaziçi Univ., Istanbul, Turkey, 2007.
18. S. Desai, E. V. Raghavendra, B. Yegnanarayana, A. W. Black, and K. Prahallad, "Voice conversion using artificial neural networks," in Proc. IEEE ICASSP, Taipei, Taiwan, Apr. 2009.
19. E.-K. Kim, S. Lee, and Y.-H. Oh, "Hidden markov model based voice conversion using dynamic characteristics of speaker," in Proc. Eurospeech, 1997, pp. 2519-2522.
20. O. Türk and L. M. Arslan, "Subband based voice conversion," in Proc. ICSLP, Denver, CO, Sep. 2002, vol.1, pp. 289-292.
21. O. Türk, O. Büyük, A. Haznedaroglu, and L. M. Arslan, "Application of voice conversion for cross-language rap singing transformation," in Proc. IEEE ICASSP, Taipei, Taiwan, Apr. 2009.
22. Z. Inanoglu and S. J. Young, "A system for transforming the emotion in speech: Combining data-driven conversion techniques for prosody and voice quality," in Proc. Interspeech, Antwerp, Belgium, Aug. 27-31, 2007.
23. T. Toda, K. Nakamura, H. Sekimoto, and K. Shikano, "Voice conversion for various types of body transmitted speech," in Proc. IEEE ICASSP, Taipei, Taiwan, Apr. 2009.
24. T. En-Najjary, O. Rosec, and T. Chonavel, "A voice conversion method based on joint pitch and spectral envelope transformation," in Proc. 8th Int. Conf. Spoken Lang. Process., Jeju, Korea, 2004.
25. O. Türk and M. Schröder, "A comparison of voice conversion methods for transforming voice quality in emotional speech synthesis," in Proc. Interspeech, Brisbane, Australia, 2008, pp. 2282-2285.
26. Y. Stylianou, "Assessment and correction of voice quality variabilities in large speech databases for concatenative speech synthesis," in Proc. IEEE ICASSP, Phoenix, AZ, 1999.
27. J. Latorre, K. Iwano, and S. Furui, "Polyglot synthesis using amixture of monolingual corpora," in Proc. IEEE ICASSP, 2005, vol.1, pp. 1-4.
28. F. Tesser, P. Cosi, C. Drioli, and G. Tisato, "Emotional festival-Mbrola TTS synthesis," in Proc. Interspeech, Lisbon, Portugal, 2005.
29. T. Dutoit and H. Leich, "Text-to-speech synthesis based on aMBE re-synthesis of segments database," Speech Commun., vol.13, pp. 435-440.
30. K. Hirose, "Improvement in corpus-based generation of f0 contours using generation process model for emotional speech synthesis," in Proc. Interspeech, 2004, pp. 1349-1352.
31. E. Zovato, A. Pacchiotti, S. Quazza, and S. Sandri, "Towards emotional speech synthesis: A rule based approach," in Proc. 5th ISCA Speech Synth. Workshop, Jun. 2004.
32. G. Beller and X. Rodet, "Content-based transformation of the expressivity in speech," in Proc. 16th Int. Congr. Phonetic Sci., Saarbrücken, Germany, Aug. 2007, pp. 2157-2160.
33. D. Jiang, W. Zhang, L. Shen, and L. Cai, "Prosody analysis and modeling for emotional speech synthesis," in Proc. IEEE ICASSP, Mar. 2005, vol.1, pp. 281-284.
34. M. Bulut, C. Busso, S. Yildirim, A. Kazemzadeh, M. C. Lee, S. Lee, and S. Narayanan, "Investigating the role of phoneme-level modifications in emotional speech resynthesis," in Proc. Interspeech, Lisbon, Portugal, 2005.
35. M. Bulut, S. Lee, and S. Narayanan, "A statistical approach for modeling prosody features using POS tags for emotional speech synthesis," in Proc. IEEE ICASSP, Honolulu, HI, Apr. 2007, vol.4, pp. 1237-1240.
36. O. Türk and L. M. Arslan, "Robust processing techniques for voice conversion," Comput. Speech Lang., vol.20, pp. 441-467, 2006.
37. E. Moulines and W. Verhelst, "Time-domain and frequency-domain techniques for prosodic modification of speech," in Speech Coding and Synthesis, Kleijn and Paliwal, Eds. Amsterdam, The Netherlands: Elsevier, 1995, pp. 519-555.
38. H. Kawanami, Y. Iwami, T. Toda, H. Saruwatari, and K. Shikano, "GMM-based voice conversion applied to emotional speech synthesis," in Proc. Eurospeech, 2003, pp. 2401-2404.
39. Z. Inanoglu and S. Young, "Data-driven emotion conversion in spoken English," Speech Commun., vol.51, no.3, pp. 268-283, Mar. 2009.
40. A. Hunecke, "Optimal design of a speech database for unit selection synthesis," Diploma thesis, Univ. des Saarlandes, Saarbrücken, Germany.
41. M. Schröder, M. Charfuelan, S. Pammi, and O. Türk, "The MARY TTS entry in the Blizzard Challenge 2008," in Proc. Blizzard Challenge 2008, Brisbane, Australia.
42. A. Kain and M. Macon, "Spectral voice conversion for text-to-speech synthesis," in Proc. IEEE ICASSP, 1998, vol.1, pp. 285-288.
43. R. Laroia, N. Phamdo, and N. Farvardin, "Robust and efficient quantization of speech LSP parameters using structured vector quantizers," in Proc. IEEE ICASSP, 1991, pp. 641-644.
44. F. Burkhardt, A. Paeschke, M. Rolfes, W. F. Sendlmeier, and B. Weiss, "A database of German emotional speech," in Proc. Interspeech, Lisbon, Portugal, 2005.
45. F. Wilcoxon, "Individual comparisons by ranking methods," Biometrics Bull. 1, pp. 80-83, 1945.
46. Y. Stylianou, "Harmonic plus noise models for speech, combined with statistical methods for speech and speaker modification," Ph.D. dissertation, Dept. Signal, Ecole Nationale Superieure des Telecomm. , ENST-Telecom Paris, Paris, France, 1996.
47. A. Syrdal, Y. Stylianou, L. Garrison, A. Conkie, and J. Schroeter, "TDPSOLA versus harmonic plus noise model in diphone based speech synthesis," in Proc. IEEE ICASSP, Seattle, WA, 1998, pp. 273-276.