Exploiting prosody hierarchy and dynamic features for pitch modeling and generation in HMM-based speech synthesis

IEEE Transactions on Audio, Speech and Language Processing

Volumn 18, Issue 8, 2010, Pages 1994-2003

  Hsia, Chi Chun ^a   Wu, Chung Hsien ^b   Wu, Jung Yun ^b  

^a INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE   (Taiwan)

^b NATIONAL CHENG KUNG UNIVERSITY   (Taiwan)

Author keywords

Dynamic features; hidden Markov model (HMM) based speech synthesis; pitch modeling and generation; prosody hierarchy

Indexed keywords

DYNAMIC FEATURES; DYNAMIC PITCH; FRAME LAYER; HMM-BASED SPEECH SYNTHESIS; MINIMUM DESCRIPTION LENGTH; PITCH CONTOURS; PITCH MODELING; PROSODIC FEATURES; PROSODIC STRUCTURE; PROSODIC WORDS; PROSODY HIERARCHY; PROSODY MODEL; STATIC FEATURES; STATISTICAL HYPOTHESIS TESTING; SUBJECTIVE EVALUATIONS; SUPERVISED CLASSIFICATION; TEMPORAL CORRELATIONS;

FUNCTION EVALUATION; HIDDEN MARKOV MODELS; SPEECH SYNTHESIS; SUBJECTIVE TESTING; TESTING;

CONTINUOUS SPEECH RECOGNITION;

EID: 77956285048     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2010.2040791     Document Type: Article

References (40)

1. J. Tejedor, D. Wang, J. Frankel, S. King, and J. Colás, "A comparison of grapheme and phoneme-based units for Spanish spoken term detection," Speech Commun., vol. 50, no. 11-12, pp. 980-991, Nov.-Dec. 2008.
2. C.-H.Wu, C.-C. Hsia, J.-F. Chen, and J.-F.Wang, "Variable-length unit selection in TTS using structural syntactic cost," IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 4, pp. 1227-1235, May 2007.
3. A. J. Hunt and A. W. Black, "Unit selection in a concatenative speech synthesis system using a large speech database," in Proc. ICASSP'96, 1996, pp. 373-376.
4. A. W. Black and N. Campbell, "Optimizing selection of units from speech database for concatenative synthesis," in Proc. Eurospeech'95, 1995, pp. 581-584.
5. H. Zen, T. Nose, J. Yamagishi, S. Sako, T. Masuko, A. W. Black, and K. Tokuda, "The HMM-based speech synthesis system version 2.0," in Proc. ISCA SSW6, Bonn, Germany, Aug. 2007.
6. K. Tokuda, T. Yoshimura, T. Masuko, T. Kobayashi, and T. Kitamura, "Speech parameter generation algorithms for HMM-based speech synthesis," in Proc. ICASSP, Jun. 2000, pp. 1315-1318.
7. J. Yamagishi, T. Nose, H. Zen, Z. Ling, T. Toda, K. Tokuda, S. King, and S. Renals, "Robust speaker-adaptive HMM-based text-to-speech synthesis," IEEE Trans. Audio, Speech, Lang. Process., vol. 17, no. 6, pp. 1208-1230, Aug. 2009.
8. L. S. Lee, C. Y. Tseng, and M. Ouh-young, "The synthesis rules in a chinese text-to-speech system," IEEE Trans. Acoust., Speech, Signal Process., vol. 37, no. 9, pp. 1309-1319, Sep. 1989.
9. A. Benijamin, S. Chilin, and S. Richard, "A corpus-based Mandarin text-to-speech synthesizer," in Proc. ICSLP'94, Yokohama, Japan, Sep. 1994, pp. 1771-1774.
10. L. Andrej and F. Frank, "Synthesis of natural sounding pitch contours in isolated utterances using hidden Markov models," IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-34, no. 5, pp. 1074-1080, Oct. 1986.
11. N. H. Pan, W. T. Jen, S. S. Yu, M. S. Yu, S. Y. Huang, and M. J. Wu, "Prosody model in a Mandarin text-to-speech system based on a hierarchical approach," in Proc. IEEE Int. Conf. Multimedia Expo, NY, Jul. 2000, vol. 1, pp. 448-451.
12. S. H. Chen, S. H. Hwang, and Y. R. Wang, "An RNN-based prosodic information synthesizer for Mandarin text-to-speech," IEEE Trans. Acoust., Speech, Signal Process., vol. 6, no. 3, pp. 226-269, May 1998.
13. S. H. Kim and J. Y. Kim, "Efficient model of establishing words tone dictionary for korean TTS system," in Proc. Eurospeech, Rhodes, Greece, Sep. 1997, pp. 243-246.
14. M. Dong and K. T. Lua, "Pitch contour model for chinese text-tospeech using CART and statistical model," in Proc. ICSLP'02, Denver, CO, Sep. 2002, pp. 2405-2408.
15. C. H. Wu and J. H. Chen, "Automatic generation of synthesis units and prosodic information for chinese concatenative synthesis," Speech Commun., vol. 35, pp. 219-237, 2001.
16. J. Tao, "F0 Prediction model of speech synthesis based on template and statistical method," in Lecture Nodes of Artificial Intelligence. New York: Springer, 2004.
17. C. Y. Tseng, S. H. Pin, Y. Lee, H. M. Wang, and Y. C. Chen, "Fluent speech prosody: Framework and modeling," Speech Commun., vol. 46, no. 3-4, pp. 284-309, 2005.
18. X. Sun, "The determination, analysis and synthesis of fundamental frequency," Ph.D. dissertating, Northwestern Univ., Evanston, IL, 2002.
19. C. Y. Tseng and Y. L. Lee, "Speech rate and prosody units: Evidence of interaction from Mandarin Chinese," in Proc. Int. Conf. Speech Prosody, Nara, Japan, Mar. 2004, pp. 251-254.
20. S. H. Chen, W. H. Lai, and Y. R. Wang, "A statistics-based pitch contour model for Mandarin speech," J. Acoust. Soc. Amer., vol. 117, no. 2, pp. 908-925, 2005.
21. M. Chu and Y. Qian, "Locating boundaries for prosodic constituents in unrestricted Mandarin texts," Comput. Linguist. Chinese Lang. Process., vol. 6, no. 1, pp. 61-82, 2001.
22. S. H. Pin, Y. L. Lee, Y. C. Chen, H. M. Wang, and C. Y. Tseng, "A Mandarin TTS system with an integrated prosodic model," in Proc. ISCSLP'04, Hong Kong, Dec. 2004, pp. 169-172.
23. H. Fujisaki and K. Hirose, "Analysis of voice fundamental frequency contours for declarative sentence of japanese," J. Acoust. Soc. Japan (E), vol. 5, no. 4, pp. 233-241, 1984.
24. T. Yoshimura, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura, "Simultaneous modeling of spectrum, pitch and duration in HMMbased speech synthesis," in Proc. Eurospeech'99, Budapest, Hungary, Sep. 1999, pp. 2347-2350.
25. K. Silverman, M. Beckman, J. Pitrelli, M. Ostendorf, C.Wightman, P. Price, J. Pierrehumbertand, and J. Hirschberg, "TOBI: A standard for labeling english prosody," in Proc. ICSLP'92, Banff, AB, Canada, Oct. 1992, pp. 867-870.
26. P. Taylor, "The tile intonation model," in Proc. ICSLP'98, Sydney, Australia, Nov. 1998, pp. 1383-1386.
27. J. Teutenberg, C. Watson, and P. Riddle, "Modeling and synthesising F0 contours with the discrete cosine transform," in Proc. ICASSP'08, Las Vegas, NV, Mar. 2008, pp. 3973-3976.
28. J. Latorre and M. Akamine, "Multilevel parametric-based F0 model for speech synthesis," in Proc. Interspeech'08, Brisbane, Australia, Sep. 2008, pp. 2274-2277.
29. S. H. Chen and Y. R. Wang, "Vector quantization of pitch information in Mandarin speech," IEEE Trans. Commun., vol. 38, no. 9, pp. 1317-1320, Sep. 1990.
30. H. Kawahara, "Speech representation and transformation using adaptive interpolation of weighted spectrum: Vocoder revisited," in Proc. ICASSP'97, Munich, Germany, Apr. 1997, vol. 2, pp. 1303-1306.
31. H. Kawahara, I. Masuda-Katsuse, and A. de Cheveigné, "Restructuring speech representations using a pitch adaptive time-frequency-based F0 extraction: Possible role of a repetitive structure in sounds," Speech Commun., vol. 27, no. 3-4, pp. 187-207, 1999.
32. T. Fukada, K. Tokuda, T. Kobayashi, and S. Imai, "An adaptive algorithm for mel-cepstral analysis of speech," in Proc. ICASSP '92, San Francisco, CA, Mar. 1992, vol. 1, pp. 137-140.
33. D. Klein and C. D. Manning, "Fast exact inference with a factored model for natural language parsing," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, pp. 3-10, 15 (NIPS 2002).
34. K. Shinoda and T.Watanabe, "MDL-based context-dependent subword modeling for speech recognition," J. Acoust. Soc. Japan (English), vol. 21, pp. 79-86, Mar. 2000.
35. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. R. Statist. Soc. B, vol. 39, pp. 1-38, 1977.
36. L. H. Cai, D. D. Cui, and R. Cai, "TH-CoSS, aMandarin speech corpus for TTS," J. Chinese Inf. Process., vol. 21, no. 2, pp. 94-99, Mar. 2007.
37. C. Huang, Y. Shi, J. Zhou, M. Chu, T.Wang, and E. Chang, "Segmental tonal modeling for phone set design in Mandarin LVCSR," in Proc. ICASSP'04, Montreal, QC, Canada, May 2004, pp. 901-904.
38. T. Lin and L. J. Wang, Phonetic Tutorials. Beijing, China: Beijing Univ. Press, 1992, pp. 103-121.
39. S. Shott, "Statistics for health professionals," W. B. Sauders Com., 1990.
40. Y.-J. Wu and R.-H. Wang, "Minimum generation error training for HMM-based speech synthesis," in Proc. ICASSP'06, Toulouse, France, May 2006, pp. 89-92.