Personalized spectral and prosody conversion using frame-based codeword distribution and adaptive CRF

IEEE Transactions on Audio, Speech and Language Processing

Volumn 21, Issue 1, 2013, Pages 51-62

  Huang, Yi Chin ^a   Wu, Chung Hsien ^a   Chao, Yu Ting ^a  

^a NATIONAL CHENG KUNG UNIVERSITY   (Taiwan)

Author keywords

Conditional random field; frame alignment; principal component analysis; prosodic boundary; voice conversion

Indexed keywords

ALIGNMENT; FUNCTION EVALUATION; IMAGE SEGMENTATION; PRINCIPAL COMPONENT ANALYSIS; RANDOM PROCESSES; SPEECH COMMUNICATION; TELEPHONE SETS;

CONDITIONAL RANDOM FIELD; CONVERSION FUNCTION; DISTANCE ESTIMATION; FRAME ALIGNMENTS; PROSODIC BOUNDARY; SPECTRAL PROPERTIES; VOICE CHARACTERISTICS; VOICE CONVERSION;

ARTIFICIAL INTELLIGENCE;

EID: 84867950508     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2012.2213247     Document Type: Article

References (35)

1. J. Yamagishi, T. Kobayashi, S. Renals, S. King, H. Zen, T. Toda, and K. Tokuda, "Improved average-voice-based speech synthesis using gender-mixed modeling and a parameter generation algorithm considering GV," in Proc. ISCA SSW6, Aug. 2007.
2. J. Yamagishi, T. Kobayashi, Y. Nakano, K. Ogata, and J. Isogai, "Analysis of speaker adaptation algorithms for HMM-based speech synthesis and a constrained SMAPLR adaptation algorithm," IEEE Trans. Audio, Speech, Lang. Process., vol. 17, no. 1, pp. 66-83, Jan. 2009.
3. Y. Stylianou, O. Cappé, and E. Moulines, "Continuous probabilistic transform for voice conversion," IEEE Trans. Speech Audio Process., vol. 6, no. 2, pp. 131-142, Mar. 1998. (Pubitemid 128720639)
4. O. Watts, J. Yamagishi, S. King, and K. Berkling, "Synthesis of child speech with HMM adaptation and voice conversion," IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 6, pp. 1005-1016, Aug. 2010.
5. A. Kain and M. W. Macon, "Spectral voice conversion for text-tospeech synthesis," in Proc. ICASSP'98, May 12-15, 1998, vol. 1, pp. 285-288, vol. 1.
6. H. Duxans, A. Bonafonte, A. Kain, and J. van Santen, "Including dynamic and phonetic information in voice conversion systems," in Proc. ICSLP '04, Jeju Island, South Korea, 2004, pp. 5-8.
7. T. Toda, A. W. Black, and K. Tokuda, "Voice conversion based on maximum likelihood estimation of spectral parameter trajectory," IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 8, pp. 2222-2235, Nov. 2007.
8. Y. Ohtani, T. Toda, H. Saruwatari, and K. Shikano, "Adaptive training for voice conversion based on eigenvoices," IEICE Trans. Inf. Syst., vol. E93-D, no. 6, pp. 1589-1598, Jun. 2010.
9. C.-C. Hsia, C.-H. Wu, and J.-Q. Wu, "Conversion function clustering and selection using linguistic and spectral information for emotional voice conversion," IEEE Trans. Comput., vol. 56, no. 9, pp. 1225-1233, Sep. 2007.
10. C.-H. Wu, C.-C. Hsia, T.-H. Liu, and J.-F. Wang, "Voice conversion using duration-embedded Bi-HMMs for expressive speech synthesis," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 4, pp. 1109-1116, Jul. 2006. (Pubitemid 46547608)
11. K. Shikano, S. Nakamura, and M. Abe, "Speaker adaptation and voice conversion by codebook mapping," in Proc. IEEE Int. Symp. Circuits Syst., Jun. 11-14, 1991, vol. 1, pp. 594-597, vol., no.
12. S. Desai, A. W. Black, B. Yegnanarayana, and K. Prahallad, "Spectral mapping using artificial neural networks for voice conversion," IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 5, pp. 954-964, Jul. 2010.
13. D. Sundermann, H. Ney, and H. Hoge, "VTLN-based cross-language voice conversion," in Proc. IEEE Workshop on ASRU'03, 30 Nov.-3 Dec. 2003, pp. 676-681.
14. F. Malfrere, O. Deroo, and T. Dutoit, "Phonetic Alignment: Speech Synthesis vs. Hybrid HMM/ANN," in Proc. ICSLP'98, Sydney, Australia, Dec. 1998, vol. 4, p. 1571.
15. N. Campbell, "Autolabelling Japanese ToBI," in Proc. ICSLP'96, Philadelphia, PA, Oct. 1996.
16. C.-H. Wu, C.-C. Hsia, C.-H. Lee, and M.-C. Lin, "Hierarchical prosody conversion using regression-based clustering for emotional speech synthesis," IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 6, pp. 1394-1405, Aug. 2010.
17. C.-C. Hsia, C.-H. Wu, and J.-Y. Wu, "Exploiting prosody hierarchy and dynamic features for pitch modeling and generation in HMM-based speech synthesis," IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 8, pp. 1994-2003, Nov. 2010.
18. C.-Y. Tseng, S.-H. Pin, Y.-L. Lee, H. M. Wang, and Y. C. Chen, "Fluent Speech Prosody: Framework and Modeling," Speech Commun., Spec. Iss. Quantitative Prosody Modeling for Natural Speech Description and Generation, vol. 46, no. 3-4, pp. 284-309, 2005. (Pubitemid 40952517)
19. S.-H. Chen, W.-H. Lai, and Y.-R. Wang, "A statistics-based pitch contour model for mandarin speech," J. Acoust. Soc. Amer., vol. 117, no. 2, pp. 908-925, 2005. (Pubitemid 40223449)
20. C. Huang, Y. Shi, J. L. Zhou, M. Chu, T. Wang, and E. Chang, "Segmental tonal modeling for phone set design in mandarin LVCSR," in Proc. ICASSP'04, 2004, pp. 901-904.
21. H. Kawahara, "Speech representation and transformation using adaptive interpolation of weighted spectrum: Vocoder revisited," in Proc. ICASSP'97, Munich, Germany, 1997, pp. 1303-1306.
22. M. Chu and Y. Qian, "Locating boundaries for prosodic constituents in unrestricted mandarin texts," Comput. Linguist. Chinese Lang. Process., vol. 6, no. 1, pp. 61-82, 2001.
23. L.-S. Lee, C.-Y. Tseng, and M. Ouh-young, "The synthesis rules in a Chinese text-to-speech system," IEEE Trans. Acoust., Speech, Signal Process., vol. 37, no. 9, pp. 1309-1319, Sep. 1989.
24. A. Benijamin, S. Chilin, and S. Richard, "A corpus-based Mandarin text-to-speech synthesizer," in Proc. ICSLP, 1994, vol. S29, no. 8. 1-8. 4, pp. 1771-1774.
25. Wagner and Agnieszka, "Analysis and recognition of accentual patterns," in Proc. Interspeech'09, 2009, pp. 2427-2430, (2009).
26. L. Andrej and F. Frank, "Synthesis of natural sounding pitch contours in isolated utterances using hidden Markov models," IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-34, no. 5, pp. 1074-1080, Oct. 1986.
27. N.-H. Pan, W.-T. Jen, S.-S. Yu, S.-Y. Huang, and M.-J. Wu, "Prosody model in a Mandarin text-to-speech system based on a hierarchical approach," in Proc. IEEE Int. Conf. Multimedia and Expo, 2000, vol. 1, pp. 448-451. (Pubitemid 33058980)
28. M. Dong and K.-T. Lua, "Pitch Contour Model for Chinese text-tospeech using CART and statistical model," in Proc. ICSLP, 2002, pp. 2405-2408.
29. J. Lafferty, A. McCallum, and F. Pereira, "Conditional random fields: Probabilistic models for segmenting and labeling sequence data," in Proc. Int. Conf. Mach. Learn., 2001.
30. Q. Zhang, X. Qiu, X. Huang, and L. Wu, "Domain Adaptation for Conditional Random Fields," in Information Retrieval Technology. New York: Springer, 2008.
31. D. C. Liu and J. Nocedal, "On the limited memory BFGS method for large scale optimization,"Math.Programming, ser. B, vol. 45, no. 3, pp. 503-528, 1989. (Pubitemid 20660315)
32. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. R. Statist. Soc. B, vol. 39, pp. 1-38, 1977.
33. Z. H. Ling, K. Richmond, J. Yamagishi, and R. H. Wang, "Articulatory control of HMM-based parametric speech synthesis driven by phonetic knowledge," in Proc. Interspeech'08, Brisbane, Australia, Sep. 2008, pp. 573-576.
34. L. H. Cai, D. D. Cui, and R. Cai, "TH-CoSS, aMandarin speech corpus for TTS," J. Chinese Inf. Process., vol. 21, no. 2, pp. 94-99, Mar. 2007.
35. H. Zen, T. Nose, J. Yamagishi, S. Sako, and K. Tokuda, The HMM-based Speech Synthesis System (HTS) Version 2. 0 2007 [Online]. Available: http://hts. sp. nitech. ac. jp/