Modeling F0 trajectories in hierarchically structured deep neural networks

Speech Communication

Volumn 76, Issue , 2016, Pages 82-92

  Yin, Xiang ^a,b   Lei, Ming ^b   Qian, Yao ^b   Soong, Frank K ^b   He, Lei ^b   Ling, Zhen Hua ^a   Dai, Li Rong ^a  

^a UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA   (China)

^b MICROSOFT   (United States)

Author keywords

Deep neural network; Discrete cosine transform; Fundamental frequency; Hidden Markov model; Speech synthesis

Indexed keywords

DISCRETE COSINE TRANSFORMS; HIDDEN MARKOV MODELS; MARKOV PROCESSES; MODEL STRUCTURES; SPEECH SYNTHESIS; SYNTHESIS (CHEMICAL);

CASCADE STRUCTURES; CONTEXTUAL FEATURE; DEEP NEURAL NETWORKS; DISCRETE COSINE TRANSFORM(DCT); FUNDAMENTAL FREQUENCIES; PARALLEL STRUCTURES; STATISTICAL PARAMETRIC SPEECH SYNTHESIS; SUBJECTIVE PERFORMANCE;

COMPLEX NETWORKS;

EID: 84950159800     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2015.10.007     Document Type: Article

References (35)

1. Chen C.J., Gopinath R.A., Monkowski M.D., Picheny M.A., and Shen K. New methods in continuous mandarin speech recognition Eurospeech 1997 1543 1546
2. Fan Y.C., Qian Y., and Soong F.K. TTS synthesis with bidirectional LSTM based recurrent neural networks Interspeech 2014 1964 1968
3. Fernandez R., Rendel A., Ramabhadran B., and Hoory R. Prosody contour prediction with long short-term memory, bi-directional, deep recurrent neural networks. Interspeech 2014 2268 2272
4. Fujisaki H. In search of models in speech communication research. Interspeech 2008 1 10
5. Kawahara H., Masuda-Katsuse I., and de Cheveigné A. Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based F0 extraction: possible role of a repetitive structure in sounds Speech Commun. 27 3 1999 187 208
6. Latorre J., and Akamine M. Multilevel parametric-base F0 model for speech synthesis Proceedings of Interspeech 2008 2274 2277
7. Lei M., Wu Y.J., Soong F.K., Ling Z.H., and Dai L.R. A hierarchical F0 modeling method for HMM-based speech synthesis. Interspeech 2010 2170 2173
8. Ling Z.H., Li D., and Yu D. Modeling spectral envelopes using restricted Boltzmann machines and deep belief networks for statistical parametric speech synthesis IEEE Trans. Audio, Speech Language Proc. 21 10 2013 2129 2139
9. Ling Z.-H., Wu Y.J., Wang Y.P., Qin L., and Wang R.H. USTC system for Blizzard Challenge 2006: an improved HMM-based speech synthesis method Proceedings of Blizzard Challenge Workshop 2006
10. Makhoul J. A fast cosine transform in one and two dimensions IEEE Trans. Acoustics, Speech Signal Proc. 28 1 1980 27 34
11. Obin N., Lacheret A., and Rodet X. Stylization and trajectory modelling of short and long term speech prosody variations Proceedings of Interspeech 2011 2029 2032
12. Qian Y., Fan Y.C., Hu W.-P., and Soong F.K. On the training aspects of deep neural network (DNN) for parametric TTS synthesis Proceedings of ICASSP 2014 3857 3861
13. Qian Y., Liang H., and Soong F.K. Generating natural F0 trajectory with additive trees. Proceedings of Interspeech 2008 2126 2129
14. Qian Y., Wu Z., Gao B., and Soong F.K. Improved prosody generation by maximizing joint probability of state and longer units IEEE Trans. Audio, Speech, Language Proc. 19 6 2011 1702 1710
15. Shinoda K., and Watanabe T. MDL-based context-dependent sub-word modeling for speech recognition J. Acoust. Soc. Jpn(E) 21 2 2000 79 86
16. Talkin D. A robust algorithm for pitch tracking (RAPT) Speech Coding Synthesis 1995 495 518
17. Teutenberg J., Watson C., and Riddle P. Modelling and synthesising F0 contours with the discrete cosine transform Proceedings of ICASSP 2008 3973 3976
18. Toda T., and Tokuda K. Speech parameter generation algorithm considering global variance for HMM-based speech synthesis Proceedings of Eurospeech 2005 1315 1318
19. Tokuda K., Kobayashi T., and Imai S. Speech parameter generation from HMM using dynamic features Proceedings of ICASSP 1995 660 663
20. Tokuda K., Masuko T., Miyazaki N., and Kobayashi T. Hidden markov models based on multi-space probability distribution for pitch pattern modeling Proceedings of ICASSP 1999 229 232
21. Tokuda K., Yoshimura T., Masuko T., Kobayashi T., and Kitamura T. Speech parameter generation algorithms for HMM-based speech synthesis Proceedings of ICASSP 2000 1315 1318
22. Wang C.C., Ling Z.H., Zhang B.F., and Dai L.R. Multi-layer F0 modeling for HMM-based speech synthesis Proceedings of ISCSLP 2008 129 132
23. Wightman C., Price P., Pierrehumbert J., and Hirschberg J. Tobi: a standard for labeling english prosody Proceedings of ICSLP 1992 12 16
24. Wu Y.J., and Soong F.K. Modeling pitch trajectory by hierarchical HMM with minimum generation error training Proceedings of ICASSP 2012 4017 4020 10.1109/ICASSP.2012.6288799
25. Wu Y.J., and Wang R.H. Minimum generation error training for HMM-based speech synthesis Proceedings of ICASSP 2006 89 92 10.1109/ICASSP.2006.1659964
26. Wu Y.J., Wang R.H., and Soong F.K. Full HMM training for minimizing generation error in synthesis ICASSP 2007 517 520
27. Wu Z., Qian Y., Soong F.K., and Zhang B. Modeling and generating tone contour with phrase intonation for mandarin chinese speech Proceedings of ISCSLP 2008 1 4 10.1109/CHINSL.2008.ECP.42
28. Yin X., Lei M., Qian Y., Soong F.-K., He L., Ling Z.H., and Dai L.R. Modeling DCT parameterized F0 trajectory at intonation phrase level with DNN or decision tree Proceedings of Interspeech 2014 2273 2277
29. Yoshimura T., Tokuda K., Masuko T., Kobayashi T., and Kitamura T. Simultaneous modeling of spectrum, pitch and duration in HMM-based speech synthesis Proceedings of Eurospeech 6 1999 2347 2350
30. Zen H., and Braunschweiler N. Context-dependent additive log F0 model for HMM-based speech synthesis. Proceedings of Interspeech 2009 2091 2094
31. Zen H., Sak H., Graves A., and Senior A. Statistical parametric speech synthesis based on recurrent neural networks Proceedings of Conference on UKSpeech 2014
32. Zen H., Senior A., and Schuster M. Statistical parametric speech synthesis using deep neural networks Proceedings of ICASSP 2013 7962 7966
33. Zen H., Toda T., Nakamura M., and Tokuda K. Details of Nitech HMM-based speech synthesis system for the Blizzard Challenge 2005 IEICE Trans. Inf. Syst. E90-D 1 2007 325 333
34. Zen H., Tokuda K., and Black A. Statistical parametric speech synthesis Speech Commun. 51 2009 1039 1064 10.1016/j.specom.2009.04.004
35. Zen H., Tokuda K., and Kitamura T. Reformulating the HMM as a trajectory model by imposing explicit relationships between static and dynamic feature vector sequences Comput. Speech Language 21 1 2007 153 173