A template-based approach for speech synthesis intonation generation using LSTMs

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn 08-12-September-2016, Issue , 2016, Pages 2463-2467

  Ronanki, Srikanth ^a   Henter, Gustav Eje ^a   Wu, Zhizheng ^a   King, Simon ^a  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

Author keywords

CTC; F0 templates; Intonation modelling; LSTM; Speech synthesis

Indexed keywords

FORECASTING; RECURRENT NEURAL NETWORKS; SPEECH; SPEECH PROCESSING; SPEECH SYNTHESIS;

F0 TEMPLATES; FUNDAMENTAL FREQUENCIES; INTONATION GENERATIONS; LSTM; RECURRENT NEURAL NETWORK (RNN); SPEECH SYNTHESIS SYSTEM; TEMPLATE-BASED APPROACHES; TEMPORAL CLASSIFICATION;

SPEECH COMMUNICATION;

EID: 84994213378     PISSN: 2308457X     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: 10.21437/Interspeech.2016-96     Document Type: Conference Paper

References (29)

1. S. King, "Measuring a decade of progress in text-to-speech," Loquens, vol. 1, no. 1, 2014.
2. H. Zen, T. Nose, J. Yamagishi, S. Sako, T. Masuko, A. Black, and K. Tokuda, "The HMM-based speech synthesis system (HTS) version 2.0," in Proc. SSW, vol. 6, 2007, pp. 294-299.
3. K. Tokuda, T. Yoshimura, T. Masuko, T. Kobayashi, and T. Kitamura, "Speech parameter generation algorithms for HMM-based speech synthesis," in Proc. ICASSP, 2000, pp. 1315-1318.
4. T. Yoshimura, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura, "Simultaneous modeling of spectrum, pitch and duration in HMM-based speech synthesis," in Proc. Eurospeech, 1999, pp. 2347-2350.
5. K. Yu and S. Young, "Continuous F0 modeling for HMM based statistical parametric speech synthesis," IEEE T. Audio Speech, vol. 19, no. 5, pp. 1071-1079, 2011.
6. H. Zen, A. Senior, and M. Schuster, "Statistical parametric speech synthesis using deep neural networks," in Proc. ICASSP, 2013, pp. 7962-7966.
7. O. Watts, G. E. Henter, T. Merritt, Z. Wu, and S. King, "From HMMs to DNNs: where do the improvements come from?" in Proc. ICASSP, 2016, pp. 5505-5509.
8. H. Zen and H. Sak, "Unidirectional long short-term memory recurrent neural network with recurrent output layer for low-latency speech synthesis," in Proc. ICASSP, 2015, pp. 4470-4474.
9. Z. Wu and S. King, "Investigating gated recurrent networks for speech synthesis," in Proc. ICASSP, 2016, pp. 5140-5144.
10. R. Fernandez, A. Rendel, B. Ramabhadran, and R. Hoory, "Prosody contour prediction with long short-term memory, bidirectional, deep recurrent neural networks," in Proc. Interspeech, 2014, pp. 2268-2272.
11. X. Yin, M. Lei, Y. Qian, F. K. Soong, L. He, Z.-H. Ling, and L.-R. Dai, "Modeling F0 trajectories in hierarchically structured deep neural networks," Speech Commun., vol. 76, pp. 82-92, 2016.
12. N. Obin, J. Beliao, C. Veaux, and A. Lacheret, "SLAM: Automatic stylization and labelling of speech melody," in Proc. Speech Prosody, 2014, pp. 246-250.
13. R. Dall and X. Gonzalvo, "JNDSLAM: A SLAM extension for speech synthesis," in Proc. Speech Prosody, 2016, pp. 1024-1028.
14. G. K. Anumanchipalli, L. C. Oliveira, and A. W. Black, "A statistical phrase/accent model for intonation modeling," in Proc. Interspeech, 2011, pp. 1813-1816.
15. A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber, "Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks," in Proc. ICML, 2006, pp. 369-376.
16. J. Latorre and M. Akamine, "Multilevel parametric-base F0 model for speech synthesis," in Proc. Interspeech, 2008, pp. 2274-2277.
17. J. Teutenberg, C. Watson, and P. Riddle, "Modelling and synthesising F0 contours with the discrete cosine transform," in Proc. ICASSP, 2008, pp. 3973-3976.
18. Y. Qian, Z.Wu, B. Gao, and F. K. Soong, "Improved prosody generation by maximizing joint probability of state and longer units," IEEE T. Audio Speech, vol. 19, no. 6, pp. 1702-1710, 2011.
19. A. Suni, D. Aalto, T. Raitio, P. Alku, and M. Vainio, "Wavelets for intonation modeling inHMMspeech synthesis," in Proc. SSW, vol. 8, 2013, pp. 285-290.
20. M. S. Ribeiro and R. A. J. Clark, "A multi-level representation of f0 using the continuous wavelet transform and the discrete cosine transform," in Proc. ICASSP, 2015, pp. 4909-4913.
21. X. Yin, M. Lei, Y. Qian, F. K. Soong, L. He, Z.-H. Ling, and L.-R. Dai, "Modeling DCT parameterized F0 trajectory at intonation phrase level with DNN or decision tree," in Proc. Interspeech, 2014, pp. 2273-2277.
22. J. O. Smith III and J. S. Abel, "Bark and ERB bilinear transforms," IEEE T. Speech Audi. P., vol. 7, no. 6, pp. 697-708, 1999.
23. S. C. Johnson, "Hierarchical clustering schemes," Psychometrika, vol. 32, no. 3, pp. 241-254, 1967.
24. K. Prahallad, A.W. Black, and R. Mosur, "Sub-phonetic modeling for capturing pronunciation variations for conversational speech synthesis," in Proc. ICASSP, 2006, pp. I-853-I-856.
25. Z. Wu, C. Valentini-Botinhao, O. Watts, and S. King, "Deep neural networks employing multi-task learning and stacked bottleneck features for speech synthesis," in Proc. ICASSP, 2015, pp. 4460-4464.
26. H. Kawahara, "STRAIGHT, exploitation of the other aspect of VOCODER: Perceptually isomorphic decomposition of speech sounds," Acoust. Sci. Technol., vol. 27, no. 6, pp. 349-353, 2006.
27. Objective measurement of active speech level, ITU Recommendation ITU-T P.56, International Telecommunication Union, Telecommunication Standardization Sector, Geneva, Switzerland, March 2011.
28. Method for the subjective assessment of intermediate quality levels of coding systems, ITU Recommendation ITU-R BS.1534- 3, International Telecommunication Union Radiocommunication Sector, Geneva, Switzerland, October 2015.
29. S. Holm, "A simple sequentially rejective multiple test procedure," Scand. J. Stat., vol. 6, no. 2, pp. 65-70, 1979.