A deep generative architecture for postfiltering in statistical parametric speech synthesis

IEEE/ACM Transactions on Audio Speech and Language Processing

Volumn 23, Issue 11, 2015, Pages 2003-2014

  Chen, Ling Hui ^a,b   Raitio, Tuomo ^c   Valentini Botinhao, Cassia ^d   Ling, Zhen Hua ^a   Yamagishi, Junichi ^d  

^a UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA   (China)

^b IFLYTEK RESEARCH   (China)

^c AALTO UNIVERSITY   (Finland)

^d UNIVERSITY OF EDINBURGH   (United Kingdom)

Author keywords

Deep generative architecture; Hidden Markov model (HMM); Modulation spectrum; Postfilter; Segmental quality; Speech synthesis

Indexed keywords

ASSOCIATIVE PROCESSING; HIDDEN MARKOV MODELS; MARKOV PROCESSES; MEMORY ARCHITECTURE; MODULATION; NETWORK ARCHITECTURE; PLASMA DIAGNOSTICS; SPEECH SYNTHESIS; STRAIN MEASUREMENT; SYNTHESIS (CHEMICAL);

BI-DIRECTIONAL ASSOCIATIVE MEMORY; CONDITIONAL PROBABILITIES; HIDDEN MARKOV MODEL (HMM)BASED STATISTICAL PARAMETRIC SPEECH SYNTHESIS; MODULATION SPECTRUM; POSTFILTERS; RESTRICTED BOLTZMANN MACHINE; STATISTICAL PARAMETRIC SPEECH SYNTHESIS; SUBJECTIVE EVALUATIONS;

SPEECH;

EID: 84942607168     PISSN: 23299290     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASLP.2015.2461448     Document Type: Article

References (43)

1. L.-H. Chen, T. Raitio, C. Valentini-Botinhao, J. Yamagishi, and Z.-H. Ling, "DNN-based stochastic postfilter for HMM-based speech synthesis", in Proc. Interspeech, 2014, pp. 1954-1958.
2. H. Zen, K. Tokuda, and A. W. Black, "Statistical parametric speech synthesis", Speech Commun., vol. 51, no. 11, pp. 1039-1064, 2009.
3. A. Black and K. Tokuda, "The blizzard challenge 2005: Evaluating corpus-based speech synthesis on common databases", in Proc. Interspeech, 2005.
4. Z.-H. Ling, S.-Y. Kang, H. Zen, A. Senior, M. Schuster, X.-J. Qian, H. Meng, and L. Deng, "Deep learning for acoustic modeling in parametric speech generation: A systematic review of existing techniques and future trends", IEEE Signal Process. Mag., vol. 32, no. 3, pp. 35-52, May 2015.
5. H. Zen, A. Senior, and M. Schuster, "Statistical parametric speech synthesis using deep neural networks", in Proc. ICASSP, 2013, pp. 7962-7966.
6. Z.-H. Ling, L. Deng, and D. Yu, "Modeling spectral envelopes using restricted Boltzmann machines and deep belief networks for statistical parametric speech synthesis", IEEE Trans. Audio, Speech, Lang. Process., vol. 21, no. 10, pp. 2129-2139, Oct. 2013.
7. S. Kang, X. Qian, and H. Meng, "Multi-distribution deep belief network for speech synthesis", in Proc. ICASSP, Apr. 2013, pp. 8012-8016.
8. R. Fernandez, A. Rendel, B. Ramabhadran, and R. Hoory, "F0 contour prediction with a deep belief network-gaussian process hybrid model", in Proc. ICASSP, Apr. 2013, pp. 6885-6889.
9. R. Vishnubhotla, S. Fernandez, and B. Ramabhadran, "An autoencoder neural-network based low-dimensionality approach to excitation modeling for HMM-based text-to-speech", in Proc. ICASSP, 2010, pp. 4614-4617.
10. R. Fernandez, A. Rendel, B. Ramabhadran, and R. Hoory, "Prosody contour prediction with long short-term memory, bi-directional, deep recurrent neural networks", in Proc. Interspeech, 2014, pp. 2268-2272.
11. Y. Fan, Y. Qian, F. Xie, and F. K. Soong, "TTS synthesis with bidirectional LSTM based recurrent neural networks", in Proc. Interspeech, 2014, pp. 1964-1968.
12. H. Zen and H. Sak, "Unidirectional long short-term memory recurrent neural network with recurrent output layer for low-latency speech synthesis", in Proc. ICASSP, Apr. 2015, pp. 4470-4474.
13. Z. Ling, Y. Wu, Y. Wang, L. Qin, and R. Wang, "USTC system for blizzard challenge 2006an improved HMM-based speech synthesis method", in Proc. Blizzard Challenge Workshop, Pittsburgh, PA, USA, Sep. 2006.
14. T. Yoshimura, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura, "Incorporating a mixed excitation model and postfilter into HMMbased text-to-speech synthesis", Syst. Comput. Jpn., vol. 36, no. 12, pp. 43-50, 2005.
15. T. Toda and K. Tokuda, "A speech parameter generation algorithm considering global variance for HMM-based speech synthesis", IEICE Trans. Inf. Syst., vol. E90-D, no. 5, pp. 816-824, 2007.
16. S. Takamichi, T. Toda, G. Neubig, S. Sakti, and S. Nakamura, "A postfilter to modify the modulation spectrum in HMM-based speech synthesis", in Proc. ICASSP, May 2014, pp. 290-294.
17. T. Toda, A. Black, and K. Tokuda, "Voice conversion based on maximumlikelihood estimation of spectral parameter trajectory", IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 8, pp. 2222-2235, Nov. 2007.
18. A. Kain and M. Macon, "Spectral voice conversion for text-to-speech synthesis", in Proc. ICASSP, 1998, pp. 285-288.
19. K. Koishida, K. Tokuda, T. Kobayashi, and S. Imai, "CELP coding based on mel-cepstral analysis", in Proc. ICASSP, 1995, vol. 1, pp. 33-36.
20. Z.-H. Ling, Y. Hu, and L.-R. Dai, "Global variance modeling on the log power spectrum of LSPs for HMM-based speech synthesis", in Proc. Interspeech, 2010, pp. 825-828.
21. K. Tokuda, Y. Nankaku, T. Toda, H. Zen, J. Yamagishi, and K. Oura, "Speech synthesis based on hidden Markov models", Proc. IEEE, vol. 101, no. 5, pp. 1234-1252, May 2013.
22. T. Fukada, K. Tokuda, T. Kobayashi, and S. Imai, "An adaptive algorithm for mel-cepstral analysis of speech", in Proc. ICASSP, San Francisco, CA, USA, Mar. 1992, vol. 1, pp. 137-140.
23. S. Desai, A. Black, B. Yegnanarayana, and K. Prahallad, "Spectral mapping using artificial neural networks for voice conversion", IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 5, pp. 954-964, Jul. 2010.
24. L.-H. Chen, Z.-H. Ling, and L.-R. Dai, "Voice conversion using generative trained deep neural networks with multiple frame spectral envelopes", in Proc. Interspeech, 2014, pp. 2313-2317.
25. L.-H. Chen, Z.-H. Ling, L.-J. Liu, and L.-R. Dai, "Voice conversion using deep neural networks with layer-wise generative training", IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 22, no. 12, pp. 1859-1872, Dec. 2014.
26. P. Smolensky, "Information processing in dynamical systems: Foundations of harmony theory", in Parallel distributed processing: Explorations in the microstructure of cognition, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA, USA: MIT Press, 1986, vol. 1, ch. 6, pp. 194-281.
27. B. Kosko, "Bidirectional associative memories", IEEE Trans. Systems, Man, Cybern., vol. 18, no. 1, pp. 49-60, 1988.
28. G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks", Science, vol. 313, no. 5786, pp. 504-507, 2006.
29. G. Hinton, "Training products of experts by minimizing contrastive divergence", Neural Comput., vol. 12, no. 14, pp. 1711-1800, 2002.
30. G. E. Hinton, "A practical guide to training restricted Boltzmann machines", in Neural Networks: Tricks of the Trade. New York, NY, USA: Springer, 2012, pp. 599-619.
31. R. Salakhutdinov, "Learning deep generative models", Ph. D. dissertation, Univ. of Toronto, Toronto, ON, Canada, 2009.
32. J. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities", in Proc. Nat. Acad. Sci., 1982, vol. 79, no. 8, pp. 2554-2558.
33. L.-J. Liu, L.-H. Chen, Z.-H. Ling, and L.-R. Dai, "Using bidirectional associative memories for joint spectral envelope modeling in voice conversion", in Proc. ICASSP, May 2014, pp. 7884-7888.
34. L. Deng, M. Seltzer, D. Yu, A. Acero, A. Rahman Mohamed, and G. Hinton, "Binary coding of speech spectrograms using a deep auto-encoder", in Proc. Interspeech, Sep. 2010.
35. Y. Bengio, "Learning deep architectures for AI", Foundations Trends Mach. Learn., vol. 2, no. 1, pp. 1-127, 2009.
36. G. Hinton, "Products of experts", in Proc. 9th Int. Conf. Artif. Neural Netw., 1999, vol. 1, pp. 825-828.
37. K. Tokuda, T. Yoshimura, T. Masuko, T. Kobayashi, and T. Kitamura, "Speech parameter generation algorithms for HMM-based speech synthesis", in Proc. ICASSP, 2000, vol. 3, pp. 1315-1318.
38. H. Kawahara, I. Masuda-Katsuse, and A. Cheveigné, "Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based F0 extraction: Possible role of a repetitive structure in sounds", Speech Commun., vol. 27, pp. 187-207, 1999.
39. K. Tokuda, T. Kobayashi, T. Masuko, and S. Imai, "Mel-generalized cepstral analysis-A unified approach to speech spectral estimation", in Proc. ICSLP, Yokohama, Japan, Sep. 1994, vol. 3, pp. 1043-1046.
40. "Method for the subjective assessment of intermediate quality level of coding systems", ITU Rec. ITU-R BS.1534-1, Int. Telecomm. Union Radiocommunication Assembly. Geneva, Switzerland, Mar. 2003.
41. IEEE, "IEEE recommended practice for speech quality measurement", IEEE Trans. Audio Electroacoust., vol. AE-17, no. 3, pp. 225-246, Sep. 1969.
42. L.-H. Chen, Z.-H. Ling, Y.-Q. Zu, R.-Q. Yan, Y. Jiang, X.-J. Xia, and Y. Wang, "The USTC system for blizzard challenge 2014", in Proc. Blizzard Challenge Workshop, Singapore, Sep. 2014.
43. J. Anden and S. Mallat, "Deep scattering spectrum", IEEE Trans. Signal Process., vol. 62, no. 16, pp. 4114-4128, Aug. 2014.