HMM-based speech synthesis utilizing glottal inverse filtering

IEEE Transactions on Audio, Speech and Language Processing

Volumn 19, Issue 1, 2011, Pages 153-165

  Raitio, Tuomo ^a   Suni, Antti ^b   Yamagishi, Junichi ^c   Pulakka, Hannu ^a   Nurminen, Jani ^d   Vainio, Martti ^b   Alku, Paavo ^a  

^a AALTO UNIVERSITY   (Finland)

^b UNIVERSITY OF HELSINKI   (Finland)

^c UNIVERSITY OF EDINBURGH   (United Kingdom)

^d MICROSOFT   (United States)

Author keywords

Glottal inverse filtering; hidden Markov model (HMM); speech synthesis

Indexed keywords

EXCITATION SIGNALS; GLOTTAL FLOW; GLOTTAL SOURCE; HIDDEN MARKOV MODEL (HMM); HMM-BASED SPEECH SYNTHESIS; INVERSE FILTERING; SOURCE SIGNALS; SPECTRAL FEATURE; SPEECH SYNTHESIZER; SYNTHESIS STAGES; SYNTHETIC SPEECH; TEXT INPUT; VOCAL-TRACTS; VOICE SOURCES;

HIDDEN MARKOV MODELS;

SPEECH SYNTHESIS;

EID: 77957744515     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2010.2045239     Document Type: Article

References (85)

1. T. Yoshimura, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura, "Simultaneous modeling of spectrum, pitch and duration in HMM-based speech synthesis", in Proc. Eurospeech, Sep. 1999, pp. 2374-2350.
2. K. Tokuda, T. Masuko, T. Yamada, T. Kobayashi, and S. Imai, "An algorithm for speech parameter generation from continuous mixture HMMs with dynamic features", in Proc. Eurospeech, 1995, vol. 1, pp. 757-760.
3. K. Tokuda, H. Zen, and A. W. Black, "An HMM-based speech synthesis system applied to English", in Proc. IEEE Workshop Speech Synth., Sep. 2002, pp. 227-230.
4. H. Zen, K. Tokuda, and A. W. Black, "Statistical parametric speech synthesis", Speech Commun., vol. 51, no. 11, pp. 1039-1064, Nov. 2009.
5. J. Yamagishi, T. Kobayashi, Y. Nakano, K. Ogata, and J. Isogai, "Analysis of speaker adaptation algorithms for HMM-based speech synthesis and a constrained SMAPLR adaptation algorithm", IEEE Trans. Audio, Speech, Lang. Process., vol. 17, no. 1, pp. 66-83, Jan. 2009.
6. M. Tachibana, J. Yamagishi, T. Masuko, and T. Kobayashi, "Speech synthesis with various emotional expressions and speaking styles by style interpolation and morphing", IEICE Trans. Inf. Syst., vol. E88-D, no. 11, pp. 2484-2491, Nov. 2005. (Pubitemid 41816793)
7. Y.-J. Wu and R.-H. Wang, "Minimum generation error training for HMM-based speech synthesis", in Proc. ICASSP, 2006, vol. 1, pp. 889-892.
8. T. Toda and K. Tokuda, "A speech parameter generation algorithm considering global variance for HMM-based speech synthesis", IEICE Trans. Inf. Syst., vol. E90-D, no. 5, pp. 816-824, May 2007.
9. G. Fant, Acoustic Theory of Speech Production. The Hague, The Netherlands: Mouton, 1960.
10. D. H. Klatt and L. C. Klatt, "Analysis, synthesis, and perception of voice quality variations among female and male talkers", J. Acoust. Soc. Amer., vol. 87, no. 2, pp. 820-857, Feb. 1990.
11. D. G. Childers and C. K. Lee, "Vocal quality factors: Analysis, synthesis, and perception", J. Acoust. Soc. Amer., vol. 90, no. 5, pp. 2394-2410, Nov. 1991.
12. T. E. Tremain, "The government standard linear predictive coding algorithm: LPC-10", Speech Technol., vol. 1, pp. 40-49, Apr. 1982.
13. L. R. Rabiner and R. W. Schafer, Digital Processing of Speech Signals. Englewood Cliffs, NJ: Prentice-Hall, 1978.
14. T. Yoshimura, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura, "Mixed excitation for HMM-based speech synthesis", in Proc. Eurospeech, 2001, pp. 2259-2262.
15. R. Maia, T. Toda, H. Zen, Y. Nankaku, and K. Tokuda, "An excitation model for HMM-based speech synthesis based on residual modeling", in Proc. 6th ISCA Workshop Speech Synth., Aug. 2007.
16. S. J. Kim and M. Hahn, "Two-band excitation for HMM-based speech synthesis", IEICE Trans. Inf. Syst., vol. E90-D, 2007.
17. G. Fant, J. Liljencrants, and Q. Lin, "A four-parameter model of glottal flow", STL-QPSR, vol. 4, pp. 1-13, 1985.
18. R. Carlson, G. Fant, C. Gobl, B. Granström, I. Karlsson, and Q. Lin, "Voice source rules for text-to-speech synthesis", in Proc. ICASSP, May 1989, vol. 1, pp. 223-226.
19. R. Carlson, B. Granström, and I. Karlsson, "Experiments with voice modelling in speech synthesis", Speech Commun., vol. 10, pp. 481-489, 1991.
20. J. Cabral, S. Renalds, K. Richmond, and J. Yamagishi, "Towards an improved modeling of the glottal source in statistical parametric speech synthesis", in Proc. 6th ISCA Workshop Speech Synth., Aug. 2007, pp. 113-118.
21. J. Cabral, S. Renalds, K. Richmond, and J. Yamagishi, "Glottal spectral separation for parametric speech synthesis", in Proc. Interspeech, 2008, pp. 1829-1832.
22. J. Holmes, "The influence of glottal waveform on the naturalness of speech from a parallel formant synthesizer", IEEE Trans. Audio Electroacoust., vol. AES-21, no. 3, pp. 298-305, Jun. 1973.
23. K. Matsui, S. D. Pearson, K. Hata, and T. Kamai, "Improving naturalness in text-to-speech synthesis using natural glottal source", in Proc. ICASSP, Apr. 1991, vol. 2, pp. 769-772.
24. P. Alku, H. Tiitinen, and R. Näätänen, "A method for generating natural-sounding speech stimuli for cognitive brain research", Clinical Neurophysiol., vol. 110, pp. 1329-1333, 1999.
25. H. Kawahara, I. Masuda-Katsuse, and A. de Cheveigné, "Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based F0 extraction: Possible role of a repetitive structure in sounds", Speech Commun., vol. 27, Apr. 1999.
26. H. Zen, T. Toda, M. Nakamura, and K. Tokuda, "Details of the Nitech HMM-based speech synthesis for Blizzard Challenge 2005", IEICE Trans. Inf. Syst., vol. E90-D, pp. 325-333, 2007.
27. V. Karaiskos, S. King, R. A. J. Clark, and C. Mayo, "The Blizzard Challenge 2008", in Proc. Blizzard Challenge Workshop, 2008.
28. T. Raitio, "Hidden Markov model based finnish text-to-speech system utilizing glottal inverse filtering", M. S. thesis, Helsinki Univ. of Technol., Espoo, Finland, 2008.
29. T. Raitio, A. Suni, H. Pulakka, M. Vainio, and P. Alku, "HMM-based Finnish text-to-speech system utilizing glottal inverse filtering", in Proc. Interspeech, 2008, pp. 1881-1884.
30. R. L. Miller, "Nature of the vocal cord wave", J. Acoust. Soc. Amer., vol. 31, no. 6, pp. 667-677, Jun. 1959.
31. J. L. Flanagan, Speech Analysis, Synthesis and Perception, 2nd ed. New York: Springer-Verlag, 1972.
32. P. Alku, "Glottal wave analysis with pitch synchronous iterative adaptive inverse filtering", Speech Commun., vol. 11, no. 2-3, pp. 109-118, 1992.
33. D. Wong, J. Markel, and A. Gray, Jr., "Least squares glottal inverse filtering from the acoustic speech waveform", IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-27, no. 4, pp. 350-355, Aug. 1979.
34. H. Strube, "Determination of the instant of glottal closure from the speech wave", J. Acoust. Soc. Amer., vol. 56, no. 5, pp. 1625-1629, 1974.
35. M. Plumpe, T. Quatieri, and D. Reynolds, "Modeling of the glottal flow derivative waveform with application to speaker identification", IEEE Trans. Speech Audio Process., vol. 7, no. 5, pp. 569-585, Sep. 1999.
36. M. Fröhlich, D. Michaelis, and H. Strube, "SIM-Simultaneous inverse filtering and matching of a glottal flow model for acoustic speech signals", J. Acoust. Soc. Amer., vol. 110, no. 1, pp. 479-488, 2001.
37. O. Akande and P. Murphy, "Estimation of the vocal tract transfer function with application to glottal wave analysis", Speech Commun., vol. 46, pp. 15-36, 2005.
38. Q. Fu and P. Murphy, "Robust glottal source estimation based on joint source-filter model optimization", IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 2, pp. 492-501, Mar. 2006.
39. D. Childers and C. Ahn, "Modeling the glottal volume-velocity waveform for three voice types", J. Acoust. Soc. Amer., vol. 97, no. 1, pp. 505-519, 1995.
40. A. Krishnamurthy and D. Childers, "Two-channel speech analysis", IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-34, no. 4, pp. 730-743, Aug. 1986.
41. H. Strik and L. Boves, "On the relation between voice source parameters and prosodic features in connected speech", Speech Commun., vol. 11, pp. 167-174, 1992.
42. A. El-Jaroudi and J. Makhoul, "Discrete all-pole modeling", IEEE Trans. Signal Process., vol. 39, no. 2, pp. 411-423, Feb. 1991.
43. J. Makhoul, "Linear prediction: A tutorial review", Proc. IEEE, vol. 63, no. 4, pp. 561-580, Apr. 1975.
44. D. Veeneman and S. BeMent, "Automatic glottal inverse filtering from speech and electroglottographic signals", IEEE Trans. Acoustics, Speech, Signal Process., vol. ASSP-33, no. 2, pp. 369-377, Apr. 1985.
45. J. Walker and P. Murphy, "Advanced methods for glottal wave extraction", in Nonlinear Analyses and Algorithms for Speech Processing, M. Faundez-Zanuy, Ed. et al. Berlin/Heidelberg, Germany: Springer, 2005, pp. 139-149.
46. P. Alku, J. Horáček, M. Airas, F. Griffond-Boitier, and A.-M. Laukkanen, "Performance of glottal inverse filtering as tested by aeroelastic modelling of phonation and FE modelling of vocal tract", Acta Acust. United With Acust., vol. 92, pp. 717-724, 2006.
47. P. Alku, B. Story, and M. Airas, "Estimation of the voice source from speech pressure signals: Evaluation of an inverse filtering technique using physical modelling of voice production", Folia Phoniatrica et Logopaedica, vol. 58, no. 2, pp. 102-113, 2006.
48. P. Alku, T. Bäckström, and E. Vilkman, "Normalized amplitude quotient for parameterization of the glottal flow", J. Acoust. Soc. Amer., vol. 112, no. 2, pp. 701-710, 2002.
49. C. Gobl and A. Ní Chasaide, "The role of voice quality in communicating emotion, mood and attitude", Speech Commun., vol. 40, no. 1-2, pp. 189-212, 2003.
50. A.-M. Laukkanen, E. Vilkman, P. Alku, and H. Oksanen, "On the perception of emotions in speech: The role of voice quality", Logopedics Phoniatrics Vocology, vol. 22, no. 4, pp. 157-168, 1997.
51. H. Zen, T. Nose, J. Yamagishi, S. Sako, T. Masuko, A. Black, and K. Tokuda, "The HMM-based speech synthesis system (HTS) version 2.0", in Proc. 6th ISCA Workshop Speech Synth., Aug. 2007, pp. 294-299.
52. D. Klatt, "Review of text-to-speech conversion for English", J. Acoust. Soc. Amer., vol. 82, no. 3, pp. 737-793, 1987.
53. J. Deller, "On the time domain properties of the two-pole model of the glottal waveform and implications for LPC", Speech Commun., vol. 2, no. 1, pp. 57-63, 1983.
54. F. K. Soong and B.-H. Juang, "Line spectrum pair (LSP) and speech data compression", in Proc. ICASSP, Mar. 1984, vol. 9, pp. 37-40.
55. M. Marume, H. Zen, Y. Nankaku, K. Tokuda, and T. Kitamura, "An investigation of spectral parameters for HMM-based speech synthesis", in Proc. Autumn Meeting Acoust. Soc. Jpn. (in Japanese), Sep. 2006.
56. F. Soong and B.-H. Juang, "Line spectrum pair (LSP) and speech data compression", in Proc. ICASSP, Mar. 1984, vol. 9, pp. 37-40.
57. K. Paliwal and W. Kleijn, "Quantization of LPC parameters", in Speech Coding and Synthesis, W. Kleijn and K. Paliwal, Eds. Amsterdam, The Netherlands: Elsevier, 1995, ch. 12.
58. L. Rabiner, "On the use of autocorrelation analysis for pitch detection", IEEE Trans. Acoust., Speech, Signal Process., vol. 25, no. 1, pp. 24-33, 1977.
59. P. Murphy, "Perturbation-free measurement of the harmonics-to-noise ratio in voice signals using pitch synchronous harmonic analysis", J. Acoust. Soc. Amer., vol. 105, no. 5, pp. 2866-2881, 1999.
60. K. Tokuda, T. Masuko, N. Miyazaki, and T. Kobayashi, "Multi-space probability distribution HMM", IEICE Trans. Inf. Syst., vol. E85-D, no. 3, pp. 455-464, Mar. 2002.
61. S. Levinson, "Continuously variable duration hidden Markov models for automatic speech recognition", Computer Speech Lang., vol. 1, no. 1, pp. 29-45, 1986. (Pubitemid 17552445)
62. H. Zen, K. Tokuda, T. Masuko, T. Kobayashi, and T. Kitamura, "A hidden semi-Markov model-based speech synthesis system", IEICE Trans. Inf. Syst., vol. E90-D, no. 5, pp. 825-834, May 2007.
63. M. Vainio, A. Suni, and P. Sirjola, "Accent and prominence in Finnish speech synthesis", in Proc. 10th Int. Conf. Speech Comput. (Specom 2005), G. Kokkinakis, N. Fakotakis, E. Dermatos, and R. Potapova, Eds., Greece, Oct. 2005, pp. 309-312, Univ. of Patras.
64. A. Suni and M. Vainio, "Deep syntactic analysis and rule based accentuation in text-to-speech synthesis", in Proc. TSD'08: Proc. 11th Int. Conf. Text, Speech, Dialogue, 2008, pp. 535-542.
65. K. Shinoda and T. Watanabe, "MDL-based context-dependent subword modeling for speech recognition", J. Acoust. Soc. Japan (E), vol. 21, pp. 79-86, Mar. 2000.
66. F. Alipour and R. Scherer, "Characterizing glottal jet turbulence", J. Acoust. Soc. Amer., vol. 119, no. 2, pp. 1063-1073, 2006.
67. G. Engeln-Müllges and E. Uhlig, Numerical Algorithms With C. Berlin, Germany: Springer, 1996.
68. M. Galassi et al., GNU Scientific Library Reference Manual, 3rd ed. 2009.
69. Z.-H. Ling, Y. Wu, Y.-P. Wang, L. Qin, and R.-H. Wang, "USTC system for Blizzard Challenge 2006: An improved HMM-based speech synthesis method", in Proc. Blizzard Challenge Workshop, 2006.
70. H. Zen and T. Toda, "An overview of Nitech HMM-based speech synthesis system for Blizzard Challenge 2005", in Proc. Interspeech, Sep. 2005, pp. 93-96. (Pubitemid 43908009)
71. S. Imai, "Cepstral analysis synthesis on the mel frequency scale", in Proc. ICASSP, Apr. 1983, vol. 8, pp. 93-96.
72. H. Kawahara, J. Estill, and O. Fujimura, "Aperiodicity extraction and control using mixed mode excitation and group delay manipulation for a high quality speech analysis, modification and synthesis system STRAIGHT", in Proc. 2nd Int. Workshop Models Anal. Vocal Emissions for Biomed. Applicat. (MAVEBA), Sep. 2001.
73. Y. Ohtani, T. Toda, H. Saruwatari, and K. Shikano, "Maximum likelihood voice conversion based on GMM with STRAIGHT mixed excitation", in Proc. Interspeech, Sep. 2006, pp. 2266-2269.
74. D. Arifianto, T. Tanaka, T. Masuko, and T. Kobayashi, "Robust F0 estimation of speech signal using harmonicity measure based on instantaneous frequency", IEICE Trans. Inf. Syst., vol. E87-D, no. 12, pp. 2812-2820, Dec. 2004.
75. H. Kawahara, H. Katayose, A. Cheveigné, and R. Patterson, "Fixed point analysis of frequency to instantaneous frequency mapping for accurate estimation of F0 and periodicity", in Proc. Eurospeech, Sep. 1999, pp. 2781-2784.
76. D. Talkin, "A robust algorithm for pitch tracking (RAPT)", in Speech Coding and Synthesis, W. Kleijn and K. Paliwal, Eds. Amsterdam, The Netherlands: Elsevier, 1995, pp. 495-518.
77. ESPS Programs Version 5.0 Entropic Research Laboratory Inc., 1993.
78. E. Moulines and F. Charpentier, "Pitch-synchronous waveform processing techniques for text-to-speech synthesis using diphones", Speech Commun., vol. 9, no. 5-6, pp. 453-467, Dec. 1990.
79. T. Fukada, K. Tokuda, T. Kobayashi, and S. Imai, "An adaptive algorithm for mel-cepstral analysis of speech", in Proc. ICASSP, 1992, vol. 1, pp. 137-140.
80. HTS, HMM-Based Speech Synthesis System Apr. 2009 [Online]. Available: http://hts.sp. nitech.ac.jp
81. M. Vainio, "Artificial neural network based prosody models for Finnish text-to-speech synthesis", Ph. D. dissertation, Univ. of Helsinki, Espoo, Finland, Dec. 2001.
82. ITU, "Methods for subjective determination of transmission quality", Int. Telecomm. Union, Rec. ITU-T P.800, Aug. 1996.
83. C. Benoít, M. Grice, and V. Hazan, "The SUS test: A method for the assessment of text-to-speech synthesis intelligibility using semantically unpredictable sentences", Speech Commun., vol. 18, no. 4, pp. 381-392, 1996.
84. F. Wilcoxon, "Individual comparisons by ranking methods", Biometrics, vol. 1, pp. 80-83, 1945.
85. Z.-H. Ling, K. Richmond, J. Yamagishi, and R.-H. Wang, "Articulatory control of HMM-based parametric speech synthesis driven by phonetic knowledge", in Proc. Interspeech, Brisbane, Australia, Sep. 2008, pp. 573-576.