Statistical conversion of silent articulation into audible speech using full-covariance HMM

Computer Speech and Language

Volumn 36, Issue , 2016, Pages 274-293

  Hueber, Thomas ^a,b   Bailly, Gérard ^a,b  

^a UNIV GRENOBLE ALPES   (France)

^b CNRS   (France)

Author keywords

Articulatory acoustic mapping; GMM; HMM; Silent speech interface; Ultrasound

Indexed keywords

LINGUISTICS; MAPPING; SPEECH; SPEECH PROCESSING; ULTRASONIC APPLICATIONS; ULTRASONICS;

ACOUSTIC MAPPING; DISCRIMINATION TESTS; GMM; HMM; LINGUISTIC KNOWLEDGE; OBJECTIVE EVALUATION; SILENT SPEECH INTERFACES; SPECTRAL DISTORTIONS;

SPEECH INTELLIGIBILITY;

EID: 84949568613     PISSN: 08852308     EISSN: 10958363     Source Type: Journal    
DOI: 10.1016/j.csl.2015.03.005     Document Type: Article

References (47)

1. P. Birkholz, D. Jackèl, and B. Kroger Construction and control of a three-dimensional vocal tract model Proceedings of ICASSP Toulouse, France 2006 873 876
2. A.L. Buchsbaum, and J.P. van Santen Selecting training inputs via greedy rank covering Proceedings of the 7th Annual ACM-SIAM Symposium on Discrete Algorithms Philadelphia, PA, USA 1996 288 295
3. T. Chen Audiovisual speech processing Signal Process. Mag. IEEE 18 1 2001 9 21
4. P. Clarkson, and R. Rosenfeld Statistical language modeling using the CMU-Cambridge toolkit Proceedings of Eurospeech Rhodes, Greece 1997 2707 2710
5. P. Combescure Vingt listes de dix phrases phonétiquement équilibrées Rev. Acoust. 14 56 1981 34 38
6. B. Denby, Y. Oussar, G. Dreyfus, and M. Stone Prospects for a silent speech interface using ultrasound imaging Proceedings of ICASSP Toulouse, France 2006 365 368
7. B. Denby, T. Schultz, K. Honda, T. Hueber, J. Gilbert, and J. Brumberg Silent speech interfaces Speech Commun. 52 4 2010 270 287
8. M.J. Fagan, S.R. Ell, J.M. Gilbert, E. Sarrazin, and P.M. Chapman Development of a (silent) speech recognition system for patients following laryngectomy Med. Eng. Phys. 30 4 2008 419 425
9. J.M. Gilbert, S.I. Rybchenko, R. Hofe, S.R. Ell, M.J. Fagan, R.K. Moore, and P. Green Isolated word recognition of silent speech using magnetic implants and sensors Med. Eng. Phys. 32 10 2010 1189 1197
10. S. Hiroya, and M. Honda Estimation of articulatory movements from speech acoustics using an HMM-based speech production model IEEE Trans. Speech Audio Process. 12 2 2004 175 185
11. T. Hueber, G. Aversano, G. Chollet, B. Denby, G. Dreyfus, Y. Oussar, P. Roussel, and M. Stone Eigentongue feature extraction for an ultrasound-based silent speech interface Proceedings of ICASSP Honolulu, USA 2007 1245 1248
12. T. Hueber, P. Badin, C. Savariaux, C. Vilain, and G. Bailly Differences in articulatory strategies between silent, whispered and normal speech? A pilot study using electromagnetic articulography Proceedings of International Seminar on Speech Production (ISSP) Montreal, Canada 2010
13. T. Hueber, E.-L. Benaroya, G. Chollet, B. Denby, and M. Stone Development of a silent speech interface driven by ultrasound and optical images of the tongue and lips Speech Commun. 52 4 2010 288 300
14. T. Hueber, E.-L. Benaroya, B. Denby, and G. Chollet Statistical mapping between articulatory and acoustic data for an ultrasound-based silent speech interface Proceedings of Interspeech Firenze, Italia 2011 593 596
15. T. Hueber, A. Ben Youssef, G. Bailly, P. Badin, and F. Elisei Cross-speaker acoustic-to-articulatory inversion using phone-based trajectory HMM for pronunciation training Proceedings of Interspeech Portland, USA 2012
16. T. Hueber, G. Chollet, B. Denby, G. Dreyfus, and M. Stone Visuo-phonetic decoding using multi-stream and context-dependent models for an ultrasound-based silent speech interface Proceedings of Interspeech Brighton, England 2009 640 643
17. T. Hueber, G. Chollet, B. Denby, and M. Stone Acquisition of ultrasound, video and acoustic speech data for a silent-speech interface application Proceedings of International Seminar on Speech Production Strasbourg, France 2008 365 369
18. S. Imai, K. Sumita, and C. Furuichi Mel log spectrum approximation (MLSA) filter for speech synthesis Electron. Commun. Jpn. Part I Commun. 66 1983 10 18
19. M. Janke, M. Wand, and T. Schultz Impact of lack of acoustic feedback in EMG-based silent speech recognition Proceedings of Interspeech Makuhari, Japan 2010 2686 2689
20. C.T. Kello, and D.C. Plaut A neural network model of the articulatory-acoustic forward mapping trained on recordings of articulatory parameters J. Acoust. Soc. Am. 116 4 2004 2354 2364
21. Z.-H. Ling, K. Richmond, and J. Yamagishi An analysis of HMM-based prediction of articulatory movements Speech Commun. 52 10 2010 834 846
22. Z.-H. Ling, K. Richmond, J. Yamagishi, and R.-H. Wang Integrating articulatory features into HMM-based parametric speech synthesis IEEE Trans. Audio Speech Lang. Process. 17 6 2009 1171 1185
23. S. Maeda Compensatory articulation during speech: evidence from the analysis and synthesis of vocal-tract shapes using an articulatory model Speech Production and Speech Modelling 1990 Springer 131 149
24. T. Muramatsu, Y. Ohtani, T. Toda, H. Saruwatari, and K. Shikano Low-delay voice conversion based on maximum likelihood estimation of spectral parameter trajectory Proceedings of Interspeech Brisbane, Australia 2008 1076 1079
25. Y. Nakajima, H. Kashioka, N. Campbell, and K. Shikano Non-audible murmur (NAM) recognition IEICE Trans. Inf. Syst. 89 1 2006 1 8
26. Y. Nakajima, H. Kashioka, K. Shikano, and N. Campbell Non-audible murmur recognition input interface using stethoscopic microphone attached to the skin Proceedings of ICASSP Hong Kong, Hong Kong 2003 708 711
27. M. Ostendorf, V.V. Digalakis, and O.A. Kimball From HMM's to segment models: a unified view of stochastic modeling for speech recognition IEEE Trans. Speech Audio Process. 4 5 1996 360 378
28. K. Richmond A trajectory mixture density network for the acoustic-articulatory inversion mapping Proceedings of Interspeech Pittsburgh, PA, USA 2006 577 580
29. M. Russell, and R. Moore Explicit modelling of state occupancy in hidden Markov models for automatic speech recognition Proceedings of ICASSP Detroit, MI, USA 1985 5 8
30. T. Schultz, and M. Wand Modeling coarticulation in EMG-based continuous speech recognition Speech Commun. 52 4 2010 341 353
31. M. Shannon, and W. Byrne Fast, low-artifact speech synthesis considering global variance Proceedings of ICASSP Vancouver, British Columbia, Canada 2013 7869 7873
32. H. Silén, E. Helander, J. Nurminen, and M. Gabbouj Ways to implement global variance in statistical speech synthesis Proceedings of Interspeech Portland, USA 2012
33. M.M. Sondhi, and J. Schroeter A hybrid time-frequency domain articulatory speech synthesizer IEEE Trans. Acoust. Speech Signal Process. 35 7 1987 955 967
34. M. Stone A guide to analysing tongue motion from ultrasound images Clin. Linguist. Phon. 19 6-7 2005 455 501
35. T. Toda, A.W. Black, and K. Tokuda Voice conversion based on maximum-likelihood estimation of spectral parameter trajectory IEEE Trans. Audio Speech Lang. Process. 15 8 2007 2222 2235
36. T. Toda, A.W. Black, and K. Tokuda Statistical mapping between articulatory movements and acoustic spectrum using a Gaussian mixture model Speech Commun. 50 3 2008 215 227
37. T. Toda, and K. Tokuda A speech parameter generation algorithm considering global variance for HMM-based speech synthesis IEICE Trans. Inf. Syst. E90-D 2007 816 824
38. K. Tokuda, T. Yoshimura, T. Masuko, T. Kobayashi, and T. Kitamura Speech parameter generation algorithms for HMM-based speech synthesis Proceedings of ICASSP Istanbul, Turkey 2000 1315 1318
39. M. Turk, and A. Pentland Eigenfaces for recognition J. Cogn. Neurosci. 3 1991 71 86
40. M. Wand, and T. Schultz Session-independent EMG-based speech recognition Proceedings of Biosignals Rome, Italy 2011 295 300
41. A. Wrench, J. Scobbie, and M. Linden Evaluation of a helmet to hold an ultrasound probe Presented at the Ultrafest IV New York, USA 2007
42. A. Wrench, and J.M. Scobbie Spatio-temporal inaccuracies of video-based ultrasound images of the tongue Proceedings of the International Seminar on Speech Production Ubatuba, Bresil 2006 451 458
43. S. Young The HTK Book 2005 http://htk.eng.cam.ac.uk/
44. A.B. Youssef, T. Hueber, P. Badin, and G. Bailly Toward a multi-speaker visual articulatory feedback system Proceedings of Interspeech Firenze, Italia 2011 589 592
45. Y.J. Yu, and S.T. Acton Speckle reducing anisotropic diffusion IEEE Trans. Image Process. 11 11 2002 1260 1270
46. H. Zen, Y. Nankaku, and K. Tokuda Continuous stochastic feature mapping based on trajectory HMMS IEEE Trans. Audio Speech Lang. Process. 19 2 2011 417 430
47. L. Zhang, and S. Renals Acoustic-articulatory modelling with the trajectory HMM IEEE Signal Process. Lett. 15 2008 245 248