Restoring speech following total removal of the larynx by a learned transformation from sensor data to acoustics

Journal of the Acoustical Society of America

Volumn 141, Issue 3, 2017, Pages EL307-EL313

  Gilbert, James M ^a   Gonzalez, Jose A ^b   Cheah, Lam A ^a   Ell, Stephen R ^c   Green, Phil ^b   Moore, Roger K ^b   Holdsworth, Ed ^d  

^a UNIVERSITY OF HULL   (United Kingdom)

^b UNIVERSITY OF SHEFFIELD   (United Kingdom)

^c CASTLE HILL HOSPITAL   (United Kingdom)

^d Practical Control Limited   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

DATA HANDLING; LEARNING ALGORITHMS; LEARNING SYSTEMS; METADATA;

ACOUSTIC SIGNALS; LARYNGEAL CANCER; MONITORING CHANGE; REAL TIME; SENSING TECHNIQUES; SENSOR DATA; SILENT SPEECH;

SPEECH;

ACOUSTICS; BIOMECHANICS; CONVALESCENCE; DEVICES; HUMAN; LARYNGECTOMY; LARYNX PROSTHESIS; LIP; MACHINE LEARNING; MAGNET; MAGNETIC FIELD; MAGNETISM; PHYSIOLOGY; PROSTHESIS DESIGN; SIGNAL PROCESSING; SOUND DETECTION; SPEECH; SPEECH INTELLIGIBILITY; TIME FACTOR; TONGUE; TRANSDUCER; VOICE;

ACOUSTICS; BIOMECHANICAL PHENOMENA; HUMANS; LARYNGECTOMY; LARYNX, ARTIFICIAL; LIP; MACHINE LEARNING; MAGNETIC FIELDS; MAGNETICS; MAGNETS; PROSTHESIS DESIGN; RECOVERY OF FUNCTION; SIGNAL PROCESSING, COMPUTER-ASSISTED; SOUND SPECTROGRAPHY; SPEECH ACOUSTICS; SPEECH INTELLIGIBILITY; TIME FACTORS; TONGUE; TRANSDUCERS; VOICE QUALITY;

EID: 85016157373     PISSN: 00014966     EISSN: None     Source Type: Journal    
DOI: 10.1121/1.4978364     Document Type: Article

References (24)

1. Cancer Research UK, Laryngeal Cancer Statistics, http://www.cancerresearchuk.org/health-professional/cancer-statistics/statistics-by-cancer-type/laryngeal-cancer#heading-Two (Last viewed 10 March 2017).
2. J. Ferlay, H. R. Shin, F. Bray, D. Forman, C. Mathers, and D. M. Parkin, “ Estimates of worldwide burden of cancer in 2008: GLOBOCAN 2008,” Int. J. Cancer 127 (12), 2893-2917 (2010)10.1002/ijc.25516.
3. Department of Veterans Affairs, Laryngeal Cancer Study Group, “ Induction chemotherapy plus radiation compared with surgery plus radiation in patients with advanced laryngeal cancer,” New Engl. J. Med. 324, 1685-1690 (1991). 10.1056/NEJM199106133242402
4. A. J. Timmermans, B. A. Dijk, L. I. Overbeek, M. L. F. Velthuysen, H. Tinteren, F. J. Hilgers, and M. W. Brekel, “ Trends in treatment and survival for advanced laryngeal cancer: A 20-year population-based study in The Netherlands,” Head Neck 38 (S1), E1247-E1255 (2016)10.1002/hed.24200.
5. P. Sunil, S. P. Verma, and H. Mahboubi, “ The changing landscape of total laryngectomy surgery,” Otolaryng. Head Neck Surg. 150 (3), 413-418 (2014). 10.1177/0194599813514515
6. B. Denby, T. Schultz, K. Honda, T. Hueber, J. M. Gilbert, and J. S. Brumberg, “ Silent speech interfaces,” Speech Commun. 52, 270-287 (2010). 10.1016/j.specom.2009.08.002
7. J. M. Gilbert, S. I. Rybchenko, R. Hofe, S. R. Ell, M. J. Fagan, R. K. Moore, and P. Green, “ Isolated word recognition of silent speech using magnetic implants and sensors,” Med. Eng. Phys. 10, 1189-1197 (2010)10.1016/j.medengphy.2010.08.011.
8. J. A. Gonzalez, L. A. Cheah, J. M. Gilbert, J. Bai, S. R. Ell, P. D. Green, and R. K. Moore, “ A silent speech system based on permanent magnet articulography and direct speech synthesis,” Comput. Speech Lang. 39, 67-87 (2016). 10.1016/j.csl.2016.02.002
9. T. Hueber and G. Bailly, “ Statistical conversion of silent articulation into audible speech using full-covariance HMM,” Comput. Speech Lang. 36, 274-293 (2015). 10.1016/j.csl.2015.03.005
10. F. Bocquelet, T. Hueber, L. Girin, C. Savariaux, and B. Yvert, “ Real-time control of an articulatory-based speech synthesizer for brain computer interfaces,” PLOS Comput. Biol. 12 (11), e1005119 (2016). 10.1371/journal.pcbi.1005119
11. S. Aryal and R. Gutierrez-Osuna, “ Data driven articulatory synthesis with deep neural networks,” Comput. Speech Lang. 36, 260-273 (2016). 10.1016/j.csl.2015.02.003
12. S. Hochreiter and J. Schmidhuber, “ Long short-term memory,” Neural Comput. 9 (8), 1-32 (1997). 10.1162/neco.1997.9.1.1
13. F. A. Gers, N. N. Schraudolph, and J. Schmidhuber, “ Learning precise timing with LSTM recurrent retworks,” J. Mach. Learn. Res. 3 (1), 115-143 (2002).
14. A. Graves, A. Mohamed, and G. Hinton, “ Speech recognition with deep recurrent neural networks,” in Proceedings of ICASSP 2013 (2013), pp. 6645-6649.
15. Y. Fan, Y. Qian, F. Xie, and F. K. Soong, “ TTS synthesis with bidirectional LSTM based recurrent neural networks,” in Proceedings of Interspeech 2014 (2014), pp. 1964-1968.
16. A. J. Yates, “ Delayed auditory feedback,” Psychol. Bull. 60, 213-232 (1963). 10.1037/h0044155
17. J. Kominek and A. W. Black, “ CMU ARCTIC databases for speech synthesis,” technical report, Language Technologies Institute, Carnegie Mellon University (2003).
18. H. Kawahara, I. Masuda-Katsuse, and A. De Cheveigné, “ Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based F0 extraction: Possible role of a repetitive structure in sounds,” Speech Commun. 27 (3), 187-207 (1999). 10.1016/S0167-6393(98)00085-5
19. T. Fukada, K. Tokuda, T. Kobayashi, and S. Imai, “ An adaptive algorithm for Mel-cepstral analysis of speech,” in Proceedings of ICASSP 1992 (1992), pp. 137-140.
20. R. Kubichek, “ Mel-cepstral distance measure for objective speech quality assessment,” in Proceedings IEEE Pacific Rim Conference on Communications, Computers and Signal Processing (1993), pp. 125-128.
21. J. Kominek, T. Schultz, and A. W. Black, “ Synthesizer voice quality of new languages calibrated with mean Mel cepstral distortion,” in Proceedings Workshop on Spoken Language Technologies for Under-Resourced Languages (SLTU) (2008).
22. K. Chládková, P. Boersma, and V. J. Podlipský, “ On-line formant shifting as a function of F0,” in Proceedings of Interspeech 2009 (2009), pp. 464-467.
23. P. Roach, Phonetics ( Oxford University Press, Oxford, 2001).
24. J. A. Gonzalez, L. J. Cheah, J. Bai, S. R. Ell, J. M. Gilbert, R. K. Moore, and P. D. Green, “ Analysis of phonetic similarity in a silent speech interface based on permanent magnetic articulography,” Proceedings of Interspeech 2014 (2014), pp. 1018-1022.