Evaluation of a silent speech interface based on magnetic sensing and deep learning for a phonetically rich vocabulary

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn 2017-August, Issue , 2017, Pages 3986-3990

  Gonzalez, Jose A ^a   Cheah, Lam A ^b   Green, Phil D ^a   Gilbert, James M ^b   Ell, Stephen R ^c   Moore, Roger K ^a   Holdsworth, Ed ^d  

^a UNIVERSITY OF SHEFFIELD   (United Kingdom)

^b UNIVERSITY OF HULL   (United Kingdom)

^c CASTLE HILL HOSPITAL   (United Kingdom)

^d Practical Control Limited   (United Kingdom)

Author keywords

Articulatory to acoustic mapping; Recurrent neural network; Speech rehabilitation; Speech synthesis

Indexed keywords

DEEP LEARNING; DEEP NEURAL NETWORKS; MAPPING; METADATA; QUALITY CONTROL; RECURRENT NEURAL NETWORKS; SPEECH; SPEECH COMMUNICATION; SPEECH RECOGNITION; SPEECH SYNTHESIS;

ACOUSTIC MAPPING; GAUSSIAN MIXTURE MODEL (GMMS); REALTIME PROCESSING; RECURRENT NEURAL NETWORK (RNN); SILENT SPEECH INTERFACES; SIMULTANEOUS RECORDING; SPEECH REHABILITATION; TOTAL LARYNGECTOMIES;

AUDIO RECORDINGS;

EID: 85039155335     PISSN: 2308457X     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: 10.21437/Interspeech.2017-802     Document Type: Conference Paper

References (34)

1. M. J. Fagan, S. R. Ell, J. M. Gilbert, E. Sarrazin, and P. M. Chapman, "Development of a (silent) speech recognition system for patients following laryngectomy," Med. Eng. Phys., vol. 30, no. 4, pp. 419-425, 2008.
2. J. M. Gilbert, S. I. Rybchenko, R. Hofe, S. R. Ell, M. J. Fagan, R. K. Moore, and P. Green, "Isolated word recognition of silent speech using magnetic implants and sensors," Med. Eng. Phys., vol. 32, no. 10, pp. 1189-1197, 2010.
3. R. Hofe, S. R. Ell, M. J. Fagan, J. M. Gilbert, P. D. Green, R. K. Moore, and S. I. Rybchenko, "Small-vocabulary speech recognition using a silent speech interface based on magnetic sensing," Speech Commun., vol. 55, no. 1, pp. 22-32, 2013.
4. J. A. Gonzalez, L. A. Cheah, J. M. Gilbert, J. Bai, S. R. Ell, P. D. Green, and R. K. Moore, "A silent speech system based on permanent magnet articulography and direct synthesis," Comput. Speech Lang., vol. 39, pp. 67-87, 2016.
5. J. M. Gilbert, J. A. Gonzalez, L. A. Cheah, S. R. Ell, P. Green, R. K. Moore, and E. Holdsworth, "Restoring speech following total removal of the larynx by a learned transformation from sensor data to acoustics," J. Acoust. Soc. Am., vol. 141, no. 3, pp. EL307- EL313, 2017.
6. Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, May 2015.
7. A. Graves, A.-r. Mohamed, and G. Hinton, "Speech recognition with deep recurrent neural networks," in Proc. ICASSP, 2013, pp. 6645-6649.
8. Y. Fan, Y. Qian, F.-L. Xie, and F. K. Soong, "TTS synthesis with bidirectional LSTM based recurrent neural networks," in Proc. Interspeech, 2014, pp. 1964-1968.
9. H. Zen and H. Sak, "Unidirectional long short-term memory recurrent neural network with recurrent output layer for low-latency speech synthesis," in Proc. ICASSP, 2015, pp. 4470-4474.
10. L. Sun, S. Kang, K. Li, and H. Meng, "Voice conversion using deep bidirectional long short-term memory based recurrent neural networks," in Proc. ICASSP, 2015, pp. 4869-4873.
11. L. A. Cheah, J. Bai, J. A. Gonzalez, S. R. Ell, J. M. Gilbert, R. K. Moore, and P. D. Green, "A user-centric design of permanent magnetic articulography based assistive speech technology," in Proc. BioSignals, 2015, pp. 109-116.
12. L. A. Cheah, J. Bai, J. A. Gonzalez, J. M. Gilbert, S. R. Ell, P. D. Green, and R. K. Moore, "Preliminary evaluation of a silent speech interface based on intra-oral magnetic sensing," in Proc. Biodevices, 2016, pp. 108-116.
13. S. Aryal and R. Gutierrez-Osuna, "Data driven articulatory synthesis with deep neural networks," Comput. Speech Lang., vol. 36, pp. 260-273, 2016.
14. A. J. Yates, "Delayed auditory feedback," Psychological bulletin, vol. 60, no. 3, p. 213, 1963.
15. A. Stuart, J. Kalinowski, M. P. Rastatter, and K. Lynch, "Effect of delayed auditory feedback on normal speakers at two speech rates," J. Acoust. Soc. Am., vol. 111, no. 5, pp. 2237-2241, 2002.
16. S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural computation, vol. 9, no. 8, pp. 1735-1780, 1997.
17. K. Cho, B. Van Merriënboer, Ç. Gülçehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, "Learning phrase representations using RNN encoder-decoder for statistical machine translation," in Proc. EMNLP, 2014, pp. 1724-1734.
18. P. J. Werbos, "Backpropagation through time: what it does and how to do it," Proceedings of the IEEE, vol. 78, no. 10, pp. 1550- 1560, 1990.
19. J. Kominek and A. W. Black, "The CMU Arctic speech databases," in Fifth ISCA Workshop on Speech Synthesis, 2004, pp. 223-224.
20. H. Kawahara, I. Masuda-Katsuse, and A. De Cheveigne, "Restructuring speech representations using a pitch-adaptive time- frequency smoothing and an instantaneous-frequency-based f0 extraction: Possible role of a repetitive structure in sounds," Speech communication, vol. 27, no. 3, pp. 187-207, Apr. 1999.
21. T. Fukada, K. Tokuda, T. Kobayashi, and S. Imai, "An adaptive algorithm for Mel-cepstral analysis of speech," in Proc. ICASSP, 1992, pp. 137-140.
22. S. De Jong, "SIMPLS: an alternative approach to partial least squares regression," Chemometrics Intell. Lab. Syst., vol. 18, no. 3, pp. 251-263, 1993.
23. M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mane, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viegas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng. (2015) Tensorflow: Large-scale machine learning on heterogeneous distributed systems.[Online]. Available: www.tensorflow.org
24. D. Kingma and J. Ba, "Adam: A method for stochastic optimization," in International Conference on Learning Representations, 2015.
25. T. Toda, A. W. Black, and K. Tokuda, "Voice conversion based on maximum-likelihood estimation of spectral parameter trajectory," IEEE Trans. Audio Speech Lang. Process., vol. 15, no. 8, pp. 2222-2235, Nov. 2007.
26. - "Statistical mapping between articulatory movements and acoustic spectrum using a Gaussian mixture model," Speech Commun., vol. 50, no. 3, pp. 215-227, Mar. 2008.
27. F. Bocquelet, T. Hueber, L. Girin, C. Savariaux, and B. Yvert, "Real-time control of an articulatory-based speech synthesizer for brain computer interfaces," PLOS Computational Biology, vol. 12, no. 11, p. e1005119, 2016.
28. K. Tokuda, T. Yoshimura, T. Masuko, T. Kobayashi, and T. Kitamura, "Speech parameter generation algorithms for HMM-based speech synthesis," in Proc. ICASSP, 2000, pp. 1315-1318.
29. H. Zen, A. Senior, and M. Schuster, "Statistical parametric speech synthesis using deep neural networks," in Proc. ICASSP. IEEE, 2013, pp. 7962-7966.
30. K. Hashimoto, K. Oura, Y. Nankaku, and K. Tokuda, "The effect of neural networks in statistical parametric speech synthesis," in Proc. ICASSP, 2015, pp. 4455-4459.
31. M. Schuster and K. K. Paliwal, "Bidirectional recurrent neural networks," IEEE Trans. Signal Process., vol. 45, no. 11, pp. 2673- 2681, 1997.
32. A. Graves and J. Schmidhuber, "Framewise phoneme classification with bidirectional LSTM and other neural network architectures," Neural Networks, vol. 18, no. 5, pp. 602-610, 2005.
33. R. Kubichek, "Mel-cepstral distance measure for objective speech quality assessment," in Proc. IEEE Pacific Rim Conference on Communications, Computers and Signal Processing, 1993, pp. 125-128.
34. J. A. Gonzalez, L. A. Cheah, J. Bai, S. R. Ell, J. M. Gilbert, R. K. Moore, and P. D. Green, "Analysis of phonetic similarity in a silent speech interface based on permanent magnetic articulography," in Proc. Interspeech, 2014, pp. 1018-1022.