A silent speech system based on permanent magnet articulography and direct synthesis

Computer Speech and Language

Volumn 39, Issue , 2016, Pages 67-87

  Gonzalez, Jose A ^a   Cheah, Lam A ^b   Gilbert, James M ^b   Bai, Jie ^b   Ell, Stephen R ^c   Green, Phil D ^a   Moore, Roger K ^a  

^a UNIVERSITY OF SHEFFIELD   (United Kingdom)

^b UNIVERSITY OF HULL   (United Kingdom)

^c CASTLE HILL HOSPITAL   (United Kingdom)

Author keywords

Augmentative and alternative communication; Permanent magnet articulography; Silent speech interfaces; Speech rehabilitation; Speech synthesis

Indexed keywords

HUMAN REHABILITATION ENGINEERING; LINEAR TRANSFORMATIONS; MAGNETS; MATHEMATICAL TRANSFORMATIONS; METADATA; PERMANENT MAGNETS; SPEECH; SPEECH COMMUNICATION; SPEECH INTELLIGIBILITY; SPEECH RECOGNITION; SPEECH SYNTHESIS;

AUGMENTATIVE-AND-ALTERNATIVE COMMUNICATION; DIMENSIONALITY REDUCTION; NONLINEAR BEHAVIOURS; OBJECTIVE QUALITY MEASURES; SILENT SPEECH INTERFACES; SIMULTANEOUS RECORDING; SPEAKER DEPENDENTS; SPEECH REHABILITATION;

AUDIO RECORDINGS;

EID: 84962110277     PISSN: 08852308     EISSN: 10958363     Source Type: Journal    
DOI: 10.1016/j.csl.2016.02.002     Document Type: Article

References (62)

1. G. Ananthakrishnan, O. Engwall, and D. Neiberg Exploring the predictability of non-unique acoustic-to-articulatory mappings IEEE Trans. Audio Speech Lang. Process. 20 December (10) 2012 2672 2682
2. T.W. Anderson An Introduction to Multivariate Statistical Analysis 3rd ed. 2003 Wiley-Interscience
3. B.S. Atal, J.J. Chang, M.V. Mathews, and J.W. Tukey Inversion of articulatory-to-acoustic transformation in the vocal tract by a computer-sorting technique J. Acoust. Soc. Am. 63 5 1978 1535 1555
4. P. Badin, G. Bailly, L. Revret, M. Baciu, C. Segebarth, and C. Savariaux Three-dimensional linear articulatory modeling of tongue, lips and face, based on MRI and video images J. Phon. 30 July (3) 2002 533 553
5. P. Birkholz, and D. Jackel A three-dimensional model of the vocal tract for speech synthesis Proc. 15th International congress of phonetic sciences 2003 2597 2600
6. P. Birkholz, I. Steiner, and S. Breuer Control concepts for articulatory speech synthesis Proc. 6th ISCA Workshop on Speech Synthesis 2008 5 10
7. C.M. Bishop Pattern Recognition and Machine Learning 2006 Springer
8. K. Brigham, and B.V.K. Vijaya Kumar Imagined speech classification with EEG signals for silent communication: a preliminary investigation into synthetic telepathy Proc. 4th International Conference on Bioinformatics and Biomedical Engineering 2010, June 1 4
9. J.S. Brumberg, A. Nieto-Castanon, P.R. Kennedy, and F.H. Guenther Brain-computer interfaces for speech communication Speech Commun. 52 4 2010, April 367 379
10. J.S. Brumberg, E.J. Wright, D.S. Andreasen, F.H. Guenther, and P.R. Kennedy Classification of intended phoneme production from chronic intracortical microelectrode recordings in speech-motor cortex Front. Neurosci. 5 2011, May 1 12
11. J. Cai, T. Hueber, S. Manitsaris, P. Roussel, L. Crevier-Buchman, M. Stone, C. Pillot-Loiseau, G. Chollet, G. Dreyfus, and B. Denby Vocal tract imaging system for post-laryngectomy voice replacement Proc. IEEE International Instrumentation and Measurement Technology Conference (I2MTC) 2013 676 680
12. L.A. Cheah, J. Bai, J.A. Gonzalez, S.R. Ell, J.M. Gilbert, R.K. Moore, and P.D. Green A user-centric design of permanent magnetic articulography based assistive speech technology Proc. BioSignals 2015 109 116
13. S. De Jong SIMPLS: an alternative approach to partial least squares regression Chemomet. Intell. Lab. Syst. 18 3 1993 251 263
14. B. Denby, T. Schultz, K. Honda, T. Hueber, J. Gilbert, and J. Brumberg Silent speech interfaces Speech Commun. 52 April (4) 2010 270 287
15. Y. Deng, J.T. Heaton, and G.S. Meltzner Towards a practical silent speech recognition system Proc. Interspeech. 2014 1164 1168
16. S. Desai, E.V. Raghavendra, B. Yegnanarayana, A.W. Black, and K. Prahallad Voice conversion using artificial neural networks Proc. ICASSP. 2009 3893 3896
17. M.J. Fagan, S.R. Ell, J.M. Gilbert, E. Sarrazin, and P.M. Chapman Development of a (silent) speech recognition system for patients following laryngectomy Med. Eng. Phys. 30 4 2008 419 425
18. T. Fukada, K. Tokuda, T. Kobayashi, and S. Imai An adaptive algorithm for Mel-cepstral analysis of speech Proc. ICASSP. 1992 137 140
19. Z. Ghahramani, and G.E. Hinton The EM algorithm for mixtures of factor analyzers. Tech. Rep. CRG-TR-96-1 1996 University of Toronto
20. J.M. Gilbert, S.I. Rybchenko, R. Hofe, S.R. Ell, M.J. Fagan, R.K. Moore, and P. Green Isolated word recognition of silent speech using magnetic implants and sensors Med. Eng. Phys. 32 10 2010 1189 1197
21. J.A. Gonzalez, L.A. Cheah, J. Bai, S.R. Ell, J.M. Gilbert, R.K.M. 1, and P.D. Green Analysis of phonetic similarity in a silent speech interface based on permanent magnetic articulography Proc. Interspeech. 2014 1018 1022
22. J.A. Gonzalez, P.D. Green, R.K. Moore, L.A. Cheah, and J.M. Gilbert A non-parametric articulatory-to-acoustic conversion system for silent speech using shared Gaussian process dynamical models Proc. UK Speech. 2015 11
23. C. Herff, D. Heger, A. de Pesters, D. Telaar, P. Brunner, G. Schalk, and T. Schultz Brain-to-text: decoding spoken phrases from phone representations in the brain Front. Neurosci. 9 June (217) 2015
24. R. Hofe, J. Bai, L.A. Cheah, S.R. Ell, J.M. Gilbert, R.K. Moore, and P.D. Green Performance of the MVOCA silent speech interface across multiple speakers Proc. Interspeech 2013 1140 1143
25. R. Hofe, S.R. Ell, M.J. Fagan, J.M. Gilbert, P.D. Green, R.K. Moore, and S.I. Rybchenko Speech synthesis parameter generation for the assistive silent speech interface MVOCA Proc. Interspeech 2011 3009 3012
26. R. Hofe, S.R. Ell, M.J. Fagan, J.M. Gilbert, P.D. Green, R.K. Moore, and S.I. Rybchenko Small-vocabulary speech recognition using a silent speech interface based on magnetic sensing Speech Commun. 55 1 2013 22 32
27. T. Hueber, G. Bailly, and B. Denby Continuous articulatory-to-acoustic mapping using phone-baseda trajectory HMM for a silent speech interface Proc. Interspeech. 2012 723 726
28. T. Hueber, E.-L. Benaroya, G. Chollet, B. Denby, G. Dreyfus, and M. Stone Development of a silent speech interface driven by ultrasound and optical images of the tongue and lips Speech Commun. 52 4 2010 288 300
29. T. Hueber, E.-L. Benaroya, B. Denby, and G. Chollet Statistical mapping between articulatory and acoustic data for an ultrasound-based silent speech interface Proc. Interspeech. 2011 593 596
30. T. Hueber, G. Chollet, B. Denby, G. Dreyfus, and M. Stone Phone recognition from ultrasound and optical video sequences for a silent speech interface Proc. Interspeech. 2008 2032 2035
31. M. Janke, M. Wand, K. Nakamura, and T. Schultz Further investigations on EMG-to-speech conversion Proc. ICASSP. 2012 365 368
32. S.-C. Jou, T. Schultz, M. Walliczek, F. Kraft, and A. Waibel Towards continuous speech recognition using surface electromyography Proc. Interspeech. 2006 573 576
33. H. Kawahara, I. Masuda-Katsuse, and A. De Cheveigne Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based f0 extraction: possible role of a repetitive structure in sounds Speech Commun. 27 April (3) 1999 187 207
34. R. Kubichek Mel-cepstral distance measure for objective speech quality assessment Proc. IEEE Pacific Rim Conference on Communications, Computers and Signal Processing 1993 125 128
35. R. Leonard A database for speaker-independent digit recognition Proc. ICASSP. 1984 328 331
36. S. Maeda A digital simulation method of the vocal-tract system Speech Commun. 1 3 1982 199 229
37. I. Matthews, T.F. Cootes, J.A. Bangham, S. Cox, and R. Harvey Extraction of visual features for lipreading IEEE Trans. Pattern Anal. Mach. Intell. 24 February (2) 2002 198 213
38. T. Moriguchi, T. Toda, M. Sano, H. Sato, G. Neubig, S. Sakti, and S. Nakamura A digital signal processor implementation of silent/electrolaryngeal speech enhancement based on real-time statistical voice conversion Proc. Interspeech. 2013 3072 3076
39. K. Nakamura, T. Toda, H. Saruwatari, and K. Shikano Speaking-aid systems using GMM-based voice conversion for electrolaryngeal speech Speech Commun. 54 January (1) 2012 134 146
40. D. Neiberg, G. Ananthakrishnan, and O. Engwall The acoustic to articulation mapping: non-linear or non-unique? Proc. Interspeech. 2008 1485 1488
41. E. Petajan, B. Bischoff, D. Bodoff, and N.M. Brooke An improved automatic lipreading system to enhance speech recognition Proc. SIGCHI conference on Human factors in computing systems 1988 19 25
42. E.D. Petajan Automatic lipreading to enhance speech recognition (speech reading) 1984 University of Illinois at Urbana-Champaign (Ph.D. thesis)
43. C. Qin, and M.Á. Carreira-Perpiñán An empirical investigation of the nonuniqueness in the acoustic-to-articulatory mapping Proc. Interspeech. 2007 74 77
44. P. Rubin, T. Baer, and P. Mermelstein An articulatory synthesizer for perceptual research J. Acoust. Soc. Am. 70 2 1981 321 328
45. J. Schroeter, and M.M. Sondhi Techniques for estimating vocal-tract shapes from the speech signal IEEE Trans. Speech Audio Process. 2 1 1994 133 150
46. T. Schultz, and M. Wand Modeling coarticulation in EMG-based continuous speech recognition Speech Commun. 52 April (4) 2010 341 353
47. Y. Stylianou, O. Cappe, and E. Moulines Continuous probabilistic transform for voice conversion IEEE Trans. Speech Audio Process. 6 March (2) 1998 131 142
48. T. Toda, A.W. Black, and K. Tokuda Spectral conversion based on maximum likelihood estimation considering global variance of converted parameter Proc. ICASSP. 2005 9 12
49. T. Toda, A.W. Black, and K. Tokuda Voice conversion based on maximum-likelihood estimation of spectral parameter trajectory IEEE Trans. Audio Speech Lang. Process. 15 November (8) 2007 2222 2235
50. T. Toda, A.W. Black, and K. Tokuda Statistical mapping between articulatory movements and acoustic spectrum using a Gaussian mixture model Speech Commun. 50 March (3) 2008 215 227
51. T. Toda, T. Muramatsu, and H. Banno Implementation of computationally efficient real-time voice conversion Proc. Interspeech 2012 94 97
52. T. Toda, M. Nakagiri, and K. Shikano Statistical voice conversion techniques for body-conducted unvoiced speech enhancement IEEE Trans. Audio Speech Lang. Process. 20 November (9) 2012 2505 2517
53. K. Tokuda, T. Yoshimura, T. Masuko, T. Kobayashi, and T. Kitamura Speech parameter generation algorithms for HMM-based speech synthesis Proc. ICASSP. 2000 1315 1318
54. A.R. Toth, K. Kalgaonkar, B. Raj, and T. Ezzat Synthesizing speech from Doppler signals Proc. ICASSP. 2010 4638 4641
55. A. Toutios, and K.G. Margaritis A support vector approach to the acoustic-to-articulatory mapping Proc. Interspeech. 2005 3221 3224
56. A. Toutios, and S. Narayanan Articulatory synthesis of French connected speech from EMA data Proc. Interspeech. 2013 2738 2742
57. A. Toutios, S. Ouni, and Y. Laprie Estimating the control parameters of an articulatory model from electromagnetic articulograph data J. Acoust. Soc. Am. 129 5 2011 3245 3257
58. M. Wand, M. Janke, and T. Schultz Tackling speaking mode varieties in EMG-based speech recognition IEEE Trans. Bio-Med. Eng. 61 October (10) 2014 2515 2526
59. M. Wand, and T. Schultz Analysis of phone confusion in EMG-based speech recognition Proc. ICASSP. 2011 757 760
60. M. Wester Unspoken speech: speech recognition based on electroencephalography 2006 Universität Karlsruhe (Master's thesis)
61. M. Zahner, M. Janke, M. Wand, and T. Schultz Conversion from facial myoelectric signals to speech: a unit selection approach Proc. Interspeech. 2014 1184 1188
62. H. Zen, K. Tokuda, and A.W. Black Statistical parametric speech synthesis Speech Commun. 51 November (11) 2009 1039 1064