Tackling speaking mode varieties in EMG-based speech recognition

IEEE Transactions on Biomedical Engineering

Volumn 61, Issue 10, 2014, Pages 2515-2526

  Wand, Michael ^a   Janke, Matthias ^a   Schultz, And Tanja ^a  

^a KARLSRUHE INSTITUTE OF TECHNOLOGY   (Germany)

Author keywords

Electromyography (EMG); EMG based speech recognition; silent speech interfaces (SSI)

Indexed keywords

CONFORMAL MAPPING; ELECTRIC POTENTIAL; ELECTROMYOGRAPHY; PHOTOMAPPING; SPEECH;

CONTINUOUS SPEECH; ELECTROMYOGRAPHIC; MULTIMODE SYSTEM; RECOGNITION ACCURACY; SEAMLESS SWITCHING; SILENT SPEECH INTERFACES; SPECTRAL MAPPINGS; WORD ERROR RATE;

SPEECH RECOGNITION;

ARTICLE; ELECTRODE; ELECTROMYOGRAPHY; ELECTROPHYSIOLOGY; HUMAN; SPEECH AND LANGUAGE ASSESSMENT; SPEECH DISCRIMINATION; WORD ERROR RATE; ADULT; ALGORITHM; FACE MUSCLE; FEMALE; MALE; PHYSIOLOGY; PROCEDURES; SIGNAL PROCESSING; SPEECH; YOUNG ADULT;

ADULT; ALGORITHMS; ELECTROMYOGRAPHY; FACIAL MUSCLES; FEMALE; HUMANS; MALE; SIGNAL PROCESSING, COMPUTER-ASSISTED; SPEECH; YOUNG ADULT;

EID: 84907468717     PISSN: 00189294     EISSN: 15582531     Source Type: Journal    
DOI: 10.1109/TBME.2014.2319000     Document Type: Article

References (58)

1. T. Schultz and M. Wand, "Modeling coarticulation in large vocabulary EMG-based speech recognition," Speech Commun., vol. 52, no. 4, pp. 341-353, 2010
2. M. Janke, M. Wand, and T. Schultz, "A spectral mapping method for EMG-based recognition of silent speech," in Proc. B-Interface, 2010, pp. 22-31
3. V.-M. Florescu, L. Crevier-Buchman, B. Denby, T. Hueber, A. Colazo-Simon, C. Pillot-Loiseau, P. Roussel, C. Gendrot, and S. Quattrocchi, "Silent vs vocalized articulation for a portable ultrasound-based silent speech interface," in Proc. Interspeech, 2010, pp. 450-453
4. B. Denby, T. Schultz, K. Honda, T. Hueber, and J. Gilbert, "Silent speech interfaces," Speech Commun., vol. 52, no. 4, pp. 270-287, 2010
5. R. Merletti and P. A. Parker, Eds., Electromyography: Physiology, Engineering, and Noninvasive Applications. New York, NY, USA: Wiley, 2004
6. Y. Deng, R. Patel, J. T. Heaton, G. Colby, L. D. Gilmore, J. Cabrera, S. H. Roy, C. J. D. Luca, and G. S. Meltzner, "Disordered speech recognition using acoustic and sEMG signals," in Proc. Interspeech, 2009, pp. 644-647
7. N. Sugie and K. Tsunoda, "A speech prosthesis employing a speech synthesizer-Vowel discrimination from perioral muscle activities and vowel production," IEEE Trans. Biomed. Eng., vol. BME-32, no. 7, pp. 485-490, Jul. 1985
8. M. S. Morse and E. M. OBrien, "Research summary of a scheme to ascertain the availability of speech information in the myoelectric signals of neck and head muscles using surface electrodes," Comput. Biol. Med., vol. 16, no. 6, pp. 399-410, 1986
9. C. Jorgensen and K. Binsted, "Web browser control using EMG based sub vocal speech recognition," in Proc. 38th Annu. Hawaii Int. Conf. Syst. Sci., 2005, p. 294c.
10. S.-C. Jou, T. Schultz, M. Walliczek, F. Kraft, and A. Waibel, "Towards continuous speech recognition using surface electromyography," in Proc. Interspeech, 2006, pp. 573-576
11. S.-C. Jou, L. Maier-Hein, T. Schultz, and A.Waibel, "Articulatory feature classification using surface electromyography," in Proc. Int.Conf. Acoust., Speech, Signal Process., 2006, pp. I-605-I-608
12. S.-C. Jou, T. Schultz, and A. Waibel, "Continuous electromyographic speech recognition with a multi-stream decoding architecture," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2007, pp. IV-401-IV-404
13. M. Janke, M. Wand, and T. Schultz, "Impact of lack of acoustic feedback in EMG-based silent speech recognition," in Proc. Interspeech, 2010, pp. 2686-2689
14. M. Wand, M. Janke, and T. Schultz, "Investigations on Speaking Mode Discrepancies in EMG-based Speech Recognition," in Proc. Interspeech, 2011, pp. 601-604
15. M. Wand, M. Janke, and T. Schultz, "Decision-Tree based analysis of speakingmode discrepancies in EMG-based speech recognition," in Proc. Biosignals, 2012, pp. 101-109
16. M. Wand and T. Schultz, "Session-independent EMG-based Speech Recognition," in Proc. Biosignals, 2011, pp. 295-300
17. M.Wand, C. Schulte, M. Janke, and T. Schultz, "Array-based electromyographic silent speech interface," in Proc. Biosignals, 2013, pp. 89-96
18. M. Wand, A. Himmelsbach, T. Heistermann, M. Janke, and T. Schultz, "Artifact removal algorithm for an EMG-based silent speech interface," in Proc. 35th Ann. Int. Conf. Eng. Med. Biol. Soc., 2013, pp. 5750-5753
19. C. Jorgensen and S. Dusan, "Speech interfaces based upon surface electromyography," Speech Commun., vol. 52, pp. 354-366, 2010
20. E. Lopez-Larraz, O. M. Mozos, J. M. Antelis, and J. Minguez, "Syllablebased speech recognition using EMG," in Proc. Ann. Int. Conf. Eng. Med. Biol. Soc., 2010, pp. 4699-4702
21. G. S. Meltzner, G. Colby, Y. Deng, and J. T. Heaton, "Signal acquisition and processing techniques for sEMG based silent speech recognition," in Proc. Ann. Int. Conf. IEEE Eng. Med. Biol. Soc., 2011, pp. 4848-4851
22. J. Freitas, A. Teixeira, and M. S. Dias, "Towards a silent speech interface for portuguese," in Proc. Biosignals, 2012, pp. 91-100
23. A. Toth,M.Wand, and T. Schultz, "Synthesizing speech from electromyography using voice transformation techniques," in Proc. Interspeech, 2009, pp. 652-655
24. K.-S. Lee, "Prediction of acoustic feature parameters using myoelectric signals," IEEE Trans. Biomed. Eng., vol. 57, no. 7, pp. 1587-1595, Jul. 2010
25. C. Johner, M. Janke, M. Wand, and T. Schultz, "Inferring prosody from facial cues for EMG-based synthesis of silent speech," in Proc. 4th Int. Conf. Appl. Human Factors Ergon., 2012, pp. 5317-5326
26. T. Ito, K. Takeda, and F. Itakura, "Analysis and recognition of whispered speech," Speech Commun., vol. 45, pp. 139-152, 2005
27. Y. Nakajima, H. Kashioka, N. Campbell, and K. Shikano, "Non-Audible Murmur (NAM) recognition," IEICE Trans. Inf. Syst., vol. E89-D, no. 1, pp. 1-4, Jan. 2006
28. J. Frowen andA. R. Perry, "Reasons for success or failure in surgical voice restoration after total laryngectomy-An Australian study," J. Laryngol. Otol., vol. 115, no. 5, pp. 393-399, 2001
29. G. S. Meltzner and R. E. Hillman, "Impact of aberrant acoustic properties on the perception of sound quality in electrolarynx speech," J. Speech, Lang., Hear. Res., vol. 48, pp. 766-779, Aug. 2005
30. C. E. Stepp, J. T. Heaton, R. G. Rolland, and R. E. Hillman, "Neck and face surface electromyography for prosthetic voice control after total laryngectomy," IEEE Trans. Neural Syst. Rehabil. Eng., vol. 17, no. 2, pp. 146-155, Apr. 2009
31. H. S. Milner-Brown, R. B. Stein, and R. Yemm, "Changes in firing rate of human motor units during linearly changing voluntary contractions," J. Physiol., vol. 230, pp. 371-390, 1973
32. C. J. de Luca, "Physiology andmathematics of myoelectric signals," IEEE Trans. Biomed. Eng., vol. BME-26, no. 6, pp. 313-325, Jun. 1979
33. S. H. Nawab, R. P. Wotiz, and C. J. D. Luca, "Decomposition of indwelling EMG signals," J. Appl. Physiol., vol. 105, pp. 700-710, 2008
34. C. J. de Luca, A. Adam, R. Wotiz, L. D. Gilmore, and S. H. Nawab, "Decomposition of surfaceEMGsignals," J. Neurophysiol., vol. 96, pp. 1646-1657, 2006
35. M. Schünke, E. Schulte, and U. Schumacher, Prometheus-Lernatlas der Anatomie. Kopf und Neuroanatomie. Stuttgart, Germany: Thieme Verlag, 2006. vol. [3].
36. L. Maier-Hein, F. Metze, T. Schultz, and A. Waibel, "Session independent non-Audible speech recognition using surface electromyography," in Proc. IEEE Workshop Autom. Speech Recog. Understanding, 2005, pp. 331-336
37. P. R. Cavanagh and P. V. Komi, "Electromechanical delay in human skeletal muscle under concentric and eccentric contractions," Eur. J. Appl. Physiol. Occupat. Physiol., vol. 42, no. 3, pp. 159-163, 1979
38. T. Heistermann, M. Janke, M. Wand, and T. Schultz, "Spatial artifact detection for multi-channel EMG-based speech recognition," in Proc. Biosignals, 2014, pp. 189-196
39. M. Wand and T. Schultz, "Towards speaker-Adaptive speech recognition based on surface electromyography," in Proc. Biosignals, 2009, pp. 155-162
40. ,"International Phonetic Association,"Handbook of the International Phonetic Association. Cambridge, U.K.: Cambridge Univ. Press, 1999
41. F. Metze and A. Waibel, "A flexible stream architecture for ASR using articulatory features," in Proc. Int. Conf. Spoken Lang. Process., 2002, pp. 2133-2136
42. K. Kirchhoff, "Robust speech recognition using articulatory information" Ph.D. dissertation, Dept. Appl. Comput. Sci., Univ. Bielefeld, Bielefeld, Germany, 1999
43. J. Frankel, M.Wester, and S. King, "Articulatory feature recognition using dynamic Bayesian networks," in Proc. Int. Conf. Spoken Lang. Process., 2004, pp. 660-663
44. M. Finke and I. Rogina, "Wide context acoustic modeling in read versus spontaneous speech," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 1997, pp. 1743-1746
45. K.-F. Lee, "Large-vocabulary speaker-independent continuous speech recognition: The SPHINX system" Ph.D. dissertation, Dept. Comput. Sci., Carnegie Mellon Univ., Pittsburgh, PA, USA, 1988
46. N. Ueda, R. Nakano, Z. Ghahramani, and G. E. Hinton, "Split and merge EM algorithm for improving Gaussian mixture density estimates," J. VLSI Signal Process., vol. 26, pp. 133-140, 2000
47. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. New York, NY, USA: Wiley, 2000
48. M. Wand, S.-C. Jou, A. R. Toth, and T. Schultz, "Impact of different speakingmodes on EMG-based speech recognition," in Proc. Interspeech, 2009, pp. 648-651
49. E. Lombard, "Le signe de lelevation De la voix.," Ann. Mal. Oreille Larynx, vol. 37, pp. 101-119, 1911
50. J. A. Tourville, K. J. Reilly, and F. H. Guenther, "Neural mechanisms underlying auditory feedback control of speech," NeuroImage, vol. 32, pp. 1429-1443, 2008
51. J. Perkell, M. Matthies, H. Lane, F. Guenther, R. Wilhelms-Tricaricoa, J.Wozniaka, and P. Guioda, "Speech motor control: Acoustic goals, saturation effects, auditory feedback and internal models," Speech Commun., vol. 22, pp. 227-250, 1997
52. F. H. Guenther, S. S. Ghosh, and J. A. Tourville, "Neural modeling and imaging of the cortical interactions underlying syllable production," Brain Lang., vol. 96, pp. 280-301, 2006
53. C. Herff, M. Janke, M.Wand, and T. Schultz, "Impact of different feedback mechanisms in EMG-based speech recognition," in Proc. Interspeech, 2011, pp. 2213-2216
54. T. Schultz and A. Waibel, "Language independent and language adaptive acoustic modeling for speech recognition," Speech Commun., vol. 35, pp. 31-51, 2001
55. E. Eide, H. Gish, P. Jeanrenaud, and A. Mielke, "Understanding and improving speech recognition performance through the use of diagnostic tools," in Proc. Int. Conf. Acoust., Speech, Signal Process., 1995, pp. 221-224
56. T. Schaaf and F. Metze, "Analysis of gender normalization using MLP and VTLN features," in Proc. Interspeech, 2010, pp. 306-309
57. P. Welch, "The use of fast fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms," IEEE Trans. Audio Electroacoust., vol. AU-15, no. 2, pp. 70-73, Jun. 1967
58. M. J. F. Gales, "Maximum likelihood linear transformations for HMMbased speech recognition", Eng. Dept., Cambridge Univ., Cambridge, MA, USA, 1997