Making machines understand us in reverberant rooms: Robustness against reverberation for automatic speech recognition

IEEE Signal Processing Magazine

Volumn 29, Issue 6, 2012, Pages 114-126

  Yoshioka, Takuya ^a   Sehr, Armin ^b   Delcroix, Marc ^a   Kinoshita, Keisuke ^a   Maas, Roland ^a   Nakatani, Tomohiro ^a   Kellermann, Walter ^c  

^a NTT Communication Science Laboratories   (Japan)

^b UNIVERSITY OF ERLANGEN NUREMBERG   (Germany)

^c NONE

Author keywords

[No Author keywords available]

Indexed keywords

ARCHITECTURAL ACOUSTICS; CONSUMER PRODUCTS; DEEP NEURAL NETWORKS; DIGITAL TELEVISION; INFORMATION SERVICES; INTERACTIVE TELEVISION; MICROPHONES; RESEARCH LABORATORIES; REVERBERATION; SPEECH;

AUTOMATIC ANNOTATION; AUTOMATIC SPEECH RECOGNITION; DIGITAL SERVICES; INTERACTIVE TV; REVERBERANT ROOM; SPEECH RECOGNITION TECHNOLOGY; SPEECH RECOGNIZER; SPEECH-TO-SPEECH TRANSLATION;

SPEECH RECOGNITION;

EID: 85032751613     PISSN: 10535888     EISSN: None     Source Type: Journal    
DOI: 10.1109/MSP.2012.2205029     Document Type: Article

References (48)

1. M. Wölfel and J. McDonough, Distant Speech Recognition. Hoboken, NJ: Wiley, 2009.
2. J. Gauvain and C.-H. Lee, "Maximum a posteriori estimation for multivariate Gaussian mixture observation of Markov chains," IEEE Trans. Speech Audio Process., vol. 2, no. 2, pp. 291-298, 1994.
3. C. Legetter and P. Woodland, "Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov models," Comput. Speech Language, vol. 9, no. 2, pp. 171-185, 1995.
4. M. J. F. Gales and S. J. Young, "Robust continuous speech recognition using parallel model combination," IEEE Trans. Speech Audio Process., vol. 4, no. 5, pp. 352-359, 1996. (Pubitemid 126753023)
5. A. Acero, L. Deng, T. Kristjansson, and J. Zhang, "HMM adaptation using vector Taylor series for noisy speech recognition," in Proc. Int. Conf. Spoken Language Process., 2000, pp. 869-872.
6. J. Droppo and A. Acero, "Environmental robustness," in Springer Handbook of Speech Processing, J. Benesty, M. M. Sondhi, and Y. Huang, Eds. Berlin: Springer-Verlag, 2008, pp. 653-679.
7. H. Hermansky and N. Morgan, "RASTA processing of speech," IEEE Trans. Speech Audio Process., vol. 2, no. 4, pp. 578-589, 1994.
8. B. E. D. Kingsbury, N. Morgan, and S. Greenberg, "Robust speech recognition using the modulation spectrogram," Speech Commun., vol. 25, no. 1-3, pp. 117-132, 1998. (Pubitemid 128413637)
9. D. Gelbart and N. Morgan, "Double the trouble: handling noise and reverberation in far-field automatic speech recognition," in Proc. Int. Conf. Spoken Language Process., 2002, pp. 2185-2188.
10. M. L. Seltzer, B. Raj, and R. M. Stern, "Likelihood-maximizing beamforming for robust hands-free speech recognition," IEEE Trans. Speech Audio Process., vol. 12, no. 5, pp. 489-498, 2004.
11. H. Kuttruff, Room Acoustics, 5th ed., Abingdon, Oxon: Spon Press, 2009.
12. J. B. Allen and D. A. Berkley, "Image method for efficiently simulating smallroom acoustics," J. Acoust. Soc. Amer., vol. 65, no. 4, pp. 943-950, 1979.
13. T. Nishiura, Y. Hirano, Y. Denda, and M. Nakayama, "Investigations into early and late reflections on distant-talking speech recognition toward suitable reverberation criteria," in Proc. Interspeech, 2007, pp. 1082-1085.
14. X. Huang, A. Acero, and H.-W. Hon, Spoken Language Processing: A Guide to Theory, Algorithm, and System Development. Englewood Cliffs, NJ: Prentice Hall, 2001.
15. S. Furui, "Speaker-independent isolated word recognition using dynamic features of speech spectrum," IEEE Trans. Acoust., Speech, Signal Process., vol. 34, no. 1, pp. 52-59, 1986. (Pubitemid 16575387)
16. M. J. F. Gales and J.-Q. Wang, "Model-based approaches to handling additive noise in reverberant environments," in Proc. Joint Workshop Hands-free Speech Commun. Microphone Arrays, 2011, pp. 121-126.
17. P. A. Naylor and N. G. Gaubitch, Eds., Speech Dereverberation. Berlin: Springer-Verlag, 2010.
18. T. H. Falk and W.-Y. Chan, "Modulation spectral features for robust far-field speaker identification," IEEE Trans. Audio, Speech, Language Process., vol. 18, no. 1, pp. 90-100, 2010.
19. S. Ganapathy, J. Pelecanos, and M. K. Omar, "Feature normalization for speaker verification in room reverberation," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2011, pp. 4836-4839.
20. M. Miyoshi and Y. Kaneda, "Inverse filtering of room acoustics," IEEE Trans. Acoust., Speech, Signal Process., vol. 36, no. 2, pp. 145-152, 1988.
21. T. Nakatani, T. Yoshioka, K. Kinoshita, M. Miyoshi, and B.-H. Juang, "Speech dereverberation based on variance-normalized delayed linear predictor," IEEE Trans. Audio, Speech, Language Process., vol. 18, no. 7, pp. 1717-1731, 2010.
22. A. Krueger and R. Haeb-Umbach, "A model-based approach to joint compensation of noise and reverberation for speech recognition," in Robust Speech Recognition of Uncertain or Missing Data: Theory and Applications, D. Kolossa and R. Haeb-Umbach, Eds. Berlin: Springer-Verlag, 2011, pp. 257-290.
23. S. Haykin, Adaptive Filter Theory, 4th ed. Englewood Cliffs, NJ: Prentice Hall, 2001.
24. B. W. Gillespie and L. E. Atlas, "Strategies for improving audible quality and speech recognition accuracy of reverberant speech," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2003, pp. 676-679.
25. J. R. Hopgood and P. J. W. Rayner, "Blind single channel deconvolution using nonstationary signal processing," IEEE Trans. Speech Audio Process., vol. 11, no. 5, pp. 476-488, 2003.
26. M. Triki and D. T. M. Slock, "Delay and predict equalization for blind speech dereverberation," in Proc. Int. Conf. Acoust, Speech, Signal Process., 2006, pp. V-97-V-100. (Pubitemid 46500976)
27. H. Buchner and W. Kellermann, "TRINICON for dereverberation of speech and audio signals," in Speech Dereverberation, P. A. Naylor and N. D. Gaubitch, Eds. Berlin: Springer-Verlag, 2010, pp. 311-385.
28. K. Kinoshita, M. Delcroix, T. Nakatani, and M. Miyoshi, "Suppression of late reverberation effect on speech signal using long-term multiple-step linear prediction," IEEE Trans. Audio, Speech, Language Process., vol. 17, no. 4, pp. 534-545, 2009.
29. T. Hori, S. Araki, T. Yoshioka, M. Fujimoto, S. Watanabe, T. Oba, A. Ogawa, K. Otsuka, D. Mikami, K. Kinoshita, T. Nakatani, A. Nakamura, and J. Yamato, "Low-latency real-time meeting recognition and understanding using distant microphones and omni-directional camera," IEEE Trans. Audio, Speech, Language Process., vol. 20, no. 2, pp. 499-513, 2012.
30. M. Delcroix, S. Watanabe, and T. Nakatani, "Variance compensation for recognition of reverberant speech with dereverberation preprocessing," in Robust Speech Recognition of Uncertain or Missing Data, R. Haeb-Umbach and D. Kolossa, Eds. Berlin: Springer-Verlag, 2011, pp. 225-256.
31. T. Yoshioka, T. Nakatani, M. Miyoshi, and H. G. Okuno, "Blind separation and dereverberation of speech mixtures by joint optimization," IEEE Trans. Audio, Speech, Language Process., vol. 19, no. 1, pp. 69-84, 2011.
32. E. A. P. Habets, "Single- and multi-microphone speech dereverberation using spectral enhancement," Ph.D. dissertation, Eindhoven Univ. Technology, Eindhoven, The Netherlands, 2006.
33. J. S. Erkelens and R. Heusdens, "Correlation-based and model-based blind single-channel late-reverberation suppression in noisy time-varying acoustical environments," IEEE Trans. Audio, Speech, Language Process., vol. 18, no. 7, pp. 1746-1765, 2010.
34. H. Kameoka, T. Nakatani, and T. Yoshioka, "Robust speech dereverberation based on non-negativity and sparse nature of speech spectrograms," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2009, pp. 45-48.
35. K. Kumar, B. Raj, R. Singh, and R. Stern, "An iterative least-squares technique for dereverberation," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2011, pp. 5488-5491.
36. K. Lebart, J. M. Boucher, and P. N. Denbigh, "A new method based on spectral subtraction for speech dereverberation," Acta Acustica United with Acustica, vol. 87, no. 3, pp. 359-366, 2001. (Pubitemid 32699291)
37. M. Wölfel, "Enhanced speech features by single-channel joint compensation of noise and reverberation," IEEE Trans. Audio, Speech, Language Process., vol. 17, no. 2, pp. 312-323, 2009.
38. L. Deng, J. Droppo, and A. Acero, "Dynamic compensation of HMM variances using the feature enhancement uncertainty computed from a parametric model of speech distortion," IEEE Trans. Speech Audio Process., vol. 13, no. 3, pp. 412-421, 2005. (Pubitemid 40666175)
39. B. Raj and R. M. Stern, "Missing-feature approaches in speech recognition," IEEE Signal Processing Mag., vol. 22, no. 5, pp. 101-116, 2005. (Pubitemid 41488524)
40. K. J. Palomäki, G. J. Brown, and J. P. Barker, "Techniques for handling convolutional distortion with 'missing data' automatic speech recognition," Speech Commun., vol. 43, no. 1-2, pp. 123-142, 2004.
41. C. K. Raut, T. Nishimoto, and S. Sagayama, "Model adaptation for long convolutional distortion by maximum likelihood based state filtering approach," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2006, pp. I-1133-I-1136. (Pubitemid 46500021)
42. H.-G. Hirsch and H. Finster, "A new approach for the adaptation of HMMs to reverberation and background noise," Speech Commun., vol. 50, no. 3, pp. 244-263, 2008. (Pubitemid 351172473)
43. A. Sehr, M. Gardill, and W. Kellermann, "Adapting HMMs of distant-talking ASR systems using feature-domain reverberation models," in Proc. European Signal Process. Conf., 2009, pp. 540-543.
44. Y.-Q. Wang and M. J. F. Gales, "Improving reverberant VTS for hands-free robust speech recognition," in Proc. Workshop. Automat. Speech Recognition, Understanding, 2011, pp. 113-118.
45. T. Takiguchi, M. Nishimura, and Y. Ariki, "Acoustic model adaptation using first-order linear prediction for reverberant speech," IEICE Trans. Inform. Syst., vol. E89-D, no. 3, pp. 908-914, 2006.
46. A. Sehr, R. Maas, and W. Kellermann, "Reverberation model-based decoding in the logmelspec domain for robust distant-talking speech recognition," IEEE Trans. Audio, Speech, Language Process., vol. 18, no. 7, pp. 1676-1691, 2010.
47. D. Povey, B. Kingsbury, L. Mangu, G. Saon, H. Soltau, and G. Zweig, "fMPE: discriminatively trained features for speech recognition," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2005, pp. 961-964.
48. M. J. F. Gales, "Semi-tied covariance matrices for hidden Markov models," IEEE Trans. Speech Audio Process., vol. 7, no. 3, pp. 272-281, 1999.