Memory-enhanced neural networks and NMF for robust ASR

IEEE Transactions on Audio, Speech and Language Processing

Volumn 22, Issue 6, 2014, Pages 1037-1046

  Geiger, Jürgen T ^a   Weninger, Felix ^a   Gemmeke, Jort F ^d   Wöllmer, Martin ^c   Schuller, Björn ^a,b   Rigoll, Gerhard ^a  

^a TECHNICAL UNIVERSITY OF MUNICH   (Germany)

^b IMPERIAL COLLEGE LONDON   (United Kingdom)

^c BMW GROUP   (Germany)

^d UNIVERSITY OF LEUVEN   (Belgium)

Author keywords

Long short term memory; Multi stream recognition; Noise robust speech recognition; Non negative matrix factorization

Indexed keywords

ACOUSTIC NOISE; ARTS COMPUTING; BRAIN; FACTORIZATION; MATRIX ALGEBRA; RECURRENT NEURAL NETWORKS; REVERBERATION; SPEECH; SPEECH ENHANCEMENT;

DISCRIMINATIVE TRAINING; DISTANT SPEECH RECOGNITION; LONG SHORT-TERM MEMORY; MULTI-STREAM; NOISE ROBUST SPEECH RECOGNITION; NONNEGATIVE MATRIX FACTORIZATION; ROBUSTNESS AGAINST NOISE; SPEECH ENHANCEMENT METHODS;

SPEECH RECOGNITION;

EID: 84910095643     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASLP.2014.2318514     Document Type: Article

References (44)

1. T. Virtanen, R. Singh, and B. Raj, Techniques for noise robustness in automatic speech recognition. New York, NY, USA: Wiley, 2012.
2. R. Martin, "Noise power spectral density estimation based on optimal smoothing and minimum statistics," IEEE Trans. Speech Audio Process., vol. 9, no. 5, pp. 504-512, Jul. 2001.
3. S. J. Rennie, J. R. Hershey, and P. A. Olsen, "Efficient model-based speech separation and denoising using non-negative subspace analysis," in Proc. ICASSP, Las Vegas, NV, USA, 2008, pp. 1833-1836.
4. P. Smaragdis, "Convolutive speech bases and their application to supervised speech separation," IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 1, pp. 1-14, Jan. 2007.
5. H. Hermansky, N. Morgan, A. Bayya, and P. Kohn, "RASTA-PLP speech analysis technique," in Proc. ICASSP, San Francisco, CA,USA, 1992, vol. 1, pp. 121-124.
6. A. Krueger and R. Haeb-Umbach, "Model-based feature enhancement for reverberant speech recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 7, pp. 1692-1707, Sep. 2010.
7. R. Haeb-Umbach and H. Ney, "Linear discriminant analysis for improved large vocabulary continuous speech recognition," in Proc. ICASSP, San Francisco, CA, USA, 1992, pp. 13-16.
8. D. Povey, D. Kanevsky, B. Kingsbury, B. Ramabhadran, G. Saon, and K. Visweswariah, "BoostedMMI for model and feature-space discriminative training," in Proc. ICASSP, Las Vegas, NV, USA, 2008, pp. 4057-4060.
9. D. Y. Kim, C. Kwan Un, and N. S. Kim, "Speech recognition in noisy environments using first-order vector Taylor series," Speech Commun., vol. 24, no. 1, pp. 39-49, 1998.
10. G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-R. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, and B. Kingsbury, "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups," IEEE Signal Process. Mag., vol. 29, no. 6, pp. 82-97, Nov. 2012.
11. M. Seltzer, D. Yu, and Y. Wang, "An investigation of deep neural networks for noise robust speech recognition," in Proc. ICASSP, Vancouver, BC, Canada, 2013, pp. 7398-7402.
12. O. Vinyals, S. V. Ravuri, and D. Povey, "Revisiting recurrent neural networks for robust ASR," in Proc. ICASSP, Kyoto, Japan, 2012, pp. 4085-4088.
13. A. Graves, A.-R. Mohamed, and G. Hinton, "Speech recognition with deep recurrent neural networks," in Proc. ICASSP, 2013, pp. 6645-6649.
14. S. Hochreiter, Y. Bengio, P. Frasconi, and J. Schmidhuber, "Gradient flow in recurrent nets: The difficulty of learning long-term dependencies," in Field Guide to Dynamical Recurrent Networks, S. C. Kremer and J. F. Kolen, Eds. Piscataway, NJ, USA: IEEE Press, 2001.
15. S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural Comput., vol. 9, no. 8, pp. 1735-1780, 1997.
16. M. Wöllmer, F. Eyben, B. Schuller, Y. Sun, T. Moosmayr, and N. Nguyen-Thien, "Robust in-car spelling recognition-a tandem BLSTM-HMM approach," in Proc. Interspeech, Brighton, U.K., 2009, pp. 2507-2510.
17. M. Wöllmer, F. Eyben, B. Schuller, and G. Rigoll, "A multi-stream ASR framework for BLSTM modeling of conversational speech," in Proc. ICASSP, Prague, Czech Republic, 2011, pp. 4860-4863.
18. J. A. Bilmes and C. Bartels, "Graphical model architectures for speech recognition," IEEE Signal Process. Mag., vol. 22, no. 5, pp. 89-100, Sep. 2005.
19. A. Hagen and A. Morris, "Recent advances in the multi-stream HMM/ANN hybrid approach to noise robust ASR," Comput. Speech Lang., vol. 19, no. 1, pp. 3-30, 2005.
20. Y. Sun, J. F. Gemmeke, B. Cranen, L. tenBosch, and L. Boves, "Using a DBN to integrate Sparse Classification and GMM-based ASR," in Proc. Interspeech, Makuhari, Japan, 2010, pp. 2098-2101.
21. J. P. Barker, E. Vincent, N. Ma, H. Christensen, and P. D. Green, "The PASCAL CHiME speech separation and recognition challenge," Comput. Speech Lang., vol. 27, no. 3, pp. 621-633, 2013.
22. E. Vincent, J. Barker, S. Watanabe, J. Le Roux, F. Nesta, and M. Matassoni, "The second 'CHiME' speech separation and recognition challenge: Datasets, tasks and baselines," in Proc. ICASSP, Vancouver, BC, Canada, 2013, pp. 126-130.
23. M. Wöllmer, F. Weninger, J. Geiger,B. Schuller, and G. Rigoll, "Noise Robust ASR in Reverberated Multisource Environments Applying Convolutive NMF and Long Short-Term Memory," Comput. Speech Lang., Special Issue Speech Separat. Recogn. Multisource Environ., vol. 27, pp. 780-797, 2013.
24. J. T. Geiger, F. Weninger, A. Hurmalainen, J. F. Gemmeke, M. Wöllmer, B. Schuller, G. Rigoll, and T. Virtanen, "The TUM+TUT+KUL Approach to the 2nd CHiME Challenge: Multi-Stream ASR Exploiting BLSTM Networks and Sparse NMF," in Proc. CHiME Workshop, Vancouver, BC, Canada, 2013, pp. 25-30.
25. J. Gemmeke, T. Virtanen, and A. Hurmalainen, "Exemplar-based sparse representations for noise robust automatic speech recognition," IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 7, pp. 2067-2080, Sep. 2011.
26. M. Delcroix, Y. Kubo, T. Nakatani, and A. Nakamura, "Is speech enhancement pre-processing still relevant when using deep neural networks for acoustic modeling?," in Proc. Interspeech, Lyon, France, 2013, pp. 2992-2996.
27. Y. Tachioka, S. Watanabe, and J. R. Hershey, "Effectiveness of discriminative training and feature transformation for reverberated and noisy speech," in Proc. ICASSP, Vancouver, BC, Canada, 2013, pp. 6935-6939.
28. D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlícek, Y. Qian, P. Schwarz, J. Silovsky, G. Stemmer, and K. Vesely, "The Kaldi speech recognition toolkit," in Proc. ASRU, Honolulu, HI, USA, 2011.
29. G. Saon, M. Padmanabhan, R. Gopinath, and S. Chen, "Maximum likelihood discriminant feature spaces," in Proc. ICASSP, Istanbul, Turkey, 2000, pp. 1129-1132.
30. M. J. Gales, "Maximum likelihood linear transformations for HMM-based speech recognition," Comput. Speech Lang., vol. 12, no. 2, pp. 75-98, 1998.
31. H. Kallasjoki, U. Remes, J. F. Gemmeke, T. Virtanen, and K. J. Palomäki, "Uncertainty measures for improving exemplar-based source separation," in Proc. INTERSPEECH, Florence, Italy, 2011, pp. 469-472.
32. T. Sainath, B. Ramabhadran, D. Nahamoo, D. Kanevsky, D. Van Compernolle, K. Demuynck, J. Gemmeke, J. Bellegarda, and S. Sundaram, "Exemplar-based processing for speech recognition: An overview," IEEE Signal Process. Mag., vol. 29, no. 6, pp. 98-113, Nov. 2012.
33. A. Hurmalainen, J. F. Gemmeke, and T. Virtanen, "Compact long context spectral factorisation models for noise robust recognition of medium vocabulary speech," in Proc. CHiME Workshop, Vancouver, BC, Canada, 2013, pp. 13-18.
34. F. Weninger, M. Wöllmer, and B. Schuller, "Combining Bottleneck-BLSTM and Semi-Supervised Sparse NMF for Recognition of Conversational Speech in Highly Instationary Noise," in Proc. Interspeech, Portland, OR, USA, 2012, pp. 302-305.
35. M. Schuster and K. K. Paliwal, "Bidirectional recurrent neural networks," IEEE Trans. Signal Process., vol. 45, no. 11, pp. 2673-2681, Nov. 1997.
36. A. Graves and J. Schmidhuber, "Framewise phoneme classification with bidirectional LSTM and other neural network architectures," Neural Netw., vol. 18, no. 5-6, pp. 602-610, 2005.
37. A. Graves, "Supervised sequence labelling with recurrent neural networks," Ph.D. dissertation, Technische Univ. München, Munich, Germany, 2008.
38. A. Mohamed, G. Dahl, and G. Hinton, "Acoustic modeling using deep belief networks," IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 1, pp. 14-22, Jan. 2012.
39. Y. Tachioka, S. Watanabe, J. Le Roux, and J. R. Hershey, "Discriminative methods for noise robust speech recognition: A CHiME challenge benchmark," in Proc. CHiME Workshop, Vancouver, BC, Canada, 2013, pp. 19-24.
40. M. Wöllmer, A. Metallinou, N. Katsamanis, B. Schuller, and S. Narayanan, "Analyzing the memory of BLSTM neural networks for enhanced emotion classification in dyadic spoken interactions," in Proc. ICASSP, Kyoto, Japan, 2012, pp. 4157-4160.
41. J. F. Gemmeke, A. Hurmalainen, T. Virtanen, and Y. Sun, "Toward a practical implementation of exemplar-based noise robust ASR," in Proc. EUSIPCO, Barcelona, Spain, 2011, pp. 1490-1494.
42. T. Virtanen, J. Gemmeke, and B. Raj, "Active-set Newton algorithm for overcomplete non-negative representations of audio," IEEE Trans. Audio, Speech, Lang. Process., vol. 21, no. 11, pp. 2277-2289, Nov. 2013.
43. F. Nesta, M. Matassoni, and R. F. Astudillo, "A flexible spatial blind source extraction framework for robust speech recognition in noisy environments," in Proc. CHiME Workshop, Vancouver, BC, Canada, 2013, pp. 33-38.
44. C. Weng, D. Yu, S. Watanabe, and B.-H. Juang, "Recurrent deep neural networks for robust speech recognition," in Proc. ICASSP, Florence, Italy, 2014, to be published.