Noise robust ASR in reverberated multisource environments applying convolutive NMF and Long Short-Term Memory

Computer Speech and Language

Volumn 27, Issue 3, 2013, Pages 780-797

  Wöllmer, Martin ^a   Weninger, Felix ^a   Geiger, Jürgen ^a   Schuller, Björn ^a   Rigoll, Gerhard ^a  

^a TECHNICAL UNIVERSITY OF MUNICH   (Germany)

Author keywords

Automatic speech recognition; Long Short Term Memory; Non negative matrix factorization; Tandem feature extraction

Indexed keywords

BRAIN; EXTRACTION; FACTORIZATION; FEATURE EXTRACTION; HIDDEN MARKOV MODELS; MARKOV PROCESSES; MATRIX ALGEBRA; MEMORY ARCHITECTURE; REVERBERATION; SIGNAL TO NOISE RATIO; SOURCE SEPARATION; SPEECH RECOGNITION;

AUTOMATIC SPEECH RECOGNITION; CONTEXT SENSITIVE; FEATURE GENERATION; MULTI-SOURCE ENVIRONMENT; NONNEGATIVE MATRIX FACTORIZATION; SPARSE CLASSIFICATION; TEMPORAL CLASSIFICATION; TEMPORAL CONTEXT MODELS;

LONG SHORT-TERM MEMORY;

EID: 84883396653     PISSN: 08852308     EISSN: 10958363     Source Type: Journal    
DOI: 10.1016/j.csl.2012.05.002     Document Type: Article

References (45)

1. Barker, J.P., Vincent, E., Ma, N., Christensen, H., Green, P.D. The PASCAL CHiME speech separation and recognition challenge. Computer Speech and Language, submitted for publication.
2. Y. Bengio, P. Simard, and P. Frasconi Learning long-term dependencies with gradient descent is difficult IEEE Transactions on Neural Networks 5 2 1994 157 166
3. M. Cooke, J. Barker, S. Cunningham, and X. Shao An audio-visual corpus for speech perception and automatic speech recognition The Journal of the Acoustical Society of America 120 5 2006 2421 2424
4. M. Cooke, J.R. Hershey, and S.J. Rennie Monaural speech separation and recognition challenge Computer Speech and Language 24 2010 1 15
5. M. Delcroix, K. Kinoshita, T. Nakatani, S. Araki, A. Ogawa, T. Hori, S. Watanabe, M. Fujimoto, T. Yoshioka, T. Oba, Y. Kubo, M. Souden, S.J. Hahm, and A. Nakamura Speech recognition in the presence of highly non-stationary noise based on spatial, spectral and temporal speech/noise modeling combined with dynamic variance adaptation. Proc. of Machine Listening in Multisource Environments (CHiME 2011), Satellite Workshop of Interspeech 2011 Florence, Italy 2011 12 17
6. T.G. Dietterich Approximate statistical tests for comparing supervised classification learning algorithms Neural Computation 10 1998 1895 1923
7. G. Evangelista, S. Marchand, M. Plumbley, and E. Vincent Sound source separation U. Zölzer, DAFX - Digital Audio Effects 2nd edition 2011 Wiley
8. S. Fernandez, A. Graves, and J. Schmidhuber An application of recurrent neural networks to discriminative keyword spotting Proc. of ICANN Porto, Portugal 2007 220 229
9. J. Gemmeke, T. Virtanen, and A. Hurmalainen Exemplar-based sparse representations for noise robust automatic speech recognition IEEE Transactions on Audio, Speech, and Language Processing 19 7 2011 2067 2080
10. J.F. Gemmeke, T. Virtanen, and A. Hurmalainen Exemplar-based speech enhancement and its application to noise-robust automatic speech recognition Proc. of CHiME Workshop Florence, Italy 2011 53 57
11. F. Gers, J. Schmidhuber, and F. Cummins Learning to forget: continual prediction with LSTM Neural Computation 12 10 2000 2451 2471
12. A. Graves, S. Fernandez, F. Gomez, and J. Schmidhuber Connectionist temporal classification: labelling unsegmented data with recurrent neural networks Proc. of ICML Pittsburgh, USA 2006 369 376
13. A. Graves, S. Fernandez, M. Liwicki, H. Bunke, and J. Schmidhuber Unconstrained online handwriting recognition with recurrent neural networks Advances in Neural Information Processing Systems 20 2008 1 8
14. A. Graves, and J. Schmidhuber Framewise phoneme classification with bidirectional LSTM and other neural network architectures Neural Networks 18 5-6 2005 602 610
15. F. Grezl, M. Karafiat, K. Stanislav, and J. Cernocky Probabilistic and bottle-neck features for LVCSR of meetings Proc. of ICASSP 2007 757 760
16. M. Helen, and T. Virtanen Separation of drums from polyphonic music using non-negative matrix factorization and support vector machine Proc. of EUSIPCO Antalya, Turkey 2005
17. H. Hermansky, D.P.W. Ellis, and S. Sharma Tandem connectionist feature extraction for conventional HMM systems Proc. of ICASSP Istanbul, Turkey 2000 1635 1638
18. S. Hochreiter, Y. Bengio, P. Frasconi, and J. Schmidhuber Gradient flow in recurrent nets: the difficulty of learning long-term dependencies S.C. Kremer, J.F. Kolen, A Field Guide to Dynamical Recurrent Neural Networks 2001 IEEE Press 1 15
19. S. Hochreiter, and J. Schmidhuber Long short-term memory Neural Computation 9 8 1997 1735 1780
20. A. Hurmalainen, K. Mahkonen, J.F. Gemmeke, and T. Virtanen Exemplar-based recognition of speech in highly variable noise Proc. of Machine Listening in Multisource Environments (CHiME 2011), Satellite Workshop of Interspeech 2011 Florence, Italy 2011 1 5
21. Jaeger, H., 2001. The echo state approach to analyzing and training recurrent neural networks. Tech. Rep. German National Research Center for Information Technology, Bremen (Tech. Rep. No. 148).
22. K.J. Lang, A.H. Waibel, and G.E. Hinton A time-delay neural network architecture for isolated word recognition Neural Networks 3 1 1990 23 43
23. T. Lin, B.G. Horne, P. Tino, and C.L. Giles Learning long-term dependencies in NARX recurrent neural networks IEEE Transactions on Neural Networks 7 6 1996 1329 1338
24. N. Ma, J. Barker, H. Christensen, and P. Green Distant microphone speech recognition in a noisy indoor environment: combining soft missing data and speech fragment decoding Proc. of ISCA Workshop on Statistical and Perceptual Audition (SAPA) Makuhari, Japan 2010
25. B. Raj, R. Singh, and T. Virtanen Phoneme-dependent NMF for speech enhancement in monaural mixtures Proc. of Interspeech ISCA, Florence, Italy 2011 1217 1220
26. B. Raj, T. Virtanen, S. Chaudhuri, and R. Singh Non-negative matrix factorization based compensation of music for automatic speech recognition Proc. of Interspeech Makuhari, Japan 2010 717 720
27. S.J. Rennie, J.R. Hershey, and P.A. Olsen Efficient model-based speech separation and denoising using non-negative subspace analysis Proc. of ICASSP Las Vegas, NV, USA 2008 1833 1836
28. A.M. Schaefer, S. Udluft, and H.G. Zimmermann Learning long-term dependencies with recurrent neural networks Neurocomputing 71 13-15 2008 2481 2488
29. J. Schmidhuber Learning complex extended sequences using the principle of history compression Neural Computing 4 2 1992 234 242
30. M.N. Schmidt, and R.K. Olsson Single-channel speech separation using sparse non-negative matrix factorization Proc. of Interspeech Pittsburgh, PA, USA 2006
31. B. Schuller, M. Wöllmer, T. Moosmayr, and G. Rigoll Recognition of noisy speech: a comparative survey of robust model architecture and feature enhancement Journal on Audio, Speech, and Music Processing 2009 (ID 942617)
32. M. Schuster, and K.K. Paliwal Bidirectional recurrent neural networks IEEE Transactions on Signal Processing 45 1997 2673 2681
33. P. Smaragdis Discovering auditory objects through non-negativity constraints Proc. of SAPA Jeju, Korea 2004
34. P. Smaragdis Convolutive speech bases and their application to supervised speech separation IEEE Transactions on Audio, Speech and Language Processing 15 1 2007 1 14
35. W. Wang, A. Cichocki, and J.A. Chambers A multiplicative algorithm for convolutive non-negative matrix factorization based on squared Euclidean distance IEEE Transactions on Signal Processing 57 7 2009 July 2858 2864
36. F. Weninger, J. Geiger, M. Wöllmer, B. Schuller, and G. Rigoll The Munich 2011 CHiME Challenge Contribution: NMF-BLSTM Speech Enhancement and Recognition for Reverberated Multisource Environments Proc. of Machine Listening in Multisource Environments (CHiME 2011), Satellite Workshop of Interspeech 2011 Florence, Italy 2011 24 29
37. F. Weninger, A. Lehmann, and B. Schuller openBliSSART: design and evaluation of a research toolkit for blind source separation in audio recognition tasks Proc. of ICASSP Prague, Czech Republic 2011 1625 1628
38. F. Weninger, M. Wöllmer, J. Geiger, B. Schuller, J. Gemmeke, A. Hurmalainen, T. Virtanen, and G. Rigoll Non-Negative Matrix Factorization for Highly Noise-Robust ASR: to Enhance or to Recognize? Proc. of ICASSP Kyoto, Japan 2012 4681 4684
39. K.W. Wilson, B. Raj, P. Smaragdis, and A. Divakaran Speech denoising using nonnegative matrix factorization with priors Proc. of ICASSP Las Vegas, NV, USA 2008 4029 4032
40. M. Wöllmer, C. Blaschke, T. Schindl, B. Schuller, B. Färber, S. Mayer, and B. Trefflich On-line driver distraction detection using long short-term memory IEEE Transactions on Intelligent Transportation Systems 12 2 2011 574 582
41. M. Wöllmer, F. Eyben, A. Graves, B. Schuller, and G. Rigoll Bidirectional LSTM networks for context-sensitive keyword detection in a cognitive virtual agent framework Cognitive Computation 2 3 2010 180 190
42. M. Wöllmer, F. Eyben, B. Schuller, and G. Rigoll A multi-stream ASR framework for BLSTM modeling of conversational speech Proc. of ICASSP Prague, Czech Republic 2011 4860 4863
43. M. Wöllmer, and B. Schuller Enhancing spontaneous speech recognition with BLSTM features Proc. of NOLISP Las Palmas de Gran Canaria, Spain 2011 17 24
44. M. Wöllmer, B. Schuller, F. Eyben, and G. Rigoll Combining long short-term memory and dynamic Bayesian networks for incremental emotion-sensitive artificial listening IEEE Journal of Selected Topics in Signal Processing 4 5 2010 867 881
45. M. Wöllmer, B. Schuller, and G. Rigoll Feature frame stacking in RNN-based Tandem ASR systems - learned vs. predefined context Proc. of Interspeech Florence, Italy 2011 1233 1236