Joint Optimization of Masks and Deep Recurrent Neural Networks for Monaural Source Separation

IEEE/ACM Transactions on Audio Speech and Language Processing

Volumn 23, Issue 12, 2015, Pages 2136-2147

  Huang, Po Sen ^a,b   Kim, Minje ^c   Hasegawa Johnson, Mark ^a   Smaragdis, Paris ^c,d  

^a University of Illinois at Urbana Champaign   (United States)

^b Clarifai   (United States)

^c University of Illinois at Urbana Champaign   (United States)

^d ADOBE RESEARCH   (United States)

Author keywords

Deep recurrent neural network (DRNN); discriminative training; monaural source separation; time frequency masking

Indexed keywords

RECURRENT NEURAL NETWORKS; SEPARATION; SPEECH; SPEECH ANALYSIS;

DEEP RECURRENT NEURAL NETWORK (DRNN); DISCRIMINATIVE TRAINING; JOINT OPTIMIZATION; SEPARATION PERFORMANCE; SINGING VOICE SEPARATIONS; SPEECH DENOISING; SPEECH SEPARATION; TIME-FREQUENCY MASKING;

SOURCE SEPARATION;

EID: 84941334839     PISSN: 23299290     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASLP.2015.2468583     Document Type: Article

References (40)

1. P.-S. Huang, S. D. Chen, P. Smaragdis, and M. Hasegawa-Johnson, "Singing-voice separation from monaural recordings using robust principal component analysis, " in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2012, pp. 57-60.
2. A. L. Maas, Q. V. Le, T. M. O'Neil, O. Vinyals, P. Nguyen, and A. Y. Ng, "Recurrent neural networks for noise reduction in robust ASR, " in Proc. 13th Annu. Conf. Int. Speech Commun. Assoc. (INTERSPEECH), 2012, pp. 22-25.
3. P. Sprechmann, A. Bronstein, and G. Sapiro, "Real-time online singing voice separation from monaural recordings using robust low-rank modeling, " in Proc. 13th Int. Soc. Music Inf. Retrieval (ISMIR), 2012.
4. Y.-H. Yang, "Low-rank representation of both singing voice and music accompaniment via learned dictionaries, " in Proc. 14th Int. Soc. Music Inf. Retrieval Conf. (ISMIR), 2013.
5. Y.-H. Yang, "On sparse and low-rank matrix decomposition for singing voice separation, " in Proc. 20th ACM Int. Conf. Multimedia, 2012, pp. 757-760.
6. S. Boll, "Suppression of acoustic noise in speech using spectral subtraction, " IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-27, no. 2, pp. 113-120, Apr. 1979.
7. Y. Ephraim and D. Malah, "Speech enhancement using a minimum-mean square error short-time spectral amplitude estimator, " IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-32, no. 6, pp. 1109-1121, Dec. 1984.
8. D. D. Lee and H. S. Seung, "Learning the parts of objects by non-negative matrix factorization, " Nature, vol. 401, no. 6755, pp. 788-791, Oct. 1999.
9. T. Hofmann, "Probabilistic latent semantic indexing, " in Proc. 22nd Annu. Int. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, 1999, pp. 50-57.
10. P. Smaragdis, B. Raj, and M. Shashanka, "A probabilistic latent variable model for acoustic modeling, " in Proc. Adv. Models Acoust. Process., Neural Inf. Process. Syst. Workshop, 2006, vol. 148.
11. P.-S. Huang, M. Kim, M. Hasegawa-Johnson, and P. Smaragdis, "Deep learning for monaural speech separation, " in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2014, pp. 1562-1566.
12. P.-S. Huang, M. Kim, M. Hasegawa-Johnson, and P. Smaragdis, "Singing-voice separation from monaural recordings using deep recurrent neural networks, " in Proc. 15th Int. Soc. Music Inf. Retrieval (ISMIR), 2014.
13. P. Kabal, TSP speech database McGill Univ., Montreal, QC, Canada, Tech. Rep. 2002.
14. C.-L. Hsu and J.-S. Jang, "On the improvement of singing voice separation for monaural recordings using the MIR-1 K dataset, " IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 2, pp. 310-319, Feb. 2010.
15. J. Garofolo, L. Lamel, W. Fisher, J. Fiscus, D. Pallett, N. Dahlgren, and V. Zue, TIMIT: Acoustic-Phonetic Continuous Speech Corpus. Philadelphia, PA, USA: Linguistic Data Consortium, 1993.
16. Y. Xu, J. Du, L.-R. Dai, and C.-H. Lee, "A regression approach to speech enhancement based on deep neural networks, " IEEE Trans. Audio, Speech, Lang. Process., vol. 23, no. 1, pp. 7-19, Jan. 2015.
17. D. Liu, P. Smaragdis, and M. Kim, "Experiments on deep learning for speech denoising, " in Proc. 15th Annu. Conf. Int. Speech Commun. Assoc. (INTERSPEECH), 2014, pp. 2685-2689.
18. D. Wang, "Time-frequency masking for speech separation and its potential for hearing aid design, " Trends in Amplificat., vol. 12, pp. 332-353, 2008.
19. F. Weninger, F. Eyben, and B. Schuller, "Single-channel speech separation with memory-enhanced recurrent neural networks, " in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2014, pp. 3709-3713.
20. S. Nie, H. Zhang, X. Zhang, and W. Liu, "Deep stacking networks with time series for speech separation, " in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2014, pp. 6667-6671.
21. A. Narayanan and D. Wang, "Ideal ratio mask estimation using deep neural networks for robust speech recognition, " in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2013, pp. 7092-7096.
22. Y. Wang and D. Wang, "Towards scaling up classification-based speech separation, " IEEE Trans. Audio, Speech, Lang. Process., vol. 21, no. 7, pp. 1381-1390, Jul. 2013.
23. Y. Wang, A. Narayanan, and D. Wang, "On training targets for supervised speech separation, " IEEE/ACM Trans. Audio, Speech, Lang. Process., vol. 22, no. 12, pp. 1849-1858, Dec. 2014.
24. Y. Tu, J. Du, Y. Xu, L. Dai, and C.-H. Lee, "Deep neural network based speech separation for robust speech recognition, " in Proc. Int. Symp. Chinese Spoken Lang. Process., 2014, pp. 532-536.
25. E. Grais, M. Sen, and H. Erdogan, "Deep neural networks for single channel source separation, " in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2014, pp. 3734-3738.
26. M. Hermans and B. Schrauwen, "Training and analysing deep recurrent neural networks, " in Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2013, pp. 190-198.
27. R. Pascanu, C. Gulcehre, K. Cho, and Y. Bengio, "How to construct deep recurrent neural networks, " in Proc. Int. Conf. Learn. Represent., 2014.
28. X. Glorot, A. Bordes, and Y. Bengio, "Deep sparse rectifier neural networks, " in Proc. 14th Int. Conf. Artif. Intell. Statist.(AISTATS), 2011, vol. 15, pp. 315-323.
29. M. C. Mozer, "A focused back-propagation algorithm for temporal pattern recognition, " Complex Syst., vol. 3, no. 4, pp. 349-381, 1989.
30. P. J. Werbos, "Backpropagation through time: What it does and how to do it, " Proc. IEEE, vol. 78, no. 10, pp. 1550-1560, Oct. 1990.
31. R. J. Williams and D. Zipser, "Gradient-based learning algorithms for recurrent networks and their computational complexity, " in Back-propagation: Theory, Architectures, and Applications. Mahwah, NJ, USA: Lawrence Erlbaum Associates, 1995, pp. 433-486.
32. O. Yilmaz and S. Rickard, "Blind separation of speech mixtures via time-frequency masking, " IEEE Trans. Signal Process., vol. 52, no. 7, pp. 1830-1847, Jul. 2004.
33. E. Vincent, R. Gribonval, and C. Fevotte, "Performance measurement in blind audio source separation, " IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 4, pp. 1462-1469, Jul. 2006.
34. J. Bruna, P. Sprechmann, and Y. Lecun, "Source separation with scattering non-negative matrix factorization, " in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2015, pp. 1876-1880.
35. C. Taal, R. Hendriks, R. Heusdens, and J. Jensen, "An algorithm for intelligibility prediction of time-frequency weighted noisy speech, " IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 7, pp. 2125-2136, Sep. 2011.
36. R. H. Byrd, P. Lu, J. Nocedal, and C. Zhu, "A limited memory algorithm for bound constrained optimization, " SIAM J. Sci. Comput., vol. 16, no. 5, pp. 1190-1208, Sep. 1995.
37. J. Li, D. Yu, J.-T. Huang, and Y. Gong, "Improving wideband speech recognition using mixed-bandwidth training data in CD-DNN-HMM, " in Proc. IEEE Spoken Lang. Technol. Workshop (SLT), 2012, pp. 131-136.
38. A. Ozerov, P. Philippe, F. Bimbot, and R. Gribonval, "Adaptation of Bayesian models for single-channel source separation and its application to voice/music separation in popular songs, " IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 5, pp. 1564-1578, Jul. 2007.
39. F. Weninger, J. R. Hershey, J. Le Roux, and B. Schuller, "Discrimi-natively trained recurrent neural networks for single-channel speech separation, " in Proc. IEEE Global Conf. Signal and Inf. Process. (GlobalSIP) Symp. Mach. Learn. Applicat. Speech Process., 2014, pp. 577-581.
40. S. Hochreiter and J. Schmidhuber, "Long short-term memory, " Neural Comput., vol. 9, no. 8, pp. 1735-1780, Nov. 1997.