Single-channel multitalker speech recognition

IEEE Signal Processing Magazine

Volumn 27, Issue 6, 2010, Pages 66-80

  Rennie, Steven ^a   Hershey, John ^b   Olsen, Peder ^c  

^a IBM T J WATSON RESEARCH CENTER   (United States)

^b MITSUBISHI ELECTRIC RESEARCH LABORATORIES   (United States)

^c UNIVERSITY OF MICHIGAN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AUDITION; DEEP NEURAL NETWORKS;

ACOUSTIC ENVIRONMENT; AUTOMATIC SPEECH RECOGNITION; GRAND CHALLENGE; HUMAN AUDITORY SYSTEM; MICROPHONE ARRAYS; MULTIPLE INPUTS; ROBUSTNESS TO NOISE; SIGNAL OF INTERESTS;

SPEECH RECOGNITION;

EID: 85032751986     PISSN: 10535888     EISSN: None     Source Type: Journal    
DOI: 10.1109/MSP.2010.938081     Document Type: Article

References (49)

1. [I] D. L. Wang and G. J. Brown, Eds., Computational Auditory Scene Analysis: Principles, Algorithms and Applications. New York: Wiley-IEEE Press, 2006.
2. M. J. Reyes-Gómez and N. Jojic, "Signal separation by efficient combinatorial optimization," in Proc. Workshop on Advances in Models for Acoustic Processing, Neural Information Processing Systems (NIPS'06), Whistler, British Columbia, Dec. 2006.
3. J. Barker, N. Ma, A. Coy, and M. Cooke, "Speech fragment decoding techniques for simultaneous speaker identification and speech recognition," Comput. Speech Lang., vol. 24, no. 1, pp. 94-111, Jan. 2010.
4. M. Cooke, P. Green, L. Josifovski, and A. Vizinho, "Robust automatic speech recognition with missing and unreliable acoustic data," Speech Commun., vol. 34, no. 3, pp. 267-285, 2001.
5. J. Ming, T. Hazen, and J. Glass, "Combining missing-feature theory, speech enhancement, and speaker-dependent/independent modeling for speech separation," Comput. Speech Lang., vol. 24, no. 1, pp. 67-76, Jan. 2010.
6. F. R. Bach and M. I. Jordan, "Learning spectral clustering, with application to speech separation," J. Mach. Learn. Res., vol. 7, pp. 1963-2001, Oct. 2006.
7. M. N. Schmidt and R. K. Olsson, "Single-channel speech separation using sparse non-negative matrixfactorization," in Proc. ISCA 9th Int. Conf. Spoken Language Processing (ICSLP'06), Pittsburgh, PA, Sept. 2006, pp. 2614-2617.
8. G.-J. Jang and T.-W. Lee, "A maximum likelihood approach to single-channel source separation," J. Mach. Learn. Res., vol. 4, pp. 1365-1392, Dec. 2003.
9. S. J. Rennie, J. R. Hershey, and P. A. Olsen, "Efficient model-based speech separation and denoising using non-negative subspace analysis," in Proc. IEEE Int. Conf. Acoustics, Speechand Signal Processing (ICASSP ' 0 8 ) , Las Vegas, N V, Apr. 2008, pp. 1833-1836.
10. B. Raj and P. Smaragdis, "Latent variable decomposition of spectrograms for single channel speaker separation," in Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz, NY, Oct. 2005, pp. 17-20.
11. [II] T. Virtanen, "Monaural sound source separation by nonnegative matrix fac torization with temporal continuity and sparseness criteria," IEEE Trans. Audio Speech Lang. Processing, vol. 15, no. 3, pp. 1066-1074, Mar. 2007.
12. K. W. Wilson, B. Raj, and P. Smaragdis, "Regularized non-negative matrix factorization with temporal dependencies for speech denoising," in Proc. ISCA 9th Annu. Conf. Int. Speech Communication Association (Interspeech'08), Brisbane, Australia, Sept. 2008, pp. 411-414.
13. T. Virtanen, A. T. Cemgil, and S. Godsill, "Bayesian extensions to non-negative matrix factorisation for audio signal modelling," in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP'08), Las Vegas, NV, Apr. 2008, pp. 1825-1828.
14. A. P. Varga and R. K. Moore, "Hidden Markov model decomposition of speech and noise," in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP'90), Albuquerque, NM, Apr. 1990, pp. 845-848.
15. Y. Ephraim, "A Bayesian estimation approach for speech enhancement using hidden Markov models," IEEE Trans. Signal Processing, vol. 40, no. 4, pp. 725-735, Apr. 1992.
16. B. J. Frey, T. T. Kristjansson, L. Deng, and A. Acero, "ALGONQUIN\-learning dynamic noise models from noisy speech for robust speech recognition," in Proc. Neural Information Processing Systems (NIPS'01), Vancouver, British Columbia, Canada, Dec. 2001, pp. 1165-1171.
17. S. T. Roweis, "Factorial models and refiltering for speech separation and de-noising," in Proc. ISCA 8th European Conf. Speech Communication and Technology (EUROSPEECH'03), Geneva, Switzerland, Sept. 2003, pp. 1009-1012.
18. T. Virtanen, "Speech recognition using factorial hidden Markov models for separation in the feature space," in Proc. ISCA 9th Int. Conf. Spoken Language Processing (ICSLP'06), Pittsburgh, PA, Sept. 2006, pp. 89-92.
19. J. R. Hershey, T. Kristjansson, S. Rennie, and P. A. Olsen, "Single channel speech separation using factorial dynamics," in Proc. Advances in Neural Information Processing Systems 19 (NIPS'06), B. Schölkopf, J. Platt, and T. Hoffman, Eds. Cambridge, MA: MIT Press, 2007, pp. 593-600.
20. J. R. Hershey, S. J. Rennie, P. A. Olsen, and T. T. Kristjansson, "Super-human multi-talker speech recognition: A graphical modeling approach," Comput. Speech Lang., vol. 24, no. 1, pp. 45-66, Jan. 2010.
21. M. Cooke, J. R. Hershey, and S. J. Rennie, "Monaural speech separation and recognition challenge," Comput. Speech Lang., vol. 24, no. 1, pp. 1-15, Jan. 2010.
22. Z. Ghahramani and M. I. Jordan, "Factorial hidden Markov models," in Proc. Advances in Neural Information Processing Systems (NIPS'95), vol. 8., D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, Eds. Cambridge, MA: MIT Press, 1995, pp. 472-478.
23. F. R. Kschischang, B. J. Frey, and H.-A. Loeliger, "Factor graphs and the sum-product algorithm," IEEE Trans. Inform. Theory, vol. 47, no. 2, pp. 498-519, Feb. 2001.
24. Y. Weiss and W. T. Freeman, "On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs," IEEE Trans. Inform. Theory, vol. 47, no. 2, pp. 736-744, Feb. 2001.
25. L. A. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, no. 2, pp. 257-286, Feb. 1989.
26. L. E. Baum and T. Petrie, "Statistical inference for probabilistic functions of finite state Markov chains," Ann. Math. Statist., vol. 37, no. 6, pp. 1554-1563, 1966.
27. F. Jelinek, "Continuous speech recognition by statistical methods," Proc. IEEE, vol. 64, no. 4, pp. 532-556, Apr. 1976.
28. M. J. Wainwright and M. I. Jordan, "Graphical models, exponential families, and variational inference," Found. Trends Mach. Learn., vol. 1, no. 1-2, pp. 1-305, 2008.
29. R. P. Lippmann, E. A. Martin, and D. B. Paul, "Multi-style training for robust isolated-word speech recognition," in Proc. IEEE Int. Conf. Acoustics Speech and Signal Processing (ICASSP'87), Dallas, TX, Apr. 1987, pp. 705-708.
30. J. R. Hershey, P. A. Olsen, and S. J. Rennie, "Signal interaction and the devil function," in Proc. ISCA 11th Annu. Conf. Int. Speech Communication Association (Interspeech'10), Makuhari, Japan, Sept. 2010, submitted for publication.
31. G. D. Durgin, T. S. Rappaport, and D. A. De Wolf, "New analytical models and probability density functions for fading in wireless communications," IEEE Trans. Commun., vol. 50, no. 6, pp. 1005-1015, 2002.
32. S. F. Boll, "Suppression of acoustic noise in speech using spectral subtraction," IEEE Trans. Acoust. Speech Signal Processing, vol. 27, no. 3, pp. 113-120, Apr. 1979.
33. P. J. Moreno, B. Raj, and R. M. Stern, "A vector Taylor series approach for environment-independent speech recognition," in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, Atlanta, GA, May 1996, vol. 2, pp. 733-736.
34. L. Deng, J. Droppo, and A. Acero, "Enhancement of log Mel power spectra of speech using a phase-sensitive model of the acoustic environment and sequential estimation of the corrupting noise," IEEE Trans. Speech Audio Processing, vol. 12, no. 2, pp. 133-143, Mar. 2004.
35. M. J. F. Gales and S. J. Young, "Robust continuous speech recognition using parallel model combination," IEEE Trans. Speech Audio Processing, vol. 4, no. 5, pp. 352-359, Sept. 1996.
36. B. J. Frey, L. Deng, A. Acero, and T. Kristjansson, "ALGONQUIN: Iterating Laplace's method to remove multiple types of acoustic distortion for robust speech recognition," in Proc. ISCA 7th European Conf. Speech Communication and Technology 2nd Interspeech Event, Aalborg, Denmark, Sept. 2001, pp. 901-904.
37. T. Kristjansson, H. Attias, and J. Hershey, "Single microphone source separation using high resolution signal reconstruction," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing, Montreal, Canada, May 2004, pp. 817-820.
38. A. Nádas, D. Nahamoo, and M. A. Picheny, "Speech recognition using noise-adaptive prototypes," IEEE Trans. Acoust. Speech Signal Processing, vol. 37, no. 10, pp. 1495-1503, Oct. 1989.
39. M. H. Radfar, A. H. Banihashemi, R. M. Dansereau, and A. Sayadiyan, "Nonlinear minimum mean square error estimator for mixture-maximisation approximation," Electron. Lett., vol. 42, no. 12, pp. 724-725, June 2006.
40. S. J. Rennie, J. R. Hershey, and P. A. Olsen, "Single-channel speech separation and recognition using loopy belief propagation," in Proc. IEEE Int. Conf. Acoustic, Speech, and Signal Processing, Taipei, Taiwan, Apr. 2009, pp. 3845-3848.
41. M. I. Jordan, Z. Ghahramani, T. S. Jaakkola, and L. K. Saul, "An introduction to variational methods for graphical models," Mach. Learn., vol. 37, no. 2, pp. 183-233, Nov. 1999.
42. S. J. Rennie, J. R. Hershey, and P. A. Olsen, "Variational loopy belief propagation for multi-talker speech recognition," in Proc. ISCA 10th Annu. Conf. Int. Speech Communication Association (Interspeech'09), Brighton, U.K., Sept. 2009, pp. 1331-1334.
43. T. El-Hay and N. Friedman, "Incorporating expressive graphical models in variational approximations: Chain-graphs and hidden variables," in Proc. 18th Conf. Uncertainty and Artificial Intelligence, 2002, pp. 136-143.
44. S. J. Rennie, J. R. Hershey, and P. A. Olsen, "Hierarchical variational loopy belief propagation for multi-talker speech recognition," in Proc. IEEE Workshop on Automatic Speech Recognition and Understanding, Merano, Italy, Dec. 2009, pp. 176-181.
45. J. Hershey and P. Olsen, "Approximating the Kullback Leibler divergence between Gaussian mixture models," in Proc. Int. Conf. Acoustics, Speech and Signal Processing (ICASSP'07), Honolulu, Hawaii, Apr. 2007, pp. 317-320.
46. P. L. Dognin, J. R. Hershey, V. Goel, and P. A. Olsen, "Refactoring acoustic models using variational density approximation," in Proc. Int. Conf. Acoustics, Speech and Signal Processing (ICASSP'09), Apr. 2009, pp. 4473-4476.
47. J. Baker, L. Deng, J. Glass, S. Khudanpur, C.-H. Lee, N. Morgan, and D. O'Shaughnessy, "Developments and directions in speech recognition and understanding, Part 1," IEEE Signal Processing Mag., vol. 26, no. 3, pp. 75-80, 2009.
48. J. Baker, L. Deng, J. Glass, S. Khudanpur, C.-H. Lee, N. Morgan, and D. O'Shaughnessy, "Developments and directions in speech recognition and understanding, Part 2," IEEE Signal Processing Mag., vol. 26, no. 4, pp. 78-85, 2009.
49. J. Bilmes, "Dynamic graphical models," IEEE Signal Processing Mag., vol. 27, no. 6, pp. 29-42, Nov. 2010.