A binaural scene analyzer for joint localization and recognition of speakers in the presence of interfering noise sources and reverberation

IEEE Transactions on Audio, Speech and Language Processing

Volumn 20, Issue 7, 2012, Pages 2016-2030

  May, Tobias ^a   Van De Par, Steven ^b   Kohlrausch, Armin ^c  

^a UNIVERSITY OF OLDENBURG   (Germany)

^b PHILIPS RESEARCH LABORATORIES   (Netherlands)

^c EINDHOVEN UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

Automatic speaker recognition; binaural processing; computational auditory scene analysis (CASA); mask estimation; missing data

Indexed keywords

AUTOMATIC SPEAKER RECOGNITION; BINARY MASKS; BINAURAL LOCALIZATION; BINAURAL PROCESSING; BUILDING BLOCKES; COCKTAIL PARTY; COMPUTATIONAL AUDITORY SCENE ANALYSIS; MISSING DATA; NOISE SOURCE; PRIORI KNOWLEDGE; SOUND SOURCE; SOURCE DETECTION; SPEAKER IDENTIFICATION; SPEAKER RECOGNITION; SPEECH DETECTION; STATE OF THE ART; TARGET SPEAKER;

ACOUSTIC GENERATORS; REVERBERATION; SPEECH PROCESSING;

SPEECH RECOGNITION;

EID: 84861514871     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2012.2193391     Document Type: Article

References (55)

1. E. C. Cherry, "Some experiments on the recognition of speech, with one and two ears," J. Acoust. Soc. Amer., vol. 25, no. 5, pp. 975-979, Sep. 1953.
2. A. W. Bronkhorst, "The cocktail party phenomenon: A review of research on speech intelligibility in multiple-talker conditions," Acustica, vol. 86, pp. 117-128, 2000. (Pubitemid 34103984)
3. A. S. Bregman, Auditory Scene Analysis: The Perceptual Organization of Sound. Cambridge, MA: MIT Press, 1990.
4. Computational Auditory Scene Analysis: Principles, Algorithms and Applications, D. L. Wang and G. Brown, Eds. Hoboken, NJ: Wiley, 2006.
5. M. Cooke, P. Green, L. Josifovski, and A. Vizinho, "Robust automatic speech recognition with missing and unreliable acoustic data," Speech Commun., vol. 34, pp. 267-285, 2001. (Pubitemid 32284867)
6. D. Pullella, M. Kühne, and R. Togneri, "Robust speaker identification using combined feature selection and missing data recognition," in Proc. ICASSP, Las Vegas, NV, 2008, pp. 4833-4836.
7. T. May, S. van de Par, and A.Kohlrausch, "Noise-robust speaker recognition combining missing data techniques and universal background modeling," IEEE Trans. Audio, Speech, Lang. Process., vol. 20, no. 1, pp. 108-121, Jan. 2012.
8. D. S. Brungart, P. S. Chang, B. D. Simpson, and D. L. Wang, "Isolating the energetic component of speech-on-speech masking with ideal time-frequency segregation," J. Acoust. Soc. Amer., vol. 120, no. 6, pp. 4007-4018, Dec. 2006. (Pubitemid 44888096)
9. N. Li and P. C. Loizou, "Factors influencing glimpsing of speech in noise," J. Acoust. Soc. Amer., vol. 122, no. 2, pp. 1165-1172, Aug. 2007. (Pubitemid 47205513)
10. D. L. Wang, U. Kjems, M. S. Pedersen, and J. B. Boldt, "Speech intelligibility in background noise with ideal binary time-frequency masking," J. Acoust. Soc. Amer., vol. 125, no. 4, pp. 2336-2347, Apr. 2009.
11. D. L.Wang, "On ideal binary masks as the computational goal of auditory scene analysis," in Speech Separation by Humans and Machines, P. Divenyi, Ed. Norwell, MA: Kluwer, 2005, ch. 12, pp. 181-197.
12. M. L. Hawley and R. Y. Litovsky, "The benefit of binaural hearing in a cocktail party: Effect of location and type of interferer," J. Acoust. Soc. Amer., vol. 115, no. 2, pp. 833-843, Feb. 2004. (Pubitemid 38200670)
13. B. D. Simpson, D. S. Brungart, N. Iyer, R. H. Gilkey, and J. T. Hamil, "Detection and localization of speech in the presence of competing speech signals," in Proc. ICAD, London, U.K., Jun. 2006, pp. 129-133.
14. K. J. Palomäki, G. J. Brown, and D. L. Wang, "A binaural processor for missing data speech recognition in the presence of noise and smallroom reverberation," Speech Commun., vol. 43, no. 4, pp. 361-378, 2004.
15. S. Harding, J. Barker, and G. Brown, "Mask estimation for missing data speech recognition based on statistics of binaural interaction," IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 1, pp. 58-67, Jan. 2006. (Pubitemid 43863453)
16. N. Roman, D. L. Wang, and G. J. Brown, "Speech segregation based on sound localization," J. Acoust. Soc. Amer., vol. 114, no. 4, pp. 2236-2252, Oct. 2003. (Pubitemid 37266649)
17. H. Christensen, N. Ma, S. N. Wrigley, and J. Barker, "A speech fragment approach to localising multiple speakers in reverberant environments," in Proc. ICASSP, Taipei, Taiwan, Apr. 2009, pp. 4593-4596.
18. T. May, S. van de Par, and A. Kohlrausch, "A probabilistic model for robust localization based on a binaural auditory front-end," IEEE Trans. Audio, Speech, Lang. Process., vol. 19, no. 1, pp. 1-13, Jan. 2011.
19. J. Woodruff and D. L.Wang, "Sequential organization of speech in reverberant environments by integrating monaural grouping and binaural localization," IEEE Trans. Audio, Speech, Lang. Process., vol. 18, no. 7, pp. 1856-1866, Sep. 2010.
20. G. Kidd, Jr., T. L. Arbogast, C. R. Mason, and F. J. Gallun, "The advantage of knowing where to listen," J. Acoust. Soc. Amer., vol. 188, no. 6, pp. 3804-3815, Dec. 2005. (Pubitemid 41820881)
21. N. Kopčo, V. Best, and S. Carlile, "Speech localization in a multitalker mixture," J. Acoust. Soc. Amer., vol. 127, no. 3, pp. 1450-1457, Mar. 2010.
22. K. J. Palomäki, G. J. Brown, and J. P. Barker, "Techniques for handling convolutional distortion with 'missing data' automatic speech recognition," Speech Commun., vol. 43, no. 1-2, pp. 123-142, Jun. 2004.
23. B. R. Glasberg and B. C. J. Moore, "Derivation of auditory filter shapes from notched-noise data," Hear. Res., vol. 47, no. 1-2, pp. 103-138, Aug. 1990. (Pubitemid 20244652)
24. G. J. Brown and M. Cooke, "Computational auditory scene analysis," Comput. Speech Lang., vol. 8, no. 4, pp. 297-336, Oct. 1994.
25. R. M. Stern, A. S. Zeiberg, and C. Trahiotis, "Lateralization of complex binaural stimuli: A weighted-image model," J. Acoust. Soc. Amer., vol. 84, no. 1, pp. 156-165, Jul. 1988.
26. T. M. Shackleton, R. Meddis, and M. J. Hewitt, "Across frequency integration in a model of lateralization," J. Acoust. Soc. Amer., vol. 91, no. 4, pp. 2276-2279, Apr. 1992.
27. T. May, S. van de Par, and A.Kohlrausch, "Binaural detection of speech sources in complex acoustic scenes," in Proc.WASPAA, NewPaltz, NY, Oct. 2011, pp. 241-244.
28. B. Raj and R. M. Stern, "Missing-feature approaches in speech recognition," IEEE Signal Process. Mag., vol. 22, no. 5, pp. 101-116, Sep. 2005. (Pubitemid 41488524)
29. M. El-Maliki and A. Drygajlo, "Missing features detection and handling for robust speaker verification," in Proc. Eurospeech, Budapest, Hungary, Sep. 1999, pp. 975-978.
30. M. Cooke and T.-W. Lee, "Speech separation and recognition competition," 2006, accessed on 12th Oct. 2010, [Online]. Available: http://staffwww.dcs.shef.ac.uk/people/M.Cooke/SpeechSeparation Challenge.htm
31. A. P. Varga, H. J. M. Steeneken, M. Tomlinson, and D. Jones, "The NOISEX-92 study on the effect of additive noise on automatic speaker recognition," Speech Research Unit, Defence Research Agency, Malvern, U.K., 1992, Tech. Rep..
32. S. Lloyd, "Least squares quantization in PCM," IEEE Trans. Inf. Theory., vol. 28, no. 2, pp. 129-137, Mar. 1982.
33. A. Dempster,N. Laird, and D. Rubin, "Maximum likelihood estimation from incomplete data via the EM algorithm," J. R. Statist. Soc. B, vol. 39, no. 1, pp. 1-38, 1977.
34. M. Cooke, "A glimpsing model of speech perception in noise," J. Acoust. Soc. Amer., vol. 199, no. 3, pp. 1562-1573, Mar. 2006. (Pubitemid 43326025)
35. D. A. Reynolds, T. F. Quatieri, and R. B. Dunn, "Speaker verification using adapted Gaussian mixture models," Digital Signal Process., vol. 10, pp. 19-41, 2000. (Pubitemid 30592166)
36. B. G. Shinn-Cunningham, J. Schickler, N. Kopčo, and R. Litovsky, "Spatial unmasking of nearby speech sources in a simulated anechoic environment," J. Acoust. Soc. Amer., vol. 110, no. 2, pp. 1118-1129, Aug. 2001. (Pubitemid 32734745)
37. W. G. Gardner and K. D. Martin, "HRTF measurements of a KEMAR dummy-head microphone," MIT Media Lab, Perceptual Computing, Tech. Rep. #280, 1994.
38. J. B. Allen and D. A. Berkley, "Image method for efficiently simulating small-room acoustics," J. Acoust. Soc. Amer., vol. 65, no. 4, pp. 943-950, Apr. 1979.
39. S. M. Schimmel, M. F. Müller, and N. Dillier, "A fast and accurate "shoebox" room acoustics simulator," in Proc. ICASSP, Taipei, Taiwan, Apr. 2009, pp. 241-244.
40. American National Standard Specification for Sound Level Meters, ANSI/ASA S1.4-1983 (R2001), Amer. Nat. Stand. Inst., 1983.
41. D. P. W. Ellis, PLP and RASTA (and MFCC, and Inversion) in Matlab, 2005, accessed on 11th October 2011 [Online]. Available: http://www.ee.columbia.edu/ ~dpwe/resources/matlab/rastamat
42. F. K. Soong and A. E. Rosenberg, "On the use of instantaneous and transitional spectral information in speaker recognition," IEEE Trans. Acoust., Speech, Signal Process., vol. 36, no. 6, pp. 871-879, Jun. 1988.
43. J. P. Openshaw and J. S. Mason, "On the limitations of cepstral features in noise," in Proc. ICASSP, Adelaide, South Australia, Australia, Apr. 1994, pp. 49-52.
44. S. Tibrewala and H. Hermansky, "Multi-band and adaptation approaches to robust speech recognition," in Proc. Eurospeech, Rhodes, Greece, Sep. 1997, pp. 2619-2622.
45. H. G. Hirsch and C. Ehrlicher, "Noise estimation techniques for robust speech recognition," in Proc. ICASSP, Detroit, MI, 1995, vol. 1, pp. 153-156.
46. Y. Ephraim and D. Malah, "Speech enhancement using a minimum mean-square error log-spectral amplitude estimator," IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-33, no. 2, pp. 443-445, Apr. 1985.
47. R. Saeidi, P. Mowlaee, T. Kinnunen, Z.-H. Tan, M. G. Christensen, S. H. Jensen, and P. Fränti, "Signal-to-signal ratio independent speaker identification for co-channel speech signals," in Proc. ICPR, Istanbul, Turkey, Aug. 2010, pp. 4565-4568.
48. T. Fawcett, "An introduction to ROC analysis," Pattern Recog. Lett., vol. 27, no. 8, pp. 861-874, Jun. 2006.
49. G. Kim, Y. Lu, Y. Hu, and P. C. Loizou, "An algorithm that improves speech intelligibility in noise for normal-hearing listeners," J. Acoust. Soc. Amer., vol. 126, no. 3, pp. 1486-1494, Sep. 2009.
50. A. Ihlefeld and B. G. Shinn-Cunningham, "Effect of source location and listener location on ILD cues in a reverberant room," J. Acoust. Soc. Amer., vol. 115, no. 5, p. 2598, 2004.
51. B. G. Shinn-Cunningham, N. Kopčo, and T. J. Martin, "Localizing nearby sound sources in a classroom: Binaural room impulse responses," J. Acoust. Soc. Amer., vol. 117, no. 5, pp. 3100-3115, May 2005. (Pubitemid 40675172)
52. S. Srinivasan, N. Roman, and D. L. Wang, "Binary and ratio time-frequency masks for robust speech recognition," Speech Commun., vol. 48, no. 11, pp. 1486-1501, Nov. 2006. (Pubitemid 44634774)
53. E. Shriberg, A. Stolcke, and D. Baron, "Observations on overlap: Findings and implications for automatic processing of multi-party conversation," in Proc. Eurospeech, Aalborg, Denmark, Sep. 2001, pp. 1359-1362.
54. K. J. Palomäki, G. J. Brown, and J. P. Barker, "Recognition of reverberant speech using full cepstral features and spectral missing data," in Proc. ICASSP, Toulouse, France, 2006, pp. 289-292.
55. C. Kim, K. Kumar, and R. M. Stern, "Binaural sound source separation motivated by auditory processing," in Proc. ICASSP, Prague, Czech Republic, 2011, pp. 5072-5075.