Time-Frequency Masking for Speech Separation and Its Potential for Hearing Aid Design

Trends in Amplification

Volumn 12, Issue 4, 2008, Pages 332-353

  Wang, Deliang ^a  

^a Ohio State University   (United States)

Author keywords

computational auditory scene analysis; hearing aids; ideal binary mask; time frequency masking

Indexed keywords

ALGORITHM; AUDITORY DISCRIMINATION; AUDITORY MASKING; HEARING AID; HUMAN; NOISE REDUCTION; REVIEW; SPEECH ANALYSIS; SPEECH DISCRIMINATION;

ALGORITHMS; AUDITORY THRESHOLD; EQUIPMENT DESIGN; HEARING AIDS; HEARING IMPAIRED PERSONS; HEARING LOSS; HUMANS; MODELS, BIOLOGICAL; PERCEPTUAL MASKING; PITCH PERCEPTION; REHABILITATION OF HEARING IMPAIRED; SIGNAL PROCESSING, COMPUTER-ASSISTED; SPEECH INTELLIGIBILITY; SPEECH PERCEPTION; TIME PERCEPTION;

EID: 56249144201     PISSN: 10847138     EISSN: 19405588     Source Type: Journal    
DOI: 10.1177/1084713808326455     Document Type: Article

References (88)

1. Aarabi P. Shi G. (2004). Phase-based dual-microphone robust speech enhancement. IEEE Transactions on Systems, Man, and Cybernetics—Part B: Cybernetics, 34, 1763–1773.
2. Anzalone M. C. Calandruccio L. Doherty K. A. Carney L. H. (2006). Determination of the potential benefit of time-frequency gain manipulation. Ear and Hearing, 27, 480–492.
3. Araki S. Makino S. Blin A. Mukai R. Sawada H. (2004, May). Underdetermined blind separation for speech in speech in real environments with sparseness and ICA. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal processing (Vol. III, pp. 881–884), Montreal, Quebec, Canada.
4. Araki S. Makino S. Sawada H. Mukai R. (2004). Underdetermined blind separation of convolutive mixtures of speech with directivity pattern based mask and ICA. In Puntonet C. G. Prieto A. (Eds.), Lecture notes in computer science: 3195. Independent component analysis and blind signal separation: Proceedings of the Fifth International Congress, ICA 2004 (pp. 898–905). Berlin: Springer.
5. Araki S. Makino S. Sawada H. Mukai R. (2005, March). Reducing musical noise by a fine-shift overlap-and-add method applied to source separation using a time-frequency mask. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (Vol. III, pp. 81–84), Philadelphia, PA.
6. Araki S. Sawada H. Mukai R. Makino S. (2006, September). Blind sparse source separation with spatially smoothed time-frequency masking. In Proceedings of the 10th International Workshop Acoustic Echo and Noise Control, Paris, France.
7. Araki S. Sawada H. Mukai R. Makino S. (2007). Underdetermined blind sparse source separation for arbitrarily arranged multiple sensors. Signal Processing, 87, 1833–1847.
8. Barker J. Josifovski L. Cooke M. Green P. (2000, October). Soft decisions in missing data techniques for robust automatic speech recognition. In Proceedings of Sixth International Conference on Spoken Language Processing (Vol. 1, pp. 373–376), Beijing, China.
9. Bench J. Bamford J. (1979). Speech hearing tests and the spoken language of hearing-impaired children. London: Academic Press.
10. Blin A. Araki S. Makino S. (2005). Underdetermined blind separation of convolutive mixtures of speech using time-frequency mask and mixing matrix estimation. IEICE Transactions on Fundamentals of Electronics Communications and Computer Sciences, E88, 88–1700.
11. Bodden M. (1993). Modeling human sound-source localization and the cocktail-party-effect. Acta Acustica, 1, 43–55.
12. Boldt J. B. Kjems U. Pedersen M. S. Lunner T. Wang D. L. (2008, September). Estimation of the ideal binary mask using directional systems. In Proceedings of the 11th International Workshop on Acoustic Echo and Noise Control, Seattle, WA.
13. Bolia R. S. Nelson W. T. Ericson M. A. Simpson B. D. (2000). A speech corpus for multitalker communications research. Journal of the Acoustical Society of America, 107, 1065–1066.
14. Bregman A. S. (1990). Auditory scene analysis. Cambridge: MIT Press.
15. Brown G. J. Cooke M. (1994). Computational auditory scene analysis. Computer Speech and Language, 8, 297–336.
16. Brown G. J. Palomäki K. J. (2006). Reverberation. In Wang D. L. Brown G. J. (Eds.), Computational auditory scene analysis: Principles, algorithms, and applications (pp. 209–250). Hoboken, NJ: Wiley/IEEE Press.
17. Brungart D. Chang P. S. Simpson B. D. Wang D. L. (2006). Isolating the energetic component of speech-on-speech masking with ideal time-frequency segregation. Journal of the Acoustical Society of America, 120, 4007–4018.
18. Buchler M. Allegro S. Launer S. Dillier N. (2005). Sound classification in hearing aids inspired by auditory scene analysis. EURASIP Journal on Applied Signal Processing, 18, 2991–3002.
19. Burkhard M. D. Sachs R. M. (1975). Anthropometric manikin for acoustic research. Journal of the Acoustical Society of America, 58, 214–222.
20. Cappe O. (1994). Elimination of the musical noise phenomenon with the Ephraim and Malah noise suppressor. IEEE Transactions on Speech and Audio Processing, 2, 345–349.
21. Cermak J. Araki S. Sawada H. Makino S. (2006, September). Blind speech separation by combining beamformers and a time frequency binary mask. In Proceedings of the 10th International Workshop Acoustic Echo and Noise Control, Paris, France.
22. Chang P. (2004). Exploration of behavioral, physiological, and computational approaches to auditory scene analysis. Unpublished master's thesis, Department of Computer Science and Engineering, The Ohio State University, Columbus.
23. Cooke M. Green P. Josifovski L. Vizinho A. (2001). Robust automatic speech recognition with missing and unreliable acoustic data. Speech Communication, 34, 267–285.
24. Deshmukh O. D. Espy-Wilson C. Y. Carney L. H. (2007). Speech enhancement using the modified phase-opponency model. Journal of the Acoustical Society of America, 121, 3886–3898.
25. Dillon H. (2001). Hearing aids. New York: Thieme.
26. Dubnov S. Tabrikian J. Arnon-Targan M. (2004, May). A method for directionally-disjoint source separation in convolutive environment. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (Vol. V, pp. 489–492), Montreal, Quebec, Canada.
27. Dubnov S. Tabrikian J. Arnon-Targan M. (2006). Speech source separation in convolutive environments using space-time-frequency analysis. EURASIP Journal on Applied Signal Processing, 2006, Article 38412, 11 pages.
28. Duda R. O. Hart P. E. Stork D. G. (2001). Pattern classification (2nd ed.). New York: Wiley.
29. Ellis D. (2006). Model-based scene analysis. In Wang D. L. Brown G. J. (Eds.), Computational auditory scene analysis: Principles, algorithms, and applications (pp. 115–146). Hoboken, NJ: Wiley/IEEE Press.
30. Gagne J.-P. (1988). Excess masking among listeners with a sensorineural hearing loss. Journal of the Acoustical Society of America, 83, 2311–2321.
31. Greenberg J. E. Zurek P. M. (2001). Microphone-array hearing aids. In Brandstein M. Ward D. (Eds.), Microphone arrays: Signal processing techniques and applications (pp. 229–253). New York: Springer.
32. Harding S. Barker J. Brown G. J. (2006). Mask estimation for missing data speech recognition based on statistics of binaural interaction. IEEE Transactions on Audio, Speech, and Language Processing, 14, 58–67.
33. Helmholtz H. (1863). On the sensation of tone (Ellis A. J., Trans., 2nd English ed.). New York: Dover.
34. Hu G. Wang D. L. (2001, October). Speech segregation based on pitch tracking and amplitude modulation. In Proceedings of IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (pp. 79–82), New Paltz, NY.
35. Hu G. Wang D. L. (2004). Monaural speech segregation based on pitch tracking and amplitude modulation. IEEE Transactions on Neural Networks, 15, 1135–1150.
36. Hu G. Wang D. L. (2006). An auditory scene analysis approach to monaural speech segregation. In Hansler E. Schmidt G. (Eds.), Topics in acoustic echo and noise control (pp. 485–515). Heidelberg, Germany: Springer.
37. Hu G. Wang D. L. (2008). Segregation of unvoiced speech from nonspeech interference. Journal of the Acoustical Society of America, 124, 1306–1319.
38. Hyvärinen A. Karhunen J. Oja E. (2001). Independent component analysis. New York: Wiley.
39. IEEE. (1969). IEEE recommended practice for speech quality measurements. IEEE Transactions on Audio and Electroacoustics, 17, 225–246.
40. Jourjine A. Rickard S. Yilmaz O. (2000, June). Blind separation of disjoint orthogonal signals: Demixing N sources from 2 mixtures. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (Vol. 5, pp. 2985–2988), Istanbul, Turkey.
41. Kates J. M. Arehart K. H. (2005). Multichannel dynamic-range compression using digital frequency warping. EURASIP Journal on Applied Signal Processing, 18, 3003–3014.
42. Kollmeier B. Peissig J. Hohmann V. (1993). Real-time multiband dynamic compression and noise reduction for binaural hearing aids. Journal of Rehabilitation Research and Development, 30, 82–94.
43. Kolossa D. Klimas A. Orglmeister R. (2005, October). Separation and robust recognition of noisy, convolutive speech mixtures using time-frequency masking and missing data techniques. In Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (pp. 82–85), New Paltz, NY.
44. Kolossa D. Orglmeister R. (2004). Nonlinear postprocessing for blind speech separation. In Puntonet C. G. Prieto A. (Eds.), Lecture notes in computer science: 3195. Independent component analysis and blind signal separation: Proceedings of the Fifth International Congress, ICA 2004 (pp. 832–839). Berlin: Springer.
45. Li M. McAllister H. G. Black N. D. Perez T. A. D. (2001). Perceptual time-frequency subtraction algorithm for noise reduction in hearing aids. IEEE Transactions on Biomedical Engineering, 48, 979–988.
46. Li N. Loizou P. C. (2008a). Effect of spectral resolution on the intelligibility of ideal binary masked speech. Journal of the Acoustical Society of America, 123, EL59–EL64.
47. Li N. Loizou P. C. (2008b). Factors influencing intelligibility of ideal binary-masked speech: Implications for noise reduction. Journal of the Acoustical Society of America, 123, 1673–1682.
48. Li P. Guan Y. Xu B. Liu W. (2006). Monaural speech separation based on computational auditory scene analysis and objective quality assessment of speech. IEEE Transactions on Audio, Speech, and Language Processing, 14, 2014–2023.
49. Li Y. Wang D. L. (in press). On the optimality of ideal binary time-frequency masks. Speech Communication.
50. Linh-Trung N. Belouchrani A. Abed-Meraim K. Boashush B. (2005). Separating more sources than sensors using time-frequency distributions. EURASIP Journal on Applied Signal Processing, 17, 2828–2847.
51. Loizou P. C. (2007). Speech enhancement: Theory and practice. Boca Raton, FL: CRC Press.
52. Lyon R. F. (1983, April). A computational model of binaural localization and separation. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (pp. 1148–1148), Boston, MA.
53. Madhu N. Breithaupt C. Martin R. (2008). Temporal smoothing of spectral masks in the cepstral domain for speech separation. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (pp. 45–48), Las Vegas, NV.
54. Molla M. K. I. Hirose K. Minematsu N. (2006). Separation of mixed audio signals by source localization and binary masking with Hilbert spectrum. Lecture notes in computer science: 3889. Independent component analysis and blind signal separation: Proceedings of the Sixth International Congress, ICA 2006 (pp. 641–648). Berlin: Springer.
55. Moore B. C. J. (2003). An introduction to the psychology of hearing (5th ed.). San Diego, CA: Academic Press.
56. Moore B. C. J. (2007). Cochlear hearing loss (2nd ed.). Chichester, UK: Wiley.
57. Mori Y. Saruwatari H. Takatani T. Ukai S. Shikano K. Hiekata T. et al., (2006). Blind separation of acoustic signals combining SIMO-model-based independent component analysis and binary masking. EURASIP Journal on Applied Signal Processing, 2006(20), 1–17
58. Nadas A. Nahamoo D. Picheny M. A. (1989). Speech recognition using noise-adaptive prototypes. IEEE Transactions on Acoustics, Speech, and Signal Processing, 37, 1495–1503.
59. Nilsson M. Soli S. Sullivan J. A. (1994). Development of the hearing in noise test for the measurement of speech reception thresholds in quiet and in noise. Journal of the Acoustical Society of America, 95, 1085–1099.
60. Palomäki K. J. Brown G. J. Barker J. (2004). Techniques for handling convolutional distortion with “missing data” automatic speech recognition. Speech Communication, 43, 123–142.
61. Pedersen M. S. Wang D. L. Larsen J. Kjems U. (2005, September). Overcomplete blind source separation by combining ICA and binary time-frequency masking. In Proceedings of the IEEE International Workshop on Machine Learning for Signal Processing (pp. 15–20), Mystic, CT.
62. Pedersen M. S. Wang D. L. Larsen J. Kjems U. (2008). Two-microphone separation of speech mixtures. IEEE Transactions on Neural Networks, 19, 475–492.
63. Pichevar R. Rouat J. (2007). Monophonic sound source separation with an unsupervised network of spiking neurones. Neurocomputing, 71, 109–120.
64. Radfar M. H. Dansereau R. M. Sayadiyan A. (2007). A maximum likelihood estimation of vocal-tract-related filter characteristics for single channel speech separation. EURASIP Journal on Audio, Speech, and Music Processing, 2007, Article 84186, 15 pages.
65. Reddy A. M. Raj B. (2007). Soft mask methods for single-channel speaker separation. IEEE Transactions on Audio, Speech, and Language Processing, 15, 1766–1776.
66. Rickard S. Balan R. Rosca J. (2001, December). Real-time time-frequency based blind source separation. In Proceedings of the Third International Conference on Independent Component Analysis and Blind Source Separation (pp. 651–656), San Diego, CA.
67. Roman N. Srinivasan S. Wang D. L. (2006). Binaural segregation in multisource reverberant environments. Journal of the Acoustical Society of America, 120, 4040–4051.
68. Roman N. Wang D. L. (2004). Binaural sound separation for multisource reverberant environments. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (Vol. II, pp. 373–376), Montreal Quebec, Canada.
69. Roman N. Wang D. L. Brown G. J. (2003). Speech segregation based on sound localization. Journal of the Acoustical Society of America, 114, 2236–2252.
70. Roweis S. T. (2001). One microphone source separation. In Advances in Neural Information Processing Systems (NIPS'00) (Vol. 13, pp. 793–799). Cambridge, MA: MIT Press.
71. Saruwatari H. Mori Y. Takatani T. Ukai S. Shikano K. Hiekata T. et al., (2005, August). Two-stage blind source separation based on ICA and binary masking for real-time robot audition system. Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2005 (pp. 2303–2308), Edmont, Alberta, Canada.
72. Sawada H. Araki S. Mukai R. Makino S. (2005). Blind extraction of a dominant source signal from mixtures of many sources. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (Vol. III, pp. 61–64), Philadelphia, PA.
73. Sawada H. Araki S. Mukai R. Makino S. (2006). Blind extraction of dominant target sources using ICA and time-frequency masking. IEEE Transactions on Audio, Speech, and Language Processing, 14, 2165–2173.
74. Shannon R. V. Zeng F.-G. Kamath V. Wygonski J. Ekelid M. (1995). Speech recognition with primarily temporal cues. Science, 270, 303–304.
75. Srinivasan S. Roman N. Wang D. L. (2006). Binary and ratio time-frequency masks for robust speech recognition. Speech Communication, 48, 1486–1501.
76. Srinivasan S. Wang D. L. (2007). Transforming binary uncertainties for robust speech recognition. IEEE Transactions on Audio, Speech, and Language Processing, 15, 2130–2140.
77. Takenouchi Y. Hamada N. (2005, December). Time-frequency masking for BSS problem using equilateral triangular microphone array. In Proceedings of the 2005 International Symposium on Intelligent Signal Processing and Communication Systems (pp. 185–188), Hong Kong.
78. Thompson S. C. (2000). Directional patterns obtained from two or three microphones (Tech. Rep.). Itasca, IL: Knowles Electronics.
79. Togami M. Sumiyoshi T. Amano A. (2006, September). Sound source separation of overcomplete convolutive mixtures using generalized sparseness. In Proceedings of the 10th International Workshop Acoustic Echo and Noise Control, Paris, France.
80. van Veen B. D. Buckley K. M. (1988, April). Beamforming: A versatile approach to spatial filtering. IEEE ASSP Magazine, pp. 4–24.
81. Wagener K. Josvassen J. L. Ardenkjær R. (2003). Design, optimization and evaluation of a Danish sentence test in noise. International Journal of Audiology, 42, 10–17.
82. Wang D. L. (2005). On ideal binary mask as the computational goal of auditory scene analysis. In Divenyi P. (Ed.), Speech separation by humans and machines (pp. 181–197). Norwell, MA: Kluwer Academic.
83. Wang D. L. Brown G. J. (1999). Separation of speech from interfering sounds based on oscillatory correlation. IEEE Transactions on Neural Networks, 10, 684–697.
84. Wang D. L. Brown G. J. (Eds.). (2006). Computational auditory scene analysis: Principles, algorithms, and applications. Hoboken, NJ: Wiley/IEEE Press.
85. Wang D. L. Kjems U. Pedersen M. S. Boldt J. B. Lunner T. (2008). Speech intelligibility in background noise with ideal binary time-frequency masking. Journal of the Acoustical Society of America, conditionally accepted.
86. Wang D. L. Kjems U. Pedersen M. S. Boldt J. B. Lunner T. (in press). Speech perception of noise with binary gains. Journal of the Acoustical Society of America.
87. Weintraub M. (1985). A theory and computational model of auditory monaural sound separation. Unpublished doctoral dissertation, Department of Electrical Engineering, Stanford University, CA.
88. Yilmaz O. Rickard S. (2004). Blind separation of speech mixtures via time-frequency masking. IEEE Transactions on Signal Processing, 52, 1830–1847.