Robust speech processing using multi-sensor multi-source information fusion - An overview of the state of the art

Information Fusion

Volumn 5, Issue 2, 2004, Pages 77-80

  Aarabi, Parham ^a   Dasarathy, Belur V ^b  

^a UNIVERSITY OF TORONTO   (Canada)

^b NONE

Author keywords

Multisensor information fusion; Robust speech processing; Speech recognition

Indexed keywords

CAMERAS; COMPUTATIONAL COMPLEXITY; COMPUTATIONAL METHODS; DATA PROCESSING; MICROPHONES; SENSOR DATA FUSION; SIGNAL TO NOISE RATIO; SPEECH RECOGNITION;

MULTISENSOR INFORMATION FUSION; ROBUST SPEECH PROCESSING;

SPEECH PROCESSING;

EID: 1842843686     PISSN: 15662535     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.inffus.2004.02.001     Document Type: Article

References (39)

1. P. Aarabi, N.H. Khameneh, Robust speech separation using visually derived speech signals, in: Proceedings of Sensor Fusion: Architectures, Algorithms, and Applications, 2002, pp. 239-247.
2. Aarabi P., Zaky S. Robust sound localization using multi-source audio-visual information fusion. Information Fusion. 3(2):2001;209-223.
3. P. Aarabi, S. Zaky, Integrated vision and sound localization, in: Proceedings of the 3rd International Conference on Information Fusion, 2000, pp. ThB3:21-27.
4. Dasarathy B.V. Elucidative fusion systems - an exposition. Information Fusion. 1(1):2000;5-15.
5. A. Guerin-Dugue, P. Teissier, J.L. Schwartz, J. Herault, Nonlinear representation for audio-visual fusion in a noisy-vowel recognition task, in: Proceedings of the International Conference on Neural Networks and Their Applications, 1997, pp. 31-40.
6. B.V. Dasarathy, Information fusion benefits delineation under off-nominal scenarios, in: Proceedings of the SPIE Conference on Sensor Fusion: Architectures, Algorithms, and Applications, 1999, pp. 2-13.
7. P. Teissier, J.L. Schwartz, A. Guerin-Dugue, Models for audiovisual fusion in a noisy-vowel recognition task, in: Proceedings of the IEEE Signal Processing Society Workshop on Multimedia Signal Processing, 1997, pp. 37-44.
8. J. Luettin, S. Dupont, Continuous audio-visual speech recognition, in: Proceedings of the 5th European Conference on Computer Vision, 1998.
9. P. deCuetos, C. Neti, A. Senior, Audio-visual intent-to-speak detection for human-computer interaction, in: Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, 2000, pp. 2373-2376.
10. Rogozan A., Deleglise P. Adaptive fusion of acoustic and visual sources for automatic speech recognition. Speech Communication. 26(1-2):1998;149-161.
11. M.J. Tomlinson, M.J. Russell, N.M. Brooke, Integrating audio and visual information to provide highly robust speech recognition, in: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, 1996, pp. 821-824.
12. C. Breglar, S.M. Omohundro, Nonlinear manifold learning for visual speech recognition, in: Proceedings of the IEEE International Conference on Computer Vision, 1995, pp. 494-499.
13. Yuhas B.P., Goldstein M.H., Sejnowski T.J., Jenkins R.E. Neural network models of sensory integration for improved vowel recognition. Proceedings of the IEEE. 78(10):1990;1658-1668.
14. E.D. Petajan, Automatic lip-reading to enhance speech recognition, in: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 1985, pp. 40-47.
15. Silsbee P.L., Bovik A.C. Computer lip-reading for improved accuracy in automatic speech recognition. IEEE Transactions on Speech and Audio Processing. 4(5):1996;337-351.
16. B.J. Frey, T. Kristjansson, L. Deng, A. Acero, Learning dynamic noise models from noisy speech for robust noise recognition, in: Proceedings of Neural Information Processing Systems (NIPS), 2001.
17. Hyvrinen A., Oja E. Independent component analysis: algorithms and applications. Neural Networks. 13(4-5):2000;411-430.
18. G. Shi, P. Aarabi, Robust digit recognition using phase-dependent time-frequency masking, in: Proceedings of the IEEE Conference on Acoustics, Speech, and Signal Processing, 2003, pp. I:684-687.
19. M.S. Brandstein, H. Silverman, A robust method for speech signal time-delay estimation in reverberant rooms, in: Proceedings of the IEEE Conference on Acoustics, Speech, and Signal Processing, 1997, pp. I:375-378.
20. Bell A., Sejnowski T. An information maximization approach to blind separation and blind deconvolution. Neural Computation. 7:1995;1129-1159.
21. S. Rennie, P. Aarabi, T. Kristjansson, B. Frey, K. Achan, Robust variational speech separation using fewer microphones than speakers, in: Proceedings of the IEEE Conference on Acoustics, Speech, and Signal Processing, 2003, pp. I:88-91.
22. Aarabi P. The fusion of distributed microphone arrays for sound localization. EURASIP Journal of Applied Signal Processing Special Issue on Sensor Networks. 2003(4):2003;338-347.
23. Teissier P., Robert-Ribes J., Schwartz J.L., Guerin-Dugue A. Comparing models for audiovisual fusion in a noisy-vowel recognition task. IEEE Transactions on Speech and Audio Processing. 7(6):1999;629-642.
24. Enzo M., Massimiliano N., Gianni V. Algorithms for acoustic localization based on microphone array in service robotics. Robotics and Autonomous Systems. 42(2):2003;69-88.
25. Potamianos G., Neti C., Gravier G., Garg A., Senior A.W. Recent advances in the automatic recognition of audiovisual speech. Proceedings of the IEEE. 91(9):2003;1306-1326.
26. S.M. Chu, T.S. Huang, Audio-visual speech modeling using coupled hidden Markov models, in: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 2002, pp. 2009-2012.
27. H. Glotin, D. Vergyr, C. Neti, G. Potamianos, J. Luettin, Weighting schemes for audio-visual fusion in speech recognition, in: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 2001, pp. 173-176.
28. J. Luettin, G. Potamianos, C. Neti, Asynchronous stream modeling for large vocabulary audio-visual speech recognition, in: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 2001, pp. 169-172.
29. M. Heckmann, F. Berthommier, K. Kroschel, Optimal weighting of posteriors for audio-visual speech recognition, in: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 2001, pp. 161-164.
30. Y. Keren, J. Xiaoyi, H. Bunke, Combining acoustic and visual classifiers for the recognition of spoken sentences, in: Proceedings of the 15th International Conference on Pattern Recognition, 2000, pp. 491-494.
31. N. Strobel, S. Spors, R. Rabenstein, Joint audio-video object localization using a recursive multi-state multi-sensor estimator, in: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 2000, pp. 2397-2400.
32. Dupont S., Luettin J. Audio-visual speech modeling for continuous speech recognition. IEEE Transactions on Multimedia. 2(3):2000;141-151.
33. Berdugo B., Rosenhouse J., Azhari H. Speakers' direction finding using estimated time delays in the frequency domain. Signal Processing. 82(1):2002;19-30.
34. Kumatani K., Nakamura S. Audio-visual speech recognition based on optimized product HMMs and GMM based-MCE-GPD stream weight estimation. IEICE Transactions on Information and Systems E. 86-D(3):2003;454-463.
35. K. Nakadai, K. Hidai, H.G. Okuno, H. Kitano, Real-time speaker localization and speech separation by audio-visual integration, in: Proceedings of the 2002 IEEE International Conference on Robotics and Automation, 2002, pp. 1043-1049.
36. A. Ghosh, A. Verma, A. Sarkar, Using likelihood L-statistics to measure confidence in audio-visual speech recognition, in: Proceedings of the 2001 IEEE Fourth Workshop on Multimedia Signal Processing, 2001, pp. 27-32.
37. G. Potamianos, J. Luettin, C. Neti, Hierarchical discriminant features for audio-visual LVCSR, in: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 2001, pp. 165-168.
38. S. Lucey, S. Sridharan, V. Chandran, Improved speech recognition using adaptive audio-visual fusion via a stochastic secondary classifier, in: Proceedings of 2001 International Symposium on Intelligent Multimedia, Video and Speech Processing, 2001, pp. 551-554.
39. L. Girin, L. Varin, G. Feng, J.L. Schwartz, Audiovisual speech enhancement: new advances using multi-layer perceptrons, in: Proceedings of the IEEE Second Workshop on Multimedia Signal Processing, 1998, pp. 77-82.