An audio-visual fusion framework with joint dimensionality reduction

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2008, Pages 4437-4440

  Liu, Ming ^a   Fu, Yun ^a   Huang, Thomas S ^a  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

Audio visual fusion; Audio visual person verification; Canonical correlation analysis; Dimensionality reduction

Indexed keywords

ACOUSTICS; CORRELATION METHODS; FACE RECOGNITION; FEATURE EXTRACTION; POWDERS; SIGNAL PROCESSING; SPEECH; SPEECH RECOGNITION;

AUDIO-VISUAL FUSION; AUDIO-VISUAL PERSON VERIFICATION; CANONICAL CORRELATION ANALYSIS; DIMENSIONALITY REDUCTION; VISUAL FEATURES;

AUDIO SYSTEMS;

EID: 51449116849     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2008.4518640     Document Type: Conference Paper

References (18)

1. E.D. Petajan, "Automatic lipreading to enhance speech recognition," in Proc. of Global Telecommunications Conference, 1984, pp. 265-272.
2. Y. Gong, "Speech recognition in noisy environment: A survey," Speech Communication, vol. 16, pp. 261-291, 1995.
3. G. Potamianos, C. Neti, J. Luettin, and I. Matthews, "Audiovisual automatic speech recognition: An overview," in Issues in Visual and Audio-Visual Speech Processing. MIT, 2004.
4. M. T. Chan, Y. Zhang, and T. S. Huang, "Integrating visual and acoustic features for speech recognition.," in Proc. of Army Research Laboratory Advanced Displays and Interactive Displays, Federated Laboratory Second Annual Symposium, 1998, pp. 64-68.
5. A. Adjoudani and C. Benoit, Speechreading by Humans and Machines, chapter On the integration of auditory and visual parameters in an HMM-based ASR, pp. 461-471, 1996.
6. S. Dupont and J. Luettin, "Audio-visual speech modeling for continuous speech recognition," IEEE Transactions on Multimedia, vol. 2, pp. 141-151, 2000.
7. M. Liu, Z. Xiong, S. M. Chu, Z. Zhang, and T. S. Huang, "Audio visual word spotting," in Proc. of IEEE Conf. on ICASSP'04, 2004, pp. 785-788.
8. S.M. Chu and T.S. Huang, "Audio-visual speech modeling using coupled hidden markov models," in Proc. of IEEE Conf. on ICASSP'02, 2002, pp. 2009-2012.
9. S.M. Chu and T.S. Huang, "An experimental study of coupled hidden markov models," in Proc. of IEEE Conf. on ICASSP'02, 2002, pp. 4100-4103.
10. I. Matthews, G. Potamianos, C. Neti, and J. Luettin, "A comparison of model and transform-based visual features for audio-visual lvcsr," in Proc. of IEEE Conf. on ICME'01, 2001, pp. 825-828.
11. H. Hotelling, "Relations between two sets of variates," Biometrika, vol. 28, pp. 321-377, 1936.
12. S. Becker, "Mutual information maximization: models of cortical self-organization network," Computation in Neural Systems, vol. 7, pp. 7-31, 1996.
13. J. Kay, "Feature discovery under contextual supervision using mutual information," in Proc. of International Joint Conference on Neural Networks, 1992, vol. 4, pp. 79-84.
14. P. W. Fieguth, W. W. Irving, and A. S. Willsky, "Multiresolution model development for overlapping trees via canonical correlation analysis," in Proc. of IEEE Conf. on ICIP'95, 1995, pp. 45-48.
15. P. Li, J. Sun, and B. Yu., "Direction finding using interpolated arrays in unknown noise fields," Signal Processing, vol. 58, pp. 319-325, 1997.
16. D. A. Reynolds, "Speaker identification and verification using Gaussian mixture speaker models," Speech Communication, vol. 17, pp. 91-108, 1995.
17. D.A. Reynolds, T.F. Quatieri, and R.B. Dunn, "Speaker verification using adapted gaussian mixture models," Digital Signal Processing, pp. 19-41, 2000.
18. J.M. Buenaposada, E. Munoz, and L. Baumela, "Efficient appearance-based tracking," in Proc. of IEEE Workshop on Articulated and Nonrigid Motion (with IEEE CVPR'04), 2004.