Adaptive multimodal fusion by uncertainty compensation

INTERSPEECH 2006 and 9th International Conference on Spoken Language Processing, INTERSPEECH 2006 - ICSLP

Volumn 5, Issue , 2006, Pages 2458-2461

  Pitsikalis, Vassilis ^a   Katsamanis, Athanassios ^a   Papandreou, George ^a   Maragos, Petros ^a  

^a NATIONAL TECHNICAL UNIVERSITY OF ATHENS   (Greece)

Author keywords

Active appearance models; Audiovisual speech recognition; Multimodal fusion; Product HMMs; Stream weights; Uncertainty compensation

Indexed keywords

DEEP NEURAL NETWORKS; PATTERN RECOGNITION; SPEECH ANALYSIS; UNCERTAINTY ANALYSIS;

ACTIVE APPEARANCE MODELS; AUDIO VISUAL SPEECH RECOGNITION; MULTI-MODAL FUSION; PRODUCT HMMS; STREAM WEIGHTS;

SPEECH RECOGNITION;

EID: 44949227080     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (18)

1. G. Potamianos, C. Neti. G. Gravier, and A. Garg, "Automatic recognition of audio-visual speech: Recent progress and challenges," Proc. of the IEEE, vol. 91, no. 9, pp. 1306-1326, 2003.
2. A. Morris, A. Hagen, H. Glotin, and H. Bourlard, "Multi-stream adaptive evidence combination for noise robust ASR," Speech Communication, vol. 34, pp. 25-40, 2001.
3. A. Potamianos, E. Sanchez-Soto, and K. Daoudi, "Stream weight computation for multi-stream classifiers," in Proc. ICASSP, 2006.
4. H. Glotin, D. Vergyri, C. Neti, G. Potamianos, and J. Luettin, "Weighting schemes for audio-visual fusion in speech recognition," in Proc. ICASSP, 2001.
5. V. Digalakis, J.R. Rohlicek, and M. Ostendorf, "ML estimation of a stochastic linear system with the EM algorithm and its application to speech recognition," IEEE TSAP, pp. 431-442, 1993.
6. R. C. Rose, E. M. Hofstetter, and D. A. Reynolds, "Integrated models of signal and background with application to speaker identification in noise," IEEE TSAP, vol. 2, no. 2, pp. 245-257, 1994.
7. N.B Yoma and M. Villar, "Speaker verification in noise using a stochastic version of the weighted viterbi algorithm," IEEE TSAP, vol. 10, no. 3, pp. 158-166, 2002.
8. L. Deng, J. Dropo, and A. Acero, "Dynamic compensation of HMM variances using the feature enhancement uncertainty computed from a parametric model of speech distortion," IEEE TSAP, vol. 13, no. 3, pp. 412-421, 2005.
9. S. Dupont and J. Luettin, "Audio-visual speech modeling for continuous speech recognition," IEEE Trans. on Multimedia, vol. 2, no. 3, pp. 141-151, 2000.
10. J. Luettin, G. Potamianos, and C. Neti, "Asynchronous stream modeling for large vocabulary audio-visual speech recognition," in Proc. ICASSP, 2001.
11. Q. Huo and C. Lee, "A bayesian predictive approach to robust speech recognition," IEEE TSAP, pp. 200-204, 2000.
12. A.V. Nefian, L. Liang, X. Pi, X. Liu, and K. Murphy, "Dynamic bayesian networks for audio-visual speech recognition," EURASIP Journal on Applied Signal Processing, vol. 11, pp. 1-15, 2002.
13. T.F. Cootes, G.J. Edwards, and Taylor C.J., "Active appearance models," IEEE PAMI, vol. 23, no. 6, pp. 681-685, 2001.
14. I. Matthews, T. F. Cootes, J. A. Bangham, S. Cox, and R. Harvey, "Extraction of visual features for lipreading," IEEE PAMI, vol. 24, no. 2, pp. 198-213, 2002.
15. W. Press, S. Teukolsky, W. Vetterling, and B. Flannery, Numerical Recipes, Cambridge Univ. Press, 1992.
16. A. Blake and M. Isard, Active contours : the application of techniques from graphics, vision, control theory and statistics to visual tracking of shapes in motion, Springer, 1998.
17. E. K. Patterson, S. Gurbuz, Z. Tufekci, and J. N. Gowdy, "CUAVE: A new audio-visual database for multimodal human-computer interface research," in Proc. ICASSP, 2002.
18. A. Varga and H.J.M. Steeneken, "Assessment for automatic speech recognition: II. noisex-92: A database and an experiment to study the effect of additive noise on speech recognition systems," Speech Communication, vol. 12, no. 3, pp. 247-252, 1993.