An asynchronous DBN for audio-visual speech recognition

2006 IEEE ACL Spoken Language Technology Workshop, SLT 2006, Proceedings

Volumn , Issue , 2006, Pages 154-157

  Saenko, Kate ^a   Livescu, Karen ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Speech recognition

Indexed keywords

AUDIO ACOUSTICS; BAYESIAN NETWORKS; ERROR ANALYSIS; INFERENCE ENGINES; LINGUISTICS; RIVERS; SPEECH; SPEECH ANALYSIS; UNDERWATER ACOUSTICS; ACOUSTIC NOISE;

ASYNCHRONOUS MODELS; ASYNCHRONY; AUDIO VISUAL SPEECH RECOGNITION (AVSR); AUDIO-VISUAL; AUDIO-VISUAL CORPORA; DE SYNCHRONIZATION; DYNAMIC BAYESIAN NETWORK (DBN); ERROR RATE (ER); LIP READING; NUMBER OF STATES; PRONUNCIATION MODELLING; SPOKEN LANGUAGES; TWO-STREAM; VISUAL SPEECH RECOGNITION;

SPEECH RECOGNITION;

ASYNCHRONY; AUDIOVISUAL SPEECH RECOGNITION; DESYNCHRONIZATION; DYNAMIC BAYESIAN NETWORKS; LIPREADING; NETWORK-BASED MODELING; NUMBER OF STATE; PRONUNCIATION MODELLING; TWO-STREAM; VISUAL SPEECH RECOGNITION;

EID: 48749083240     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SLT.2006.326841     Document Type: Conference Paper

References (16)

1. J. Bilmes, "The Graphical Models Toolkit", http://ssli.ee. washington.edu/bilmes/gmtk/.
2. J. Bilmes and G. Zweig, "The Graphical Models Toolkit: An open source software system for speech and time-series processing," in Proc. ICASSP, 2002.
3. M. Brand, N. Oliver, and A. Pentland, "Coupled hidden Markov models for complex action recognition," in Proc. IEEE International Conference on Computer Vision and Pattern Recognition, pp. 994-999, San Juan, Puerto Rico, June 1997.
4. A. Dempster, N. Laird, and D. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," Journal of the Royal Statistical Society, 39:1-38, 1977.
5. Z. Ghahramani and M. Jordan, "Factorial hidden Markov models," in Proc. Conference Advances in Neural Information Processing Systems, D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, eds., vol. 8, pp. 472-478, MIT Press, Cambridge, MA, USA, 1995.
6. J. Gowdy, A. Subramanya, C. Bartels, and J. Bilmes, "DBN based multi-stream models for audio-visual speech recognition," in Proc. ICASSP, 2004.
7. G. Gravier, G. Potamianos, and C. Neti, "Asynchrony modeling for audio-visual speech recognition," in Proc. Human Language Technology Conference, San Diego, 2002.
8. K. Livescu and J. Glass, "Feature-based pronunciation modeling for speech recognition," in Proc. HLT/NAACL, 2004.
9. K. Livescu and J. Glass, "Feature-based pronunciation modeling with trainable asynchrony probabilities," in Proc. ICSLP, 2004.
10. K. Murphy, Dynamic Bayesian Networks: Representation, Inference and Learning. Ph.D. thesis, U.C. Berkeley CS Division, 2002.
11. C. Neti, G. Potamianos, J. Luettin, I. Matthews, H. Glotin, and D. Vergyri, "Large-vocabulary audio-visual speech recognition: A summary of the Johns Hopkins Summer 2000 Workshop," in Proc. Works. Signal Processing, pp. 619-624, Cannes, France, 2001.
12. H. Nock and S. Young, "Loosely-Coupled HMMs for ASR," in Proc. ICSLP, 2000.
13. E.K. Patterson, S. Gurbuz, Z. Tufekci, and J.N. Gowdy, "CUAVE: A new audio-visual database for multimodal human-computer interface research," in Proc. ICASSP, 2002.
14. G. Potamianos, C. Neti, G. Gravier, A. Garg, and A. Senior, "Recent Advances in the Automatic Recognition of Audio-Visual Speech", in Proc. IEEE, 2003.
15. K. Saenko, M. Siracusa, K.Wilson, K. Livescu, J. Glass, and T. Darrell, "Visual Speech Recognition with Loosely Synchronized Feature Streams," in Proc. International Conference on Computer Vision, 2006.
16. Y. Zhang, Q. Diao, S. Huang, W. Hu, C. Bartels, and J. Bilmes, "DBN based multi-stream models for speech", in Proc. ICASSP, 2003.