Dynamic stream weighting for turbo-decoding-based audiovisual ASR

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn 08-12-September-2016, Issue , 2016, Pages 2135-2139

  Gergen, Sebastian ^a   Zeiler, Steffen ^a   Abdelaziz, Ahmed Hussen ^b   Nickel, Robert ^c   Kolossa, Dorothea ^a  

^a RUHR UNIVERSITY BOCHUM   (Germany)

^b INTERNATIONAL COMPUTER SCIENCE INSTITUTE   (United States)

^c BUCKNELL UNIVERSITY   (United States)

Author keywords

Audiovisual speech recognition; Stream weighting; Turbo decoding

Indexed keywords

AUDIO ACOUSTICS; COST FUNCTIONS; DECODING; FACE RECOGNITION; HIDDEN MARKOV MODELS; MARKOV PROCESSES; SPEECH COMMUNICATION; SPEECH PROCESSING; VIDEO STREAMING;

AUDIO VISUAL SPEECH RECOGNITION; AUTOMATIC SPEECH RECOGNITION; COUPLED HIDDEN MARKOV MODELS; HUMAN MACHINE INTERACTION; RECOGNITION ACCURACY; SIGNAL DEGRADATION; STREAM WEIGHTING; TURBO DECODING;

SPEECH RECOGNITION;

EID: 84994339086     PISSN: 2308457X     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: 10.21437/Interspeech.2016-166     Document Type: Conference Paper

References (22)

1. A. V. Nefian, L. Liang, X. Pi, X. Liu, and K. Murphy, "Dynamic Bayesian networks for audio-visual speech recognition, " EURASIP Journal on Applied Signal Processing, vol. 11, pp. 1-15, 2002.
2. G. Potamianos, C. Neti, G. Gravier, A. Garg, and A. W. Senior, "Recent advances in the automatic recognition of audiovisual speech, " Proceedings of IEEE, vol. 91, no. 9, pp. 1306-1326, Sep 2003.
3. A. Vorwerk, S. Zeiler, D. Kolossa, R. F. Astudillo, and D. Lerch, "Use of missing and unreliable data for audiovisual speech recognition, " in Robust Speech Recognition of Uncertain or Missing Data, D. Kolossa and R. Haeb-Umbach, Eds. Springer, 2011, pp. 345-375. [Online]. Available: http://dx.doi.org/10.1007/978-3-642-21317-5
4. S. Receveur, D. Scheler, and T. Fingscheidt, "A turbo-decoding weighted forward-backward algorithm for multimodal speech recognition, " in 5th International Workshop on Spoken Dialog Systems, 2014, pp. 4-15.
5. J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Y. Ng, "Multimodal deep learning, " in Proceedings of the 28th International Conference on Machine Learning, 2011, pp. 1-8.
6. S. Tamura, H. Ninomiya, N. Kitaoka, S. Osuga, Y. Iribe, K. Takeda, and S. Hayamizu, "Audio-visual speech recognition using deep bottleneck features and high-performance lipreading, " in Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2015, pp. 575-582.
7. Y. Mroueh, E. Marcheret, and V. Goel, "Deep multimodal learning for audio-visual speech recognition, " in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2015, pp. 2130-2134.
8. S. Dupont and J. Luettin, "Audio-visual speech modeling for continuous speech recognition, " IEEE Transactions on Multimedia, vol. 2, no. 3, pp. 141-151, 2000. [Online]. Available: http://dx.doi.org/10.1109/6046.865479
9. A. H. Abdelaziz, S. Zeiler, and D. Kolossa, "A new EM estimation of dynamic stream weights for coupled-HMM-based audio-visual ASR, " in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2014, pp. 1527-1531.
10. L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition, " Proceedings of the IEEE, vol. 77, no. 2, pp. 257-286, 1989.
11. R. Haeb-Umbach and D. Kolossa, Robust Speech Recognition of Uncertain or Missing Data-Theory and Applications, D. Kolossa and R. Haeb-Umbach, Eds. Springer, 2011.
12. S. Receveur, R. Weiss, and T. Fingscheidt, "Turbo automatic speech recognition, " IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP), vol. 99, pp. 1-1, 2016.
13. C. Berrou, A. Glavieux, and P. Thitimajshima, "Near Shannon limit error-correcting coding and decoding: Turbo codes, " in IEEE International Conference on Communications, 1993, pp. 1064-1070.
14. C. Berrou and A. Glavieux, "Near optimum error-correcting coding and decoding: Turbo Codes, " IEEE Transactions on Communications, vol. 44, no. 10, pp. 1261-1271, Oct. 1996.
15. S. T. Shivappa, B. D. Rao, and M. M. Trivedi, "Multimodal information fusion using the iterative decoding algorithm and its application to audio-visual speech recognition, " in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2008, pp. 2241-2244.
16. S. Zeiler, R. Nickel, N. Ma, G. J. Brown, and D. Kolossa, "Robust audiovisual speech recognition using noise-adaptive linear discriminant analysis, " in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2016, pp. 1-2.
17. A. H. Abdelaziz, S. Zeiler, and D. Kolossa, "Learning dynamic stream weights for coupled-HMM-based audio-visual speech recognition, " IEEE Transactions on Audio, Speech & Language Processing, vol. 23, no. 5, pp. 863-876, 2015.
18. M. Cooke, J. Barker, S. Cunningham, and X. Shao, "An audiovisual corpus for speech perception and automatic speech recognition, " The Journal of the Acoustical Society of America, vol. 120, no. 5, pp. 2421-2424, 2006.
19. L. Bahl, P. Brown, P. de Souza, and R. Mercer, "Maximum mutual information estimation of hidden Markov model parameters for speech recognition, " in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1986, pp. 999-999.
20. D. Povey, D. Kanevsky, B. Kingsbury, B. Ramabhadran, G. Saon, and K. Visweswariah, "Boosted MMI for model and featurespace discriminative training, " in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2008, pp. 4057-4060.
21. L. Deng and X. Li, "Machine learning paradigms for speech recognition: An overview, " IEEE Transactions on Audio, Speech, and Language Processing, vol. 21, no. 5, pp. 1060-1089, 2013.
22. H. J. M. Steeneken and F. W. M. Geurtsen, "Description of the RSG-10 noise database, " in Technical Report IZF 1988-3, TNO Institute for Perception, Soesterberg, The Netherlands, 1988.