Dynamic Bayesian networks for audio-visual speech recognition

Eurasip Journal on Applied Signal Processing

Volumn 2002, Issue 11, 2002, Pages 1274-1288

  Nefian, Ara V ^a   Liang, Luhong ^b   Pi, Xiaobo ^b   Liu, Xiaoxing ^b   Murphy, Kevin ^c  

^a INTEL CORPORATION   (United States)

^b INTEL CORPORATION   (China)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Audio visual speech recognition; Coupled hidden Markov models; Dynamic Bayesian networks; Factorial hidden Markov models; Hidden Markov models

Indexed keywords

ACOUSTIC NOISE; ACOUSTIC SIGNAL PROCESSING; ALGORITHMS; CORRELATION THEORY; MARKOV PROCESSES; SPEECH SYNTHESIS; VIDEO SIGNAL PROCESSING;

AUDIO-VISUAL SPEECH RECOGNITION (AVSR); BAYESIAN NETWORKS;

SPEECH RECOGNITION;

EID: 0036874999     PISSN: 11108657     EISSN: None     Source Type: Journal    
DOI: 10.1155/S1110865702206083     Document Type: Article

References (30)

1. H. McGurk and J. MacDonald, "Hearing lips and seeing voices," Nature, vol. 264, no. 5588, pp. 746-748, 1976.
2. C. Neti, G. Potamianos, J. Luettin, et al., "Audio visual speech recognition, Final workshop 2000 report," Tech. Rep., Center for Language and Speech Processing, The Johns Hopkins University, Baltimore, Md, USA, 2000.
3. M. Brand, N. Oliver, and A. Pentland, "Coupled hidden Markov models for complex action recognition," in Proc. IEEE International Conference on Computer Vision and Pattern Recognition, pp. 994-999, San Juan, Puerto Rico, June 1997.
4. Z. Ghahramani and M. I. Jordan, "Factorial hidden Markov models," in Proc. Conf. Advances in Neural Information Processing Systems, D. S. Touretzky, M. C. Mozer, and M. E. Hasselmo, Eds., vol. 8, pp. 472-478, MIT Press, Cambridge, Mass, USA, 1995.
5. T. Dean and K. Kanazawa, "A model for reasoning about persistence and causation," Artificial Intelligence, vol. 93, no. 1-2, pp. 1-27, 1989.
6. L. Rabiner and B.-H. Juang, Fundamentals of Speech Recognition, Prentice-Hall, Englewood Cliffs, NJ, USA, 1993.
7. J. Luettin, N. Thacker, and S. Beet, "Speechreading using shape and intensity information," in Proc. the 4th IEEE International Conf. on Spoken Language Processing, vol. 1, pp. 58-61, Philadelphia, Pa, USA, 1996.
8. T. Chen, "Audiovisual speech processing," IEEE Signal Processing Magazine, vol. 18, pp. 9-21, January 2001.
9. [9] C. Bregler and S. Omohundro, "Nonlinear manifold learning for visual speech recognition," in Proc. IEEE International Conf. on Computer Vision, pp. 494-499, Boston, Mass, USA, 1995.
10. R. Kober, U. Harz, and J. Schiffers, "Fusion of visual and acoustic signals for command-word recognition," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing, pp. 1495-1497, Munich, Germany, April 1997.
11. S. Dupont and J. Luettin, "Audio-visual speech modeling for continuous speech recognition," IEEE Trans. Multimedia, vol. 2, no. 3, pp. 141-151, 2000.
12. A. Adjoudani and C. Benoît, "Audio-visual speech recognition compared across two architectures," in European Conference on Speech Communication and Technology, pp. 1563-1566, Madrid, Spain, 1995.
13. G. Potamianos, J. Luettin, and C. Neti, "Hierarchical discriminant features for audio-visual LVCSR," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing, pp. 165-168, Salt Lake City, Utah, USA, May 2001.
14. J. Luettin, G. Potamianos, and C. Neti, "Asynchronous stream modeling for large vocabulary audio-visual speech recognition," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing, pp. 169-172, Salt Lake City, Utah, USA, 2001.
15. M. J. Tomlinson, M. J. Russell, and N. M. Brooke, "Integrating audio and visual information to provide highly robust speech recognition," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing, pp. 821-824, Atlanta, Ga, USA, May 1996.
16. Y. Zhang, S. Levinson, and T. Huang, "Speaker independent audio-visual speech recognition," in IEEE International Conference on Multimedia and Expo, vol. 2, pp. 1073-1076, New York, NY, USA, 2000.
17. G. Gravier, G. Potamianos, and C. Neti, "Asynchrony modeling for audio-visual speech recognition," in Proc. Human Language Technology Conference, San Diego, Calif, USA, March 2002.
18. Advanced Multimedia Processing Lab, http://amp.ece.cmu.edu/projects/AudioVisualSpeechProcessing/, Carnegie Mellon University, Pittsburgh, Pa, USA.
19. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, John Wiley & Sons, New York, NY, USA, 2nd edition, 2000.
20. K. R. Castleman, Digital Image Processing, Prentice-Hall, Englewood Cliffs, NJ, USA, 1996.
21. U. Kjaerulff, "A computational scheme for reasoning in dynamic probabilistic networks," in Proc. the 8th International Conference on Uncertainty in Artificial Intelligence, pp. 121-129, Stanford, Calif, USA, 1992.
22. Z. Ghahramani, "Learning dynamic Bayesian networks," in Adaptive Processing of Sequences and Data Structures, C. Giles and M. Gori, Eds., Lecture Notes in Artificial Intelligence, pp. 168-197, Springer-Verlag, Berlin, Germany, 1998.
23. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference, Morgan Kaufmann Publishers, San Mateo, Calif, USA, 1988.
24. F. V. Jensen, Bayesian Networks and Decision Graphs, Springer-Verlag, New York, USA, 2001.
25. V. Pavlovic, Dynamic Bayesian networks for information fusion with applications to human-computer interfaces, Ph.D. thesis, University of Illinois, Urbana-Champaign, III, USA, 1999.
26. S. Chu and T. Huang, "Bimodal speech recognition using coupled hidden Markov models," in Proc. IEEE International Conf. on Spoken Language Processing, vol. 2, pp. 747-750, Beijing, China, 2000.
27. S. Chu and T. Huang, "Audio-visual speech modeling using coupled hidden Markov models," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing, pp. 2009-2012, Orlando, Fla, USA, May 2002.
28. M. E. Hennecke, D. G. Stork, and K. V. Prasad, "Visionary speech: Looking ahead to practical speechreading systems," in Speechreading by Humans and Machines: Models, Systems and Applications, D. G. Stork and M. E. Hennecke, Eds., vol. 150 of NATO ASI Series F: Computer and Systems Sciences, pp. 331-349, Springer-Verlag, Berlin, Germany, 1996.
29. A. Nefian, L. Liang, X. Pi, X. Liu, C. Mao, and K. Murphy, "A coupled HMM for audio-visual speech recognition," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing, pp. 2013-2016, Orlando, Fla, USA, May 2002.
30. L. Liang, X. Liu, Y. Zhao, X. Pi, and A. Nefian, "Speaker independent audio-visual continuous speech recognition," in IEEE International Conference on Multimedia and Expo, Lausanne, Switzerland, August 2002.