Recent advances in the automatic recognition of audiovisual speech

Proceedings of the IEEE

Volumn 91, Issue 9, 2003, Pages 1306-1325

  Potamianos, Gerasimos ^a   Neti, Chalapathy ^a   Gravier, Guillaume ^b   Garg, Ashutosh ^c   Senior, Andrew W ^a  

^a IBM T J WATSON RESEARCH CENTER   (United States)

^b INRIA   (France)

^c IBM ALMADEN RESEARCH CENTER   (United States)

Author keywords

Adaptation; Audiovisual fusion; Audiovisual speech recognition (ASR); Face tracking; Hidden Markov models (HMM); Multimedia databases; Multistream hidden Markov models; Product hidden Markov models; Speechreading; Stream reliability; Visual feature extraction

Indexed keywords

ALGORITHMS; COMPUTER SIMULATION; DATABASE SYSTEMS; FACE RECOGNITION; FEATURE EXTRACTION; HUMAN COMPUTER INTERACTION; IMAGE PROCESSING; MARKOV PROCESSES; RELIABILITY; SPEECH ANALYSIS; VISUAL COMMUNICATION;

ADAPTATION; AUDIOVISUAL FUSION; AUDIOVISUAL SPEECH RECOGNITION (ASR); FACE TRACKING; HIDDEN MARKOV MODELS (HMM); MULTIMEDIA DATABASES; MULTISTREAM HIDDEN MARKOV MODELS; PRODUCT HIDDEN MARKOV MODELS; SPEECHREADING; STREAM RELIABILITY;

SPEECH RECOGNITION;

EID: 4544290191     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/JPROC.2003.817150     Document Type: Conference Paper

References (134)

1. R. P. Lippmann, "Speech recognition by machines and humans," Speech Commun., vol. 22, pp. 1-15, 1997.
2. B. H. Juang, "Speech recognition in adverse environments," Comput. Speech Lang., vol. 5, pp. 275-294, 1991.
3. R. Stern, A. Acero, F.-H. Liu, and Y. Ohshima, "Signal processing for robust speech recognition," in Automatic Speech and Speaker Recognition. Advanced Topics, C.-H. Lee, F. K. Soong, and Y. Ohshima, Eds. Norwell, MA: Kluwer, 1997, ch. 15, pp. 357-384.
4. D. G. Stork and M. E. Hennecke, Eds., Speechreading by Humans and Machines. Berlin, Germany: Springer-Verlag, 1996.
5. R. Campbell, B. Dodd, and D. Burnham, Eds., Hearing by Eye II. Hove, U.K.: Psychology, 1998.
6. W. H. Sumby and I. Pollack, "Visual contribution to speech intelligibility in noise," J. Acoust. Soc. Amer., vol. 26, pp. 212-215, 1954.
7. H. MacGurk and J. MacDonald, "Hearing lips and seeing voices," Nature, vol. 264, pp. 746-748, 1976.
8. M. Marschark, D. LePoutre, and L. Bernent, "Mouth movement and signed communication," in Hearing by Eye II, R. Campbell, B. Dodd, and D. Burnham, Eds. Hove, U.K.: Psychology, 1998, ch. 13, pp. 245-266.
9. L. E. Bernstein, M. E. Demorest, and P. E. Tucker, "What makes a good speechreader? First you have to find one," in Hearing by Eye II, R. Campbell, B. Dodd, and D. Burnham, Eds. Hove, U.K.: Psychology, 1998, ch. 11, pp. 211-227.
10. A. Q. Summerfield, "Some preliminaries to a comprehensive account of audio visual speech perception," in Hearing by Eye: The Psychology of Lip-Reading, R. Campbell and B. Dodd, Eds. London, U.K.: Lawrence Erlbaum, 1987, pp. 3-51.
11. D. W. Massaro and D. G. Stork, "Speech recognition and sensory integration," Amer. Sci., vol. 86, pp. 236-244, 1998.
12. H. Yehia, P. Rubin, and E. Vatikiotis-Bateson, "Quantitative association of vocal-tract and facial behavior," Speech Commun., vol. 26, pp. 23-43, 1998.
13. J. P. Barker and F. Berthommier, "Estimation of speech acoustics from visual speech features: A comparison of linear and nonlinear models," in Proc. Conf. Audio-Visual Speech Processing, 1999, pp. 112-117.
14. J. Jiang, A. Alwan, P. A. Keating, B. Chaney, E. T. Auer Jr., and L. E. Bernstein, "On the relationship between face movements, tongue movements, and speech acoustics," EURASIP J. Appl. Signal Processing, vol. 2002, pp. 1174-1188, Nov. 2002.
15. Q. Summerfield, A. MacLeod, M. McGrath, and M. Brooke, "Lips, teeth, and the benefits of lipreading," in Handbook of Research on Face Processing, H. D. Ellis and A. W. Young, Eds. Amsterdam, The Netherlands: Elsevier, 1989, pp. 223-233.
16. P. M. T. Smeele, "Psychology of human speechreading," in Speechreading by Humans and Machines, D. G. Stork and M. E. Hennecke, Eds. Berlin, Germany: Springer-Verlag, 1996, pp. 3-15.
17. M. E. Hennecke, D. G. Stork, and K. V. Prasad, "Visionary speech: Looking ahead to practical speechreading systems," in Speechreading by Humans and Machines, D. G. Stork and M. E. Hennecke, Eds. Berlin, Germany: Springer-Verlag, 1996, pp. 331-349.
18. E. D. Petajan, "Automatic lipreading to enhance speech recognition," in Proc. Global Telecommunications Conf., 1984, pp. 265-272.
19. L. Rabiner and B.-H. Juang, Fundamentals of Speech Recognition. Englewood Cliffs, NJ: Prentice-Hall, 1993.
20. C. C. Chibelushi, F. Deravi, and J. S. D. Mason, "A review of speech-based bimodal recognition," IEEE Trans. Multimedia, vol. 4, pp. 23-37, Mar. 2002.
21. A. Adjoudani and C. Benoît, "On the integration of auditory and visual parameters in an HMM-based ASR," in Speechreading by Humans and Machines, D. G. Stork and M. E. Hennecke, Eds. Berlin, Germany: Springer-Verlag, 1996, pp. 461-471.
22. Q. Su and P. L. Silsbee, "Robust audiovisual integration using semicontinuous hidden Markov models," in Proc. Int. Conf. Spoken Language Processing, 1996, pp. 42-45.
23. J. R. Movellan and G. Chadderdon, "Channel separability in the audio visual integration of speech: A Bayesian approach," in Speechreading by Humans and Machines, D. G. Stork and M. E. Hennecke, Eds. Berlin, Germany: Springer-Verlag, 1996, pp. 473-487.
24. S. Cox, I. Matthews, and A. Bangham, "Combining noise compensation with visual information in speech recognition," in Proc. Eur. Workshop Audio-Visual Speech Processing, 1997, pp. 53-56.
25. M. T. Chan, Y. Zhang, and T. S. Huang, "Real-time lip tracking and bimodal continuous speech recognition," in Proc. Workshop Multimedia Signal Processing, 1998, pp. 65-70.
26. S. Dupont and J. Luettin, "Audio-visual speech modeling for continuous speech recognition," IEEE Trans. Multimedia, vol. 2, pp. 141-151, Sept. 2000.
27. S. Gurbuz, Z. Tufekci, E. Patterson, and J. N. Gowdy, "Application of affine-invariant Fourier descriptors to lipreading for audio-visual speech recognition," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 2001, pp. 177-180.
28. F. J. Huang and T. Chen, "Consideration of Lombard effect for speechreading," in Proc. Workshop Multimedia Signal Processing, 2001, pp. 613-618.
29. A. V. Nefian, L. Liang, X. Pi, X. Liu, and K. Murphy, "Dynamic Bayesian networks for audio-visual speech recognition," EURASIP J. Appl. Signal Process., vol. 2002, pp. 1274-1288, Nov. 2002.
30. G. Potamianos and H. P. Graf, "Discriminative training of HMM stream exponents for audio-visual speech recognition," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 1998, pp. 3733-3736.
31. Y. Zhang, S. Levinson, and T. Huang, "Speaker independent audiovisual speech recognition," in Proc. Int. Conf. Multimedia Expo, 2000, pp. 1073-1076.
32. A. J. Goldschen, O. N. Garcia, and E. D. Petajan, "Rationale for phoneme-viseme mapping and feature selection in visual speech recognition," in Speechreading by Humans and Machines, D. G. Stork and M. E. Hennecke, Eds. Berlin, Germany: Springer-Verlag, 1996, pp. 505-515.
33. A. Rogozan, P. Deléglise, and M. Alissali, "Adaptive determination of audio and visual weights for automatic speech recognition," in Proc. Eur. Workshop Audio-Visual Speech Processing, 1997, pp. 61-64.
34. C. Bregler and Y. Konig, "'Eigenlips' for robust speech recognition," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 1994, pp. 669-672.
35. G. Krone, B. Talle, A. Wichen, and G. Palm, "Neural architectures for sensorfusion in speech recognition," in Proc. Eur. Workshop Audio-Visual Speech Processing, 1997, pp. 57-60.
36. S. Nakamura, H. Ito, and K. Shikano, "Stream weight optimization of speech and lip image sequence for audio-visual speech recognition," in Proc. Int. Conf. Spoken Language Processing, vol. 3, 2000, pp. 20-23.
37. C. Neti, G. Potamianos, J. Luettin, I. Matthews, H. Glotin, D. Vergyri, J. Sison, A. Mashari, and J. Zhou, "Audio-visual speech recognition," Center Lang. Speech Process., Johns Hopkins, Univ., Baltimore, MD, 2000.
38. G. Potamianos and C. Neti, "Automatic speechreading of impaired speech," in Proc. Conf. Audio-Visual Speech Processing, 2001, pp. 177-182.
39. P. Duchnowski, U. Meier, and A. Waibel, "See me, hear me: Integrating automatic speech recognition and lip-reading," in Proc. Int. Conf. Spoken Language Processing, 1994, pp. 547-550.
40. G. Potamianos, H. P. Graf, and E. Cosatto, "An image transform approach for HMM based automatic lipreading," in Proc. Int. Conf. image Processing, vol. 1, 1998, pp. 173-177.
41. G. Chiou and J.-N. Hwang, "Lipreading from color video," IEEE Trans. Image Processing, vol. 6, pp. 1192-1195, Aug. 1997.
42. N. Li, S. Dettmer, and M. Shah, "Lipreading using eigensequences," in Proc. Int. Workshop Automatic Face Gesture Recognition, 1995, pp. 30-34.
43. P. Scanlon and R. Reilly, "Feature analysis for automatic speechreading," in Proc. Workshop Multimedia Signal Processing, 2001, pp. 625-630.
44. G. Potamianos, C. Neti, G. Iyengar, A. W. Senior, and A. Verma, "A cascade visual front end for speaker independent automatic speechreading," Int. J. Speech Technol., vol. 4, pp. 193-208, July/Oct. 2001.
45. P. Jourlin, "Word dependent acoustic-labial weights in HMM-based speech recognition," in Proc. Eur. Workshop Audio-Visual Speech Processing, 1997, pp. 69-72.
46. P. Teissier, J. Robert-Ribes, and J. L. Schwartz, "Comparing models for audiovisual fusion in a noisy-vowel recognition task," IEEE Trans. Speech Audio Processing, vol. 7, pp. 629-642, Nov. 1999.
47. M. Heckmann, F. Berthommier, and K. Kroschel, "Noise adaptive stream weighting in audio-visual speech recognition," EURASIP J. Appl. Signal Processing, vol. 2002, pp. 1260-1273, Nov. 2002.
48. L. Czap, "Lip representation by image ellipse," in Proc. Int. Conf. Spoken Language Processing, vol. 4, 2000, pp. 93-96.
49. J. Luettin and N. A. Thacker, "Speechreading using probabilistic models," Comput. Vision Image Understanding, vol. 65, pp. 163-178, 1997.
50. D. Chandramohan and P. L. Silsbee, "A multiple deformable template approach for visual speech recognition," in Proc. Int. Conf. Spoken Language Processing, 1996, pp. 50-53.
51. B. Dalton, R. Kaucic, and A. Blake, "Automatic speechreading using dynamic contours," in Speechreading by Humans and Machines, D. G. Stork and M. E. Hennecke, Eds. Berlin, Germany: Springer-Verlag, 1996, pp. 373-382.
52. P. S. Aleksic, J. J. Williams, Z. Wu, and A. K. Katsaggelos, "Audio-visual speech recognition using MPEG-4 compliant visual features," EURASIP J. Appl. Signal Processing, vol. 2002, pp. 1213-1227, Nov. 2002.
53. M. T. Chan, "HMM based audio-visual speech recognition integrating geometric- and appearance-based visual features," in Proc. Workshop Multimedia Signal Processing, 2001, pp. 9-14.
54. T. F. Cootes, G. J. Edwards, and C. J. Taylor, "Active appearance models," in Proc. Eur. Conf. Computer Vision, 1998, pp. 484-498.
55. A. Adjoudani, T. Guiard-Marigny, B. L. Goff, L. Reveret, and C. Benoît, "A multimedia platform for audio-visual speech processing," in Proc. Eur. Conf. Speech Communication Technology, 1997, pp. 1671-1674.
56. M. Kass, A. Witkin, and D. Terzopoulos, "Snakes: Active contour models," Int. J. Comput. Vision, vol. 4, pp. 321-331, 1988.
57. A. L. Yuille, P. W. Hallinan, and D. S. Cohen, "Feature extraction from faces using deformable templates," Int. J. Comput. Vision, vol. 8, pp. 99-111, 1992.
58. T. F. Cootes, C. J. Taylor, D. H. Cooper, and J. Graham, "Active shape models - their training and application," Comput. Vision image Understanding, vol. 61, pp. 38-59, Jan. 1995.
59. H. P. Graf, E. Cosatto, and G. Potamianos, "Robust recognition of faces and facial features with a multi-modal system," in Proc. Int. Conf. Systems, Man, Cybernetics, 1997, pp. 2034-2039.
60. X. Zhang, C. C. Broun, R. M. Mersereau, and M. Clements, "Automatic speechreading with applications to human-computer interfaces," EURASIP J. Appl. Signal Processing, vol. 2002, pp. 1228-1247, Nov. 2002.
61. P. Daubias and P. Deléglise, "Automatically building and evaluating statistical models for lipreading," EURASIP J. Appl. Signal Processing, vol. 2002, pp. 1202-1212, Nov. 2002.
62. H. A. Rowley, S. Batuja, and T. Kanade, "Neural network-based face detection," IEEE Trans. Pattern Anal. Machine Intell., vol. 20, pp. 23-38, Jan. 1998.
63. K.-K. Sung and T. Poggio, "Example-based learning for view-based human face detection," IEEE Trans. Pattern Anal. Machine Intell., vol. 20, pp. 39-51, Jan. 1998.
64. A. W. Senior, "Face and feature finding for a face recognition system," in Proc. Int. Conf. Audio Video-based Biometric Person Authentication, 1999, pp. 154-159.
65. C. R. Rao, Linear Statistical Inference and its Applications. New York: Wiley, 1965.
66. C. Chatfield and A. J. Collins, Introduction to Multivariate Analysis. London, U.K.: Chapman & Hall, 1991.
67. G. Potamianos and C. Neti, "Improved ROI and within frame discriminant features for lipreading," in Proc. Int. Conf. Image Processing, vol. 3, 2001, pp. 250-253.
68. R. C. Gonzalez and P. Wintz, Digital Image Processing. Reading, MA: Addison-Wesley, 1977.
69. I. Daubechies, Wavelets. Philadelphia, PA: SIAM, 1992.
70. M. J. Tomlinson, M. J. Russell, and N. M. Brooke, "Integrating audio and visual information to provide highly robust speech recognition," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 1996, pp.821-824.
71. M. S. Gray, J. R. Movellan, and T. J. Sejnowski, "Dynamic features for visual speech-reading: A systematic comparison," in Advances in Neural Information Processing Systems, M. C. Mozer, M. I. Jordan, and T. Petsche, Eds. Cambridge. MA: MIT Press, 1997, vol. 9, pp. 751-757.
72. S. Young, D. Kershaw, J. Odell, D. Ollason, V. Valtchev, and P. Woodland, The HTK Book. Cambridge, U.K.: Entropic, 1999.
73. L. D. Rosenblum and H. M. Saldana, "Time-varying information for visual speech perception," in Hearing by Eye II, R. Campbell, B. Dodd, and D. Burnham, Eds. Hove, U.K.: Psychology, 1998, ch. 3. pp. 61-81.
74. R. A. Gopinath, "Maximum likelihood modeling with Gaussian distributions for classification," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 1998, pp. 661-664.
75. L. Polymenakos, P. Olsen, D. Kanevsky, R. A. Gopinath, P. Gopalakrishnan, and S. Chen, "Transcription of broadcast news - some recent improvements to IBM's LVCSR system," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 1998, pp. 901-904.
76. U. Meier, W. Hurst, and P. Duchnowski, "Adaptive bimodal sensor fusion for automatic speechreading," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 1996, pp. 833-836.
77. O. Vanegas, A. Tanaka, K. Tokuda, and T. Kitamura, "HMM-based visual speech recognition using intensity and location normalization," in Proc. Int. Conf. Spoken Language Processing, 1998, pp. 289-292.
78. T. Chen, "Audiovisual speech processing. Lip reading and lip synchronization," IEEE Signal Processing Mag., vol. 18, pp. 9-21, Jan. 2001.
79. M. Gordan, C. Kotropoulos, and I. Pitas, "A support vector machine-based dynamic network for visual speech recognition applications," EURASIP J. Appl. Signal Processing, vol. 2002, pp. 1248-1259, Nov. 2002.
80. |80] S. M. Chu and T. S. Huang, "Audio-visual speech modeling using coupled hidden Markov models," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 2002, pp. 2009-2012.
81. E. K. Patterson, S. Gurbuz, Z. Tufekci, and J. N. Gowdy, "Moving talker, speaker-independent feature study, and baseline results using the CUAVE multimodal speech corpus," EURASIP J. Appl. Signal Processing, vol. 2002, pp. 1189-1201, Nov. 2002.
82. A. Rogozan, "Discriminative learning of visual data for audiovisual speech recognition," Int. J. Artif. Intell. Tools, vol. 8, pp. 43-52, 1999.
83. P. L. Silsbee and A. C. Bovik, "Computer lipreading for improved accuracy in automatic speech recognition," IEEE Trans. Speech Audio Processing, vol. 4, pp. 337-351, Sept. 1996.
84. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum likelihood from incomplete data via the EM algorithm," J. Royal Statist. Soc., vol. 39, pp. 1-38, 1977.
85. L. R. Bahl, P. F. Brown, P. V. DeSouza, and R. L. Mercer, "Maximum mutual information estimation of hidden Markov model parameters for speech recognition," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 1986, pp. 49-52.
86. W. Chou, B.-H. Juang, C.-H. Lee, and F. Soong, "A minimum error rate pattern recognition approach to speech recognition," Int. J. Pattern Recognit. Artif. Intell., vol. 8, pp. 5-31, Jan. 1994.
87. L. Girin, G. Feng, and J.-L. Schwartz, "Noisy speech enhancement with filters estimated from the speaker's lips," in Proc. Eur. Conf. Speech Communication Technology, 1995, pp. 1559-1562.
88. L. Girin, J.-L. Schwartz, and G. Feng, "Audio-visual enhancement of speech in noise," J. Acoust. Soc. Amer., vol. 109, pp. 3007-3020, 2001.
89. R. Goecke, G. Potamianos, and C. Neti, "Noisy audio feature enhancement using audio-visual speech data," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 2002, pp. 2025-2028.
90. S. Deligne, G. Potamianos, and C. Neti, "Audio-visual speech enhancement with AVCDCN (audio-visual codebook dependent cepstral normalization)," in Proc. Int. Conf. Spoken Language Processing, 2002, pp. 1449-1452.
91. L. Xu, A. Krzyzak, and C. Y. Suen, "Methods of combining multiple classifiers and their applications in handwritten recognition," IEEE Trans. Syst., Man, Cybern., vol. 22, pp. 418-435, May/June 1992.
92. J. Kittler, M. Halef, R. P. W. Duin, and J. Matas, "On combining classifiers," IEEE Trans. Pattern Anal. Machine Intell., vol. 20, pp. 226-239, Mar. 1998.
93. A. K. Jain, R. P. W. Duin, and J. Mao, "Statistical pattern recognition: A review," IEEE Trans. Pattern Anal. Machine Intell., vol. 22, pp. 4-37, Jan. 2000.
94. A. Jain, R. Bolle, and S. Pankanti, "Introduction to biometrics," in Biometrics. Personal Identification in Networked Society, A. Jain, R. Bolle, and S. Pankanti, Eds. Norwell, MA: Kluwer, 1999, ch. 1, pp. 1-41.
95. H. Bourlard and S. Dupont, "A new ASR approach based on independent processing and recombination of partial frequency bands," in Proc. Int. Conf. Spoken Language Processing, 1996, pp. 426-429.
96. H. Glotin and F. Berthommier, "Test of several external posterior weighting functions for multiband full combination ASR," in Proc. Int. Conf. Spoken Language Processing, vol. 1, 2000, pp. 333-336.
97. C. Miyajima, K. Tokuda, and T. Kitamura, "Audio-visual speech recognition using MCE-based HMM's and model-dependent stream weights," in Proc. Int. Conf. Spoken Language Processing, vol. 2, 2000, pp. 1023-1026.
98. P. Beyerlein, "Discriminative model combination," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 1998, pp. 481-484.
99. H. Glotin, D. Vergyri, C. Neti, G. Potamianos, and J. Luettin, "Weighting schemes for audio-visual fusion in speech recognition," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 2001, pp. 173-176.
100. P. Varga and R. K. Moore, "Hidden Markov model decomposition of speech and noise," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 1990, pp. 845-848.
101. M. Brand, N. Oliver, and A. Pentland, "Coupled hidden Markov models for complex action recognition," in Proc. Conf. Computer Vision Pattern Recognition, 1997, pp. 994-999.
102. K. W. Grant and S. Greenberg, "Speech intelligibility derived from asynchronous processing of auditory-visual information," in Proc. Conf. Audio-Visual Speech Processing, 2001, pp. 132-137.
103. X. Liu, Y. Zhao, X. Pi, L. Liang, and A. V. Nefian, "Audio-visual continuous speech recognition using a coupled hidden Markov model," in Proc. Int. Conf. Spoken Language Processing, 2002, pp. 213-216.
104. G. Gravier, G. Potamianos, and C. Neti, "Asynchrony modeling for audio-visual speech recognition," in Proc. Human Language Technology Conf., 2002, pp. 1-6.
105. S. Nakamura, "Fusion of audio-visual information for integrated speech processing," in Audio- and Video-Based Biometric Person Authentication, J. Bigun and F. Smeraldi, Eds. Berlin, Germany: Springer-Verlag, 2001, pp. 127-143.
106. G. Gravier, S. Axelrod, G. Potamianos, and C. Neti, "Maximum entropy and MCE based HMM stream weight estimation for audio-visual ASR," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 2002, pp. 853-856.
107. J. A. Nelder and R. Mead, "A simplex method for function minimization," Comput. J., vol. 7, pp. 308-313, 1965.
108. F. Berthommier and H. Glotin, "A new SNR-feature mapping for robust multistream speech recognition," in Proc. Int. Congress Phonetic Sciences, 1999, pp. 711-715.
109. G. Polamianos and C. Neti, "Stream confidence estimation for audio-visual speech recognition," in Proc. Int. Conf. Spoken Language Processing, vol. 3, 2000, pp. 746-749.
110. J.-L. Gauvain and C.-H. Lee, "Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains," IEEE Trans. Speech Audio Processing, vol. 2, pp. 291-298, Apr. 1994.
111. C. J. Leggetter and P. C. Woodland, "Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov models," Comput. Speech Lang., vol. 9, pp. 171-185, 1995.
112. L. Neumeyer, A. Sankar, and V. Digalakis, "A comparative study of speaker adaptation techniques," in Proc. Eur. Conf. Speech Communication Technology, 1995, pp. 1127-1130.
113. T. Anastasakos, J. McDonough, and J. Makhoul, "Speaker adaptive training: A maximum likelihood approach to speaker normalization," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 1997, pp. 1043-1046.
114. M. J. F. Gales, "Maximum likelihood multiple projection schemes for hidden Markov models," Cambridge University, Cambridge, U.K., 1999.
115. G. Potamianos and A. Potamianos, "Speaker adaptation for audio-visual speech recognition," in Proc. Eur. Conf. Speech Communication Technology, 1999, pp. 1291-1294.
116. S. Pigeon and L. Vandendorpe, "The M2VTS multimodal face database," in Audio-and Video-based Biometric Person Authentication, J. Bigün, G. Chollet, and G. Borgefors, Eds. Berlin, Germany: Springer-Verlag, 1997, pp. 403-109.
117. K. Messer, J. Matas, J. Kittler, J. Luettin, and G. Maitre, "XM2VTS: The extended M2VTS database," in Proc. Int. Conf. Audio Video-based Biometric Person Authentication, 1999, pp. 72-76.
118. G. Iyengar and C. Neti, "Detection of faces under shadows and lighting variations," in Proc. Workshop Multimedia Signal Processing, 2001, pp. 15-20.
119. L. R. Bahl, S. V. De Gennaro, P. S. Gopalakrishnan, and R. L. Mercer, "A fast approximate acoustic match for large vocabulary speech recognition," IEEE Trans. Speech Audio Processing, vol. 1, pp. 59-67, Jan. 1993.
120. T. Chen and R. R. Rao, "Audio-visual integration in multimodal communication," Proc. IEEE, vol. 86, pp. 837-852, May 1998.
121. T. Wark and S. Sridharan, "A syntactic approach to automatic lip feature extraction for speaker identification," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 1998, pp. 3693-3696.
122. B. Fröba, C. Küblbeck, C. Rothe, and P. Plankensteiner, "Multisensor biometric person recognition in an access control system," in Proc. Int. Conf. Audio Video-based Biometric Person Authentication, 1999, pp. 55-59.
123. B. Maison, C. Neti, and A. Senior, "Audio-visual speaker recognition for broadcast news: Some fusion techniques," in Proc. Workshop Multimedia Signal Processing, 1999, pp. 161-167.
124. M. M. Cohen and D. W. Massaro, "What can visual speech synthesis tell visual speech recognition?," presented at the Asilomar Conf. Signals, Systems, Computers, Pacific Grove, CA, 1994.
125. T. Chen, H. P. Graf, and K. Wang, "Lip synchronization using speech-assisted video processing," IEEE Signal Processing Lett., vol.2, pp. 57-59, Apr. 1995.
126. E. Cosatto, G. Potamianos, and H. P. Graf, "Audio-visual unit selection for the synthesis of photo-realistic talking-heads," in Proc. Int. Conf. Multimedia Expo, 2000, pp. 1097-1100.
127. E. Cosatto and H. P. Graf, "Photo-realistic talking-heads from image samples," IEEE Trans. Multimedia, vol. 2, pp. 152-163, Sept. 2000.
128. J. J. Williams and A. K. Katsaggelos, "An HMM-based speech-to-video synthesizer," IEEE Trans. Neural Networks, vol. 13, pp. 900-915, July 2002.
129. P. De Cuetos, C. Neti, and A. Senior, "Audio-visual intent to speak detection for human computer interaction," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 2000, pp. 1325-1328.
130. J. Huang, Z. Liu, Y. Wang, Y. Chen, and E. Wong, "Integration of multimodal features for video scene classification based on HMM," in Proc. Workshop Multimedia Signal Processing, 1999, pp. 53-58.
131. E. Foucher, L. Girin, and G. Feng, "Audiovisual speech coder: Using vector quantization to exploit the audio/video correlation," in Proc. Conf. Audio-Visual Speech Processing, 1998, pp. 67-71.
132. D. Sodoyer, J.-L. Schwartz, L. Girin, J. Klinkisch, and C. Jutten, "Separation of audio-visual speech sources: A new approach exploiting the audio-visual coherence of speech stimuli," EURASIP J. Appl. Signal Processing, vol. 2002, pp. 1165-1173, Nov. 2002.
133. U. Bub, M. Hunke, and A. Waibel, "Knowing who to listen 10 in speech recognition: Visually guided beamforming," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing, 1995, pp. 848-851.
134. D. N. Zotkin, R. Duraiswami, and L. S. Davis, "Joint audio-visual tracking using particle filters," EURASIP J. Appl. Signal Processing, vol. 2002, pp. 1154-1164, Nov. 2002.