Multimodal speaker/speech recognition using lip motion, lip texture and audio

Signal Processing

Volumn 86, Issue 12, 2006, Pages 3549-3558

  Cetingul H E ^a   Erzin, E ^a   Yemez, Y ^a   Tekalp, A M ^a  

^a KOÇ UNIVERSITY   (Turkey)

Author keywords

Decision fusion; Isolated word recognition; Lip motion; Lip reading; Speaker identification

Indexed keywords

DECISION FUSION; ISOLATED WORD RECOGNITION; LIP MOTION; LIP READING; SPEAKER IDENTIFICATION;

CHARACTER RECOGNITION; DECISION THEORY; MODAL ANALYSIS; MOTION ESTIMATION;

SPEECH RECOGNITION;

EID: 33749436578     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2006.02.045     Document Type: Article

References (34)

1. Campbell J. Speaker recognition: a tutorial. Proc. IEEE 85 9 (1997) 1437-1462
2. Zhang Y.Y.J., and Lades M. Face recognition: eigenface, elastic matching, and neural nets. Proc. IEEE 85 9 (1997) 1423-1435
3. Turk M., and Pentland A. Eigenfaces for recognition. J. Cognitive Neurosci. 3 1 (1991) 586-591
4. Kittler J., Hatef M., Duin R., and Matas J. On combining classifiers. IEEE Trans. Pattern Anal. Machine Intell. 20 3 (1998) 226-239
5. Matthews I., Cootes T., Bangham J., Cox S., and Harvey R. Extraction of visual features for lipreading. IEEE Trans. Pattern Anal. Machine Intell. 24 2 (2002) 198-213
6. Chibelushi C., Deravi F., and Mason J. A review of speech-based bimodal recognition. IEEE Trans. Multimedia 4 1 (2002) 23-37
7. X. Zhang, C. Broun, R. Mersereau, M. Clements, Automatic speechreading with applications to human-computer interfaces, EURASIP J. Appl. Signal Process. (2002) 1228-1247.
8. G. Potamianos, C. Neti, G. Gravier, A. Garg, A. Senior, Recent advances in the automatic recognition of audio-visual speech, Proc. IEEE 91(9).
9. J. Perez, A. Frangi, E. Solano, K. Lukas, Lip reading for robust speech recognition on embedded devices, Proceedings of the International Conference on Acoustics, Speech and Signal Processing 2005 (ICASSP '05), vol. I, 2005, pp. 473-476.
10. Frischholz R., and Dieckmann U. BioID: a multimodal biometric identification system. J. IEEE Comput. 33 2 (2000) 64-68
11. Erzin E., Yemez Y., and Tekalp A. Multimodal speaker identification using an adaptive classifier cascade based on modality reliability. IEEE Trans. Multimedia 7 5 (2005) 840-852
12. Wark T., and Sridharan S. Adaptive fusion of speech and lip information for robust speaker identification. Digital Signal Process. 11 3 (2001) 169-186
13. Jourlin P., Luettin J., Genoud D., and Wassner H. Acoustic-labial speaker verification. Pattern Recognition Lett. 18 9 (1997) 853-858
14. L. Mok, W. Lau, S. Leung, S. Wang, H. Yan, Lip features selection with application to person authentication, Proceedings of the International Conference on Acoustics, Speech and Signal Processing 2004 (ICASSP '04), vol. III, 2004, pp. 397-400.
15. M. Civanlar, T. Chen, Password-free network security through joint use of audio and video, Proc. SPIE Photonic (1996) 120-125.
16. Rabiner L., and Juang B.-H. Fundamentals of Speech Recognition (1993), Prentice-Hall, Englewood Cliffs, NJ
17. Dupont S., and Luettin J. Audio-visual speech modeling for continuous speech recognition. IEEE Trans. Multimedia 2 3 (2000) 141-151
18. C. Bregler, Y. Konig, Eigenlips for robust speech recognition, Proceedings of IEEE Conference on Acoustics, Speech and Signal Processing, 1994, pp. 669-672.
19. H. Cetingul, Y. Yemez, E. Erzin, A. Tekalp, Discriminative lip-motion features for biometric speaker identification, Proceedings of the International Conference on Image Processing 2004 (ICIP 2004), 2004, pp. 2023-2026.
20. S. Lucey, S. Sridharan, V. Chandran, Initialised eigenlip estimator for fast lip tracking using linear regression, Proceedings of the 15th International Conference on Pattern Recognition 2000, vol. 3, 2000, pp. 178-181.
21. S. Wang, W. Lau, S. Leung, H. Yan, A real-time automatic lipreading system, Proceedings of the 2004 International Symposium on Circuits and Systems (ISCAS 2004), vol. 2, 2004, pp. 101-104.
22. T. Wakasugi, M. Nishiura, K. Fukui, Robust lip contour extraction using separability of multi-dimensional distributions, Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition (FGR'04), 2004, pp. 415-420.
23. P. Aleksic, J. Williams, Z. Wu, A. Katsaggelos, Audio-visual speech recognition using MPEG-4 compliant visual features, EURASIP J. Appl. Signal Process. (2002) 1213-1227.
24. Eveno N., Caplier A., and Coulon P.-Y. Accurate and quasi-automatic lip tracking. IEEE Trans. Circuits Systems Video Technol. 14 5 (2004) 706-715
25. Sanderson C., and Paliwal K. Noise compensation in a person verification system using face and multiple speech features. Pattern Recognition 36 2 (2003) 293-302
26. Brunelli R., and Falavigna D. Person identification using multiple clues. IEEE Trans. Pattern Anal. Machine Intell. 17 (1995) 955-966
27. U. Chaudhari, G. Ramaswamy, G. Potamianos, C. Neti, Information fusion and decision cascading for audio-visual speaker recognition based on time-varying stream reliability prediction, Proceedings of the International Conference on Multimedia & Expo 2003 (ICME2003), vol. 3, 2003, pp. 9-12.
28. Zhang D. Automated Biometrics (2000), Kluwer Academic Publishers, Dordrecht
29. Huang X., Acero A., and Hon H.-W. Spoken Language Processing: A Guide to Theory, Algorithm, and System Development (2001), Prentice-Hall, Englewood Cliffs, NJ
30. Odobez J.-M., and Bouthemy P. Robust multiresolution estimation of parametric motion models. J. Visual Comm. Image Representation 6 4 (1995) 348-365
31. H. Cetingul, Y. Yemez, E. Erzin, A. Tekalp, Robust lip-motion features for speaker identification, Proceedings of the International Conference on Acoustics, Speech and Signal Processing 2005 (ICASSP '05), vol. I, 2005, pp. 509-512.
32. Puri A., Chen X., and Luthra A. Video coding using the H.264/MPEG-4 AVC compression standard. Signal Processing: Image Communications 19 (2004) 793-849
33. Reynolds D. Speaker identification and verification using gaussian mixture speaker models. Speech Comm. 17 (1995) 91-108
34. Erzin E., Yemez Y., and Tekalp A. Joint audio-video processing for robust biometric speaker identification in car. DSP in Mobile and Vehicular Systems (2005), Springer, Berlin