Audio-visual person authentication using lip-motion from orientation maps

Pattern Recognition Letters

Volumn 28, Issue 11, 2007, Pages 1368-1382

  Faraj, Maycel Isaac ^a   Bigun, Josef ^a  

^a HALMSTAD UNIVERSITY   (Sweden)

Author keywords

Audio visual recognition; Biometric recognition; Biometrics; Gaussian Markov model; Hidden Markov model; Lip movements; Motion; Optical flow; Orientation; Person identification; Speaker authentication; Speaker verification; Structure tensor

Indexed keywords

AUDIO ACOUSTICS; BIOMETRICS; COMPUTATIONAL EFFICIENCY; DATABASE SYSTEMS; HIDDEN MARKOV MODELS; OPTICAL FLOWS;

AUDIOVISUAL RECOGNITION; BIOMETRIC RECOGNITION; GAUSSIAN MARKOV MODELS; LIP-MOVEMENTS; PERSON IDENTIFICATION; SPEAKER AUTHENTICATION; SPEAKER VERIFICATION; STRUCTURE TENSORS;

SPEECH RECOGNITION;

EID: 34249752774     PISSN: 01678655     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patrec.2007.02.017     Document Type: Article

References (39)

1. Bigun J. Vision with Direction (2006), Springer, Heidelberg
2. Bigun J., and Granlund G. Optimal orientation detection of linear symmetry. First International Conference on Computer Vision, ICCV, London, June 8-11 (1987), IEEE Computer Society 433-438
3. Bigun J., Granlund G., and Wiklund J. Multidimensional orientation estimation with applications to texture analysis of optical flow. IEEE Trans. Pattern Anal. Machine Intell. 13 8 (1991) 775-790
4. Bigun, E., Bigun, J., Duc, B., Fischer, S., 1997. Expert conciliation for multi modal person authentication systems by bayesian statistics. In: Bigun, J., Chollet, G., Borgefors, G. (Eds.), Audio and Video Based Person Authentication - AVBPA97, pp. 291-300.
5. Brunelli K.R., and Falavigna D. Person identification using multiple cues. IEEE Trans. Pattern Anal. Machine Intell. 17 10 (1995) 955-966
6. Chen T. Audiovisual speech processing. IEEE Signal Process. Mag. 18 1 (2001) 9-21
7. Chibelushi C., Deravi F., and Mason J. A review of speech-based bimodal recognition. IEEE Trans. Multimedia 4 1 (2002) 23-37
8. Dieckmann, U., Plankensteiner, P., Wagner, T., 1997. Acoustic-labial speaker verification. In: Proc. First Internat. Conf. on Audio- and Video-Based Biometric Person Authentication, LNCS 1206, pp. 301-310.
9. Duc B., Fischer S., and Bigun J. Face authentication with sparse grid gabor information. IEEE Int. Conf. Acoust. Speech Signal Process. 4 21 (1997) 3053-3056
10. Dupont S., and Luettin J. Audio-visual speech modelling for continuous speech recognition. IEEE Trans. Multimedia 2 3 (2000) 141-151
11. Faraj, M.I., Bigun, J., 2006. Person verification by lip-motion. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition workshop - CVPR 2006, pp. 37-45.
12. Fasel B., and Luettin J. Automatic facial expression analysis: A survey. J. Pattern Recognition Soc. 36 1 (2003) 259-275
13. Frischholz, R., Dieckmann, U., 2000. Bioid: A multimodal biometric identification system. IEEE-Computer Society Press, vol. 33(2), 2000, pp. 64-68.
14. Furui, S., 1997. Recent advances in speaker recognition. In: Proc. First Internat. Conf. on Audio- and Video-Based Biometric Person Authentication, LNCS 1206, pp. 237-252.
15. Granlund G.H. In search of a general picture processing operator. Computer Graphics Image Process. 8 2 (1978) 155-173
16. Hazen T.J. Visual model structures and synchrony constraints for audio-visual speech recognition. IEEE Trans. Audio Speech Lang. Process. 14 3 (2006) 1082-1089
17. Horn B., and Schunck B. Determining optical flow. J. Art. Intell. 17 1 (1981) 185-203
18. Jourlin, P., Luettin, J., Genoud, D., Wassner, H., 1997. Acoustic-labial speaker verification. In: Proc. First Internat. Conf. on Audio- and Video-Based Biometric Person Authentication, LNCS 1206, pp. 319-326.
19. Kollreider, K., Fronthaler, H., Bigun, J., 2005. Evaluating liveness by face images and the structure tensor. In: AutoID 2005: Fourth Workshop on Automatic Identification Advanced Technologies. IEEE Computer Society, pp. 75-80.
20. Lucas B.D., and Kanade T. An iterative image registration technique with an application to stereo vision. Int. Joint Conf. Art. Intell. (1981) 674-679
21. Lucey S., Chen T., Sridharan S., and Chandran V. Integration strategies for audiovisual speech processing: Applied to text-dependent speaker recognition. IEEE Trans. Multimedia 7 3 (2005) 495-506
22. Luettin, J., Maitre, G., 1998. Evaluation protocol for the extended m2vts database xm2vtsdb. In: IDIAP Communication 98-054, Technical report R R-21, number = IDIAP - 1998.
23. Luettin, J., Thacker, N., Beet, S., 1996. Speaker identification by lipreading. In: Proc. 4th Internat. Conf. on Spoken Language Processing ICSLP'96, pp. 62-65.
24. Mase K., and Pentland A. Automatic lip-reading by optical-flow analysis. Systems Comput. Jpn. 22 6 (1991) 67-76
25. Messer, K., Matas, J., Kittler, J., Luettin, J., 1999. Xm2vtsdb: The extended m2vts database. In: Second International Conference of Audio and Video-Based Biometric Person Authentication ICSLP'96, pp. 72-77.
26. Pal N., and Bezdek J. On cluster validity for the fuzzy c-means model. IEEE Trans. Fuzzy Systems 3 3 (1995) 370-379
27. Potamianos G., Neti C., Gravier G., Garg A., and Senior A. Recent advances in the automatic recognition of audiovisual speech. Proc. IEEE 91 9 (2003) 1306-1326
28. Reynolds D., and Rose R. Robust text-independent speaker identification using gaussian mixture models. IEEE Trans. Speech Audio Process. ICASSP090 3 1 (1995) 72-83
29. Sanchez, M.R., Matas, J., Kittler, J., 1997. Statistical chromaticity-based lip tracking with b-splines. In: Proc. First Internat. Conf. on Audio- and Video-Based Biometric Person Authentication, LNCS 1206, pp. 69-76.
30. Smeraldi F., and Bigun J. Retinal vision applied to facial features detection and face authentication. Pattern Recognition Lett. 23 (2002) 463-475
31. Stover J., Hall D., and Gibson R. A fuzzy-logic architecture for autonomous multisensor data fusion. IEEE Trans. Industr. Electron. 43 3 (1996) 403-410
32. Tamura S., Iwano K., and Furui S. Multi-modal speech recognition using optical flow analysis for lip images. J. VLSI Signal Process. 36 2 (2004) 117-124
33. Tang, X., Li, X., 2001. Fusion of audio-visual information integrated speech processing. In: Third Internat. Conf. on Audio- and Video-Based Biometric Person Authentication AV BPA02001, LNCS 2091, pp. 127-143.
34. Tang, X., Li, X., 2004. Video based face recognition using multiple classifiers. In: Sixth IEEE Internat. Conf. on Automatic Face and Gesture Recognition FGR2004 - IEEE Computer Society, pp. 345-349.
35. Varshney P. Multisensor data fusion. Electron. Commun. Eng. J. 9 6 (1997) 245-253
36. Veeravalli, A.G., Pan, W., Adhami, R., Cox, P.G., 2005. A tutorial on using hidden markov models for phoneme recognition. In: Proc. Thirty-Seventh Southeastern Symp. on System Theory, SSST 2005.
37. Wark T., Sridharan S., and Chandran V. The use of speech and lip modalities for robust speaker verification under adverse conditions. IEEE Int. Conf. Multimedia Comput. Systems 1 (1999)
38. Yamamoto E., Nakamura S., and Shikano K. Lip movement synthesis from speech based on hidden markov models. J. Speech Commun. 26 1 (1998) 105-115
39. Young, S., Kershaw, D., Odell, J., Ollason, D., Valtchev, V., Woodland, P., 2000. The htk book (for htk version 3.0) http://htk.eng.cam.ac.uk/docs/docs.shtml.