Robust biometric person identification using automatic classifier fusion of speech, mouth, and face experts

IEEE Transactions on Multimedia

Volumn 9, Issue 4, 2007, Pages 701-713

  Fox, Niall A ^a,b   Gross, Ralph ^a,c   Cohn, Jeffrey F ^a,d   Reilly, Richard B ^a,b  

^a IEEE   (Ireland)

^b UNIVERSITY COLLEGE DUBLIN   (Ireland)

^c CARNEGIE MELLON UNIVERSITY   (United States)

^d UNIVERSITY OF PITTSBURGH   (United States)

Author keywords

Biometric fusion; Expert reliability; Hidden Markov models; Image information loss; Mouth features; Multi modal; Person recognition; Robustness; Tri expert

Indexed keywords

BIOMETRIC FUSION; EXPERT RELIABILITY; PERSON IDENTIFICATION; PERSON RECOGNITION; TRI-EXPERT;

BIOMETRICS; CLASSIFICATION (OF INFORMATION); DATABASE SYSTEMS; FEATURE EXTRACTION; HIDDEN MARKOV MODELS; PATTERN MATCHING; ROBUSTNESS (CONTROL SYSTEMS);

FACE RECOGNITION;

EID: 34249312658     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2007.893339     Document Type: Article

References (52)

1. N. A. Fox and R. B. Reilly, "Audio-visual speaker identification based on the use of dynamic audio and visual features," in Proc. 4th Int. Conf. Audio- and Video-Based Biometric Person Authentication, Guildford, U.K., Jun. 2003, pp. 743-751.
2. C. Sanderson and K. K. Paliwal, "Identity verification using speech and face information," Dig. Signal Process., vol. 14, no. 5, pp. 449-480, Sep. 2004.
3. T. J. Wark, S. Sridharan, and V. Chandran, "The use of speech and lip modalities for robust speaker verification under adverse conditions," in Proc. IEEE Int. Conf. Multimedia Computing and Systems, Florence, Italy, Jun. 1999, vol. 1, pp. 812-816.
4. C. Sanderson, S. Bengio, and Y. Gao, "On transforming statistical models for non-frontal face verification," Pattern Recognit., vol. 39, no. 2, pp. 288-302, 2006.
5. D. Blackburn, M. Bone, and P. J. Phillips, Facial Recognition Vendor Test 2000: Evaluation Report Feb. 2001 [Online]. Available: http://www.frvt.org, Tech. Rep.
6. R. Gross, J. Shi, and J. F. Cohn, "Quo vadis face recognition," in Proc. 3rd Workshop on Empirical Evaluation Methods in Computer Vision, Kauai, HI, Dec. 2001, pp. 119-132.
7. N. A. Fox, R. Gross, P. de Chazal, J. F. Cohn, and R. B. Reilly, "Person identification using automatic integration of speech, lip, and face experts," in ACM SIGMM Workshop on Biometrics Methods and Applications, Berkley, CA, Nov. 2003, pp. 25-32.
8. G. Potamianos, C. Neti, G. Gravier, A. Garg, and A. W. Senior, "Recent advances in the automatic recognition of audiovisual speech," Proc. IEEE, vol. 91, no. 9, pp. 1306-1324, Sep. 2003.
9. C. C. Chibelushi, F. Deravi, and J. S. D. Mason, "A review of speech-based bimodal recognition," IEEE Trans. Multimedia, vol. 4, no. 1, pp. 23-35, Mar. 2002.
10. U. Dieckmann, P. Plankensteiner, and T. Wagner, "SESAM: A biometric person identification system using sensor fusion," Pattern Recognit. Lett., vol. 18, no. 9, pp. 827-833, Sept. 1997.
11. Y. Yemez, A. Kanak, E. Erzin, and A. M. Tekalp, "Multimodal speaker identification with audio-video processing," in Proc. Int. Conf. Image Processing, Barcelona, Spain, Sep. 2003, vol. 3, pp. 5-8.
12. S. Ben-Yacoub, Y. Abdeljaoued, and E. Mayoraz, "Fusion of face and speech data for person identity verification," IEEE Trans. Neural Netw., vol. 10, no. 5, pp. 1065-1074, May 1999.
13. T. Wark and S. Sridharan, "Adaptive fusion of speech and lip information for robust speaker identification," Dig. Sig. Process., vol. 11, no. 3, pp. 169-186, July 2001.
14. A. Kanak, E. Erzin, Y. Yemez, and A. M. Tekalp, "Joint audio-video processing for biometric speaker identification," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing, Hong Kong, Apr. 2003, vol. 2, pp. 377-380.
15. R. Brunelli and D. Falavigna, "Person identification using multiple cues," IEEE Trans. Pattern Anal. Mach. Intell., vol. 17, no. 10, pp. 955-966, Oct. 1995.
16. C. C. Chibelushi, F. Deravi, and J. S. D. Mason, "Adaptive classifier integration for robust pattern recognition," IEEE Trans. Syst., Man, Cybern. B; Cybern., vol. 29, pp. 902-907, Dec. 1999.
17. A. K. Jain and A. Ross, "Learning user-specific parameters in a multi-biometric system," in Proc. Int. Conf. Image Processing., Rochester, NY, Sep. 22-25, 2002, vol. 1, pp. 57-60.
18. D. A. Reynolds and R. C. Rose, "Robust text-independent speaker identification using gaussian mixture speaker models," IEEE Trans. Speech Audio Process., vol. 3, no. 1, pp. 72-83, Jan. 1995.
19. S. Young, G. Evermann, D. Kershaw, G. Moore, J. Odell, D. Ollason, V. Valtchev, and P. Woodland, The HTK Book (for HTK Version 3.1). Cambridge, U.K.: Cambridge Univ. Eng. Dept.: Microsoft Corporation, 2001.
20. S. Lucey, T. Chen, S. Sridharan, and V. Chandran, "Integration strategies for audio-visual speech processing: Applied to text dependent speaker recognition," IEEE Trans. Multimedia, vol. 7, no. 3, pp. 495-506, Jun. 2005.
21. L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, no. 2, pp. 257-286, Feb. 1989.
22. I. Matthews, T. F. Cootes, J. A. Bangham, J. A. Cox, and R. Harvey, "Extraction of visual features for lipreading," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 2, pp. 198-213, Feb. 2002.
23. G. Potamianos, H. Graf, and E. Cosatto, "An image transform approach for HMM based automatic lipreading," in Proc. IEEE Int. Conf. Image Processing, Chicago, IL, Oct. 1998, vol. 3, pp. 173-177.
24. I. Matthews, G. Potamianos, C. Neti, and J. Luettin, "A comparison of model and transform-based visual features for audio-visual LVCSR," in Proc. IEEE Int. Conf. Multimedia and Expo, Tokyo, Japan, Aug. 2001, pp. 825-828.
25. A. N. Netravali and B. G. Haskell, Digital Pictures. New York: Plenum, 1988.
26. V. Vapnik, The Nature of Statistical Learning Theory. New York: Springer Verlag, 1995.
27. Y. Li, S. Gong, and H. Liddell, "Support vector regression and classification based multi-view face detection and recognition," in Proc. 4th IEEE Int. Conf. Automatic Face and Gesture Recognition, Grenoble, France, Mar. 2000, pp. 300-305.
28. P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman, "Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 711-720, Jul. 1997.
29. L. Sirovich and M. Kirby, "Low-dimensional procedure for the characterization of human faces," J. Opt. Soc. Amer. A, vol. 4, no. 3, pp. 519-524, 1987.
30. M. Turk and A. Pentland, "Eigenfaces for recognition," J. Cog. Neurosci., vol. 3, no. 1, pp. 71-86, 1991.
31. S. Lawrence, C. L. Giles, A. C. Tsoi, and A. D. Back, "Face recognition: A convolutional neural-network approach," IEEE Trans. Neural Netw., vol. 8, no. 1, pp. 98-113, 1997.
32. W. Zhao, R. Chellappa, P. Phillips, and A. Rosenfeld, "Face recognition: A literature survey," ACM Comput. Surv., vol. 35, no. 4, pp. 399-458, 2003.
33. A. Lanitis, C. J. Taylor, and T. F. Cootes, "Automatic interpretation and coding of face images using flexible models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 743-756, Jun. 1997.
34. A. Yuille, "Deformable templates for face recognition," J. Cog. Neurosci., vol. 3, no. 1, pp. 59-70, 1991.
35. L. Wiskott, J.-M. Fellous, N. Kruger, and C. von der Malsburg, "Face recognition by elastic bunch graph matching," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 775-779, Jul. 1997.
36. A. V. Nefian, L. H. Liang, T. Fu, and X. X. Liu, "A bayesian approach to audio-visual speaker identification," in Proc. 4th Int. Conf. Audio-and Video-Based Biometric Person Authentication, Guildford, U.K., Jun. 2003, pp. 761-769.
37. F. Cardinaux, C. Sanderson, and S. Bengio, "Face verification using adapted generative models," in Proc. 6th IEEE Int. Conf. Automatic Face and Gesture Recognition (AFGR), Seoul, 2004, pp. 825-830.
38. P. Penev and J. Atick, "Local feature analysis: A general statistical theory for object representation," Network: Comput. in Neural Syst., vol. 7, no. 3, pp. 477-500, 1996.
39. C. BenAbdelkader and P. Griffin, "Comparing and combining depth and texture cues for face recognition," Image Vis. Comput., vol. 23, pp. 339-352, 2005.
40. P. J. Phillips, P. Grother, P. Michaels, D. Blackburn, E. Tabassi, and M. Bone, "Face recognition vendor test 2002," in Proc. IEEE Int. Workshop on Analysis and Modeling of Faces and Gestures, (AMFG), Nice, France, Oct. 2003, p. 44.
41. J. Kittler, M. Hatef, R. P. W. Duin, and J. Matas, "On combining classifiers," IEEE Trans. Pattern Anal Mach. Intell., vol. 20, no. 3, pp. 226-239, Mar. 1998.
42. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
43. S. Dupont and J. Luettin, "Audio-visual speech modeling for continuous speech recognition," IEEE Trans. Multimedia, vol. 2, no. 3, pp. 141-151, Sep. 2000.
44. S. Tamura, K. Iwano, and S. Furui, "A stream-weight optimization method for audio-visual speech recognition using multi-stream HMMs," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing, Montreal, Canada, May 2004, vol. 1, pp. 857-860.
45. N. A. Fox and R. B. Reilly, "Robust multi-modal person identification with tolerance of facial expression," in Proc. IEEE Int. Conf. Systems, Man and Cybernetics, The Hague, The Netherlands, Oct. 10-13, 2004, vol. 1, pp. 580-585.
46. R. W. Frischholz and U. Dieckmann, "BiolD: A multimodal biometric identification system," Computer, vol. 33, no. 2, pp. 64-68, 2000.
47. A. Jain, K. Nandakumar, and A. Ross, "Score normalization in multi-modal biometric systems," Pattern Recognit., 2005.
48. M. Heckmann, F. Berthommier, and K. Kroschel, "Noise adaptive stream weighting in audio-visual speech recognition," EURASIP J. Appl. Signal Process., vol. 2002, no. 11, pp. 1260-1273, Nov. 2002.
49. J. Fierrez-Aguilar, Y. Chen, J. Ortega-Garcia, and A. Jain, "Incorporating image quality in multi-algorithm fingerprint verification," in Proc. Int. Conf. Biometrics (ICB), Hong Kong, China, 2006, pp. 213-220.
50. K. Messer, J. Kittler, J. Luettin, and G. Maitre, "XM2VTSDB: "The extended M2VTS database"," in Proc. 2nd Int. Conf. Audio and Video-based Biometric Person Authentication, Washington, DC, Mar. 1999, pp. 72-77.
51. A. P. Varga, H. J. M. Steeneken, M. Tomlinson, and D. Jones, The Noisex-92 Study on the Effect of Additive Noise on Automatic Speech Recognition Speech Res. Unit, Defence Research Agency, Malvern, U.K., 1992, Tech. Rep..
52. T. Wark, S. Sridharan, and V. Chandran, "An approach to statistical lip modelling for speaker identification via chromatic feature extraction," in Proc. 14th Int. Conf. Pattern Recognition, Brisbane, Australia, Aug. 1998, vol. 1, pp. 123-125.