Multiscale fusion of visible and thermal IR images for illumination- invariant face recognition

International Journal of Computer Vision

Volumn 71, Issue 2, 2007, Pages 215-233

  Kong, Seong G ^a   Heo, Jingu ^b   Boughorbel, Faysal ^a   Zheng, Yue ^a   Abidi, Besma R ^a   Koschan, Andreas ^a   Yi, Mingzhong ^a   Abidi, Mongi A ^a  

^a University of Tennessee Knoxville   (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Eyeglass replacement; Face recognition; Multisensor image registration; Personal identification; Security; Thermal infrared imaging; Visible thermal image fusion

Indexed keywords

EYEGLASS REPLACEMENT; MULTISENSOR IMAGE REGISTRATION; PERSONAL IDENTIFICATION; SECURITY; THERMAL INFRARED IMAGING; VISIBLE-THERMAL IMAGE FUSION;

COMPUTER SOFTWARE; CURVE FITTING; FACE RECOGNITION; INFRARED IMAGING; SENSOR DATA FUSION; SPECTRUM ANALYSIS;

THERMOGRAPHY (IMAGING);

EID: 33748483465     PISSN: 09205691     EISSN: 15731405     Source Type: Journal    
DOI: 10.1007/s11263-006-6655-0     Document Type: Article

References (61)

1. Adini, Y., Moses, Y., and Ullman, S. 1997. Face recognition: The problem of compensating for changes in illumination direction. IEEE Trans. Pattern Analysis and Machine Intelligence, 19(7):721-732.
2. Ben-Yacoub, S., Abdeljaoued, Y, and Mayoraz, E. 1999. Fusion of face and speech data for person identity verification. IEEE Trans. on Neural Networks, 10(5): 1065-1074.
3. Besl, P.J. and McKay, N.D. 1992. A method for registration of 3-D shapes. IEEE Trans. Pattern Analysis and Machine Intelligence, 14(2):239-256.
4. Blackburn, D.M., Bone, J.M., and Phillips, P.J. 2001. Face recognition vendor test 2000. Evaluation Report, National Institute of Standards and Technology, pp. 1-70.
5. Bone, M. and Blackburn, D. 2002. Face recognition at a chokepoint: Scenario evaluation results. Evaluation Report, Department of Defense.
6. Bookstein, F.L. 1979. Fitting conic sections to scattered data. Computer Graphics and Image Processing, 9(1):56-71.
7. Boughorbel, F., Koschan, A., Abidi, B., and Abidi, M. 2004a. Gaussian fields: A new criterion for 3D rigid registration. Pattern Recognition, 37(7):1567-1571.
8. Boughorbel, F., Koschan, A., Abidi, B., and Abidi, M. 2004b. Gaussian energy functions for registration without correspondences. In Proc. of 17th Int'l Conf. on Pattern Recognition, pp. 24-27 Cambridge, UK.
9. Brunelli R. and Poggio, T. 1993. Face recognition: Features versus templates. IEEE Trans. Pattern Analysis and Machine Intelligence, 15(10): 1042-1052.
10. Brunelli, R. and Falavigna D. 1995. Personal identification using multiple cues. IEEE Trans. Pattern Analysis and Machine Intelligence, 17(10):955-966.
11. Burton, A.M., Bruce, V., and Hancock, P.J.B. 1999. From pixels to people: A model of familiar face recognition. Cognitive. Science, 23(1):1-31.
12. Chang, K., Bowyer, K.W., Sarkar, S., and Victor, B. 2003. Comparison and combination of ear and face image in appearance-based biometrics. IEEE Trans. Pattern Analysis and Machine. Intelligence, 25(9): 1160-1165.
13. Chellappa, R., Wilson, C.L., and Sirohey, S. 1995. Human and machine recognition of faces: A survey. In Proceedings of the IEEE, Vol. 83, no. 5, pp. 705-740.
14. Cox, I.J., Ghosn, J., and Yianilos, P.N. 1996. Feature-Based Face Recognition Using Mixture-Distance. In Proc. Int'l Conf. Computer Vision and Pattern Recognition, pp. 209-216.
15. Charpiat, G., Faugeras, O., and Keriven, R. 2003. Shape Metrics, Warping and Statistics. In Proc. Int'l Conf. on Image Processing, Barcelona, Vol. 2, pp. 627-630.
16. Chen, X., Flynn, P. and Bowyer, K. 2003. Visible-light and infrared face recognition. In Proc, of Workshop on Multimodal User Authentication, pp. 48-55.
17. Craw, L, Costen, N., Kato, T., and Akamatsu, S. 1999. How should we represent faces for automatic recognition'?. IEEE Trans. Pattern Analysis and Machine Intelligence, 21(8):725-736.
18. Dalley, G. and Flynn, P. 2002. Pair-wise range image registration: A study in outlier classification. Computer Vision and Image Understanding, 87(1-3):104-115.
19. Dasarathy, B.V. 1994. Decision Fusion, Washington. DC: IEEE Computer Society Press.
20. Daubechies, I. 1992. Ten lectures on wavelets. CBMS-NSF Lecture Notes, No. 61, Society for Industrial and Applied Mathematics.
21. Elgammal, A., Duraiswami, R., and Davis, L. 2003. Efficient kernel density estimation using the fast gauss transform with applications to color modeling and tracking. IEEE Trans. Pattern Analysis and Machine Intelligence, 25(11):1499-1504.
22. Fang, Y., Tan, T., and Wang, Y. 2002. Fusion of global and local features for face verification. In Proc. of Int'l Conf. on Pattern Recognition, pp. 382-385.
23. Fitzgibbon, A.W. 2003. Robust registration of 2D and 3D Point Sets. Image and Vision Computing, 21(13):1145-1153.
24. Fitzgibbon, A., Pilu, M., and Fisher, R.B. 1999. Direct least square fitting of ellipses. IEEE Trans. on Pattern Analysis and Machine Intelligence, 21(5):476-480.
25. Freeman, H. and Davis, L.S. 1977. A corner-finding algorithm for chain-coded curves. IEEE Trans. on Computer, 26(3):297-303.
26. Greengard, L. 1988. The Rapid Evaluation of Potential Fields in Particle. Systems. Cambridge, MA: MIT Press.
27. Greengard, L. and Strain, J. 1991. The fast gauss transform. SIAM Journal on Scientific Computing, 12:79-94.
28. Gutta, S., Huang, J.R.J., Jonathon, P., and Wechsler, H. 2000. Mixture of experts for classification of gender, ethnic origin, and pose of human faces. IEEE Trans. Neural Networks, 11(4):948-960.
29. Hall, D.L. and Llinas, J. 2001. Handbook of Multisensor Data Fusion. CRC Press.
30. Heo, J., Abidi, B., Kong, S.G., and Abidi, M. 2003. Performance comparison of visual and thermal signatures for face recognition. Biometric Consortium Conference, Arlington, VA.
31. Ho, T.K., Hull, J.J., and Srihari, S.N. 1994. Decision combination in multiple classifier systems. IEEE Trans. on Pattern Analysis and Machine Intelligence, 16(1):66-75.
32. Hong, L. and Jain, A. 1998. Integrating faces and fingerprints for personal identification. IEEE Trans. Pattern Analysis and Machine Intelligence, 20(12): 1295-1307.
33. http://www.cse.ohio-state.edu/otcbvs-bench/
34. http://www.equinoxsensors.com/products/HID.html
35. Huttenlocher, D.P., Klanderman, G.A., and Rucklidge, W.J. 1993. Comparing images using the Hausdorff distance. IEEE Trans. Pattern Analysis and Machine Intelligence, 15(9):850-863.
36. Irani, M. and Anandan, P. 1998. Robust multi-sensor image alignment. In Proc, of the 6th Int'l Conf on Computer Vision, pp. 959-965.
37. Jain, A.K., Ross, A., and Prabhakar, S. 2004. An introduction to biometric recognition. IEEE Trans. Circuits and Systems for Video Technology, 14(1):4-20.
38. Kanade, T. 1973. Picture processing by computer complex and recognition of human faces. Technical Report, Kyoto University.
39. Kong, S.G., Heo, J., Abidi, B.R., Paik, J., and Abidi, M.A. 2005. Recent advances in visual and infrared face recognition-A review. Computer Vision and Image. Understanding, 97(1):103-135.
40. Manjunath, B.S., Chellappa, R., and von der Malsburg, C. 1992. A feature based approach to face recognition. In Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 373-378.
41. Murio, D.A. 1993. The Mollification Method and the Numerical Solution of Ill-Posed Pmblems. New York, NY: John Wiley & Sons.
42. Pavlidis, I. and Symosek, P. 2000. The imaging issue in an automatic face/disguise detection system. In Proc. IEEE Workshop on Computer Vision Beyond the Visible Spectrum: Methods and Applications, pp. 15-24.
43. Penev, P.S. 1998. Local feature analysis: A statistical theory for information representation and transmission. Ph.D. Thesis, The Rockefeller University.
44. Penev, P.S. 1999. Dimensionality reduction by sparsification in a local-features representation of human faces. Technical Report, The Rockefeller University.
45. Phillips, P.J., Grother, P., Micheals, R.J., Blackburn, D.M., Tabassi, E., and Bone, M. 2003. Face recognition vendor test 2002. Evaluation Report, National Institute of Standards and Technology, pp. 1-56.
46. Phillips, P.J., Moon, H., Rizvi, S.A., and Rauss, P.J. 2000. The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Analysis and Machine Intelligence, 22(10): 1090-1104.
47. Press, W.H., Teukolsky, S.A., Vetterling, W.T., and Flannery, B.P. 1992. Numerical Recipes in C: The Art of Scientific Computing. Second edition. Cambridge University Press.
48. Prokoski, F. 2000. History, current status, and future of infrared identification. In Proc. IEEE Workshop on Computer Vision Beyond the Visible. Spectrum: Methods and Applications, pp. 5-14.
49. Ross, A. and Jain, A. 2003. Information fusion in biometrics. Pattern Recognition. Letters, 24(13):2115-2125.
50. Selinger, A. and Socolinsky, D.A. 2001. Appearance-based facial recognition using visible and thermal imagery: A comparative study. Technical Report 02-01, Equinox Corporation.
51. Sharp, G.C., Lee, S.W., and Wehe, D.K. 2002. TCP Registration using invariant features. IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 24(1):90-102.
52. Singh, S., Gyaourova, A., Bebis, G., and Pavlidis, I. 2004. Infrared and visible image fusion for face recognition. In Proc. SPIE Defense and Security Symposium (Biometric Technology for Human Identification), pp. 585-596.
53. Socolinsky, D.A., Selinger, A., and Neuheisel, J.D. 2003. Face recognition with visible and thermal infrared imagery. Computer Vision and Image Understanding, 91(1-2):72-114.
54. Snelick, R., Uludag, U., Mink, A., Indovina, M., and Jain, A. 2005. Large-scale evaluation of multimodal biometric authentication using state-of-the-art systems. IEEE Trans. Pattern Analysis and Machine Intelligence, 27(3):450-455.
55. Störring, M., Andersen, H.J., and Granum, E. 2001. Physics-based modelling of human skin colour under mixed illuminants. Journal of Robotics and Autonomous Systems, 35(3/4):131-142.
56. Turk, M. and Pentland, A. 1991. Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1):72-86.
57. Wilder, J., Phillips, P.J., Jiang, C. and Wiener, S. 1996. Comparison of visible and infrared imagery for face recognition. In Proc. Int. Conf. Automatic Face and Gesture Recognition, pp. 182-187.
58. Wolff, L.B., Socolinsky, D.A., and Eveland, C.K. 2001. Quantitative measurement of illumination invariance for face recognition using thermal infrared imagery. In Proc. IEEE Workshop on Computer Vision Beyond the Visible Spectrum,
59. Yang, C., Duraiswami, R., Gumerov, N.A., and Davis, L. 2003. Improved fast Gauss transform and efficient kernel density estimation. In Proc. 9th. Int'l Conf. on Computer Vision, Nice, France, pp. 464-471.
60. Yoshitomi, Y., Miyaura, T., Tomita, S., and Kimura, S. 1997. Face identification using thermal image processing. In Proc. IEEE Int'l Workshop on Robot and Human Communication, pp. 374-379.
61. Zitova, B. and Flusser, J. 2003. Image registration methods: a survey. Image and Vision Computing, 21(11):977-1000.