3D-2D face recognition with pose and illumination normalization

Computer Vision and Image Understanding

Volumn 154, Issue , 2017, Pages 137-151

  Kakadiaris, Ioannis A ^a,c   Toderici, George ^a   Evangelopoulos, Georgios ^a   Passalis, Georgios ^a,b   Chu, Dat ^a   Zhao, Xi ^a,d   Shah, Shishir K ^a   Theoharis, Theoharis ^a,b  

^a Univ of Houston ^*  (United States)

^b UNIVERSITY OF ATHENS   (Greece)

^c UNIVERSITY OF HOUSTON   (United States)

^d XI'AN JIAOTONG UNIVERSITY   (China)

Author keywords

3D 2D face recognition; 3D 2D model fitting; Biometrics; Computer vision; Face and gesture recognition; Illumination normalization; Model based face recognition; Object recognition; Physically based modeling

Indexed keywords

BIOMETRICS; COMPUTER VISION; GESTURE RECOGNITION; LARGE DATASET; OBJECT RECOGNITION; PROBES; TEXTURES;

FACE AND GESTURE RECOGNITION; ILLUMINATION NORMALIZATION; MODEL FITTING; MODEL-BASED OPC; PHYSICALLY BASED MODELING;

FACE RECOGNITION;

EID: 84979671321     PISSN: 10773142     EISSN: 1090235X     Source Type: Journal    
DOI: 10.1016/j.cviu.2016.04.012     Document Type: Article

References (60)

1. Abate, A., Nappi, M., Riccio, D., Sabatino, G., 2D and 3D face recognition: a survey. Pattern Recognit. Lett. 28:14 (2007), 1885–1906.
2. Al-Osaimi, F.R., Bennamoun, M., Mian, A.S., Illumination normalization of facial images by reversing the process of image formation. Mach. Vis. Appl. 22:6 (2011), 899–911.
3. Atick, J.J., Griffin, P.A., Redlich, A.N., Statistical approach to Shape from Shading: reconstruction of three-dimensional face surfaces from single two-dimensional images. Neural Comput. 8:6 (1996), 1321–1340, 10.1162/neco.1996.8.6.1321.
4. Biswas, S., Aggarwal, G., Chellappa, R., Robust estimation of albedo for illumination-invariant matching and shape recovery. IEEE Trans. Pattern Anal. Mach. Intell. 31 (2009), 884–899, 10.1109/TPAMI.2008.135.
5. Blanz, V., Vetter, T., Face recognition based on fitting a 3D morphable model. IEEE Trans. Pattern Anal. Mach. Intell. 25:9 (2003), 1063–1074.
6. Bowyer, K., Chang, K., Flynn, P., A survey of approaches and challenges in 3D and multi-modal 3D+2D face recognition. Comput. Vis. Image Understand. 101:1 (2006), 1–15.
7. Bregler, C., Hertzmann, A., Biermann, H., Recovering non-rigid 3D shape from image streams. Proceedings IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Hilton Head, SC, vol. 2, 2000, 690–696.
8. Efraty, B., Huang, C., Shah, S.K., Kakadiaris, I.A., Facial landmark detection in uncontrolled conditions. Proceedings International Joint Conference on Biometrics. Washington, DC, 2011.
9. Fang, T., Zhao, X., Ocegueda, O., Shah, S.K., Kakadiaris, I.A., 3D/4D facial expression analysis: an advanced annotated face model approach. Image Vis. Comput. 30:10 (2012), 738–749.
10. Georghiades, A., Belhumeur, P., Kriegman, D., From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Analys. Mach. Intell. 23:6 (2001), 643–660, 10.1109/34.927464.
11. Gu, L., Kanade, T., 3D alignment of face in a single image. Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York, NY, 2006, 1305–1312, 10.1109/CVPR.2006.11.
12. Hartley, R.I., Zisserman, A., Multiple View Geometry in Computer Vision. second, 2004, Cambridge University Press ISBN: 0521540518.
13. Huang, D., Ardabilian, M., Wang, Y., Chen, L., Automatic asymmetric 3D-2D face recognition. Proceedings 20th International Conference on Pattern Recognition. Istanbul, Turkey, 2010, 1225–1228.
14. Huang, D., Shan, C., Ardabilian, M., Wang, Y., Chen, L., Local binary patterns and its applications on facial image: a survey. IEEE Trans. Syst. Man Cybern. Part C 41:6 (2011), 765–781.
15. Jahanbin, S., Choi, H., Bovik, A., Passive multimodal 2-D+3-D face recognition using Gabor features and landmark distances. IEEE Trans. Inf. Forensics Security 6:4 (2011), 1287–1304, 10.1109/TIFS.2011.2162585.
16. Jain, A., Nandakumar, K., Ross, A., Score normalization in multimodal biometric systems. Pattern Recognit. 38:12 (2005), 2270–2285, 10.1016/j.patcog.2005.01.012.
17. Kakadiaris, I.A., Passalis, G., Toderici, G., Murtuza, M.N., Lu, Y., Karampatziakis, N., Theoharis, T., Three-dimensional face recognition in the presence of facial expressions: an annotated deformable model approach. IEEE Trans. Pattern Anal. Mach. Intell. 29:4 (2007), 640–649, 10.1109/TPAMI.2007.1017.
18. Kazemi, V., Sullivan, J., One millisecond face alignment with an ensemble of regression trees. Proceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Columbus, OH, 2014, 1867–1874.
19. Kemelmacher-Shlizerman, I., Basri, R., 3D face reconstruction from a single image using a single reference face shape. IEEE Trans. Pattern Anal. Mach. Intell. 33:2 (2011), 394–405.
20. King, D.E., Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10 (2009), 1755–1758.
21. L1 Identity Solutions, L1 FaceIt SDK.
22. Lee, J., Moghaddam, B., Pfister, H., Machiraju, R., A bilinear illumination model for robust face recognition. Proceedings IEEE International Conference on Computer Vision, Beijing, China, 2, 2005, 1177–1184.
23. Lee, M.W., Ranganath, S., Pose-invariant face recognition using a 3D deformable model. Pattern Recognit. 36:8 (2003), 1835–1846.
24. Levine, M.D., Yua, Y., State-of-the-art of 3D facial reconstruction methods for face recognition based on a single 2D training image per person. Pattern Recognt. Lett. 30:10 (2009), 908–913.
25. Lourakis, M., 2005. Levenberg-Marquardt nonlinear least squares algorithms in C/C++. http://www.ics.forth.gr/~lourakis/levmar/.
26. Matthews, I., Xiao, J., Baker, S., 2D vs. 3D deformable face models: representational power, construction, and real-time fitting. Int. J. Comput. Vis. 75:1 (2007), 93–113.
27. Mian, A., Bennamoun, M., Owens, R., An efficient multimodal 2D-3D hybrid approach to automatic face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 29:11 (2007), 1927–1943.
28. Moeini, A., Moeini, H., Faez, K., Unrestricted pose-invariant face recognition by sparse dictionary matrix. Image Vis. Comput. 36 (2015), 9–22, 10.1016/j.imavis.2015.01.007.
29. Mpiperis, I., Malassiotis, S., Strintzis, M., Bilinear models for 3D face and facial expression recognition. IEEE Trans. Inf. Forensics Security 3:3 (2008), 498–511.
30. Ocegueda, O., Passalis, G., Theoharis, T., Shah, S., Kakadiaris, I.A., UR3D-C: linear dimensionality reduction for efficient 3D face recognition. Proceedings International Joint Conference on Biometrics. Washington D.C, 2011.
31. Ocegueda, O., Shah, S., Kakadiaris, I.A., Which parts of the face give out your identity?. Proceedings IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Colorado Springs, CO, 2011, 641–648.
32. Osadchy, M., Jacobs, D., Lindenbaum, M., Surface dependent representations for illumination insensitive image comparison. IEEE Trans. Pattern Anal. Mach. Intell. 29:1 (2007), 98–111.
33. Park, U., Hong, C., Jain, A.K., 3D model-assisted face recognition in video. Proceedings Second Canadian Conference on Computer and Robot Vision. Victoria, Canada, 2005, 322–329.
34. Pekalska, E., Paclik, P., Duin, R., A generalized kernel approach to dissimilarity-based classification. J. Mach. Learn. Res. 2:2 (2002), 175–211.
35. Phillips, P., Flynn, P., Scruggs, T., Bowyer, K., Chang, J., Hoffman, K., Marques, J., Min, J., Worek, W., Overview of the face recognition grand challenge. Proceedings IEEE Conference on Computer Vision and Pattern Recognition, San Diego, CA, Vol. 1, 2005, 947–954, 10.1109/CVPR.2005.268.
36. Phillips, P., Scruggs, W., O'Toole, A., Flynn, P., Bowyer, K., Schott, C., Sharpe, M., FRVT 2006 and ICE 2006 large-scale experimental results. IEEE Trans. Pattern Anal. Mach. Intell. 32:5 (2010), 831–846, 10.1109/TPAMI.2009.59.
37. Pittsburgh Pattern Recognition, 2011. PittPatt face recognition software development kit (PittPatt SDK) v5.2.
38. Prabhu, U., Heo, J., Savvides, M., Unconstrained pose-invariant face recognition using 3D generic elastic models. IEEE Trans. Pattern Anal. Mach. Intell. 33:10 (2011), 1952–1961, 10.1109/TPAMI.2011.123.
39. Rama, A., Tarres, F., Onofrio, D., Tubaro, S., Mixed 2D-3D information for pose estimation and face recognition. Proceedings IEEE International Conference on Acoustics, Speech and Signal Processing, 2006, 361–364.
40. Riccio, D., Dugelay, J.-L., Geometric invariants for 2D/3D face recognition. Pattern Recognit. Lett. 28:14 (2007), 1907–1914.
41. Romdhani, S., Ho, J., Vetter, T., Kriegman, D.J., Face recognition using 3-D models: pose and illumination. Proc. IEEE 94:11 (2006), 1977–1999, 10.1109/JPROC.2006.886019.
42. Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M., 300 faces in-the-wild challenge: the first facial landmark localization challenge. IEEE International Conference on Computer Vision Workshops. Sydney, Australia, 2013, 397–403, 10.1109/ICCVW.2013.59.
43. Savran, A., Alyüz, N., Dibeklioğlu, H., Çeliktutan, O., Gökberk, B., Sankur, B., Akarun, L., Bosphorus database for 3D face analysis. Biometrics and Identity Management Lecture Notes in Computer Science, vol. 5372, 2008, 47–56.
44. Smith, W., Hancock, E., Estimating the albedo map of the face from a single image. Proceedings IEEE International Conference on Image Processing, Genoa, Italy, vol. 3, 2005, 780–783.
45. Theoharis, T., Passalis, G., Toderici, G., Kakadiaris, I.A., Unified 3D face and ear recognition using wavelets on geometry images. Pattern Recognit. 41:3 (2008), 796–804, 10.1016/j.patcog.2007.06.024.
46. Toderici, G., Evangelopoulos, G., Fang, T., Theoharis, T., Kakadiaris, I.A., UHDB11 database for 3D-2D face recognition. Proceedings 6th Pacific-Rim Symposium on Image and Video Technology. Guanajuato, Mexico, 2013, 73–86.
47. Toderici, G., Passalis, G., Zafeiriou, S., Tzimiropoulos, G., Petrou, M., Theoharis, T., Kakadiaris, I.A., Bidirectional relighting for 3D-aided 2D face recognition. Proceedings IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, CA, 2010, 2721–2728 978-1-4244-6983-3/10.
48. Tzimiropoulos, G., Argyriou, V., Zafeiriou, S., Stathaki, T., Robust FFT-based scale-invariant image registration with image gradients. IEEE Trans. Pattern Anal. Mach. Intell. 32:10 (2010), 1899–1906.
49. Tzimiropoulos, G., Zafeiriou, S., Pantic, M., Subspace learning from image gradient orientations. IEEE Trans. Pattern Anal. Mach. Intell. 34:12 (2012), 2454–2466.
50. Vijayan, V., Bowyer, K., Flynn, P., Huang, D., Chen, L., Ocegueda, O., Shah, S., Kakadiaris, I.A., Twins 3D face recognition challenge. Proceedings International Joint Conference on Biometrics. Washington, DC, 2011.
51. Wang, Y., Liu, J., Tang, X., Robust 3D face recognition by local shape difference boosting. IEEE Trans. Pattern Anal. Mach. Intell. 32:10 (2010), 1858–1870.
52. Wang, Y., Zhang, L., Liu, Z., Hua, G., Wen, Z., Zhang, Z., Samaras, D., Face relighting from a single image under arbitrary unknown lighting conditions. IEEE Trans. Pattern Anal. Mach. Intell. 31:11 (2009), 1968–1984.
53. Wolf, L., Hassner, T., Taigman, Y., Effective unconstrained face recognition by combining multiple descriptors and learned background statistics. IEEE Trans. Pattern Anal. Mach. Intell. 33:10 (2011), 1978–1990, 10.1109/TPAMI.2010.230.
54. Yin, L., Yourst, M., 3D face recognition based on high-resolution 3D face modeling from frontal and profile views. Proc. ACM SIGMM Workshop on Biometrics Methods and Applications. New York, NY, USA, 2003, 1–8.
55. Zhang, W., Huang, D., Samaras, D., Morvan, J.-M., Wang, Y., Chen, L., 3D assisted face recognition via progressive pose estimation. Proc. IEEE International Conference on Image Processing. Paris, France, 2014, 728–732.
56. Zhang, W., Huang, D., Wang, Y., Chen, L., 3D-aided face recognition across pose variations. Chinese Conference on Biometric Recognition, 2012, 58–66.
57. Zhao, X., Evangelopoulos, G., Chu, D., Shah, S., Kakadiaris, I.A., Minimizing illumination differences for 3-D to 2-D face recognition using lighting maps. IEEE Trans. Cybern. 44:5 (2014), 725–736, 10.1109/TCYB.2013.2291196.
58. Zhao, X., Shah, S., Kakadiaris, I.A., Illumination normalization using self-lighting ratios for 3D-2D face recognition. Proc. European Conference on Computer Vision Workshop. Firenze, Italy, 2012, 220–229.
59. Zhao, X., Shah, S., Kakadiaris, I.A., Illumination alignment using lighting ratio: application to 3D-2D face recognition. Proc. 10th International Conference on Automatic Face and Gesture Recognition. Shanghai, China, 2013, 1–6, 10.1109/FG.2013.6553782.
60. Zhou, Z., Ganesh, A., Wright, J., Tsai, S.-F., Ma, Y., Nearest-subspace patch matching for face recognition under varying pose and illumination. Proceedings IEEE International Conference on Automatic Face Gesture Recognition. Amsterdam, The Netherlands, 2008, 10.1109/AFGR.2008.4813452.