A Benchmark and Comparative Study of Video-Based Face Recognition on COX Face Database

IEEE Transactions on Image Processing

Volumn 24, Issue 12, 2015, Pages 5967-5981

  Huang, Zhiwu ^a   Shan, Shiguang ^a,b   Wang, Ruiping ^a   Zhang, Haihong ^c   Lao, Shihong ^d   Kuerban, Alifu ^e   Chen, Xilin ^a  

^a INSTITUTE OF COMPUTING TECHNOLOGY   (China)

^b Chinese Academy of Sciences   (China)

^c Development Center   (Japan)

^d OMRON CORPORATION   (Japan)

^e XINJIANG UNIVERSITY   (China)

Author keywords

benchmarking; COX Face DB; point to set correlation learning; still to video; Video based face recognition; video to still; video to video

Indexed keywords

BENCHMARKING; DATABASE SYSTEMS; QUERY PROCESSING; SECURITY SYSTEMS;

COX FACE DB; SET CORRELATION; STILL-TO-VIDEO; VIDEO TO VIDEOS; VIDEO-BASED FACE RECOGNITION; VIDEO-TO-STILL;

FACE RECOGNITION;

ANATOMY AND HISTOLOGY; BIOMETRY; FACE; FACTUAL DATABASE; FEMALE; HUMAN; IMAGE PROCESSING; MALE; PROCEDURES; QUALITY CONTROL; STATISTICAL MODEL; VIDEORECORDING;

BENCHMARKING; BIOMETRIC IDENTIFICATION; DATABASES, FACTUAL; FACE; FEMALE; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MALE; MODELS, STATISTICAL; VIDEO RECORDING;

EID: 84959499967     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2015.2493448     Document Type: Article

References (69)

1. T.-K. Kim, J. Kittler, and R. Cipolla, "Discriminative learning and recognition of image set classes using canonical correlations," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 6, pp. 1005-1018, Jun. 2007.
2. M. Kim, S. Kumar, V. Pavlovic, and H. Rowley, "Face tracking and recognition with visual constraints in real-world videos," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2008, pp. 1787-1794.
3. R. Wang, S. Shan, X. Chen, and W. Gao, "Manifold-manifold distance with application to face recognition based on image set," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2008, pp. 2940-2947.
4. H. Cevikalp and B. Triggs, "Face recognition based on image sets," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 2567-2573.
5. S. Chen, C. Sanderson, M. T. Harandi, and B. C. Lovell, "Improved image set classification via joint sparse approximated nearest subspaces," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 452-459.
6. S. Zhou, V. Krueger, and R. Chellappa, "Probabilistic recognition of human faces from video," Comput. Vis. Image Understand., vol. 91, nos. 1-2, pp. 214-245, Jul. 2003.
7. S. Biswas, G. Aggarwal, and P. J. Flynn, "Pose-robust recognition of low-resolution face images," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 601-608.
8. Z. Huang, S. Shan, H. Zhang, H. Lao, A. Kuerban, and X. Chen, "Benchmarking still-to-video face recognition via partial and local linear discriminant analysis on COX-S2V dataset," in Proc. Asian Conf. Comput. Vis., 2012, pp. 589-600.
9. Z. Huang, X. Zhao, S. Shan, R. Wang, and X. Chen, "Coupling alignments with recognition for still-to-video face recognition," in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2013, pp. 3296-3303.
10. J. H. Barr, K. W. Bowyer, P. J. Flynn, and S. Biswas, "Face recognition from video: A review," Int. J. Pattern Recognit. Artif. Intell., vol. 26, no. 5, p. 1266002, 2012.
11. M. Grgic, K. Delac, and S. Grgic, "SCface - Surveillance cameras face database," Multimedia Tools Appl., vol. 51, no. 3, pp. 863-879, Feb. 2011.
12. A. J. O'Toole, P. Phillips, S. Weimer, D. Roark, J. Ayyad, R. Barwick, and J. Dunlop, "Recognizing people from dynamic and static faces and bodies: Dissecting identity with a fusion approach," Vis. Res., vol. 51, no. 1, pp. 74-83, Jan. 2011.
13. L. Wolf, T. Hassner, and I. Maoz, "Face recognition in unconstrained videos with matched background similarity," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 529-534.
14. J. R. Beveridge, P. Phillips, D. Bolme, B. A. Draper, G. Givens, Y. Lui, M. N. Teli, H. Zhang, W. T. Scruggs, K. W. Bowyer, P. J. Flynn, and S. Cheng, "The challenge of face recognition from digital point-andshoot cameras," in Proc. IEEE 6th Int. Conf. Biometrics, Theory, Appl. Syst., Sep./Oct. 2013, pp. 1-8.
15. R. Gross and J. Shi, "The CMU motion of body (MoBo) database," Tech. Rep. CMU-RI-TR-01-18, Robot. Inst., Carnegie Mellon Univ., Pittsburgh, PA, USA, 2001.
16. K.-C. Lee, J. Ho, M.-H. Yang, and D. Kriegman, "Video-based face recognition using probabilistic appearance manifolds," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2003, pp. 313-320.
17. K.-C. Lee, J. Ho, M.-H. Yang, and D. Kriegman, "Visual tracking and recognition using probabilistic appearance manifolds," Comput. Vis. Image Understand., vol. 99, no. 3, pp. 303-331, Sep. 2005.
18. R. Goh, L. Liu, X. Liu, and T. Chen, "The CMU face in action (FIA) database," in Proc. Int. Conf. Autom. Face Gesture Recognit., 2005, pp. 255-263.
19. University of Cambridge Face Database. [Online]. Available: http://mi.eng.cam.ac.uk/~oa214/academic/, accessed Oct. 24, 2015.
20. Faces96 Database. [Online]. Available: http://cswww.essex.ac.uk/mv/ allfaces/faces96.html, accessed Oct. 24, 2015.
21. C. Sanderson. (2008). Biometric Person Recognition: Face, Speech and Fusion. [Online]. Available: http://www.itee.uq.edu.au/~onrad/vidtimit/
22. J. Phillips, "Video challenge problem multiple biometric grand challenge: Preliminary results of version 2," Nat. Inst. Standards Technol., 2009. [Online]. Available: http://biometrics.nist.gov/cs-links/ face/mbgc/FINAL-VIDEO-MBGC-Workshop2-v1.pdf., accessed Oct. 24, 2015.
23. J. R. Barr, K. W. Bowyer, and P. J. Flynn, "Detecting questionable observers using face track clustering," in Proc. IEEE Workshop Appl. Comput. Vis., Jan. 2011, pp. 182-189.
24. Y. Wong, S. Chen, S. Mau, C. Sanderson, and B. C. Lovell, "Patch-based probabilistic image quality assessment for face selection and improved video-based face recognition," in Proc. IEEE Conf. Workshop Comput. Vis. Pattern Recognit., Jun. 2011, pp. 74-81.
25. R. Chellappa, J. Ni, and V. M. Patel, "Remote identification of faces: Problems, prospects, and progress," Pattern Recognit. Lett., vol. 33, no. 14, pp. 1849-1859, Oct. 2012.
26. L. Liu, L. Zhang, H. Liu, and S. Yan, "Toward large-population face identification in unconstrained videos," IEEE Trans. Circuits Syst. Video Technol., vol. 24, no. 11, pp. 1874-1884, Nov. 2014.
27. X. Liu and T. Chen, "Video-based face recognition using adaptive hidden Markov models," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2003, pp. 340-345.
28. S. K. Zhou, R. Chellappa, and B. Moghaddam, "Visual tracking and recognition using appearance-adaptive models in particle filters," IEEE Trans. Image Process., vol. 13, no. 11, pp. 1491-1506, Nov. 2004.
29. N. Ye and T. Sim, "Towards general motion-based face recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 2598-2605.
30. O. Yamaguchi, K. Fukui, and K. Maeda, "Face recognition using temporal image sequence," in Proc. IEEE Int. Conf. Autom. Face Gesture Recognit., Apr. 1998, pp. 318-323.
31. M. Hayat, M. Bennamoun, and S. An, "Learning non-linear reconstruction models for image set classification," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 1915-1922.
32. J. Hamm and D. D. Lee, "Grassmann discriminant analysis: A unifying view on subspace-based learning," in Proc. Int. Conf. Mach. Learn., 2008, pp. 376-383.
33. J. Hamm and D. D. Lee, "Extended Grassmann kernels for subspacebased learning," in Proc. Adv. Neural Inf. Process., 2009, pp. 601-608.
34. R. Wang, H. Guo, L. S. Davis, and Q. Dai, "Covariance discriminative learning: A natural and efficient approach to image set classification," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2012, pp. 2496-2503.
35. Z. Huang, R. Wang, S. Shan, and X. Chen, "Learning Euclidean-to-Riemannian metric for point-to-set classification," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 1677-1684.
36. D. R. Hardoon, S. Szedmak, and J. Shawe-Taylor, "Canonical correlation analysis: An overview with application to learning methods," Dept. Comput. Sci., Roy. Holloway, Univ. London, London, U.K., Tech. Rep. CSD-TR-03-02, 2003.
37. J.-T. Chien and C.-C. Wu, "Discriminant waveletfaces and nearest feature classifiers for face recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 12, pp. 1644-1649, Dec. 2002.
38. P. Vincent and Y. Bengio, "K-local hyperplane and convex distance nearest neighbor algorithms," in Proc. Adv. Neural Inf. Process. Syst., 2001, pp. 985-992.
39. H. Cevikalp, B. Triggs, and R. Polikar, "Nearest hyperdisk methods for high-dimensional classification," in Proc. Int. Conf. Mach. Learn., 2008, doi: 10.1145/1390156.1390172.
40. P. Zhu, L. Zhang, W. Zuo, and D. Zhang, "From point to set: Extend the learning of distance metrics," in Proc. Int. Conf. Comput. Vis., Dec. 2013, pp. 2664-2671.
41. P. C. Mahalanobis, "On the generalized distance in statistics," in Proc. Nat. Inst. Sci. India, vol. 2, no. 1, pp. 49-55, 1936.
42. K. Fukui and O. Yamaguchi, "Face recognition using multi-viewpoint patterns for robot vision," in Proc. Int. Symp. Robot. Res., 2003, pp. 192-201.
43. M. T. Harandi, C. Sanderson, S. Shirazi, and B. C. Lovell, "Graph embedding discriminant analysis on Grassmannian manifolds for improved image set matching," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 2705-2712.
44. R. Wang and X. Chen, "Manifold discriminant analysis," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 429-436.
45. Z. Cui, S. Shan, H. Zhang, S. Lao, and X. Chen, "Image sets alignment for video-based face recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2012, pp. 2626-2633.
46. Y. Hu, A. S. Mian, and R. Owens, "Sparse approximated nearest points for image set classification," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 121-128.
47. M. Yang, P. Zhu, L. Van Gool, and L. Zhang, "Face recognition based on regularized nearest points between image sets," in Proc. IEEE Int. Conf. Autom. Face Gesture Recognit., Apr. 2013, pp. 1-7.
48. P. Zhu, W. Zuo, L. Zhang, S. C.-K. Shiu, and D. Zhang, "Image setbased collaborative representation for face recognition," IEEE Trans. Inf. Forensics Security, vol. 9, no. 7, pp. 1120-1132, Jul. 2014.
49. G. Shakhnarovich, J. W. Fisher, and T. Darrell, "Face recognition from long-term observations," in Proc. Eur. Conf. Comput. Vis., 2002, pp. 851-865.
50. O. Arandjelović, G. Shakhnarovich, J. Fisher, R. Cipolla, and T. Darrell, "Face recognition with image sets using manifold density divergence," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2005, pp. 581-588.
51. R. Vemulapalli, J. K. Pillai, and R. Chellappa, "Kernel learning for extrinsic classification of manifold features," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 1782-1789.
52. J. Lu, G. Wang, and P. Moulin, "Image set classification using holistic multiple order statistics features and localized multi-kernel metric learning," in Proc. IEEE Conf. Int. Conf. Comput. Vis., Dec. 2013, pp. 329-336.
53. B. K. Gunturk, A. U. Batur, Y. Altunbasak, M. H. Hayes, and R. M. Mersereau, "Eigenface-domain super-resolution for face recognition," IEEE Trans. Image Process., vol. 12, no. 5, pp. 597-606, May 2003.
54. O. Arandjelović and R. Cipolla, "A manifold approach to face recognition from low quality video across illumination and pose using implicit super-resolution," in Proc. Int. Conf. Comput. Vis., 2007, pp. 1-8.
55. U. Park and A. K. Jain, "3D model-based face recognition in video," in Proc. Int. Conf. Biometrics, 2007, pp. 1085-1094.
56. D. Thomas, K. W. Bowyer, and P. J. Flynn, "Multi-factor approach to improving recognition performance in surveillance-quality video," in Proc. IEEE Conf. Biometrics, Theory, Appl. Syst., Sep./Oct. 2008, pp. 1-7.
57. X. Liu and T. Chen, "Face mosaicing for pose robust video-based recognition," in Proc. Asian Conf. Comput. Vis., 2007, pp. 662-671.
58. Y. Zhang and A. M. Martínez, "A weighted probabilistic approach to face recognition from multiple images and video sequences," Image Vis. Comput., vol. 24, no. 6, pp. 626-638, Jun. 2006.
59. J. Stallkamp, H. K. Ekenel, and R. Stiefelhagen, "Video-based face recognition on real-world data," in Proc. Int. Conf. Comput. Vis., Oct. 2007, pp. 1-8.
60. M. A. Turk and A. P. Pentland, "Face recognition using eigenfaces," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 1991, pp. 586-591.
61. B. Moghaddam and A. Pentland, "Probabilistic visual learning for object representation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 696-710, Jul. 1997.
62. X. Wang and X. Tang, "Dual-space linear discriminant analysis for face recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun./Jul. 2004, pp. 564-569.
63. H. Hotelling, "Relations between two sets of variates," Biometrika, vol. 28, nos. 3-4, pp. 312-377, 1936.
64. M. T. Harandi, M. Salzmann, S. Jayasumana, R. Hartley, and H. Li, "Expanding the family of Grassmannian kernels: An embedding perspective," in Proc. Eur. Conf. Comput. Vis., 2014, pp. 408-423.
65. V. Arsigny, P. Fillard, X. Pennec, and N. Ayache, "Fast and simple computations on tensors with Log-Euclidean metrics," French Inst. Res. Comput. Sci. Autom., Rocquencourt, France, Tech. Rep. RR-5584, 2005.
66. X. Pennec, P. Fillard, and N. Ayache, "A Riemannian framework for tensor computing," Int. J. Comput. Vis., vol. 66, no. 1, pp. 41-66, Jan. 2006.
67. S. Jayasumana, R. Hartley, M. Salzmann, H. Li, and M. T. Harandi, "Kernel methods on the Riemannian manifold of symmetric positive definite matrices," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 73-80.
68. V. Arsigny, P. Fillard, X. Pennec, and N. Ayache, "Geometric means in a novel vector space structure on symmetric positive-definite matrices," SIAM J. Matrix Anal. Appl., vol. 29, no. 1, pp. 328-347, 2007.
69. W. Förstner and B. Moonen. (2003). A metric for covariance matrices. Geodesy-The Challenge of the 3rd Millennium, pp. 299-309. [Online]. Available: http://link.springer.com/chapter/10.1007/978-3-662-05296-9-31