Duplex Metric Learning for Image Set Classification

IEEE Transactions on Image Processing

Volumn 27, Issue 1, 2018, Pages 281-292

  Cheng, Gong ^a   Zhou, Peicheng ^a   Han, Junwei ^a  

^a NORTHWESTERN POLYTECHNICAL UNIVERSITY   (China)

Author keywords

deep learning; feature learning; Image set classification; metric learning

Indexed keywords

DEEP LEARNING; FACE RECOGNITION; IMAGE RECONSTRUCTION; LINEAR PROGRAMMING; MEASUREMENTS; NEURONS; OBJECT RECOGNITION; PERSONNEL TRAINING;

CLASSIFICATION ERRORS; FEATURE LEARNING; FEATURE REPRESENTATION; IMAGE SETS; METRIC LEARNING; OBJECTIVE FUNCTIONS; RECONSTRUCTION ERROR; REGULARIZATION TERMS;

IMAGE CLASSIFICATION;

EID: 85038262495     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2017.2760512     Document Type: Article

References (74)

1. R. Wang, H. Guo, L. S. Davis, and Q. Dai, "Covariance discriminative learning: A natural and efficient approach to image set classification," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2012, pp. 2496-2503.
2. Y. Hu, A. S. Mian, and R. Owens, "Sparse approximated nearest points for image set classification," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 121-128.
3. J. Lu, G. Wang, and P. Moulin, "Image set classification using holistic multiple order statistics features and localized multi-kernel metric learning," in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2013, pp. 329-336.
4. J. Lu, G. Wang, W. Deng, P. Moulin, and J. Zhou, "Multi-manifold deep metric learning for image set classification," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 1137-1145.
5. A. Mahmood, A. Mian, and R. Owens, "Semi-supervised spectral clustering for image set classification," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 121-128.
6. M. Hayat, M. Bennamoun, and S. An, "Deep reconstruction models for image set classification," IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 4, pp. 713-727, Apr. 2015.
7. R. Wang and X. Chen, "Manifold discriminant analysis," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 429-436.
8. J. Lu, G. Wang, and P. Moulin, "Localized multifeature metric learning for image-set-based face recognition," IEEE Trans. Circuits Syst. Video Technol., vol. 26, no. 3, pp. 529-540, Mar. 2016.
9. J. Lu, G. Wang, W. Deng, and P. Moulin, "Simultaneous feature and dictionary learning for image set based face recognition," in Proc. Eur. Conf. Comput. Vis., 2014, pp. 265-280.
10. Y. Hu, A. Mian, and R. Owens, "Face recognition using sparse approximated nearest points between image sets," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 10, pp. 1992-2004, Oct. 2012.
11. A. Mian, Y. Hu, R. Hartley, and R. Owens, "Image set based face recognition using self-regularized non-negative coding and adaptive distance metric learning," IEEE Trans. Image Process., vol. 22, no. 12, pp. 5252-5262, Dec. 2013.
12. E. G. Ortiz, A. Wright, and M. Shah, "Face recognition in movie trailers via mean sequence sparse representation-based classification," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 3531-3538.
13. P. Zhu, L. Zhang, W. Zuo, and D. Zhang, "From point to set: Extend the learning of distance metrics," in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2013, pp. 2664-2671.
14. Z. Cui, W. Li, D. Xu, S. Shan, and X. Chen, "Fusing robust face region descriptors via multiple metric learning for face recognition in the wild," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 3554-3561.
15. J. Hu, J. Lu, and Y.-P. Tan, "Discriminative deep metric learning for face verification in the wild," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 1875-1882.
16. W. Wang, R. Wang, Z. Huang, S. Shan, and X. Chen, "Discriminant analysis on Riemannian manifold of Gaussian distributions for face recognition with image sets," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 2048-2057.
17. Z. Huang, R. Wang, S. Shan, and X. Chen, "Projection metric learning on Grassmann manifold with application to video based face recognition," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 140-149.
18. S. Liao, Y. Hu, X. Zhu, and S. Z. Li, "Person re-identification by local maximal occurrence representation and metric learning," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., May 2015, pp. 2197-2206.
19. J. Hu, J. Lu, and Y.-P. Tan, "Deep transfer metric learning," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 325-333.
20. T.-K. Kim, J. Kittler, and R. Cipolla, "Learning discriminative canonical correlations for object recognition with image sets," in Proc. Eur. Conf. Comput. Vis., 2006, pp. 251-262.
21. Y.-C. Chen, V. M. Patel, P. J. Phillips, and R. Chellappa, "Dictionarybased face recognition from video," in Proc. Eur. Conf. Comput. Vis., 2012, pp. 766-779.
22. Y. Yan, F. Nie, W. Li, C. Gao, Y. Yang, and D. Xu, "Image classification by cross-media active learning with privileged information," IEEE Trans. Multimedia, vol. 18, no. 12, pp. 2494-2502, Dec. 2016.
23. Y. Yang, D. Xu, F. Nie, S. Yan, and Y. Zhuang, "Image clustering using local discriminant models and global integration," IEEE Trans. Image Process., vol. 19, no. 10, pp. 2761-2773, Oct. 2010.
24. Y. Yang, F. Nie, D. Xu, J. Luo, Y. Zhuang, and Y. Pan, "A multimedia retrieval framework based on semi-supervised ranking and relevance feedback," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 4, pp. 723-742, Apr. 2012.
25. G. Cheng, J. Han, L. Guo, Z. Liu, S. Bu, and J. Ren, "Effective and efficient midlevel visual elements-oriented land-use classification using VHR remote sensing images," IEEE Trans. Geosci. Remote Sens., vol. 53, no. 8, pp. 4238-4249, Aug. 2015.
26. X. Lu, X. Zheng, and Y. Yuan, "Remote sensing scene classification by unsupervised representation learning," IEEE Trans. Geosci. Remote Sens., vol. 55, no. 9, pp. 5148-5157, Sep. 2017.
27. G. Cheng, J. Han, and X. Lu, "Remote sensing image scene classification: Benchmark and state of the art," Proc. IEEE, vol. 105, no. 10, pp. 1865-1883, Oct. 2017.
28. G. Cheng, Z. Li, X. Yao, L. Guo, and Z. Wei, "Remote sensing image scene classification using bag of convolutional features," IEEE Geosci. Remote Sens. Lett., vol. 14, no. 10, pp. 1735-1739, Oct. 2017.
29. J. Lu, G. Wang, W. Deng, and K. Jia, "Reconstruction-based metric learning for unconstrained face verification," IEEE Trans. Inf. Forensics Security, vol. 10, no. 1, pp. 79-89, Jan. 2015.
30. T. Matsuzawa, R. Relator, J. Sese, and T. Kato, "Stochastic Dykstra algorithms for metric learning with positive definite covariance descriptors," in Proc. Eur. Conf. Comput. Vis., 2016, pp. 786-799.
31. H. Cevikalp and B. Triggs, "Face recognition based on image sets," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 2567-2573.
32. J. Hamm and D. D. Lee, "Grassmann discriminant analysis: A unifying view on subspace-based learning," in Proc. IEEE Int. Conf. Mach. Learn., Jul. 2008, pp. 376-383.
33. P. Turaga, A. Veeraraghavan, A. Srivastava, and R. Chellappa, "Statistical computations on Grassmann and Stiefel manifolds for image and video-based recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 11, pp. 2273-2286, Nov. 2011.
34. L. Shao, L. Liu, and M. Yu, "Kernelized multiview projection for robust action recognition," Int. J. Comput. Vis., vol. 118, no. 2, pp. 115-129, Jun. 2016.
35. M. Yu, L. Shao, X. Zhen, and X. He, "Local feature discriminant projection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 9, pp. 1908-1914, Sep. 2016.
36. Y. Guo, G. Ding, L. Liu, J. Han, and L. Shao, "Learning to hash with optimized anchor embedding for scalable retrieval," IEEE Trans. Image Process., vol. 26, no. 3, pp. 1344-1354, Mar. 2017.
37. G. Ding, Y. Guo, J. Zhou, and Y. Gao, "Large-scale cross-modality search via collective matrix factorization hashing," IEEE Trans. Image Process., vol. 25, no. 11, pp. 5427-5440, Nov. 2016.
38. J. Shen, Y. Du, W. Wang, and X. Li, "Lazy random walks for superpixel segmentation," IEEE Trans. Image Process., vol. 23, no. 4, pp. 1451-1462, Apr. 2014.
39. W. Wang, J. Shen, X. Li, and F. Porikli, "Robust video object cosegmentation," IEEE Trans. Image Process., vol. 24, no. 10, pp. 3137-3148, Oct. 2015.
40. J. Han, G. Cheng, Z. Li, and D. Zhang, "A unified metric learning-based framework for co-saliency detection," IEEE Trans. Circuits Syst. Video Technol., to be published, doi: 10.1109/TCSVT.2017.2706264.
41. X. Yao, J. Han, G. Cheng, X. Qian, and L. Guo, "Semantic annotation of high-resolution satellite images via weakly supervised learning," IEEE Trans. Geosci. Remote Sens., vol. 54, no. 6, pp. 3660-3671, Jun. 2016.
42. R. G. Cinbis, J. Verbeek, and C. Schmid, "Unsupervised metric learning for face identification in TV video," in Proc. IEEE Int. Conf. Comput. Vis., Nov. 2011, pp. 1559-1566.
43. Q. Qian, R. Jin, S. Zhu, and Y. Lin, "Fine-grained visual categorization via multi-stage metric learning," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 3716-3724.
44. X. Han, T. Leung, Y. Jia, R. Sukthankar, and A. C. Berg, "MatchNet: Unifying feature and metric learning for patch-based matching," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 3279-3286.
45. F. Xiong, M. Gou, O. Camps, and M. Sznaier, "Person re-identification using kernel-based metric learning methods," in Proc. Eur. Conf. Comput. Vis., 2014, pp. 1-16.
46. S. Trivedi, D. McAllester, and G. Shakhnarovich, "Discriminative metric learning by neighborhood gerrymandering," in Proc. Conf. Adv. Neural Inform. Process. Syst., 2014, pp. 3392-3400.
47. Q. Cao, Y. Ying, and P. Li, "Similarity metric learning for face recognition," in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2013, pp. 2408-2415.
48. V. Zantedeschi, R. Emonet, and M. Sebban, "Metric learning as convex combinations of local models with generalization guarantees," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2016, pp. 1478-1486.
49. G. Chechik, V. Sharma, U. Shalit, and S. Bengio, "Large scale online learning of image similarity through ranking," J. Mach. Learn. Res., vol. 11, pp. 1109-1135, Jan. 2010.
50. K. Q. Weinberger and L. K. Saul, "Distance metric learning for large margin nearest neighbor classification," J. Mach. Learn. Res., vol. 10, pp. 207-244, Feb. 2009.
51. F. Schroff, D. Kalenichenko, and J. Philbin, "FaceNet: A unified embedding for face recognition and clustering," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 815-823.
52. K.-C. Lee, J. Ho, M.-H. Yang, and D. Kriegman, "Video-based face recognition using probabilistic appearance manifolds," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2003, pp. 313-320.
53. R. Gross and J. Shi, "The CMU motion of body (MoBo) database," School Robot. Inst., Carnegie Mellon Univ., Pittsburgh, PA, USA, Tech. Rep. CMU-RI-TR-01-18, 2001.
54. N. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar, "Attribute and simile classifiers for face verification," in Proc. IEEE Int. Conf. Comput. Vis., Sep./Oct. 2009, pp. 365-372.
55. M. Kim, S. Kumar, V. Pavlovic, and H. Rowley, "Face tracking and recognition with visual constraints in real-world videos," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2008, pp. 1-8.
56. B. Leibe and B. Schiele, "Analyzing appearance and contour based methods for object categorization," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2003, pp. 409-415.
57. P. Viola and M. J. Jones, "Robust real-time face detection," Int. J. Comput. Vis., vol. 57, no. 2, pp. 137-154, 2004.
58. L. van der Maaten and G. Hinton, "Visualizing data using t-SNE," J. Mach. Learn. Res., vol. 9, pp. 2579-2605, Nov. 2008.
59. Y. Taigman, M. Yang, M. A. Ranzato, and L. Wolf, "DeepFace: Closing the gap to human-level performance in face verification," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 1701-1708.
60. Y. Sun, X. Wang, and X. Tang, "Deeply learned face representations are sparse, selective, and robust," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 2892-2900.
61. Y. Wen, K. Zhang, Z. Li, and Y. Qiao, "A discriminative feature learning approach for deep face recognition," in Proc. Eur. Conf. Comput. Vis., 2016, pp. 499-515.
62. Y. Sun, X. Wang, and X. Tang, "Sparsifying neural network connections for face recognition," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2016, pp. 4856-4864.
63. O. M. Parkhi, A. Vedaldi, and A. Zisserman, "Deep face recognition," in Proc. Brit. Mach. Vis. Conf., 2015, pp. 1-12.
64. J. Yang, P. Ren, D. Chen, F. Wen, H. Li, and G. Hua, "Neural aggregation network for video face recognition," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jul. 2017, pp. 4362-4371.
65. Y. Liu, J. Yan, and W. Ouyang, "Quality aware network for set to set recognition," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., Jul. 2017, pp. 5790-5799.
66. Y. Hu, X. Wu, and R. He. (2017). "Attention-set based metric learning for video face recognition." [Online]. Available: https://arxiv.org/abs/1704.03805
67. L. Wolf, T. Hassner, and I. Maoz, "Face recognition in unconstrained videos with matched background similarity," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 529-534.
68. G. Cheng, P. Zhou, and J. Han, "Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images," IEEE Trans. Geosci. Remote Sens., vol. 54, no. 12, pp. 7405-7415, Dec. 2016.
69. D. Zhang, J. Han, C. Li, J. Wang, and X. Li, "Detection of co-salient objects by looking deep and wide," Int. J. Comput. Vis., vol. 120, no. 2, pp. 215-232, Nov. 2016.
70. G. Cheng, P. Zhou, and J. Han, "RIFD-CNN: Rotation-invariant and Fisher discriminative convolutional neural networks for object detection," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2016, pp. 2884-2893.
71. D. Zhang, J. Han, J. Han, and L. Shao, "Cosaliency detection based on intrasaliency prior transfer and deep intersaliency mining," IEEE Trans. Neural Netw. Learn. Syst., vol. 27, no. 6, pp. 1163-1176, Jun. 2016.
72. X. Yao, J. Han, D. Zhang, and F. Nie, "Revisiting co-saliency detection: A novel approach based on two-stage multi-view spectral rotation coclustering," IEEE Trans. Image Process., vol. 26, no. 7, pp. 3196-3209, Jul. 2017.
73. D. Zhang, J. Han, L. Jiang, S. Ye, and X. Chang, "Revealing event saliency in unconstrained video collection," IEEE Trans. Image Process., vol. 26, no. 4, pp. 1746-1758, Apr. 2017.
74. R. Wang, S. Shan, X. Chen, and W. Gao, "Manifold-manifold distance with application to face recognition based on image set," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2008, pp. 2940-2947.