Person re-identification by unsupervised video matching

Pattern Recognition

Volumn 65, Issue , 2017, Pages 197-210

  Ma, Xiaolong ^a,d   Zhu, Xiatian ^b   Gong, Shaogang ^b   Xie, Xudong ^a   Hu, Jianming ^a   Lam, Kin Man ^c   Zhong, Yisheng ^a  

^a TSINGHUA UNIVERSITY   (China)

^b QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

^c HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

^d China Academy of Electronics and Information Technology   (China)

Author keywords

Action recognition; Gait recognition; Person re identification; Spatio temporal pyramids; Temporal sequence matching; Time shift; Video matching

Indexed keywords

COMPUTER VISION; PATTERN RECOGNITION;

ACTION RECOGNITION; GAIT RECOGNITION; PERSON RE IDENTIFICATIONS; SPATIO TEMPORAL; TEMPORAL SEQUENCES; TIME SHIFTS; VIDEO MATCHING;

CAMERAS;

EID: 85010637305     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2016.11.018     Document Type: Article

References (98)

1. [1] Gong, S., Cristani, M., Loy, C.C., Hospedales, T.M., The Re-Identification Challenge. Gong, S., Cristani, M., Yan, S., Loy, C.C., (eds.) Person Re-Identification, 2014, Springer, Berlin, Heidelberg.
2. [3] M. Farenzena, L. Bazzani, A. Perina, V. Murino, M. Cristani, Person re-identification by symmetry-driven accumulation of local features, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2010, pp. 2360–2367.
3. [4] B. Prosser, W.-S. Zheng, S. Gong, T. Xiang, Person re-identification by support vector ranking, in: Proceedings of the British Machine Vision Conference, 2010.
4. [5] M. Hirzer, P.M. Roth, M. Köstinger, H. Bischof, Relaxed pairwise learned metric for person re-identification, in: Proceedings of the European Conference on Computer Vision, 2012, pp. 780–793.
5. [6] R. Zhao, W. Ouyang, X. Wang, Unsupervised salience learning for person re-identification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 3586–3593.
6. [7] A. Bhuiyan, A. Perina, V. Murino, Person re-identification by discriminatively selecting parts and features, in: Proceedings of the Workshop of European Conference on Computer Vision, 2014, pp. 147–161.
7. [8] Liu, C., Gong, S., Loy, C.C., On-the-fly feature importance mining for person re-identification. Pattern Recognit. 47 (2014), 1602–1615.
8. [9] Zheng, W.-S., Gong, S., Xiang, T., Towards open-world person re-identification by one-shot group-based verification. IEEE Trans. Pattern Anal. Mach. Intell. 38:3 (2016), 591–606.
9. [10] L. Zhang, T. Xiang, S. Gong, Learning a discriminative null space for person re-identification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
10. [11] H. Wang, M.M. Ullah, A. Klaser, I. Laptev, C. Schmid, et al., Evaluation of local spatio-temporal features for action recognition, in: Proceedings of the British Machine Vision Conference, 2009.
11. [12] Poppe, R., A survey on vision-based human action recognition. Image Vis. Comput. 28 (2010), 976–990.
12. [13] Sarkar, S., Phillips, P.J., Liu, Z., Vega, I.R., Grother, P., Bowyer, K.W., The humanid gait challenge problem: data sets, performance, and analysis. IEEE Trans. Pattern Anal. Mach. Intell. 27:2 (2005), 162–177.
13. [14] Bashir, K., Xiang, T., Gong, S., Gait recognition without subject cooperation. Pattern Recognit. Lett. 31 (2010), 2052–2060.
14. [15] Yang, Y., Ramanan, D., Articulated human detection with flexible mixtures of parts. IEEE Trans. Pattern Anal. Mach. Intell. 35:12 (2013), 2878–2890.
15. [16] W. Ouyang, X. Chu, X. Wang, Multi-source deep learning for human pose estimation, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 2337–2344.
16. [17] T. Wang, S. Gong, X. Zhu, S. Wang, Person re-identification by video ranking, in: Proceedings of the European Conference on Computer Vision, 2014, pp. 688–703.
17. [18] Wang, T., Gong, S., Zhu, X., Wang, S., Person re-identification by discriminative selection in video ranking. IEEE Trans. Pattern Anal. Mach. Intell. 38:12 (2016), 2501–2514.
18. [19] A. Klaser, M. Marszalek, A spatio-temporal descriptor based on 3d-gradients, in: Proceedings of the British Machine Vision Conference, 2008.
19. [20] S. Lazebnik, C. Schmid, J. Ponce, Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, 2006, pp. 2169–2178.
20. [21] H. Pirsiavash, D. Ramanan, Detecting activities of daily living in first-person camera views, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 2847–2854.
21. [22] M. Hirzer, C. Beleznai, P.M. Roth, H. Bischof, Person re-identification by descriptive and discriminative classification, in: Proceedings of the Scandinavian Conference on Image Analysis, 2011.
22. [23] S. Liao, Z. Mo, Y. Hu, S.Z. Li, Open-set person re-identification, arXiv preprint, 2014, pp. 1–16.
23. [24] R. Martín-Félez, T. Xiang, Gait recognition by ranking, in: Proceedings of the European Conference on Computer Vision, 2012, pp. 328–341.
24. [25] Rabiner, L.R., Juang, B.-H., Fundamentals of Speech Recognition, 14, 1993, PTR Prentice Hall, Englewood Cliffs.
25. [26] E. Kodirov, T. Xiang, S. Gong, Dictionary learning with iterative laplacian regularisation for unsupervised person re-identification, in: Proceedings of the British Machine Vision Conference, 2015.
26. [27] S. Liao, Y. Hu, X. Zhu, S.Z. Li, Person re-identification by local maximal occurrence representation and metric learning, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 2197–2206.
27. [28] Xu, D., Huang, Y., Zeng, Z., Xu, X., Human gait recognition using patch distribution feature and locality-constrained group sparse representation. IEEE Trans. Image Process. 21:1 (2012), 316–326.
28. [29] Hofmann, M., Geiger, J., Bachmann, S., Schuller, B., Rigoll, G., The tum gait from audio, image and depth (gaid) database: multimodal recognition of subjects and traits. J. Vis. Commun. Image Represent. 25:1 (2014), 195–206.
29. [30] Chattopadhyay, P., Roy, A., Sural, S., Mukhopadhyay, J., Pose depth volume extraction from rgb-d streams for frontal gait recognition. J. Vis. Commun. Image Represent. 25:1 (2014), 53–63.
30. [31] Choudhury, S.D., Tjahjadi, T., Robust view-invariant multiscale gait recognition. Pattern Recognit. 48:3 (2015), 798–811.
31. [32] T. Kobayashi, N. Otsu, Action and simultaneous multiple-person identification using cubic higher-order local auto-correlation, in: Proceedings of the IEEE International Conference on Pattern Recognition, Vol. 3, 2004, pp. 741–744.
32. [33] Han, J., Bhanu, B., Individual recognition using gait energy image. IEEE Trans. Pattern Anal. Mach. Intell. 28 (2006), 316–322.
33. [34] G.V. Veres, L. Gordon, J.N. Carter, M.S. Nixon, What image information is important in silhouette-based gait recognition? in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, 2004, pp. II–776.
34. [35] D. S. Matovski, M. S. Nixon, S. Mahmoodi, T. Mansfield, On including quality in applied automatic gait recognition, in: Proceedings of the IEEE International Conference on Pattern Recognition, 2012, pp. 3272–3275.
35. [36] M. Hofmann, S. Sural, G. Rigoll, Gait recognition in the presence of occlusion: a new dataset and baseline algorithms, in: Proceedings of the International Conference on Computer Graphics, Visualization and Computer Vision, 2011, pp. 99–104.
36. [37] Boulgouris, N.V., Chi, Z.X., Human gait recognition based on matching of body components. Pattern Recognit. 40:6 (2007), 1763–1770.
37. [38] Hossain, M.A., Makihara, Y., Wang, J., Yagi, Y., Clothing-invariant gait identification using part-based clothing categorization and adaptive weight control. Pattern Recognit. 43:6 (2010), 2281–2291.
38. [39] S.H. Shaikh, K. Saeed, N. Chaki, Gait recognition using partial silhouette-based approach, in: Proceedings of the IEEE International Conference on Signal Processing and Integrated Networks, 2014, pp. 101–106.
39. [40] D. Muramatsu, Y. Makihara, Y. Yagi, Gait regeneration for recognition, in: Proceedings of the IAPR International Conference on Biometrics, 2015, pp. 1–8.
40. [41] J. Xiao, H. Cheng, H. Sawhney, C. Rao, M. Isnardi, Bilateral filtering-based optical flow estimation with occlusion detection, in: Proceedings of the European Conference on Computer Vision, 2006, pp. 211–224.
41. [42] S. Yu, D. Tan, T. Tan, Modelling the effect of view angle variation on appearance-based gait recognition, in: Proceedings of the Asian Conference on Computer Vision, 2006, pp. 807–816.
42. [43] Yang, X., Zhou, Y., Zhang, T., Shu, G., Yang, J., Gait recognition based on dynamic region analysis. Signal Process. 88:9 (2008), 2350–2356.
43. [44] S. Singh, K. Biswas, Biometric gait recognition with carrying and clothing variants, in: Proceedings of the Pattern Recognition and Machine Intelligence, 2009, pp. 446–451.
44. [45] Martín-Félez, R., Xiang, T., Uncooperative gait recognition by learning to rank. Pattern Recognit. 47:12 (2014), 3793–3806.
45. [46] P. Senin, Dynamic time warping algorithm review, Proceedings of the Information and Computer Science, Department University of Hawaii at Manoa Honolulu, USA, 2008, pp. 1–23.
46. [47] T. Rakthanmanon, B. Campana, A. Mueen, G. Batista, B. Westover, Q. Zhu, J. Zakaria, E. Keogh, Searching and mining trillions of time series subsequences under dynamic time warping, in: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2012, pp. 262–270.
47. [48] E.J. Keogh, M.J. Pazzani, Derivative dynamic time warping, in: Proceedings of the SIAM International Conference on Data Mining, Vol. 1, 2001, pp. 5–7.
48. [49] Gullo, F., Ponti, G., Tagarelli, A., Greco, S., A time series representation model for accurate and fast similarity detection. Pattern Recognit. 42:11 (2009), 2998–3014.
49. [50] Jeong, Y.-S., Jeong, M.K., Omitaomu, O.A., Weighted dynamic time warping for time series classification. Pattern Recognit. 44:9 (2011), 2231–2240.
50. [51] Horng, J.-H., Li, J.T., An automatic and efficient dynamic programming algorithm for polygonal approximation of digital curves. Pattern Recognit. Lett. 23:1–3 (2002), 171–182.
51. [52] Oka, R., Spotting method for classification of real world data. Comput. J. 41:8 (1998), 559–565.
52. [53] Wu, Z., Li, Y., Radke, R.J., Viewpoint invariant human re-identification in camera networks using pose priors and subject-discriminative features. IEEE Trans. Pattern Anal. Mach. Intell. 37:5 (2015), 1095–1108.
53. [54] Y.-C. Chen, W.-S. Zheng, J. Lai, Mirror representation for modeling view-specific transform in person re-identification, in: Proceedings of the International Joint Conference of Artificial Intelligence, 2015, pp. 3402–3408.
54. [55] H. Wang, S. Gong, X. Zhu, T. Xiang, Human-in-the-loop person re-identification, in: Proceedings of the European Conference on Computer Vision, 2016, pp. 405–422.
55. [56] H. Wang, X. Zhu, T. Xiang, S. Gong, Towards unsupervised open-set person re-identification, in: Proceedings of the IEEE International Conference on Image Processing, 2016.
56. [57] E. Kodirov, T. Xiang, Z. Fu, S. Gong, Person re-identification by unsupervised l1 graph learning, in: Proceedings of the European Conference on Computer Vision, 2016.
57. [58] O. Hamdoun, F. Moutarde, B. Stanciulescu, B. Steux, Person re-identification in multi-camera system by signature based on interest point descriptors collected on short video sequences, in: Proceedings of the ACM International Conference on Distributed Smart Cameras, 2008, pp. 1–6.
58. [59] D.N.T. Cong, C. Achard, L. Khoudour, L. Douadi, Video sequences association for people re-identification across multiple non-overlapping cameras, in: Proceedings of the International Conference on Image Analysis and Processing, 2009, pp. 179–189.
59. [60] Nakajima, C., Pontil, M., Heisele, B., Poggio, T., Full-body person recognition system. Pattern Recognit. 36 (2003), 1997–2006.
60. [61] N. Gheissari, T.B. Sebastian, R. Hartley, Person reidentification using spatiotemporal appearance, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2006, pp. 1528–1535.
61. [62] D.S. Cheng, M. Cristani, M. Stoppa, L. Bazzani, V. Murino, Custom pictorial structures for re-identification, in: Proceedings of the British Machine Vision Conference, 2011.
62. [63] Y. Xu, L. Lin, W.-S. Zheng, X. Liu, Human re-identification by matching compositional template with cluster sampling, in: Proceedings of the IEEE International Conference on Computer Vision, 2013.
63. [64] Roy, A., Sural, S., Mukherjee, J., A hierarchical method combining gait and phase of motion with spatiotemporal model for person re-identification. Pattern Recognit. Lett. 33:14 (2012), 1891–1901.
64. [65] R. Kawai, Y. Makihara, C. Hua, H. Iwama, Y. Yagi, Person re-identification using view-dependent score-level fusion of gait and color features, in: IEEE International Conference on Pattern Recognition, 2012, pp. 2694–2697.
65. [66] A. Bedagkar-Gala, S. K. Shah, Gait-assisted person re-identification in wide area surveillance, in: Proceedings of the Workshop of Asian Conference on Computer Vision, 2014, pp. 633–649.
66. [67] Liu, Z., Zhang, Z., Wu, Q., Wang, Y., Enhancing person re-identification by integrating gait biometric. Neurocomputing 168:30 (2015), 1144–1156.
67. [68] J. You, A. Wu, X. Li, W.-S. Zheng, Top-push video-based person re-identification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
68. [69] N. McLaughlin, J. Martinez del Rincon, P. Miller, Recurrent convolutional network for video-based person re-identification, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
69. [70] C. Schüldt, I. Laptev, B. Caputo, Recognizing human actions: a local svm approach, in: Proceedings of the IEEE International Conference on Pattern Recognition, Vol. 3, 2004, pp. 32–36.
70. [71] P. Dollár, V. Rabaud, G. Cottrell, S. Belongie, Behavior recognition via sparse spatio-temporal features, in: Proceedings of the IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, 2005, pp. 65–72.
71. [72] I. Laptev, M. Marszalek, C. Schmid, B. Rozenfeld, Learning realistic human actions from movies, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1–8.
72. [73] Kim, T.-K., Cipolla, R., Canonical correlation analysis of video volume tensors for action categorization and detection. IEEE Trans. Pattern Anal. Mach. Intell. 31:8 (2009), 1415–1428.
73. [74] P. Scovanner, S. Ali, M. Shah, A 3-dimensional sift descriptor and its application to action recognition, in: Proceedings of the ACM International Conference on Multimedia, 2007, pp. 357–360.
74. [75] G. Willems, T. Tuytelaars, L. Van Gool, An efficient dense and scale-invariant spatio-temporal interest point detector, in: Proceedings of the European Conference on Computer Vision, 2008, pp. 650–663.
75. [76] Zhu, Y., Lucey, S., Convolutional sparse coding for trajectory reconstruction. IEEE Trans. Pattern Anal. Mach. Intell. 37:3 (2015), 529–540.
76. [77] S. Nowozin, G. Bakir, K. Tsuda, Discriminative subsequence mining for action classification, in: Proceedings of the IEEE International Conference on Computer Vision, 2007, pp. 1–8.
77. [78] K. Schindler, L. Van Gool, Action snippets: How many frames does human action recognition require? in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2008, pp. 1–8.
78. [79] J.C. Niebles, C.-W. Chen, L. Fei-Fei, Modeling temporal structure of decomposable motion segments for activity classification, in: Proceedings of the European Conference on Computer Vision, 2010, pp. 392–405.
79. [80] A. Gaidon, Z. Harchaoui, C. Schmid, Actom sequence models for efficient action detection, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2011, pp. 3201–3208.
80. [81] A. Gaidon, Z. Harchaoui, C. Schmid, A time series kernel for action recognition, in: Proceedings of the British Machine Vision Conference, 2011.
81. [82] L. Zelnik-Manor, M. Irani, Event-based analysis of video, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2001.
82. [83] Irani, M., Anandan, P.a., Bergen, J., Kumar, R., Hsu, S., Efficient representations of video sequences and their applications. Signal Process.: Image Commun. 8:4 (1996), 327–351.
83. [84] J. Choi, W.J. Jeon, S.-C. Lee, Spatio-temporal pyramid matching for sports videos, in: Proceedings of the ACM International Conference on Multimedia Information Retrieval, 2008, pp. 291–297.
84. [85] Smeulders, A.W., Chu, D.M., Cucchiara, R., Calderara, S., Dehghan, A., Shah, M., Visual tracking: an experimental survey. IEEE Trans. Pattern Anal. Mach. Intell. 36:7 (2014), 1442–1468.
85. [86] Grauman, K., Darrell, T., The pyramid match kernel: efficient learning with sets of features. J. Mach. Learn. Res. 8 (2007), 725–760.
86. [87] F. Shi, R. Laganiere, E. Petriu, H. Zhen, Lpm for fast action recognition with large number of classes, in: Proceedings of the Workshop of IEEE International Conference on Computer Vision, 2013.
87. [88] Wang, H., Oneata, D., Verbeek, J., Schmid, C., A robust and efficient video representation for action recognition. Int. J. Comput. Vis., 2015, 1–20.
88. [89] D. J. Berndt, J. Clifford, Using dynamic time warping to find patterns in time series., in: Proceedings of the Workshop of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Vol. 10, 1994, pp. 359–370.
89. [90] Fraser, A.M., Swinney, H.L., Independent coordinates for strange attractors from mutual information. Phys. Rev. A, 33(2), 1986, 1134.
90. [91] Loy, C.C., Xiang, T., Gong, S., Time-delayed correlation analysis for multi-camera activity understanding. Int. J. Comput. Vis. 90 (2010), 106–129.
91. [92] M. Shokoohi-Yekta, J. Wang, E. Keogh, On the non-trivial generalization of dynamic time warping to the multi-dimensional case, in: Proceedings of the SIAM International Conference on Data Mining, 2015, pp. 39–48.
92. [93] A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, in: Proceedings of the Advances in Neural Information Processing Systems, 2012, pp. 1097–1105.
93. [94] Müller, M., Dynamic time warping. Inf. Retr. Music Motion, 2007, 69–84.
94. [95] S. Salvador, P. Chan, Fastdtw: Toward accurate dynamic time warping in linear time and space, in: Proceedings of the Workshop of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2004.
95. [96] Chapelle, O., Keerthi, S.S., Efficient algorithms for ranking with svms. Inf. Retr. 13 (2010), 201–215.
96. [97] Lisanti, G., Masi, I., Bagdanov, A., Del Bimbo, A., Person re-identification by iterative re-weighted sparse ranking. IEEE Trans. Pattern Anal. Mach. Intell. 37:8 (2015), 1629–1642.
97. [98] H. Ding, G. Trajcevski, P. Scheuermann, X. Wang, E. Keogh, Querying and mining of time series data: experimental comparison of representations and distance measures, in: Proceedings of the Proceedings of the Very Large Data Bases Endowment, 1010, 1(2), 2008, pp. 1542–1552.
98. [99] Y.-C. Chen, W.-S. Zheng, P.C. Yuen, J. Lai, An asymmetric distance model for cross-view feature mapping in person re-identification, in: Proceedings of the IEEE Transactions on Circuits and Systems for Video Technology, Vol. PP, 2015, pp. 1–1.