Partial Occlusion Handling in Pedestrian Detection with a Deep Model

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 26, Issue 11, 2016, Pages 2123-2137

  Ouyang, Wanli ^a   Zeng, Xingyu ^a   Wang, Xiaogang ^a  

^a CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Deep model; human detection; object detection; occlusion handling; pedestrian detection

Indexed keywords

DEFORMATION; OBJECT RECOGNITION; VISIBILITY;

DEFORMABLE PART-BASED MODELS; HIDDEN VARIABLE; HUMAN DETECTION; MULTIPLE LAYERS; OCCLUSION HANDLING; PART-BASED MODELS; PARTIAL OCCLUSIONS; PEDESTRIAN DETECTION;

OBJECT DETECTION;

EID: 84991992642     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2015.2501940     Document Type: Article

References (116)

1. M. Andriluka, S. Roth, and B. Schiele, "Pictorial structures revisited: People detection and articulated pose estimation," in Proc. IEEE Conf. CVPR, Jun. 2009, pp. 1014-1021.
2. H. Azizpour and I. Laptev, "Object detection using strongly-supervised deformable part models," in Proc. 12th ECCV, 2012, pp. 836-849.
3. A. Bar-Hillel, D. Levi, E. Krupka, and C. Goldberg, "Part-based feature synthesis for human detection," in Proc. 11th ECCV, 2010, pp. 127-142.
4. O. Barinova, V. Lempitsky, and P. Kohli, "On detection of multiple object instances using Hough transforms," in Proc. IEEE Conf. CVPR, Jun. 2010, pp. 2233-2240.
5. Y. Bengio, "Learning deep architectures for AI," Found. Trends Mach. Learn., vol. 2, no. 1, pp. 1-127, 2009.
6. Y. Bengio, A. Courville, and P. Vincent, "Representation learning: A review and new perspectives," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 8, pp. 1798-1828, Aug. 2013.
7. M. Bergtholdt, J. Kappes, S. Schmidt, and C. Schnörr, "A study of parts-based object class detection using complete graphs," Int. J. Comput. Vis., vol. 87, nos. 1-2, pp. 93-117, 2010.
8. L. Bourdev, S. Maji, T. Brox, and J. Malik, "Detecting people using mutually consistent poselet activations," in Proc. 11th ECCV, 2010, pp. 168-181.
9. L. Bourdev and J. Malik, "Poselets: Body part detectors trained using 3D human pose annotations," in Proc. IEEE 12th ICCV, Sep./Oct. 2009, pp. 1365-1372.
10. Y. Chen, L. Zhu, and A. Yuille, "Active mask hierarchies for object detection," in Proc. 11th ECCV, 2010, pp. 43-56.
11. S. Dai, M. Yang, Y. Wu, and A. Katsaggelos, "Detector ensemble," in Proc. IEEE Conf. CVPR, Jun. 2007, pp. 1-8.
12. N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in Proc. IEEE Comput. Soc. Conf. CVPR, Jun. 2005, pp. 886-893.
13. N. Dalal, B. Triggs, and C. Schmid, "Human detection using oriented histograms of flow and appearance," in Proc. 9th ECCV, 2006, pp. 428-441.
14. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "ImageNet: A large-scale hierarchical image database," in Proc. IEEE Conf. CVPR, Jun. 2009, pp. 248-255.
15. C. Desai and D. Ramanan, "Detecting actions, poses, and objects with relational phraselets," in Proc. 12th ECCV, 2012, pp. 158-172.
16. P. Dollár, R. Appel, and W. Kienzle, "Crosstalk cascades for frame-rate pedestrian detection," in Proc. 12th ECCV, 2012, pp. 645-659.
17. P. Dollár, S. Belongie, and P. Perona, "The fastest pedestrian detector in the west," in Proc. BMVC, 2010, pp. 68.1-68.11.
18. P. Dollár, Z. Tu, P. Perona, and S. Belongie, "Integral channel features," in Proc. BMVC, 2009, pp. 91.1-91.11.
19. P. Dollár, Z. Tu, H. Tao, and S. Belongie, "Feature mining for image classification," in Proc. IEEE Conf. CVPR, Jun. 2007, pp. 1-8.
20. P. Dollár, C. Wojek, B. Schiele, and P. Perona, "Pedestrian detection: An evaluation of the state of the art," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 4, pp. 743-761, Apr. 2012.
21. G. Duan, H. Ai, and S. Lao, "A structural filter approach to human detection," in Proc. 11th ECCV, 2010, pp. 238-251.
22. M. Enzweiler, A. Eigenstetter, B. Schiele, and D. M. Gavrila, "Multicue pedestrian classification with partial occlusion handling," in Proc. IEEE Conf. CVPR, Jun. 2010, pp. 990-997.
23. M. Enzweiler and D. M. Gavrila, "A mixed generative-discriminative framework for pedestrian classification," in Proc. IEEE Conf. CVPR, Jun. 2008, pp. 1-8.
24. M. Enzweiler and D. M. Gavrila, "Integrated pedestrian classification and orientation estimation," in Proc. IEEE Conf. CVPR, Jun. 2010, pp. 982-989.
25. M. Enzweiler and D. M. Gavrila, "Monocular pedestrian detection: Survey and experiments," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 12, pp. 2179-2195, Dec. 2009.
26. M. Enzweiler and D. M. Gavrila, "A multilevel mixture-of-experts framework for pedestrian classification," IEEE Trans. Image Process., vol. 20, no. 10, pp. 2967-2979, Oct. 2011.
27. A. Ess, B. Leibe, and L. Van Gool, "Depth and appearance for mobile scene analysis," in Proc. IEEE 11th ICCV, Oct. 2007, pp. 1-8.
28. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman, "The Pascal visual object classes (VOC) challenge," Int. J. Comput. Vis., vol. 88, no. 2, pp. 303-338, 2010.
29. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, "Object detection with discriminatively trained part-based models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1627-1645, Sep. 2010.
30. P. F. Felzenszwalb, R. B. Girshick, and D. McAllester. Discriminatively Trained Deformable Part Models. [Online]. Available: http://www.cs.brown.edu/~pff/latent/, accessed 2012.
31. P. F. Felzenszwalb and D. P. Huttenlocher, "Pictorial structures for object recognition," Int. J. Comput. Vis., vol. 61, no. 1, pp. 55-79, 2005.
32. R. Fisher. CAVIAR: Context Aware Vision using Imagebased Active Recognition. [Online]. Available: http://homepages.inf.ed.ac.uk/rbf/ CAVIAR/, accessed 2012.
33. J. Gall and V. Lempitsky, "Class-specific Hough forests for object detection," in Proc. IEEE Conf. CVPR, Jun. 2009, pp. 1022-1029.
34. T. Gao, B. Packer, and D. Koller, "A segmentation-aware object detection model with occlusion handling," in Proc. IEEE Conf. CVPR, Jun. 2011, pp. 1361-1368.
35. D. Gerónimo, A. M. López, A. D. Sappa, and T. Graf, "Survey of pedestrian detection for advanced driver assistance systems," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 7, pp. 1239-1258, Jul. 2010.
36. R. B. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proc. IEEE Conf. CVPR, Jun. 2014, pp. 580-587.
37. R. B. Girshick, P. F. Felzenszwalb, and D. McAllester, "Object detection with grammar models," in Proc. NIPS, 2011, pp. 442-450.
38. K. He, X. Zhang, S. Ren, and J. Sun, "Spatial pyramid pooling in deep convolutional networks for visual recognition," in Proc. 13th ECCV, 2014, pp. 346-361.
39. G. E. Hinton, "Training products of experts by minimizing contrastive divergence," Neural Comput., vol. 14, no. 8, pp. 1771-1800, 2002.
40. G. E. Hinton, S. Osindero, and Y.-W. Teh, "A fast learning algorithm for deep belief nets," Neural Comput., vol. 18, no. 7, pp. 1527-1554, 2006.
41. G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, vol. 313, no. 5786, pp. 504-507, Jul. 2006.
42. A. Hyvärinen, "Estimation of non-normalized statistical models by score matching," J. Mach. Learn. Res., vol. 6, pp. 695-709, Dec. 2005.
43. K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. LeCun, "What is the best multi-stage architecture for object recognition?" in Proc. IEEE 12th Int. Conf. Comput. Vis., Sep./Oct. 2009, pp. 2146-2153.
44. Stanford Vision Lab. ImageNet Large Scale Visual Recognition Challenge. [Online]. Available: http://www.imagenet. org/challenges/LSVRC/2011/, accessed 2012.
45. C. H. Lampert, M. B. Blaschko, and T. Hofmann, "Beyond sliding windows: Object localization by efficient subwindow search," in Proc. IEEE Conf. CVPR, Jun. 2008, pp. 1-8.
46. N. Le Roux, N. Heess, J. Shotton, and J. Winn, "Learning a generative model of images by factoring appearance and shape," Neural Comput., vol. 23, no. 3, pp. 593-650, 2011.
47. H. Lee, R. Grosse, R. Ranganath, and A. Y. Ng, "Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations," in Proc. 26th Annu. ICML, 2009, pp. 609-616.
48. B. Leibe, A. Leonardis, and B. Schiele, "Combined object categorization and segmentation with an implicit shape model," in Proc. Workshop Statist. Learn. Comput. Vis. ECCV, May 2004, pp. 1-16.
49. B. Leibe, E. Seemann, and B. Schiele, "Pedestrian detection in crowded scenes," in Proc. IEEE Comput. Soc. Conf. CVPR, Jun. 2005, pp. 878-885.
50. M. Lin, Q. Chen, and S. Yan. (2013). "Network in network." [Online]. Available: http://arxiv.org/abs/1312.4400
51. Y.-Y. Lin, T.-L. Liu, and C.-S. Fuh, "Fast object detection with occlusions," in Proc. 8th ECCV, 2004, pp. 402-413.
52. Z. Lin and L. S. Davis, "A pose-invariant descriptor for human detection and segmentation," in Proc. 10th ECCV, 2008, pp. 423-436.
53. Z. Lin, L. S. Davis, D. Doermann, and D. DeMenthon, "Hierarchical part-template matching for human detection and segmentation," in Proc. IEEE 11th ICCV, Oct. 2007, pp. 1-8.
54. P. Luo, Y. Tian, X. Wang, and X. Tang, "Switchable deep network for pedestrian detection," in Proc. IEEE Conf. CVPR, Jun. 2014, pp. 899-906.
55. P. Luo, X. Wang, and X. Tang, "Hierarchical face parsing via deep learning," in Proc. IEEE Conf. CVPR, Jun. 2012, pp. 2480-2487.
56. P. Luo, X. Wang, and X. Tang, "A deep sum-product architecture for robust facial attributes analysis," in Proc. ICCV, Dec. 2013, pp. 2864-2871.
57. P. Luo, X. Wang, and X. Tang, "Pedestrian parsing via deep decompositional network," in Proc. ICCV, Dec. 2013, pp. 2648-2655.
58. S. Maji, A. C. Berg, and J. Malik, "Classification using intersection kernel support vector machines is efficient," in Proc. CVPR, Jun. 2008, pp. 1-8.
59. M. Mathias, R. Benenson, R. Timofte, and L. Van Gool, "Handling occlusions with Franken-classifiers," in Proc. CVPR, Dec. 2013, pp. 1505-1512.
60. C. Mikolajczyk, C. Schmid, and A. Zisserman, "Human detection based on a probabilistic assembly of robust part detectors," in Proc. ECCV, 2004, pp. 69-82.
61. K. Mikolajczyk, B. Leibe, and B. Schiele, "Multiple object class detection with a generative model," in Proc. CVPR, Jun. 2006, pp. 26-36.
62. S. Munder and D. M. Gavrila, "An experimental study on pedestrian classification," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 11, pp. 1863-1868, Nov. 2006.
63. W. Nam, P. Dollár, and J. H. Han, "Local decorrelation for improved pedestrian detection," in Proc. Adv. Neural Inf. Process. Syst., 2014, pp. 424-432.
64. M. Norouzi, M. Ranjbar, and G. Mori, "Stacks of convolutional restricted Boltzmann machines for shift-invariant feature learning," in Proc. CVPR, Jun. 2009, pp. 2735-2742.
65. W. Ouyang, X. Chu, and X. Wang, "Multi-source deep learning for human pose estimation," in Proc. CVPR, 2014, pp. 2337-2344.
66. W. Ouyang et al. (2014). "DeepID-Net: Multi-stage and deformable deep convolutional neural networks for object detection." [Online]. Available: http://arxiv.org/abs/1409.3505
67. W. Ouyang and X. Wang, "A discriminative deep model for pedestrian detection with occlusion handling," in Proc. CVPR, Jun. 2012, pp. 3258-3265.
68. W. Ouyang and X. Wang, "Joint deep learning for pedestrian detection," in Proc. ICCV, Dec. 2013, pp. 2056-2063.
69. W. Ouyang and X. Wang, "Single-pedestrian detection aided by multipedestrian detection," IEEE Trans. Pattern Anal. Machine Intell., vol. 37, no. 9, pp. 1875-1889, Sep. 2015.
70. W. Ouyang et al., "DeepID-Net: Deformable deep convolutional neural networks for object detection," in Proc. CVPR, 2015, pp. 1-10.
71. W. Ouyang, X. Zeng, and X. Wang, "Modeling mutual visibility relationship in pedestrian detection," in Proc. CVPR, Jun. 2013, pp. 3222-3229.
72. W. Ouyang, X. Zeng, and X. Wang, "Learning mutual visibility relationship for pedestrian detection with a deep model," Int. J. Comput. Vis., to be published.
73. W. Ouyang, X. Zeng, and X. Wang, "Single-pedestrian detection aided by two-pedestrian detection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 9, pp. 1875-1889, Sep. 2015.
74. D. Park, D. Ramanan, and C. Fowlkes, "Multiresolution models for object detection," in Proc. ECCV, 2010, pp. 241-254.
75. D. Park, C. L. Zitnick, D. Ramanan, and P. Dollár, "Exploring weak stabilization for motion feature extraction," in Proc. CVPR, Jun. 2013, pp. 2882-2889.
76. M. Pedersoli, J. Gonzàlez, A. D. Bagdanov, and J. J. Villanueva, "Recursive coarse-to-fine localization for fast object detection," in Proc. ECCV, 2010, pp. 280-293.
77. H. Poon and P. Domingos, "Sum-product networks: A new deep architecture," in Proc. UAI, Nov. 2011, pp. 689-690.
78. F. Porikli, "Integral histogram: A fast way to extract histograms in Cartesian spaces," in Proc. CVPR, Jun. 2005, pp. 829-836.
79. M. Ranzato, J. Susskind, V. Mnih, and G. Hinton, "On deep generative models with applications to recognition," in Proc. CVPR, Jun. 2011, pp. 2857-2864.
80. P. Sabzmeydani and G. Mori, "Detecting pedestrians by learning shapelet features," in Proc. CVPR, Jun. 2007, pp. 1-8.
81. W. R. Schwartz, A. Kembhavi, D. Harwood, and L. S. Davis, "Human detection using partial least squares analysis," in Proc. ICCV, Sep./Oct. 2009, pp. 24-31.
82. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun. (2013). "OverFeat: Integrated recognition, localization and detection using convolutional networks." [Online]. Available: http://arxiv.org/abs/1312.6229
83. P. Sermanet, K. Kavukcuoglu, S. Chintala, and Y. LeCun, "Pedestrian detection with unsupervised multi-stage feature learning," in Proc. CVPR, Jun. 2013, pp. 3626-3633.
84. V. D. Shet, J. Neumann, V. Ramesh, and L. S. Davis, "Bilattice-based logical reasoning for human detection," in Proc. CVPR, Jun. 2007, pp. 1-8.
85. K. Simonyan, A. Vedaldi, and A. Zisserman, "Deep inside convolutional networks: Visualising image classification models and saliency maps," in Proc. ICLR, 2014, pp. 1-8.
86. A. W. M. Smeulders, T. Gevers, K. E. A. van de Sande, and J. R. R. Uijlings, "Segmentation as selective search for object recognition," in Proc. ICCV, Nov. 2011, pp. 1879-1886.
87. P. Smolensky, Information Processing in Dynamical Systems: Foundations of Harmony Theory. Cambridge, MA, USA: MIT Press, 1986.
88. J. Sohl-Dickstein, P. Battaglino, and M. R. DeWeese, "Minimum probability flow learning," in Proc. ICML, 2011, pp. 1-8.
89. M. Sun and S. Savarese, "Articulated part-based model for joint object detection and pose estimation," in Proc. ICCV, Nov. 2011, pp. 723-730.
90. Y. Sun, X. Wang, and X. Tang, "Deep convolutional network cascade for facial point detection," in Proc. CVPR, Jun. 2013, pp. 3476-3483.
91. Y. Sun, X. Wang, and X. Tang, "Hybrid deep learning for face verification," in Proc. ICCV, Dec. 2013, pp. 1489-1496.
92. Y. Sun, X. Wang, and X. Tang, "Deep learning face representation from predicting 10, 000 classes," in Proc. CVPR, Jun. 2014, pp. 1891-1898.
93. Y. Tang, R. Salakhutdinov, and G. Hinton, "Robust Boltzmann machines for recognition and denoising," in Proc. CVPR, Jun. 2012, pp. 2264-2271.
94. O. Tuzel, F. Porikli, and P. Meer, "Pedestrian detection via classification on Riemannian manifolds," IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 10, pp. 1713-1727, Oct. 2008.
95. A. Vedaldi, V. Gulshan, M. Varma, and A. Zisserman, "Multiple kernels for object detection," in Proc. ICCV, Sep./Oct. 2009, pp. 606-613.
96. P. Viola and M. J. Jones, "Robust real-time face detection," Int. J. Comput. Vis., vol. 57, no. 2, pp. 137-154, Mar. 2004.
97. P. Viola, M. J. Jones, and D. Snow, "Detecting pedestrians using patterns of motion and appearance," Int. J. Comput. Vis., vol. 63, no. 2, pp. 153-161, 2005.
98. S. Walk, N. Majer, K. Schindler, and B. Schiele, "New features and insights for pedestrian detection," in Proc. CVPR, Jun. 2010, pp. 1030-1037.
99. X. Wang, T. X. Han, and S. Yan, "An HOG-LBP human detector with partial occlusion handling," in Proc. CVPR, Sep./Oct. 2009, pp. 32-39.
100. X. Wang, M. Wang, and W. Li, "Scene-specific pedestrian detection for static video surveillance," IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 2, pp. 361-374, Feb. 2014.
101. Y. Wang, D. Tran, and Z. Liao, "Learning hierarchical poselets for human parsing," in Proc. CVPR, Jun. 2011, pp. 1705-1712.
102. C. Wojek and B. Schiele, "A performance evaluation of single and multi-feature people detection," in Proc. DAGM, 2008, pp. 82-91.
103. C. Wojek, S. Walk, and B. Schiele, "Multi-cue onboard pedestrian detection," in Proc. CVPR, Jun. 2009, pp. 794-801.
104. B. Wu and R. Nevatia, "Detection of multiple, partially occluded humans in a single image by Bayesian combination of edgelet part detectors," in Proc. ICCV, Oct. 2005, pp. 90-97.
105. B. Wu and R. Nevatia, "Cluster boosted tree classifier for multi-view, multi-pose object detection," in Proc. ICCV, Oct. 2007, pp. 1-8.
106. B. Wu and R. Nevatia, "Detection and tracking of multiple, partially occluded humans by Bayesian combination of edgelet based part detectors," Int. J. Comput. Vis., vol. 75, no. 2, pp. 247-266, Nov. 2007.
107. B. Wu and R. Nevatia, "Detection and segmentation of multiple, partially occluded objects by grouping, merging, assigning part detection responses," Int. J. Comput. Vis., vol. 82, no. 2, pp. 185-204, Apr. 2009.
108. T. Wu and S.-C. Zhu, "A numerical study of the bottom-up and top-down inference processes in and-or graphs," Int. J. Comput. Vis., vol. 93, no. 2, pp. 226-252, Jun. 2011.
109. J. Yan, X. Zhang, Z. Lei, S. Liao, and S. Z. Li, "Robust multi-resolution pedestrian detection in traffic scenes," in Proc. CVPR, Jun. 2013, pp. 3033-3040.
110. Y. Yang and D. Ramanan, "Articulated pose estimation with flexible mixtures-of-parts," in Proc. CVPR, Jun. 2011, pp. 1385-1392.
111. M. D. Zeiler and R. Fergus. (2013). "Visualizing and understanding convolutional networks." [Online]. Available: http://arxiv.org/ abs/1311.2901
112. X. Zeng, W. Ouyang, M. Wang, and X. Wang, "Deep learning of scene-specific classifier for pedestrian detection," in Proc. ECCV, 2014, pp. 472-487.
113. X. Zeng, W. Ouyang, and X. Wang, "Multi-stage contextual deep learning for pedestrian detection," in Proc. ICCV, Dec. 2013, pp. 121-128.
114. R. Zhao, W. Ouyang, H. Li, and X. Wang, "Saliency detection by multi-context deep learning," in Proc. CVPR, 2015, pp. 1265-1274.
115. L. Zhu, Y. Chen, A. Yuille, and W. Freeman, "Latent hierarchical structural learning for object detection," in Proc. CVPR, Jun. 2010, pp. 1062-1069.
116. Q. Zhu, M.-C. Yeh, K.-T. Cheng, and S. Avidan, "Fast human detection using a cascade of histograms of oriented gradients," in Proc. CVPR, 2006, pp. 1491-1498.