Advanced Deep-Learning Techniques for Salient and Category-Specific Object Detection: A Survey

IEEE Signal Processing Magazine

Volumn 35, Issue 1, 2018, Pages 84-100

  Han, Junwei ^a   Zhang, Dingwen ^b   Cheng, Gong ^a   Liu, Nian ^a   Xu, Dong ^c  

^a NORTHWESTERN POLYTECHNICAL UNIVERSITY   (China)

^b CARNEGIE MELLON UNIVERSITY   (United States)

^c UNIVERSITY OF SYDNEY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; CONTENT BASED RETRIEVAL; DEEP LEARNING; IMAGE RETRIEVAL; LEARNING ALGORITHMS; LEARNING SYSTEMS; OBJECT RECOGNITION;

CATEGORY SPECIFICS; CONTENT BASED IMAGE RETRIEVAL; EVALUATION METRICS; LEARNING TECHNIQUES; MODERN TECHNIQUES; SALIENT OBJECT DETECTION; SCENE UNDERSTANDING; VISION COMMUNITIES;

OBJECT DETECTION;

EID: 85040657540     PISSN: 10535888     EISSN: None     Source Type: Journal    
DOI: 10.1109/MSP.2017.2749125     Document Type: Review

References (99)

1. B. Alexe, T. Deselaers, V. Ferrari, "Measuring the objectness of image windows,"IEEE Trans. Pattern Anal. Machine Intell., vol. 34, no. 11, pp. 2189-2202,2012.
2. W. Kuo, B. Hariharan, J. Malik, "Deepbox: Learning objectness with convolutionalnetworks," in Proc. IEEE Int. Conf. Computer Vision, 2015, pp. 2479-2487.
3. T. Deselaers, B. Alexe, V. Ferrari, "Weakly supervised localization andlearning with generic knowledge," Int. J. Computer Vision, vol. 100, no. 3, pp. 275-293, 2012.
4. Y. Wu, J. Lim, M.-H. Yang, "Object tracking benchmark," IEEE Trans.Pattern Anal. Machine Intell., vol. 37, no. 9, pp. 1834-1848, 2015.
5. T. Liu, Z. Yuan, J. Sun, J. Wang, N. Zheng, X. Tang, H.-Y. Shum, "Learningto detect a salient object," IEEE Trans. Pattern Anal. Machine Intell., vol. 33, no. 2,pp. 353-367, 2011.
6. M.-M. Cheng, N. J. Mitra, X. Huang, P. H. S. Torr, S.-M. Hu, "Global contrastbased salient region detection," IEEE Trans. Pattern Anal. Machine Intell.,vol. 37, no. 3, pp. 569-582, 2015.
7. A. Borji and L. Itti, "State-of-the-art in visual attention modeling," IEEE Trans.Pattern Anal. Machine Intell., vol. 35, no. 1, pp. 185-207, 2013.
8. X. Li, T. Uricchio, L. Ballan, M. Bertini, C. G. Snoek, A. D. Bimbo,"Socializing the semantic gap: A comparative survey on image tag assignment,refinement, retrieval," ACM Comput. Surveys, vol. 49, no. 1, pp. 14, 2016.
9. B. Hariharan, P. Arbelaéz, R. Girshick, J. Malik, "Hypercolumns for objectsegmentation and fine-grained localization," in Proc. IEEE Conf. Computer Visionand Pattern Recognition, 2015, pp. 447-456.
10. R. Girshick, J. Donahue, T. Darrell, J. Malik, "Rich feature hierarchies foraccurate object detection and semantic segmentation," in Proc. IEEE Conf.Computer Vision and Pattern Recognition, 2014, pp. 580-587.
11. W. Ouyang, X. Wang, X. Zeng, S. Qiu, P. Luo, Y. Tian, H. Li, S. Yang, Z.Wang, C.-C. Loy, "Deepid-net: Deformable deep convolutional neural networksfor object detection," in Proc. IEEE Conf. Computer Vision and PatternRecognition, 2015, pp. 2403-2412.
12. J. Tighe, M. Niethammer, S. Lazebnik, "Scene parsing with objectinstance inference using regions and per-exemplar detectors," Int. J. ComputerVision, vol. 112, no. 2, pp. 150-171, 2015.
13. A. Prest, C. Schmid, V. Ferrari, "Weakly supervised learning of interactionsbetween humans and objects," IEEE Trans. Pattern Anal. Machine Intell.,vol. 34, no. 3, pp. 601-614, 2012.
14. Y. LeCun, F. J. Huang, L. Bottou, "Learning methods for generic objectrecognition with invariance to pose and lighting," in Proc. IEEE Conf. ComputerVision and Pattern Recognition, 2004, pp. 97-104.
15. C. Szegedy, A. Toshev, D. Erhan, "Deep neural networks for object detection,"in Proc. Advances in Neural Information Processing Systems, 2013, pp.2553-2561.
16. M. Everingham, S. A. Eslami, L. Van Gool, C. K. Williams, J. Winn, A.Zisserman, "The Pascal visual object classes challenge: A retrospective," Int. J.Computer Vision, vol. 111, no. 1, pp. 98-136, 2015.
17. A. Ghodrati, A. Diba, M. Pedersoli, T. Tuytelaars, L. Van Gool,"Deepproposal: Hunting objects by cascading deep convolutional layers," in Proc.IEEE Int. Conf. Computer Vision, 2015, pp. 2578-2586.
18. R. Girshick, "Fast R-CNN," in Proc. IEEE Int. Conf. Computer Vision, 2015,pp. 1440-1448.
19. J. Redmon, S. Divvala, R. Girshick, A. Farhadi, "You only look once:Unified, real-time object detection," arXiv Preprint, arXiv:1506.02640, 2015.
20. N. Liu and J. Han, "DHSNet: Deep hierarchical saliency network for salientobject detection," in Proc. IEEE Conf. Computer Vision and Pattern Recognition,2016, pp. 678-686.
21. K. Fukushima, "Neocognitron: A self-organizing neural network model for amechanism of pattern recognition unaffected by shift in position," Biol. Cybernet.,vol. 36, no. 4, pp. 193-202, 1980.
22. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard,and L. D. Jackel, "Backpropagation applied to handwritten zip code recognition,"Neural Comput., vol. 1, no. 4, pp. 541-551, 1989.
23. Y. LeCun, Y. Bengio, G. Hinton, "Deep learning," Nature, vol. 521, no.7553, pp. 436-444, 2015.
24. A. Krizhevsky, I. Sutskever, G. E. Hinton, "ImageNet classification withdeep convolutional neural networks," in Proc. Advances in Neural InformationProcessing Systems, 2012, pp. 1097-1105.
25. K. Simonyan and A. Zisserman, "Very deep convolutional networks for largescaleimage recognition," arXiv Preprint, arXiv:1409.1556, 2014.
26. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V.Vanhoucke, A. Rabinovich, "Going deeper with convolutions," in Proc. IEEEConf. Computer Vision and Pattern Recognition, 2015, pp. 1-9.
27. K. He, X. Zhang, S. Ren, J. Sun, "Deep residual learning for image recognition,"in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2016, pp.770-778.
28. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, "ImageNet: Alarge-scale hierarchical image database," in Proc. IEEE Conf. Computer Vision andPattern Recognition, 2009, pp. 248-255.
29. J. R. Uijlings, K. E. Van De Sande, T. Gevers, A. W. Smeulders,"Selective search for object recognition," Int. J. Computer Vision, vol. 104, no. 2,pp. 154-171, 2013.
30. P. Krähenbühl and V. Koltun, "Geodesic object proposals," in Proc. EuropeanConf. Computer Vision, 2014, pp. 725-739.
31. L. Bazzani, A. Bergamo, D. Anguelov, L. Torresani, "Self-taught objectlocalization with deep networks," in Proc. IEEE Winter Conf. Applications ofComputer Vision, 2016, pp. 1-9.
32. X. Hou, L. Zhang, "Saliency detection: A spectral residual approach," inProc. IEEE Conf. Computer Vision and Pattern Recognition, 2007, pp. 1-8.
33. P. O. Pinheiro, R. Collobert, P. Dollar, "Learning to segment object candidates,"in Proc. Advances in Neural Information Processing Systems, 2015, pp.1990-1998.
34. P. O. Pinheiro, T.-Y. Lin, R. Collobert, P. Dollár, "Learning to refine objectsegments," in Proc. European Conf. Computer Vision, 2016, pp. 75-91.
35. C. L. Zitnick, P. Dollár, "Edge boxes: Locating object proposals fromedges," in Proc. European Conf. Computer Vision, 2014, pp. 391-405.
36. H. Hu, S. Lan, Y. Jiang, Z. Cao, F. Sha, "FastMask: Segment multi-scaleobject candidates in one shot," arXiv Preprint, arXiv:1612.08843, 2016.
37. D. Erhan, C. Szegedy, A. Toshev, D. Anguelov, "Scalable object detectionusing deep neural networks," in Proc. IEEE Conf. Computer Vision and PatternRecognition, 2014, pp. 2147-2154.
38. C. Szegedy, S. Reed, D. Erhan, D. Anguelov, S. Ioffe, "Scalable, highqualityobject detection," arXiv Preprint, arXiv:1412.1441, 2014.
39. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, "Rethinking theinception architecture for computer vision," in Proc. IEEE Conf. Computer Visionand Pattern Recognition, 2016, pp. 2818-2826.
40. S. Ren, K. He, R. Girshick, J. Sun, "Faster R-CNN: Towards real-timeobject detection with region proposal networks," in Proc. Advances in NeuralInformation Processing Systems, 2015, pp. 91-99.
41. T. Kong, A. Yao, Y. Chen, F. Sun, "HyperNet: Towards accurate regionproposal generation and joint object detection," in Proc. IEEE Conf. ComputerVision and Pattern Recognition, 2016, pp. 845-853.
42. H. Li, Y. Liu, W. Ouyang, X. Wang, "Zoom out-and-in network with recursivetraining for object proposal," arXiv Preprint, arXiv:1702.05711, 2017.
43. A. Borji, M.-M. Cheng, H. Jiang, J. Li, "Salient object detection: A survey,"arXiv Preprint, arXiv:1411.5878, 2014.
44. A. Borji, H. R. Tavakoli, D. N. Sihite, L. Itti, "Analysis of scores, data sets,and models in visual saliency prediction," in Proc. IEEE Int. Conf. ComputerVision, 2013, pp. 921-928.
45. J. Han, D. Zhang, X. Hu, L. Guo, J. Ren, F. Wu, "Background prior-basedsalient object detection via deep reconstruction residual," IEEE Trans. CircuitsSystems Video Technol., vol. 25, no. 8, pp. 1309-1321, 2015.
46. L. Wang, H. Lu, X. Ruan, M.-H. Yang, "Deep networks for saliency detectionvia local estimation and global search," in Proc. IEEE Conf. Computer Visionand Pattern Recognition, 2015, pp. 3183-3192.
47. G. Lee, Y.-W. Tai, J. Kim, "Deep saliency with encoded low level distancemap and high level features," in Proc. IEEE Conf. Computer Vision and PatternRecognition, 2016, pp. 660-668.
48. G. Li, Y. Yu, "Deep contrast learning for salient object detection," in Proc.IEEE Conf. Computer Vision and Pattern Recognition, 2016, pp. 660-668.
49. L. Wang, L. Wang, H. Lu, P. Zhang, X. Ruan, "Saliency detection withrecurrent fully convolutional networks," in Proc. European Conf. Computer Vision,2016, pp. 825-841.
50. J. Yang, M.-H. Yang, "Top-down visual saliency via joint CRF and dictionarylearning," in Proc. IEEE Conf. Computer Vision and Pattern Recognition,2012, pp. 2296-2303.
51. S. He, R. W. Lau, Q. Yang, "Exemplar-driven top-down saliency detectionvia deep association," in Proc. IEEE Conf. Computer Vision and PatternRecognition, 2016, pp. 5723-5732.
52. H. Cholakkal, J. Johnson, D. Rajan, "Backtracking ScSPM image classifierfor weakly supervised top-down saliency," in Proc. IEEE Conf. Computer Visionand Pattern Recognition, 2016, pp. 5278-5287.
53. J. Zhang, Z. Lin, J. Brandt, X. Shen, S. Sclaroff, "Top-down neural attentionby excitation backprop," in Proc. European Conf. Computer Vision, 2016, pp.543-559.
54. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, D. Ramanan, "Objectdetection with discriminatively trained part-based models," IEEE Trans. PatternAnal. Machine Intell., vol. 32, no. 9, pp. 1627-1645, 2010.
55. K. He, X. Zhang, S. Ren, J. Sun, "Spatial pyramid pooling in deep convolutionalnetworks for visual recognition," IEEE Trans. Pattern Anal. MachineIntell., vol. 37, no. 9, pp. 1904-1916, 2015.
56. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, Y. LeCun,"Overfeat: Integrated recognition, localization and detection using convolutional networks,"arXiv Preprint, arXiv:1312.6229, 2013.
57. J. Redmon, S. Divvala, R. Girshick, A. Farhadi, "You only look once:Unified, real-time object detection," in Proc. IEEE Conf. Computer Vision andPattern Recognition, 2016, pp. 779-788.
58. W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, A. C. Berg,"SSD: Single shot multibox detector," in Proc. European Conf. Computer Vision,2016, pp. 21-37.
59. M. Najibi, M. Rastegari, L. S. Davis, "G-CNN: An iterative grid basedobject detector," in Proc. IEEE Conf. Computer Vision and Pattern Recognition,2016, pp. 2369-2377.
60. J. Redmon and A. Farhadi, "YOLO9000: Better, faster, stronger," arXivPreprint, arXiv:1612.08242, 2016.
61. A. Shrivastava, A. Gupta, R. Girshick, "Training region-based objectdetectors with online hard example mining," in Proc. IEEE Conf. Computer Visionand Pattern Recognition, 2016, pp. 761-769.
62. G. Cheng, P. Zhou, J. Han, "RIFD-CNN: Rotation-invariant and fisher discriminativeconvolutional neural networks for object detection," in Proc. IEEEConf. Computer Vision and Pattern Recognition, 2016, pp. 2884-2893.
63. S. Bell, C. Lawrence Zitnick, K. Bala, R. Girshick, "Inside-outside net:Detecting objects in context with skip pooling and recurrent neural networks," inProc. IEEE Conf. Computer Vision and Pattern Recognition, 2016, pp. 2874-2883.
64. S. Gidaris, N. Komodakis, "Object detection via a multi-region and semanticsegmentation-aware CNN model," in Proc. IEEE Int. Conf. Computer Vision,2015, pp. 1134-1142.
65. A. Shrivastava, A. Gupta, "Contextual priming and feedback for fasterR-CNN," in Proc. European Conf. Computer Vision, 2016, pp. 330-348.
66. J. Feng, Y. Wei, L. Tao, C. Zhang, J. Sun, "Salient object detection bycomposition," in Proc. IEEE Int. Conf. Computer Vision, 2011, pp. 1028-1035.
67. M.-M. Cheng, Z. Zhang, W.-Y. Lin, P. Torr, "BING: Binarized normedgradients for objectness estimation at 300fps," in Proc. IEEE Conf. ComputerVision and Pattern Recognition, 2014, pp. 3286-3293.
68. K.-Y. Chang, T.-L. Liu, H.-T. Chen, S.-H. Lai, "Fusing generic objectnessand visual saliency for salient object detection," in Proc. IEEE Int. Conf. ComputerVision, 2011, pp. 914-921.
69. P. Jiang, H. Ling, J. Yu, J. Peng, "Salient region detection by UFO:Uniqueness, focusness and objectness," in Proc. IEEE Int. Conf. Computer Vision,2013, pp. 1976-1983.
70. Y. Li, X. Hou, C. Koch, J. M. Rehg, A. L. Yuille, "The secrets of salientobject segmentation," in Proc. IEEE Conf. Computer Vision and PatternRecognition, 2014, pp. 280-287.
71. T. Malisiewicz, A. Gupta, A. A. Efros, "Ensemble of exemplar-svms forobject detection and beyond," in Proc. IEEE Int. Conf. Computer Vision, 2011, pp.89-96.
72. M. Bar, "The proactive brain: Using analogies and associations to generate predictions,"Trends Cogn. Sci., vol. 11, no. 7, pp. 280-289, 2007.
73. R. M. Nosofsky, "Attention, similarity, the identification-categorizationrelationship," J. Exp. Psychol., vol. 115, no. 1, pp. 39, 1986.
74. A. Borji, "Boosting bottom-up and top-down visual features for saliency estimation,"in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2012, pp.438-445.
75. T. Judd, K. Ehinger, F. Durand, A. Torralba, "Learning to predict wherehumans look," in Proc. IEEE Int. Conf. Computer Vision, 2009, pp. 2106-2113.
76. D. Zhang, D. Meng, L. Zhao, J. Han, "Bridging saliency detection to weaklysupervised object detection based on self-paced curriculum learning," in Proc.Int. Joint Conf. Artificial Intelligence, 2016, pp. 3538-3544.
77. B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, A. Torralba, "Learning deepfeatures for discriminative localization," in Proc. IEEE Conf. Computer Vision andPattern Recognition, 2016, pp. 2921-2929.
78. M. Everingham, L. Van Gool, C. K. Williams, J. Winn, A. Zisserman,"The pascal visual object classes (VOC) challenge," Int. J. Computer Vision, vol.88, no. 2, pp. 303-338, 2010.
79. M. Everingham, A. Zisserman, C. K. Williams, L. Van Gool, M. Allan, C. M.Bishop, O. Chapelle, N. Dalal, T. Deselaers, G. Dorkó, "The PASCAL visualobject classes challenge 2007 (VOC2007) results," 2007. [Online]. Available: Http://host.robots.ox.ac.uk/pascal/VOC/voc2007/workshop/index.html
80. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár,and C. L. Zitnick, "Microsoft COCO: Common objects in context," in Proc.European Conf. Computer Vision, 2014, pp. 740-755.
81. Q. Yan, L. Xu, J. Shi, J. Jia, "Hierarchical saliency detection," in Proc.IEEE Conf. Computer Vision and Pattern Recognition, 2013, pp. 1155-1162.
82. V. Movahedi and J. H. Elder, "Design and perceptual validation of performancemeasures for salient object segmentation," in Proc. IEEE ComputerSociety Conf. Computer Vision and Pattern Recognition Workshops, 2010, pp.49-56.
83. G. Li and Y. Yu, "Visual saliency based on multiscale deep features," in Proc.IEEE Conf. Computer Vision and Pattern Recognition, 2015, pp. 5455-5463.
84. C. Yang, L. Zhang, H. Lu, X. Ruan, M.-H. Yang, "Saliency detection viagraph-based manifold ranking," in Proc. IEEE Conf. Computer Vision and PatternRecognition, 2013, pp. 3166-3173.
85. M.-M. Cheng, N. J. Mitra, X. Huang, S.-M. Hu, "Salientshape: Groupsaliency in image collections," Visual Computer, vol. 30, no. 4, pp. 443-453,2014.
86. J. Pont-Tuset, P. Arbelaez, J. T. Barron, F. Marques, J. Malik,"Multiscale combinatorial grouping for image segmentation and object proposalgeneration," IEEE Trans. Pattern Anal. Machine Intell., vol. 39, no. 1, pp. 128-140, 2017.
87. R. Zhao, W. Ouyang, H. Li, X. Wang, "Saliency detection by multi-contextdeep learning," in Proc. IEEE Conf. Computer Vision and Pattern Recognition,2015, pp. 1265-1274.
88. H. Jiang, J. Wang, Z. Yuan, Y. Wu, N. Zheng, S. Li, "Salient object detection:A discriminative regional feature integration approach," in Proc. IEEE Conf.Computer Vision and Pattern Recognition, 2013, pp. 2083-2090.
89. P. Mehrani and O. Veksler, "Saliency segmentation based on learning andgraph cut refinement," in Proc. British Machine Vision Conf., 2010, pp. 1-12.
90. P. Dollár, R. Appel, S. Belongie, P. Perona, "Fast feature pyramids forobject detection," IEEE Trans. Pattern Anal. Machine Intell., vol. 36, no. 8, pp.1532-1545, 2014.
91. Q. Hou, M.-M. Cheng, X. Hu, Z. Tu, A. Borji, "Deeply supervised salientobject detection with short connections," in Proc. IEEE Conf. Computer Vision andPattern Recognition, 2017, pp. 3203-3212.
92. D. Zhang, J. Han, J. Han, L. Shao, "Cosaliency detection based on intrasaliencyprior transfer and deep intersaliency mining," IEEE Trans. Neural NetworksLearning Syst., vol. 27, no. 6, pp. 1163-1176, 2016.
93. D. Zhang, J. Han, C. Li, J. Wang, X. Li, "Detection of co-salient objects bylooking deep and wide," Int. J. Computer Vision, vol. 120, no. 2, pp. 215-232,2016.
94. D. Zhang, D. Meng, J. Han, "Co-saliency detection via a self-paced multiple-instance learning framework," IEEE Trans. Pattern Anal. Machine Intell., vol.39, no. 5, pp. 865-878, 2017.
95. D. Zhang, J. Han, L. Jiang, S. Ye, X. Chang, "Revealing event saliency inunconstrained video collection," IEEE Trans. Image Processing, vol. 26, no. 4, pp.1746-1758, 2017.
96. D. Zhang, L. Yang, D. Meng, D. Xu, J. Han, "SPFTN: A self-paced finetuningnetwork for segmenting objects in weakly labelled videos," in Proc. IEEEConf. Computer Vision and Pattern Recognition, 2017, pp. 4429-4437.
97. D. Zhang, J. Han, Y. Yang, D. Huang, "Learning category-specific 3Dshape models from weakly labeled 2D images," in Proc. IEEE Conf. ComputerVision and Pattern Recognition, 2017, pp. 4573-4581.
98. Y. Bengio, J. Louradour, R. Collobert, J. Weston, "Curriculum learning,"in Proc. Annu. Int. Conf. Machine Learning, 2009, pp. 41-48.
99. Y. Yuan, X. Liang, X. Wang, D.-Y. Yeung, A. Gupta, "Temporal dynamicgraph LSTM for action-driven video object detection," arXiv Preprint,arXiv:1708.00666, 2017.