What is a salient object? A dataset and a baseline model for salient object detection

IEEE Transactions on Image Processing

Volumn 24, Issue 2, 2015, Pages 742-756

  Borji, Ali ^a  

^a University of Wisconsin Milwaukee   (United States)

Author keywords

bottom up attention; explicit saliency; eye movements; regions of interest; Salient object detection

Indexed keywords

EYE MOVEMENTS; OBJECT RECOGNITION;

BOTTOM-UP ATTENTION; EXPLICIT SALIENCY; PREDICTION MODEL; REGIONS OF INTEREST; SALIENCY DETECTION; SALIENT OBJECT DETECTION; SALIENT REGION DETECTIONS; STATE OF THE ART;

OBJECT DETECTION;

EID: 84921759894     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2014.2383320     Document Type: Article

References (94)

1. T. Judd, K. Ehinger, F. Durand, and A. Torralba, "Learning to predict where humans look," in Proc. IEEE 12th Int. Conf. Comput. Vis., Sep./Oct. 2009, pp. 2106-2113.
2. L. Itti, C. Koch, and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254-1259, Nov. 1998.
3. N. D. B. Bruce and J. K. Tsotsos, "Saliency based on information maximization," in Advances in Neural Information Processing Systems 18. Cambridge, MA, USA: MIT Press, 2005.
4. X. Hou and L. Zhang, "Dynamic visual attention: Searching for coding length increments," in Advances in Neural Information Processing Systems 21. Red Hook, NY, USA: Curran Associates, 2008.
5. A. Garcia-Diaz, X. R. Fdez-Vidal, X. M. Pardo, and R. Dosil, "Decorrelation and distinctiveness provide with human-like saliency," in Proc. 11th Int. Conf. ACIVS, 2009, pp. 343-354.
6. B. W. Tatler, "The central fixation bias in scene viewing: Selecting an optimal viewing position independently of motor biases and image feature distributions," J. Vis., vol. 7, no. 14, pp. 1-17, 2007.
7. B. C. Russell, A. Torralba, K. P. Murphy, and W. T. Freeman, "LabelMe: A database and web-based tool for image annotation," Int. J. Comput. Vis., vol. 77, nos. 1-3, pp. 157-173, 2008.
8. M.-M. Cheng, G.-X. Zhang, N. J. Mitra, X. Huang, and S.-M. Hu, "Global contrast based salient region detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 409-416.
9. S. Goferman, L. Zelnik-Manor, and A. Tal, "Context-aware saliency detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2010, pp. 2376-2383.
10. R. Achanta, S. Hemami, F. Estrada, and S. Süsstrunk, "Frequency-tuned salient region detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2009, pp. 1597-1604.
11. M. Wang, J. Konrad, P. Ishwar, K. Jing, and H. Rowley, "Image saliency: From intrinsic to extrinsic context," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 417-424.
12. J. Feng, Y. Wei, L. Tao, C. Zhang, and J. Sun, "Salient object detection by composition," in Proc. IEEE Int. Conf. Comput. Vis., Nov. 2011, pp. 1028-1035.
13. M. J. Bravo and H. Farid, "A scale invariant measure of clutter," J. Vis., vol. 8, no. 1, 2008, Art. ID 23.
14. K.-Y. Chang, T.-L. Liu, H.-T. Chen, and S.-H. Lai, "Fusing generic objectness and visual saliency for salient object detection," in Proc. IEEE Int. Conf. Comput. Vis., Nov. 2011, pp. 914-921.
15. H. Jiang, J. Wang, Z. Yuan, T. Liu, N. Zheng, and S. Li, "Automatic salient object segmentation based on context and shape prior," in Proc. Brit. Mach. Vis. Conf., 2011, p. 7.
16. H. Jiang, J. Wang, Z. Yuan, Y. Wu, N. Zheng, and S. Li, "Salient object detection: A discriminative regional feature integration approach," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 2083-2090.
17. T. Liu, J. Sun, N.-N. Zheng, X. Tang, and H.-Y. Shum, "Learning to detect a salient object," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2007, pp. 1-8.
18. J. Li, M. D. Levine, X. An, and H. He, "Saliency detection based on frequency and spatial domain analyses," in Proc. BMVC, 2011, pp. 86.1-86.11.
19. Q. Yan, L. Xu, J. Shi, and J. Jia, "Hierarchical saliency detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 1155-1162.
20. V. Movahedi and J. H. Elder, "Design and perceptual validation of performance measures for salient object segmentation," in Proc. IEEE Comput. Soc. Conf. (CVPRW), Jun. 2010, pp. 49-56.
21. P. F. Felzenszwalb and D. P. Huttenlocher, "Efficient graph-based image segmentation," Int. J. Comput. Vis., vol. 59, no. 2, pp. 167-181, 2004.
22. S. Alpert, M. Galun, R. Basri, and A. Brandt, "Image segmentation by probabilistic bottom-up aggregation and cue integration," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2007, pp. 1-8.
23. A. Mishra, Y. Aloimonos, and C. L. Fah, "Active segmentation with fixation," in Proc. IEEE 12th Int. Conf. Comput. Vis., Sep./Oct. 2011, pp. 468-475.
24. C. Rother, V. Kolmogorov, and A. Blake, "'GrabCut': Interactive foreground extraction using iterated graph cuts," ACM Trans. Graph., vol. 23, no. 3, pp. 309-314, 2004.
25. A. Borji and L. Itti, "Exploiting local and global patch rarities for saliency detection," in Proc. IEEE Conf. CVPR, Jun. 2012, pp. 478-485.
26. A. Borji, M.-M. Cheng, H. Jiang, and J. Li. (2014). "Salient object detection: A survey." [Online]. Available: http://arxiv.org/abs/1411.5878
27. B. Alexe, T. Deselares, and V. Ferrari, "What is an object?" in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 73-80.
28. L. Elazary and L. Itti, "Interesting objects are visually salient," J. Vis., vol. 8, no. 3, 2008, Art. ID 3.
29. F. Perazzi, P. Krähenbühl, Y. Pritch, and A. Hornung, "Saliency filters: Contrast based filtering for salient region detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2012, pp. 733-740.
30. A. Borji, D. N. Sihite, and L. Itti, "Salient object detection: A benchmark," in Proc. 12th ECCV, 2012, pp. 414-429.
31. J. Shi and J. Malik, "Normalized cuts and image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888-905, Aug. 2000.
32. A. D. Hwang, H.-C. Wang, and M. Pomplun, "Semantic guidance of eye movements in real-world scenes," Vis. Res., vol. 51, no. 10, pp. 1192-1205, 2011.
33. A. Nuthman and J. M. Henderson, "Object-based attentional selection in scene viewing," J. Vis., vol. 10, no. 8, 2010, Art. ID 20.
34. P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, "Contour detection and hierarchical image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 5, pp. 898-916, May 2011.
35. D. R. Martin, C. C. Fowlkes, and J. Malik, "Learning to detect natural image boundaries using local brightness, color, and texture cues," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 5, pp. 530-549, May 2004.
36. X. Ren, C. C. Fowlkes, and J. Malik, "Scale-invariant contour completion using conditional random fields," in Proc. 10th IEEE Int. Conf. Comput. Vis., Oct. 2003, pp. 1214-1221.
37. R. Margolin, A. Tal, and L. Zelnik-Manor, "What makes a patch distinct?" in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2013, pp. 1139-1146.
38. X. Shen and Y. Wu, "A unified approach to salient object detection via low rank matrix recovery," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2012, pp. 853-860.
39. A. Borji, D. N. Sihite, and L. Itti, "Quantitative analysis of humanmodel agreement in visual saliency modeling: A comparative study," IEEE Trans. Image Process., vol. 22, no. 1, pp. 55-69, Jan. 2013.
40. D. Batra, A. Kowdle, D. Parikh, J. Luo, and T. Chen, "iCoseg: Interactive co-segmentation with intelligent scribble guidance," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 3169-3176.
41. D. Parkhurst, K. Law, and E. Niebur, "Modeling the role of salience in the allocation of overt visual attention," Vis. Res., vol. 42, no. 1, pp. 107-123, 2002.
42. Y. Li, X. Hou, C. Koch, J. M. Rehg, and A. L. Yuille, "The secrets of salient object segmentation," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 280-287.
43. J. Xu, M. Jiang, S. Wang, M. S. Kankanhalli, and Q. Zhao, "Predicting human gaze beyond pixels," J. Vis., vol. 14, no. 1, pp. 1-20, 2014.
44. K. Koehler, F. Guo, S. Zhang, and M. P. Eckstein, "What do saliency models predict?" J. Vis., vol. 14, no. 3, 2014, Art. ID 14.
45. Z. Yang, D. Li, J. Wang, and X. Li, "Saliency detection based on manifold learning," in Proc. 8th Int. Symp. Multispectral Image Process. Pattern Recognit., 2013, p. 891906.
46. M. Spain and P. Perona, "Measuring and predicting object importance," Int. J. Comput. Vis., vol. 91, no. 1, pp. 59-76, 2011.
47. A. C. Berg et al., "Understanding and predicting importance in images," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2012, pp. 3562-3569.
48. P. Isola, J. Xiao, A. Torralba, and A. Oliva, "What makes an image memorable?" in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2011, pp. 145-152.
49. R. Rosenholtz, Y. Li, and L. Nakano, "Measuring visual clutter," J. Vis., vol. 7, no. 2, 2007, Art. ID 17.
50. Y.-G. Jiang, Y. Wang, R. Feng, X. Xue, Y. Zheng, and H. Yang, "Understanding and predicting interestingness of videos," in Proc. AAAI, 2013, pp. 29-34.
51. H. Katti, K. Y. Bin, T. S. Chua, and M. Kankanhalli, "Pre-attentive discrimination of interestingness in images," in Proc. IEEE Int. Conf. Multimedia Expo (ICME), Jun./Apr. 2008, pp. 1433-1436.
52. M. Gygli, H. Grabner, H. Riemenschneider, F. Nater, and L. Van Gool, "The interestingness of images," in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Dec. 2013, pp. 1633-1640.
53. S. Dhar, V. Ordonez, and T. L. Berg, "High level describable attributes for predicting aesthetics and interestingness," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2011, pp. 1657-1664.
54. L. Itti and P. F. Baldi, "Bayesian surprise attracts human attention," in Advances in Neural Information Processing Systems 18. Cambridge, MA, USA: MIT Press, 2005, pp. 547-554.
55. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, Apr. 2004.
56. K. A. Ehinger, J. Xiao, A. Torralba, and A. Oliva, "Estimating scene typicality from human ratings and image features," in Proc. 33rd Annu. Cognit. Sci. Conf., 2011, p. 2562.
57. J. Vogel and B. Schiele, "A semantic typicality measure for natural scene categorization," in Pattern Recognition, LNCS 3175. Berlin, Germany: Springer-Verlag, 2004, pp. 195-203.
58. A. Farhadi, I. Endres, D. Hoiem, and D. Forsyth, "Describing objects by their attributes," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2009, pp. 1778-1785.
59. D. Parikh and K. Grauman, "Relative attributes," in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Nov. 2011, pp. 503-510.
60. X. Hou, J. Harel, and C. Koch, "Image signature: Highlighting sparse salient regions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 1, pp. 194-201, Jan. 2012.
61. H. J. Seo and P. Milanfar, "Static and space-time visual saliency detection by self-resemblance," J. Vis., vol. 9, no. 12, pp. 1-27, 2009, Art. ID 15.
62. E. Erdem and A. Erdem, "Visual saliency estimation by nonlinearly integrating features using region covariances," J. Vis., vol. 13, no. 4, pp. 1-20, 2013, Art. ID 11.
63. C. M. Masciocchi, S. Mihalas, D. Parkhurst, and E. Niebur, "Everyone knows what is interesting: Salient locations which should be fixated," J. Vis., vol. 9, no. 11, pp. 1-22, 2009.
64. A. Borji, D. N. Sihite, and L. Itti, "What stands out in a scene? A study of human explicit saliency judgment," Vis. Res., vol. 91, pp. 62-77, Oct. 2013.
65. L. Itti and M. A. Arbib, "Attention and the minimal subscene," in Action to Language via the Mirror Neuron System. Cambridge, U.K.: Cambridge Univ. Press, 2006, pp. 289-346.
66. G. Kulkarni et al., "Baby talk: Understanding and generating simple image descriptions," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2011, pp. 1601-1608.
67. U. Rutishauser, D. Walther, C. Koch, and P. Perona, "Is bottomup attention useful for object recognition?" in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2. Jun./Jul. 2004, pp. II-37-II-44.
68. J. R. R. Uijlings, K. E. A. van de Sande, T. Gevers, and A. W. M. Smeulders, "Selective search for object recognition," Int. J. Comput. Vis., vol. 104, no. 2, pp. 154-171, 2013.
69. A. Borji and L. Itti, "Scene classification with a sparse set of salient regions," in Proc. IEEE Int. Conf. Robot. Autom. (ICRA), May 2011, pp. 1902-1908.
70. C. Guo and L. Zhang, "A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression," IEEE Trans. Image Process., vol. 19, no. 1, pp. 185-198, Jan. 2010.
71. L. Itti, "Automatic foveation for video compression using a neurobiological model of visual attention," IEEE Trans. Image Process., vol. 13, no. 10, pp. 1304-1318, Oct. 2004.
72. Y.-F. Ma, X.-S. Hua, L. Lu, and H.-J. Zhang, "A generic framework of user attention model and its application in video summarization," IEEE Trans. Multimedia, vol. 7, no. 5, pp. 907-919, Oct. 2005.
73. P. Bodesheim, "Spectral clustering of ROIs for object discovery," in Proc. 33rd DAGM-Symp., LNCS 6835. 2011, pp. 450-455.
74. S. Goferman, A. Tal, and L. Zelnik-Manor, "Puzzle-like collage," Comput. Graph. Forum, vol. 29, no. 2, pp. 459-468, 2010.
75. J. Wang, L. Quan, J. Sun, X. Tang, and H.-Y. Shum, "Picture collage," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (CVPR), vol. 1. Jun. 2006, pp. 347-354.
76. A. Ninassi, O. Le Meur, P. Le Callet, and D. Barbba, "Does where you gaze on an image affect your perception of quality? Applying visual attention to image quality metric," in Proc. IEEE Conf. Image Process. (ICIP), vol. 2. Sep./Oct. 2007, pp. II-169-II-172.
77. H. Liu and I. Heynderickx, "Studying the added value of visual attention in objective image quality metrics based on eye movement data," in Proc. 16th IEEE ICIP, Nov. 2009, pp. 3097-3100.
78. M. Donoser, M. Urschler, M. Hirzer, and H. Bischof, "Saliency driven total variation segmentation," in Proc. IEEE 12th Int. Conf. Comput. Vis. (ICCV), Sep./Oct. 2009, pp. 817-824.
79. T. Chen, M.-M. Cheng, P. Tan, A. Shamir, and S.-M. Hu, "Sketch2Photo: Internet image montage," ACM Trans. Graph., vol. 28, no. 5, 2009, Art. ID 124.
80. S. Feng, D. Xu, and X. Yang, "Attention-driven salient edge(s) and region(s) extraction with application to CBIR," Signal Process., vol. 90, no. 1, pp. 1-15, 2010.
81. A. Borji, S. Frintrop, D. N. Sihite, and L. Itti, "Adaptive object tracking by learning background context," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Jun. 2012, pp. 23-30.
82. A. Y.-S. Chia et al., "Semantic colorization with internet images," ACM Trans. Graph., vol. 30, no. 6, 2011, Art. ID 156.
83. H. Liu, L. Zhang, and H. Huang, "Web-image driven best views of 3D shapes," Vis. Comput., vol. 28, no. 3, pp. 279-287, 2012.
84. M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman, "The PASCAL visual object classes (VOC) challenge," Int. J. Comput. Vis., vol. 88, no. 2, pp. 303-338, 2010.
85. S. Stalder, H. Grabner, and L. Van Gool, "Dynamic objectness for adaptive tracking," in Proc. 11th ACCV, 2013, pp. 43-56.
86. J. Li, M. D. Levine, X. An, X. Xu, and H. He, "Visual saliency based on scale-space analysis in the frequency domain," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 4, pp. 996-1010, Apr. 2013.
87. A. Karpathy, S. Miller, and L. Fei-Fei, "Object discovery in 3D scenes via shape analysis," in Proc. IEEE ICRA, May 2013, pp. 2088-2095.
88. A. Borji, H. R. Tavakoli, D. N. Sihite, and L. Itti, "Analysis of scores, datasets, and models in visual saliency prediction," in Proc. IEEE ICCV, Dec. 2013, pp. 921-928.
89. S. Frintrop, G. M. Garcia, and A. B. Cremers, "A cognitive approach for object discovery," in Proc. 22nd ICPR, Aug. 2014, pp. 2329-2334.
90. D. Meger et al., "Curious George: An attentive semantic robot," Robot. Auto. Syst., vol. 56, no. 6, pp. 503-511, 2008.
91. B. Jiang, L. Zhang, H. Lu, C. Yang, and M.-H. Yang, "Saliency detection via absorbing Markov chain," in Proc. IEEE ICCV, Dec. 2013, pp. 1665-1672.
92. P. Jiang, H. Ling, J. Yu, and J. Peng, "Salient region detection by UFO: Uniqueness, focusness and objectness," in Proc. IEEE ICCV, Dec. 2013, pp. 1976-1983.
93. A. Borji, D. N. Sihite, and L. Itti, "What/where to look next? Modeling top-down visual attention in complex interactive environments," IEEE Trans. Syst., Man, Cybern., Syst., vol. 44, no. 5, pp. 523-538, May 2014.
94. J. Kim, D. Han, Y.-W. Tai, and J. Kim, "Salient region detection via high-dimensional color transform," in Proc. IEEE CVPR, Jun. 2014, pp. 883-890.