Spatiochromatic Context Modeling for Color Saliency Analysis

IEEE Transactions on Neural Networks and Learning Systems

Volumn 27, Issue 6, 2016, Pages 1177-1189

  Zhang, Jun ^a   Wang, Meng ^a   Zhang, Shengping ^b   Li, Xuelong ^c   Wu, Xindong ^a,d  

^a HEFEI UNIVERSITY OF TECHNOLOGY   (China)

^b HARBIN INSTITUTE OF TECHNOLOGY   (China)

^c XI AN INSTITUTE OF OPTICS AND PRECISION MECHANICS   (China)

^d University of Vermont ^*  (United States)

Author keywords

Color; context; region and boundary; spatiochromatic filtering; Visual saliency

Indexed keywords

IMAGE SEGMENTATION; OBJECT RECOGNITION; VISUALIZATION;

CHROMATIC STIMULI; COGNITIVE PSYCHOLOGY; NORMALIZATION METHODS; ORIENTATION INTERACTION; PERCEPTUAL METRICS; PRIMARY VISUAL CORTEX; SALIENT OBJECT DETECTION; STATE-OF-THE-ART APPROACH;

COLOR;

EID: 84940704075     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2015.2464316     Document Type: Article

References (87)

1. E. C. Potter, B. Wyble, C. E. Hagmann, E. S. McCourt, "Detecting meaning in RSVP at 13 ms per picture," Attention, Perception, Psychophys., vol. 76, no. 2, pp. 270-279, Feb. 2014.
2. W. Wang, Y. Song, A. Zhang, "Semantics-based image retrieval by region saliency," in Proc. Int. Conf. Content-Based Image Video Retr., London, U.K., Jul. 2002, pp. 29-37.
3. Y. Yang, Y. Yang, H. T. Shen, "Effective transfer tagging from image to video," ACM Trans. Multimedia Comput., Commun., Appl., vol. 9, no. 2, p. 14, May 2013.
4. F. Yu, J. Cheng, Z. Li, H. Lu, "Saliency cuts: An automatic approach to object segmentation," in Proc. 19th Int. Conf. Pattern Recognit., Tampa, FL, USA, Dec. 2008, pp. 1-4.
5. M.-M. Cheng, G.-X. Zhang, N. J. Mitra, X. Huang, S.-M. Hu, "Global contrast based salient region detection," in Proc. 24th IEEE Conf. Comput. Vis. Pattern Recognit., Colorado Springs, CO, USA, Jun. 2011, pp. 409-416.
6. H. Zheng, G. Lefebvre, C. Laurent, "Fast-learning adaptive-subspace self-organizing map: An application to saliency-based invariant image feature construction," IEEE Trans. Neural Netw., vol. 19, no. 5, pp. 746-757, May 2008.
7. G. Sharma, F. Jurie, C. Schmid, "Discriminative spatial saliency for image classification," in Proc. 24th IEEE Conf. Comput. Vis. Pattern Recognit., Providence, RI, USA, Jun. 2012, pp. 3506-3513.
8. T.-H. Tsai, W.-H. Cheng, C.-W. You, M.-C. Hu, A. W. Tsui, H.-Y. Chi, "Learning and recognition of on-premise signs from weakly labeled street view images," IEEE Trans. Image Process., vol. 23, no. 3, pp. 1047-1059, Mar. 2014.
9. S. Han and N. Vasconcelos, "Object recognition with hierarchical discriminant saliency networks," Frontiers Comput. Neurosci., vol. 8, p. 109, Sep. 2014.
10. D. Gao and N. Vasconcelos, "Bottom-up saliency is a discriminant process," in Proc. 11th IEEE Int. Conf. Comput. Vis., Rio de Janeiro, Brazil, Oct. 2007, pp. 1-6.
11. N. D. B. Bruce and J. K. Tsotsos, "Saliency based on information maximization," in Proc. 19th Annu. Conf. Neural Inf. Process. Syst., Montreal, QC, Canada, Dec. 2005, pp. 155-162.
12. W.-H. Cheng, W.-T. Chu, J.-L. Wu, "A visual attention based region-of-interest determination framework for video sequences," IEICE Trans. Inf. Syst., vol. E88-D, no. 7, pp. 1578-1586, 2005.
13. X. Hou and L. Zhang, "Dynamic visual attention: Searching for coding length increments," in Proc. 22nd Annu. Conf. Neural Inf. Process. Syst., Vancouver, BC, Canada, Dec. 2008, pp. 681-688.
14. L. Itti, C. Koch, E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254-1259, Nov. 1998.
15. D. Gao, V. Mahadevan, N. Vasconcelos, "The discriminant centersurround hypothesis for bottom-up saliency," in Proc. 22nd Annu. Conf. Neural Inf. Process. Syst., Vancouver, BC, Canada, Dec. 2008, pp. 497-504.
16. W.-H. Cheng, C.-W. Wang, J.-L. Wu, "Video adaptation for small display based on content recomposition," IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 1, pp. 43-58, Jan. 2007.
17. E. Vazquez, T. Gevers, M. Lucassen, J. van de Weijer, R. Baldrich, "Saliency of color image derivatives: A comparison between computational models and human perception," J. Opt. Soc. Amer. A, vol. 27, no. 3, pp. 613-621, 2010.
18. H. R. Tavakoli, E. Rahtu, J. Heikkilä, "Fast and efficient saliency detection using sparse sampling and kernel density estimation," in Proc. 17th Scandin. Conf. Image Anal., Ystad, Sweden, May 2011, pp. 666-675.
19. E. Erdem and A. Erdem, "Visual saliency estimation by nonlinearly integrating features using region covariances," J. Vis., vol. 13, no. 4, pp. 1-20, Mar. 2013.
20. X. Hou and L. Zhang, "Saliency detection: A spectral residual approach," in Proc. 20th IEEE Conf. Comput. Vis. Pattern Recognit., Minneapolis, MN, USA, Jun. 2007, pp. 1-8.
21. R. Achanta, S. Hemami, F. Estrada, S. Susstrunk, "Frequency-tuned salient region detection," in Proc. 22nd IEEE Conf. Comput. Vis. Pattern Recognit., Miami, FL, USA, Jun. 2009, pp. 1597-1604.
22. S. Lu, C. Tan, J.-H. Lim, "Robust and efficient saliency modeling from image co-occurrence histograms," IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 1, pp. 195-201, Jan. 2014.
23. F. Perazzi, P. Krähenbühl, Y. Pritch, A. Hornung, "Saliency filters: Contrast based filtering for salient region detection," in Proc. 25th IEEE Conf. Comput. Vis. Pattern Recognit., Providence, RI, USA, Jun. 2012, pp. 733-740.
24. X. Shen and Y. Wu, "A unified approach to salient object detection via low rank matrix recovery," in Proc. 25th IEEE Conf. Comput. Vis. Pattern Recognit., Providence, RI, USA, Jun. 2012, pp. 853-860.
25. Y. Wei, F. Wen, W. Zhu, J. Sun, "Geodesic saliency using background priors," in Proc. 12th Eur. Conf. Comput. Vis., Florence, Italy, Oct. 2012, pp. 29-42.
26. L. Zhang, M. H. Tong, T. K. Marks, H. Shan, G. W. Cottrell, "SUN: A Bayesian framework for saliency using natural statistics," J. Vis., vol. 8, no. 7, p. 32, Dec. 2008.
27. C. Scharfenberger, A. Wong, K. Fergani, D. A. Clausi, J. S. Zelek, "Statistical textural distinctiveness for salient region detection in natural images," in Proc. 26th IEEE Conf. Comput. Vis. Pattern Recognit., Portland, OR, USA, Jun. 2013, pp. 979-986.
28. W. Zhu, S. Liang, Y. Wei, J. Sun, "Saliency optimization from robust background detection," in Proc. 27th IEEE Conf. Comput. Vis. Pattern Recognit., Columbus, OH, USA, Jun. 2014, pp. 2814-2821.
29. Y. Liu, X. Li, L. Wang, Y. Niu, "Interpolation-tuned salient region detection," Sci. China Inf. Sci., vol. 57, no. 1, pp. 0121041-0121049, 2014.
30. D. Chen and H. Chu, "Scale-invariant amplitude spectrum modulation for visual saliency detection," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 8, pp. 1206-1214, Aug. 2012.
31. S. Goferman, L. Zelnik-Manor, A. Tal, "Context-aware saliency detection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 10, pp. 1915-1926, Oct. 2012.
32. Q. Zhao and C. Koch, "Learning saliency-based visual attention: A review," Signal Process., vol. 93, no. 6, pp. 1401-1407, Jun. 2013.
33. Q. Wang, Y. Yuan, P. Yan, X. Li, "Saliency detection by multipleinstance learning," IEEE Trans. Cybern., vol. 43, no. 2, pp. 660-672, Apr. 2013.
34. L. Zhang, Y. Xia, R. Ji, X. Li, "Spatial-aware object-level saliency prediction by learning graphlet hierarchies," IEEE Trans. Ind. Electron, vol. 62, no. 2, pp. 1301-1308, Feb. 2015.
35. F. Zhu and L. Shao, "Weakly-supervised cross-domain dictionary learning for visual recognition," Int. J. Comput. Vis., vol. 109, nos. 1-2, pp. 42-59, Aug. 2014.
36. J. Han, S. He, X. Qian, D. Wang, L. Guo, T. Liu, "An objectoriented visual saliency detection framework based on sparse coding representations," IEEE Trans. Circuits Syst. Video Technol., vol. 23, no. 12, pp. 2009-2021, Dec. 2013.
37. J. Han et al., "Efficient, simultaneous detection of multi-class geospatial targets based on visual saliency modeling and discriminative learning of sparse coding," ISPRS J. Photogramm. Remote Sens., vol. 89, pp. 37-48, Mar. 2014.
38. X. Li, H. Lu, L. Zhang, X. Ruan, M.-H. Yang, "Saliency detection via dense and sparse reconstruction," in Proc. 14th IEEE Int. Conf. Comput. Vis., Sydney, NSW, Australia, Dec. 2013, pp. 2976-2983.
39. C. Shen and Q. Zhao, "Learning to predict eye fixations for semantic contents using multi-layer sparse network," Neurocomputing, vol. 138, pp. 61-68, Aug. 2014.
40. J. Han, D. Zhang, S. Wen, L. Guo, T. Liu, X. Li, "Two-stage learning to predict human eye fixations via sdaes," IEEE Trans. Syst., Man, Cybern., vol. PP, no. 99, Feb. 2015, doi: 10.1109/TCYB.2015.2404432.
41. S. Wen, J. Han, D. Zhang, L. Guo, "Saliency detection based on feature learning using deep Boltzmann machines," in Proc. IEEE Int. Conf. Multimedia Expo, Chengdu, China, Jul. 2014, pp. 1-6.
42. Q. Zhao and C. Koch, "Learning visual saliency by combining feature maps in a nonlinear manner using AdaBoost," J. Vis., vol. 12, no. 6, p. 22, Jun. 2012.
43. J. Han, D. Zhang, X. Hu, L. Guo, J. Ren, F. Wu, "Background prior-based salient object detection via deep reconstruction residual," IEEE Trans. Circuits Syst. Video Technol., vol. 25, no. 8, pp. 1309-1321, Aug. 2015.
44. J. Han, K. N. Ngan, M. Li, H.-J. Zhang, "Unsupervised extraction of visual attention objects in color images," IEEE Trans. Circuits Syst. Video Technol., vol. 16, no. 1, pp. 141-145, Jan. 2006.
45. T. Liu et al., "Learning to detect a salient object," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 2, pp. 353-367, Feb. 2011.
46. L. Yang and Y. Zhou, "Exploring feature sets for two-phase biomedical named entity recognition using semi-CRFs," Knowl. Inf. Syst., vol. 40, no. 2, pp. 439-453, 2014.
47. P. Balamurugan, S. Shevade, T. R. Babu, "Scalable sequential alternating proximal methods for sparse structural SVMs and CRFs," Knowl. Inf. Syst., vol. 38, no. 3, pp. 599-621, Mar. 2014.
48. Y. Yang, Y. Yang, Z. Huang, H. T. Shen, F. Nie, "Tag localization with spatial correlations and joint group sparsity," in Proc. 25th IEEE Conf. Comput. Vis. Pattern Recognit., Providence, RI, USA, Jun. 2011, pp. 881-888.
49. H. Zhao, J. Chen, Y. Han, X. Cao, "Image aesthetics enhancement using composition-based saliency detection," Multimedia Syst., vol. 21, no. 2, pp. 159-168, Mar. 2015.
50. K. E. A. van de Sande, T. Gevers, C. G. M. Snoek, "Evaluating color descriptors for object and scene recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1582-1596, Sep. 2010.
51. S.-J. Wang, J. Yang, M.-F. Sun, X.-J. Peng, M.-M. Sun, C.-G. Zhou, "Sparse tensor discriminant color space for face verification," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 6, pp. 876-888, Jun. 2012.
52. X. Cao, Y. Zhang, X. Guo, Y.-M. Cheung, "Video color conceptualization using optimization," Sci. China Inf. Sci., vol. 57, no. 7, pp. 0721141-07211411, 2014.
53. J. Zhang, Y. Barhomi, T. Serre, "A new biologically inspired color image descriptor," in Proc. 12th Eur. Conf. Comput. Vis., Florence, Italy, Oct. 2012, pp. 312-324.
54. M. S. Livingstone and D. H. Hubel, "Anatomy and physiology of a color system in the primate visual cortex," J. Neurosci., vol. 4, no. 1, pp. 309-356, 1984.
55. C. Koch and S. Ullman, "Shifts in selective visual attention: Towards the underlying neural circuitry," Human Neurobiol., vol. 4, no. 4, pp. 219-227, 1985.
56. L. Itti and C. Koch, "Computational modelling of visual attention," Nature Rev. Neurosci., vol. 2, no. 3, pp. 194-203, 2001.
57. Z. Li, "A saliency map in primary visual cortex," Trends Cognit. Sci., vol. 6, no. 1, pp. 9-16, Jan. 2002.
58. L. Zhaoping and L. Zhe, "Properties of V1 neurons tuned to conjunctions of visual features: Application of the V1 saliency hypothesis to visual search behavior," PLoS ONE, vol. 7, no. 6, p. e36223, 2012.
59. A. R. Koene and L. Zhaoping, "Feature-specific interactions in salience from combined feature contrasts: Evidence for a bottom-up saliency map in V1," J. Vis., vol. 7, no. 7, pp. 1-14, May 2007.
60. B. Li, J. K. Thompson, T. Duong, M. R. Peterson, R. D. Freeman, "Origins of cross-orientation suppression in the visual cortex," J. Neurophysiol., vol. 96, no. 4, pp. 1755-1764, 2006.
61. R. Shapley and M. J. Hawken, "Color in the cortex: Single-and doubleopponent cells," Vis. Res., vol. 51, no. 7, pp. 701-717, Apr. 2011.
62. A. M. Treisman and G. Gelade, "A feature-integration theory of attention," Cognit. Psychol., vol. 12, no. 1, pp. 97-136, Jan. 1980.
63. D. J. Heeger, "Normalization of cell responses in cat striate cortex," J. Neurosci., vol. 9, no. 2, pp. 181-197, 1992.
64. T. Judd, K. Ehinger, F. Durand, A. Torralba, "Learning to predict where humans look," in Proc. 12th IEEE Int. Conf. Comput. Vis., Kyoto, Japan, Sep./Oct. 2009, pp. 2106-2113.
65. B. R. Conway and M. S. Livingstone, "Spatial and temporal properties of cone signals in alert macaque primary visual cortex," J. Neurosci., vol. 26, no. 42, pp. 10826-10846, 2006.
66. G. J. Brouwer and D. J. Heeger, "Cross-orientation suppression in human visual cortex," J. Neurophysiol., vol. 106, no. 5, pp. 2108-2119, 2011.
67. J. H. Reynolds and D. J. Heeger, "The normalization model of attention," Neuron, vol. 61, no. 2, pp. 168-185, Jan. 2009.
68. M. Carandini and D. J. Heeger, "Normalization as a canonical neural computation," Nature Rev. Neurosci., vol. 13, pp. 51-62, Jan. 2012.
69. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, Nov. 2004.
70. O. Chomat, V. C. de Verdière, D. Hall, J. L. Crowley, "Local scale selection for Gaussian based description techniques," in Proc. 6th Eur. Conf. Comput. Vis., Dublin, Ireland, Jun. 2000, pp. 117-134.
71. E. H. Adelson and J. R. Bergen, "Spatiotemporal energy models for the perception of motion," J. Opt. Soc. Amer. A, vol. 2, no. 2, pp. 284-299, 1985.
72. M. Carandini and D. J. Heeger, "Summation and division by neurons in primate visual cortex," Science, vol. 264, no. 5163, pp. 1333-1336, 1994.
73. X. Hou, J. Harel, C. Koch, "Image signature: Highlighting sparse salient regions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 1, pp. 194-201, Jan. 2012.
74. S. Kopf, B. Guthier, P. Schaber, T. Dittrich, W. Effelsberg, "Analysis of disparity maps for detecting saliency in stereoscopic video," Dept. Comput. Sci., Univ. Mannheim, Mannheim, Germany, Tech. Rep. TR-2013-003, 2013.
75. J. Zhang and S. Sclaroff, "Saliency detection: A Boolean map approach," in Proc. 14th IEEE Int. Conf. Comput. Vis., Sydney, NSW, Australia, Dec. 2013, pp. 153-160.
76. G. Kootstra, B. de Boer, L. R. B. Schomaker, "Predicting eye fixations on complex visual stimuli using local symmetry," Cognit. Comput., vol. 3, no. 1, pp. 223-240, Mar. 2011.
77. J. Harel, C. Koch, P. Perona, "Graph-based visual saliency," in Proc. 21th Annu. Conf. Neural Inf. Process. Syst., Vancouver, BC, Canada, Dec. 2007, pp. 545-552.
78. L. Zhaoping and R. J. Snowden, "A theory of a saliency map in primary visual cortex (V1) tested by psychophysics of colour-orientation interference in texture segmentation," Vis. Cognit., vol. 14, nos. 4-8, pp. 911-933, 2006.
79. Q. Zhao and C. Koch, "Learning a saliency map using fixated locations in natural scenes," J. Vis., vol. 11, no. 3, pp. 1-15, 2011.
80. A. Olmos and F. A. Kingdom, "A biologically inspired algorithm for the recovery of shading and reflectance images," Perception, vol. 33, no. 12, pp. 1463-1473, 2004.
81. A. Borji, H. R. Tavakoli, D. N. Sihite, L. Itti, "Analysis of scores, datasets, models in visual saliency prediction," in Proc. 14th IEEE Int. Conf. Comput. Vis., Sydney, NSW, Australia, Dec. 2013, pp. 921-928.
82. Z. Bylinskii, T. Judd, A. Borji, L. Itti, F. Durand, A. Oliva, A. Torralba. (Jan. 2012). MIT Saliency Benchmark. [Online]. Available: http://saliency.mit.edu/
83. T. Judd, F. Durand, A. Torralba, "A benchmark of computational models of saliency to predict human fixations," Dept. Comput. Graph., Massachusetts Inst. Technol., Cambridge, MA, USA, Tech. Rep. MIT-CSAIL-TR-2012-001, 2012.
84. X. Li, Y. Li, C. Shen, A. Dick, A. van den Hengel, "Contextual hypergraph modeling for salient object detection," in Proc. 14th IEEE Int. Conf. Comput. Vis., Sydney, NSW, Australia, Dec. 2013, pp. 3328-3335.
85. C. M. Stoughton and B. R. Conway, "Neural basis for unique hues," Current Biol., vol. 18, no. 6, pp. R698-R699, Aug. 2008.
86. X. Zhang, Y. Wang, J. Zhang, L. Hu, M. Wang, "Light field saliency vs. 2D saliency: A comparative study," Neurocomputing, vol. 166, pp. 389-396, Oct. 2015.
87. J. Zhang, M. Wang, J. Gao, Y. Wang, X. Zhang, X. Wu, "Saliency detection with a deeper investigation of light field," in Proc. 24th Int. Joint Conf. Artif. Intell., Buenos Aires, Argentina, Jul. 2015, pp. 2212-2218.