Bottom-Up Visual Saliency Estimation with Deep Autoencoder-Based Sparse Reconstruction

IEEE Transactions on Neural Networks and Learning Systems

Volumn 27, Issue 6, 2016, Pages 1227-1240

  Xia, Chen ^a   Qi, Fei ^a   Shi, Guangming ^a  

^a XIDIAN UNIVERSITY   (China)

Author keywords

Autoencoder; center surround (C S) difference; deep learning; reconstruction; saliency; Unsupervised feature learning.

Indexed keywords

FEATURE EXTRACTION; IMAGE RECONSTRUCTION; LEARNING SYSTEMS;

BOTTOM-UP SALIENCIES; COMPREHENSIVE EVALUATION; CONTRAST COMPUTATION; GENERALIZATION ABILITY; RECONSTRUCTION PARAMETERS; RECONSTRUCTION PATTERNS; SPARSE RECONSTRUCTION; STATE-OF-THE-ART ALGORITHMS;

IMAGE PROCESSING;

EID: 84955106472     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2015.2512898     Document Type: Article

References (82)

1. L. Itti and P. Baldi, "Bayesian surprise attracts human attention," Vis. Res., vol. 49, no. 10, pp. 1295-1306, Jun. 2009.
2. J. M. Wolfe and T. S. Horowitz, "What attributes guide the deployment of visual attention and how do they do it?" Nature Rev. Neurosci., vol. 5, no. 6, pp. 495-501, 2004.
3. L. Itti, C. Koch, E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254-1259, Nov. 1998.
4. L. Itti and C. Koch, "Computational modelling of visual attention," Nature Rev. Neurosci., vol. 2, no. 3, pp. 194-203, Mar. 2001.
5. Z. Li, "A saliency map in primary visual cortex," Trends Cognit. Sci., vol. 6, no. 1, pp. 9-16, Jan. 2002.
6. L. Itti, "Automatic foveation for video compression using a neurobiological model of visual attention," IEEE Trans. Image Process., vol. 13, no. 10, pp. 1304-1318, Oct. 2004.
7. C. Guo and L. Zhang, "A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression," IEEE Trans. Image Process., vol. 19, no. 1, pp. 185-198, Jan. 2010.
8. T. Liu et al., "Learning to detect a salient object," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 2, pp. 353-367, Feb. 2011.
9. D. Erhan, C. Szegedy, A. Toshev, D. Anguelov, "Scalable object detection using deep neural networks," in Proc. 27th IEEE Conf. Comput. Vis. Pattern Recognit., Columbus, OH, USA, Jun. 2014, pp. 2155-2162.
10. D. Gao and N. Vasconcelos, "Discriminant saliency for visual recognition from cluttered scenes," in Proc. Adv. Neural Inf. Process. Syst., Vancouver, BC, Canada, Dec. 2004, pp. 481-488.
11. A. A. Salah, E. Alpaydin, L. Akarun, "A selective attention-based method for visual pattern recognition with application to handwritten digit recognition and face recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 3, pp. 420-425, Mar. 2002.
12. Y.-F. Ma and H.-J. Zhang, "Contrast-based image attention analysis by using fuzzy growing," in Proc. 11th ACM Int. Conf. Multimedia, Berkeley, CA, USA, Nov. 2003, pp. 374-381.
13. D. Gao and N. Vasconcelos, "Bottom-up saliency is a discriminant process," in Proc. 11th IEEE Int. Conf. Comput. Vis., Rio de Janeiro, Brazil, Oct. 2007, pp. 185-190.
14. D. A. Klein and S. Frintrop, "Center-surround divergence of feature statistics for salient object detection," in Proc. 13th IEEE Int. Conf. Comput. Vis., Barcelona, Spain, Nov. 2011, pp. 2214-2219.
15. C. Xia, P. Wang, F. Qi, G. Shi, "Nonlocal center-surround reconstruction-based bottom-up saliency estimation," in Proc. 20th IEEE Int. Conf. Image Process., Melbourne, VIC, Australia, Sep. 2013, pp. 206-210.
16. J. Wu, F. Qi, G. Shi, Y. Lu, "Non-local spatial redundancy reduction for bottom-up saliency estimation," J. Vis. Commun. Image Represent., vol. 23, no. 7, pp. 1158-1166, Oct. 2012.
17. C. Xia, F. Qi, G. Shi, P. Wang, "Nonlocal center-surround reconstruction-based bottom-up saliency estimation," Pattern Recognit., vol. 48, no. 4, pp. 1337-1348, Apr. 2015.
18. A. Borji and L. Itti, "Exploiting local and global patch rarities for saliency detection," in Proc. 25th IEEE Conf. Comput. Vis. Pattern Recognit., Providence, RI, USA, Jun. 2012, pp. 478-485.
19. M.-M. Cheng, G.-X. Zhang, N. J. Mitra, X. Huang, S.-M. Hu, "Global contrast based salient region detection," in Proc. 24th IEEE Conf. Comput. Vis. Pattern Recognit., Colorado Springs, CO, USA, Jun. 2011, pp. 409-416.
20. S. Goferman, L. Zelnik-Manor, A. Tal, "Context-aware saliency detection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 10, pp. 1915-1926, Oct. 2012.
21. L. Duan, C. Wu, J. Miao, L. Qing, Y. Fu, "Visual saliency detection by spatially weighted dissimilarity," in Proc. 24th IEEE Conf. Comput. Vis. Pattern Recognit., Colorado Springs, CO, USA, Jun. 2011, pp. 473-480.
22. N. D. B. Bruce and J. K. Tsotsos, "Saliency based on information maximization," in Proc. Annu. Conf. Neural Inf. Process. Syst., Montreal, QC, Canada, Dec. 2005, pp. 155-162.
23. X. Hou and L. Zhang, "Dynamic visual attention: Searching for coding length increments," in Proc. Annu. Conf. Neural Inf. Process. Syst., Vancouver, BC, Canada, Dec. 2008, pp. 681-688.
24. L. Zhang, M. H. Tong, T. K. Marks, H. Shan, G. W. Cottrell, "SUN: A Bayesian framework for saliency using natural statistics," J. Vis., vol. 8, no. 7, pp. 1-20, Dec. 2008.
25. Z. Ren, S. Gao, L.-T. Chia, D. Rajan, "Regularized feature reconstruction for spatio-temporal saliency detection," IEEE Trans. Image Process., vol. 22, no. 8, pp. 3120-3132, Aug. 2013.
26. X. Li, H. Lu, L. Zhang, X. Ruan, M.-H. Yang, "Saliency detection via dense and sparse reconstruction," in Proc. 14th IEEE Int. Conf. Comput. Vis., Sydney, NSW, Australia, Dec. 2013, pp. 2976-2983.
27. G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, vol. 313, no. 5786, pp. 504-507, 2006.
28. Q. V. Le et al., "Building high-level features using large scale unsupervised learning," in Proc. 29th Int. Conf. Mach. Learn., Edinburgh, Scotland, Jun. 2012, pp. 1-11.
29. A. Krizhevsky and G. E. Hinton, "Using very deep autoencoders for content-based image retrieval," in Proc. 19th Eur. Symp. Artif. Neural Netw., Bruges, Belgium, Apr. 2011, pp. 1-7.
30. A. Torralba, A. Oliva, M. S. Castelhano, J. M. Henderson, "Contextual guidance of eye movements and attention in real-world scenes: The role of global features in object search," Psychol. Rev., vol. 113, no. 4, pp. 766-786, 2006.
31. T. Judd, K. Ehinger, F. Durand, A. Torralba, "Learning to predict where humans look," in Proc. 12th IEEE Int. Conf. Comput. Vis., Kyoto, Japan, Sep. 2009, pp. 2106-2113.
32. H. J. Seo and P. Milanfar, "Static and space-time visual saliency detection by self-resemblance," J. Vis., vol. 9, no. 12, pp. 1-27, 2009.
33. G. Kootstra, A. Nederveen, B. de Boer, "Paying attention to symmetry," in Proc. 19th Brit. Mach. Vis. Conf., Leeds, U.K., Sep. 2008, pp. 1115-1125.
34. A. Borji, "Boosting bottom-up and top-down visual features for saliency estimation," in Proc. 25th IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Providence, RI, USA, Jun. 2012, pp. 438-445.
35. E. Erdem and A. Erdem, "Visual saliency estimation by nonlinearly integrating features using region covariances," J. Vis., vol. 13, no. 4, pp. 1-20, Mar. 2013.
36. Q. Zhao and C. Koch, "Learning saliency-based visual attention: A review," Signal Process., vol. 93, no. 6, pp. 1401-1407, Jun. 2013.
37. Y. Lu, W. Zhang, C. Jin, X. Xue, "Learning attention map from images," in Proc. 25th IEEE Conf. Comput. Vis. Pattern Recognit., Providence, RI, USA, Jun. 2012, pp. 1067-1074.
38. E. Vig, M. Dorr, D. Cox, "Large-scale optimization of hierarchical features for saliency prediction in natural images," in Proc. 27th IEEE Conf. Comput. Vis. Pattern Recognit., Columbus, OH, USA, Jun. 2014, pp. 2798-2805.
39. C. Shen, M. Song, Q. Zhao, "Learning high-level concepts by training a deep network on eye fixations," in Proc. NIPS Deep Learn. Unsupervised Feature Learn. Workshop, Lake Tahoe, NV, USA, Dec. 2012, pp. 1-8.
40. C. Shen and Q. Zhao, "Learning to predict eye fixations for semantic contents using multi-layer sparse network," Neurocomputing, vol. 138, pp. 61-68, Aug. 2014.
41. A. Borji and L. Itti, "State-of-the-art in visual attention modeling," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 1, pp. 185-207, Jan. 2013.
42. B. A. Olshausen and D. J. Field, "Emergence of simple-cell receptive field properties by learning a sparse code for natural images," Nature, vol. 381, no. 6583, pp. 607-609, 1996.
43. X. Hou and L. Zhang, "Saliency detection: A spectral residual approach," in Proc. 20th IEEE Conf. Comput. Vis. Pattern Recognit., Minneapolis, MN, USA, Jun. 2007, pp. 1-8.
44. P. Siva, C. Russell, T. Xiang, L. Agapito, "Looking beyond the image: Unsupervised learning for object saliency and detection," in Proc. 26th IEEE Conf. Comput. Vis. Pattern Recognit., Portland, OR, USA, Jun. 2013, pp. 3238-3245.
45. A. M. Treisman and G. Gelade, "A feature-integration theory of attention," Cognit. Psychol., vol. 12, no. 1, pp. 97-136, Jan. 1980.
46. X. Shen and Y. Wu, "A unified approach to salient object detection via low rank matrix recovery," in Proc. 25th IEEE Conf. Comput. Vis. Pattern Recognit., Providence, RI, USA, Jun. 2012, pp. 853-860.
47. H. Jiang, J. Wang, Z. Yuan, Y. Wu, N. Zheng, S. Li, "Salient object detection: A discriminative regional feature integration approach," in Proc. 26th IEEE Conf. Comput. Vis. Pattern Recognit., Portland, OR, USA, Jun. 2013, pp. 2083-2090.
48. Q. Zhao and C. Koch, "Learning a saliency map using fixated locations in natural scenes," J. Vis., vol. 11, no. 3, pp. 1-15, 2011.
49. A. Krizhevsky, I. Sutskever, G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Proc. Annu. Conf. Adv. Neural Inf. Process. Syst., Lake Tahoe, NV, USA, Dec. 2012, pp. 1106-1114.
50. P. Sermanet, K. Kavukcuoglu, S. Chintala, Y. LeCun, "Pedestrian detection with unsupervised multi-stage feature learning," in Proc. 26th IEEE Conf. Comput. Vis. Pattern Recognit., Portland, OR, USA, Jun. 2013, pp. 3626-3633.
51. M. Osadchy, Y. Le Cun, M. L. Miller, "Synergistic face detection and pose estimation with energy-based models," J. Mach. Learn. Res., vol. 8, pp. 1197-1215, Jan. 2007.
52. D. Ciresan, U. Meier, J. Masci, J. Schmidhuber, "A committee of neural networks for traffic sign classification," in Proc. Int. Joint Conf. Neural Netw., San Jose, CA, USA, Jul. 2011, pp. 1918-1921.
53. C. Szegedy, A. Toshev, D. Erhan, "Deep neural networks for object detection," in Proc. Adv. Neural Inf. Process. Syst., Lake Tahoe, NV, USA, Dec. 2013, pp. 2553-2561.
54. R. Girshick, J. Donahue, T. Darrell, J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proc. 27th IEEE Conf. Comput. Vis. Pattern Recognit., Columbus, OH, USA, Jun. 2014, pp. 580-587.
55. I. Lenz, H. Lee, A. Saxena, "Deep learning for detecting robotic grasps," Int. J. Robot. Res., vol. 34, nos. 4-5, pp. 705-724, 2015.
56. V. Mnih and G. E. Hinton, "Learning to detect roads in high-resolution aerial images," in Proc. 11th Eur. Conf. Comput. Vis., Heraklion, Greece, Sep. 2010, pp. 210-223.
57. Y. Bengio, "Learning deep architectures for AI," Found. Trends Mach. Learn., vol. 2, no. 1, pp. 1-127, 2009.
58. B. A. Olshausen and D. J. Field, "Sparse coding with an overcomplete basis set: A strategy employed by V1?" Vis. Res., vol. 37, no. 23, pp. 3311-3325, Dec. 1997.
59. S. Ramanathan, H. Katti, N. Sebe, M. Kankanhalli, T.-S. Chua, "An eye fixation database for saliency detection in images," in Proc. 11th Eur. Conf. Comput. Vis., Heraklion, Greece, Sep. 2010, pp. 30-43.
60. M. D. Zeiler, G. W. Taylor, R. Fergus, "Adaptive deconvolutional networks for mid and high level feature learning," in Proc. 13th IEEE Int. Conf. Comput. Vis. (ICCV), Barcelona, Spain, Nov. 2011, pp. 2018-2025.
61. J. van de Weijer, T. Gevers, J.-M. Geusebroek, "Edge and corner detection by photometric quasi-invariants," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 4, pp. 625-630, Apr. 2005.
62. G. E. Hinton, "Learning multiple layers of representation," Trends Cognit. Sci., vol. 11, no. 10, pp. 428-434, Oct. 2007.
63. G. E. Hinton, "Training products of experts by minimizing contrastive divergence," Neural Comput., vol. 14, no. 8, pp. 1771-1800, 2002.
64. G. Hinton, "A practical guide to training restricted Boltzmann machines," Dept. Comput. Sci., Univ. Toronto, Toronto, ON, Canada, Tech. Rep. UTML-TR-2010-003, Aug. 2010.
65. N. Riche, M. Duvinage, M. Mancas, B. Gosselin, T. Dutoit, "Saliency and human fixations: State-of-the-art and study of comparison metrics," in Proc. 14th IEEE Int. Conf. Comput. Vis., Sydney, NSW, Australia, Dec. 2013, pp. 1153-1160.
66. A. Borji, D. N. Sihite, L. Itti, "Quantitative analysis of humanmodel agreement in visual saliency modeling: A comparative study," IEEE Trans. Image Process., vol. 22, no. 1, pp. 55-69, Jan. 2013.
67. B. W. Tatler, "The central fixation bias in scene viewing: Selecting an optimal viewing position independently of motor biases and image feature distributions," J. Vis., vol. 7, no. 14, pp. 1-17, 2007.
68. X. Hou, J. Harel, C. Koch, "Image signature: Highlighting sparse salient regions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 1, pp. 194-201, Jan. 2012.
69. R. J. Peters, A. Iyer, L. Itti, C. Koch, "Components of bottom-up gaze allocation in natural images," Vis. Res., vol. 45, no. 8, pp. 2397-2416, 2005.
70. B. W. Tatler, R. J. Baddeley, I. D. Gilchrist, "Visual correlates of fixation selection: Effects of scale and time," Vis. Res., vol. 45, no. 5, pp. 643-659, Mar. 2005.
71. J. Wang, M. P. DaSilva, P. LeCallet, V. Ricordel, "Computational model of stereoscopic 3D visual saliency," IEEE Trans. Image Process., vol. 22, no. 6, pp. 2151-2165, Jun. 2013.
72. A. Garcia-Diaz, V. Leborán, X. R. Fdez-Vidal, X. M. Pardo, "On the relationship between optical variability, visual saliency, eye fixations: A computational approach," J. Vis., vol. 12, no. 6, pp. 1-22, 2012.
73. J. Harel, C. Koch, P. Perona, "Graph-based visual saliency," in Proc. Adv. Neural Inf. Process. Syst., Vancouver, BC, Canada, Dec. 2006, pp. 545-552.
74. T. Judd, F. Durand, A. Torralba, "A benchmark of computational models of saliency to predict human fixations," Comput. Sci. Artif. Intell. Lab., Massachusetts Inst. Technol., Cambridge, MA, USA, Tech. Rep. MIT-CSAIL-TR-2012-001, Jan. 2012.
75. R. P. N. Rao and D. H. Ballard, "Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects," Nature Neurosci., vol. 2, no. 1, pp. 79-87, 1999.
76. Y. Huang and R. P. N. Rao, "Predictive coding," Wiley Interdiscipl. Rev., Cognit. Sci., vol. 2, no. 5, pp. 580-593, Sep./Oct. 2011.
77. J. Zhang and S. Sclaroff, "Saliency detection: A Boolean map approach," in Proc. 14th IEEE Int. Conf. Comput. Vis., Sydney, NSW, Australia, Dec. 2013, pp. 153-160.
78. J. Pan and X. Giró-i-Nieto. (Jul. 2015). "End-to-end convolutional network for saliency prediction." [Online]. Available: http://arxiv.org/abs/1507.01422
79. M. Kümmerer, L. Theis, M. Bethge. (Apr. 2015). "Deep gaze I: Boosting saliency prediction with feature maps trained on ImageNet." [Online]. Available: http://arxiv.org/abs/1411.1045
80. Y. Li, X. Hou, C. Koch, J. M. Rehg, A. L. Yuille, "The secrets of salient object segmentation," in Proc. 27th IEEE Conf. Comput. Vis. Pattern Recognit., Columbus, OH, USA, Jun. 2014, pp. 280-287.
81. M. W. Spratling, "Predictive coding as a model of the V1 saliency map hypothesis," Neural Netw., vol. 26, pp. 7-28, Feb. 2012.
82. V. Beaucousin, G. Simon, M. Cassotti, A. Pineau, O. Houdé, N. Poirel, "Global interference during early visual processing: ERP evidence from a rapid global/local selective task," Front. Psychol., vol. 4, pp. 1-6, 2013, Art. ID 539.