Two-Stage Learning to Predict Human Eye Fixations via SDAEs

IEEE Transactions on Cybernetics

Volumn 46, Issue 2, 2016, Pages 487-498

  Han, Junwei ^a   Zhang, Dingwen ^a   Wen, Shifeng ^a   Guo, Lei ^a   Liu, Tianming ^b   Li, Xuelong ^c  

^a NORTHWESTERN POLYTECHNICAL UNIVERSITY   (China)

^b University of Georgia ^*  (United States)

^c XI AN INSTITUTE OF OPTICS AND PRECISION MECHANICS   (China)

Author keywords

Deep networks; eye fixation prediction; saliency detection; stacked denoising autoencoders (SDAEs)

Indexed keywords

EYE MOVEMENTS;

FEATURE CONTRASTS; INFERENCE MECHANISM; OPTIMAL MECHANISM; RAW IMAGE DATA; RESEARCH INTERESTS; SALIENCY DETECTION; SECOND LEARNING; STATE-OF-THE-ART APPROACH;

FORECASTING;

ALGORITHM; AUTOMATED PATTERN RECOGNITION; EYE FIXATION; HUMAN; PHYSIOLOGY; PROCEDURES; STATISTICAL MODEL;

ALGORITHMS; FIXATION, OCULAR; HUMANS; MODELS, STATISTICAL; PATTERN RECOGNITION, AUTOMATED;

EID: 84923658437     PISSN: 21682267     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCYB.2015.2404432     Document Type: Article

References (60)

1. J. William, The Principles of Psychology. Cambridge, MA, USA: Harvard Univ. Press, 1890.
2. J. Han, K. N. Ngan, M. Li, and H.-J. Zhang, "Unsupervised extraction of visual attention objects in color images, " IEEE Trans. Circuits Syst. Video Technol., vol. 16, no. 1, pp. 141-145, Jan. 2006.
3. T. Liu et al., "Picture collage, " IEEE Trans. Multimedia, vol. 11, no. 7, pp. 1225-1239, Aug. 2009.
4. J. Han et al., "Image visual attention computation and application via the learning of object attributes, " Mach. Vis. Appl., vol. 25, no. 7, pp. 1-13, Oct. 2013.
5. D. Zhang et al., "Weakly supervised learning for target detection in remote sensing images, " IEEE Geosci. Remote Sens. Lett., vol. 12, no. 4, pp. 701-705, Apr. 2015.
6. J. Han, D. Zhang, G. Cheng, L. Guo, and J. Ren, "Object detection in optical remote sensing images based on weakly supervised learning and high-level feature learning, " IEEE Trans. Geosci. Remote Sens., vol. 53, no. 6, pp. 3325-3337, Jun. 2015.
7. F. Zhu and L. Shao, "Weakly-supervised cross-domain dictionary learning for visual recognition, " Int. J. Comput. Vis., vol. 109, nos. 1-2, pp. 42-59, Aug. 2014.
8. L. Shao and M. Brady, "Specific object retrieval based on salient regions, " Patt. Recognit., vol. 39, no. 10, pp. 1932-1948, Oct. 2006.
9. L. Shao and M. Brady, "Invariant salient regions based image retrieval under viewpoint and illumination variations, " J. Vis. Commun. Image Represent., vol. 17, no. 6, pp. 1256-1272, Dec. 2006.
10. J. Han et al., "Efficient, simultaneous detection of multi-class geospatial targets based on visual saliency modeling and discriminative learning of sparse coding, " ISPRS J. Photogramm. Remote Sens., vol. 89, pp. 37-48, Mar. 2014.
11. J. Han et al., "Video abstraction based on fMRI-driven visual attention model, " Inf. Sci., vol. 281, pp. 781-796, Oct. 2014.
12. J. Han et al., "Learning computational models of video memorability from fMRI brain imaging" IEEE Trans. Cybern., 2014. Doi: 10.1109/TCYB.2014.2358647.
13. A. Garcia-Diaz, X. R. Fdez-Vidal, X. M. Pardo, and R. Dosil, "Saliency from hierarchical adaptation through decorrelation and variance normalization, " Image Vis. Comput., vol. 30, no. 1, pp. 51-64, Jan. 2012.
14. A. Borji and L. Itti, "Exploiting local and global patch rarities for saliency detection, " in Proc. IEEE Int. Conf. Comput. Vis. Patt. Recognit., Providence, RI, USA, Jun. 2012, pp. 478-485.
15. S. Goferman, L. Zelnik-Manor, and A. Tal, "Context-Aware saliency detection, " IEEE Trans. Patt. Anal. Mach. Intell., vol. 34, no. 10, pp. 1915-1926, Oct. 2012.
16. J. Han et al., "Background prior based salient object detection via deep reconstruction residual, " IEEE Trans. Circuits Syst. Video Technol., 2014. Doi: 10.1109/TCSVT.2014.2381471.
17. L. Itti, C. Koch, and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis, " IEEE Trans. Patt. Anal. Mach. Intell., vol. 20, no. 11, pp. 1254-1259, Nov. 1998.
18. T. Judd, K. Ehinger, F. Durand, and A. Torralba, "Learning to predict where humans look, " in Proc. 12th Int. Conf. Comput. Vision, Kyoto, Japan, Oct. 2009, pp. 2106-2113.
19. X. Hou and L. Zhang, "Dynamic visual attention: Searching for coding length increments, " in Proc. Conf. Adv. Neural Inform. Process. Syst., 2008, pp. 681-688.
20. J. Han et al., "An object-oriented visual saliency detection framework based on sparse coding representations, " IEEE Trans. Circuits Syst. Video Technol., vol. 23, no. 12, pp. 2009-2021, Dec. 2013.
21. X. Shen and Y. Wu, "A unified approach to salient object detection via low rank matrix recovery, " in Proc. IEEE Int. Conf. Comput. Vis. Patt. Recognit., 2012, pp. 853-860.
22. X. Hou and L. Zhang, "Saliency detection: A spectral residual approach, " in Proc. IEEE Int. Conf. Comput. Vis. Patt. Recognit., 2007, pp. 1-8.
23. C. Guo, Q. Ma, and L. Zhang, "Spatio-Temporal saliency detection using phase spectrum of quaternion Fourier transform, " in Proc. IEEE Int. Conf. Comput. Vis. Patt. Recognit., 2008, pp. 1-8.
24. X. Hou, J. Harel, and C. Koch, "Image signature: Highlighting sparse salient regions, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 1, pp. 194-201, Jan. 2012.
25. B. Schauerte and R. Stiefelhagen, "Quaternion-based spectral saliency detection for eye fixation prediction, " in Proc. Eur. Conf. Comput. Vis., 2012, pp. 116-129.
26. B. Han, H. Zhu, and Y. Ding, "Bottom-up saliency based on weighted sparse coding residual, " in Proc. ACM Int. Conf. Multimedia, Scottsdale, AZ, USA, Dec. 2011, pp. 1117-1120.
27. Q. Zhao and C. Koch, "Learning a saliency map using fixated locations in natural scenes, " J. Vis., vol. 11, no. 3, p. 9, Mar. 2011.
28. T. Liu et al., "Learning to detect a salient object, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 2, pp. 353-367, Feb. 2011.
29. A. Borji, "Boosting bottom-up and top-down visual features for saliency estimation, " in Proc. IEEE Int. Conf. Comput. Vis. Patt. Recognit., Providence, RI, USA, Jun. 2012, pp. 438-445.
30. M. Cerf, J. Harel, W. Einhäuser, and C. Koch, "Predicting human gaze using low-level saliency combined with face detection, " in Proc. Conf. Adv. Neural Inform. Process. Syst., 2008, pp. 241-248.
31. C. Shen, M. Song, and Q. Zhao, "Learning high-level concepts by training a deep network on eye fixations, " in Proc. Deep Learn. Unsupervised Feature Learn. Workshop, Lake Tahoe, NV, USA, 2012.
32. L. Itti and C. Koch, "Computational modelling of visual attention, " Neuroscience, vol. 2, no. 3, pp. 194-203, Mar. 2001.
33. A. Borji, D. N. Sihite, and L. Itti, "Quantitative analysis of human-model agreement in visual saliency modeling: A comparative study, " IEEE Trans. Image Process., vol. 22, no. 1, pp. 55-69, Jan. 2013.
34. D. Gao, V. Mahadevan, and N. Vasconcelos, "On the plausibility of the discriminant center-surround hypothesis for visual saliency, " J. Vis., vol. 8, no. 7, p. 13, Jun. 2008.
35. H. J. Seo and P. Milanfar, "Static and space-Time visual saliency detection by self-resemblance, " J. Vis., vol. 9, no. 12, p. 15, Nov. 2009.
36. L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, "SUN: A Bayesian framework for saliency using natural statistics, " J. Vis., vol. 8, no. 7, p. 32, Dec. 2008.
37. N. D. Bruce and J. K. Tsotsos, "Saliency, attention, and visual search: An information theoretic approach, " J. Vis., vol. 9, no. 3, p. 5, Mar. 2009.
38. J. Li, M. D. Levine, X. An, X. Xu, and H. He, "Visual saliency based on scale-space analysis in the frequency domain, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 4, pp. 996-1010, Apr. 2013.
39. Y. Bengio, "Deep learning of representations: Looking forward, " in Proc. Stat. Lang. Speech Process., 2013, pp. 1-37.
40. W. Hou, X. Gao, D. Tao, and X. Li, "Blind image quality assessment via deep learning, " IEEE Trans. Neural Netw. Learn. Syst., 2014. Doi: 10.1109/TNNLS.2014.2336852.
41. P. Sermanet et al., "OverFeat: Integrated recognition, localization and detection using convolutional networks, " arXiv preprint arXiv:1312.6229, Dec. 2013.
42. Y. Yang et al., "Multi-feature fusion via hierarchical regression for multimedia analysis, " IEEE Trans. Multimedia, vol. 15, no. 3, pp. 572-581, Apr. 2013.
43. Y. Bengio, "Deep learning of representations for unsupervised and transfer learning, " in Proc. JMLR Workshops Conf., vol. 27. 2012, pp. 17-36.
44. Y. Bengio et al., "Deep learners benefit more from out-of-distribution examples, " in Proc. Worshop. Artif. Intell. Stat., 2011, pp. 164-172.
45. M. Oquab, L. Bottou, I. Laptev, and J. Sivic, "Learning and transferring mid-level image representations using convolutional neural networks, " in Proc. IEEE Int. Conf. Comput. Vis. Patt. Recognit., 2013, pp. 1717-1724.
46. K. He, X. Zhang, S. Ren, and J. Sun, "Spatial pyramid pooling in deep convolutional networks for visual recognition, " in Proc. Eur. Conf. Comput. Vision, 2014, pp. 346-361.
47. G. E. Hinton and R. S. Zemel, "Autoencoders, minimum description length, and Helmholtz free energy, " in Proc. Conf. Adv. Neural Inform. Process. Syst., 1994, p. 3.
48. Y. Bengio, "Learning deep architectures for AI, " Found. Trends Mach. Learn., vol. 2, no. 1, pp. 1-127, Jan. 2009.
49. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol, "Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion, " J. Mach. Learn. Res., vol. 11, pp. 3371-3408, Jan. 2010.
50. Y. MarcAurelio Ranzato, L. Boureau, and Y. LeCun, "Sparse feature learning for deep belief networks, " in Proc. Conf. Adv. Neural Inform. Process. Syst., 2007, pp. 1185-1192.
51. Y. Bengio, A. Courville, and P. Vincent, "Representation learning: A review and new perspectives, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 8, pp. 1798-1828, Aug. 2013.
52. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, Learning Representations by Back-Propagating Errors. Cambridge, MA, USA: MIT Press, 1988.
53. G. E. Hinton, S. Osindero, and Y.-W. Teh, "A fast learning algorithm for deep belief nets, " Neural Comput., vol. 18, no. 7, pp. 1527-1554, Jul. 2006.
54. J. Zhang and S. Sclaroff, "Saliency detection: A Boolean map approach, " in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 153-160.
55. J. Harel, C. Koch, and P. Perona, "Graph-based visual saliency, " in Proc. Conf. Adv. Neural Inform. Process. Syst., 2006, pp. 545-552.
56. H. Shin, M. Orton, D. Collins, S. Doran, and M. Leach, "Stacked autoencoders for unsupervised feature learning and multiple organ detection in a pilot study using 4D patient data, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 8, pp. 1930-1943, Aug. 2013.
57. J. Li, Y. Tian, and T. Huang, "Probabilistic multi-Task learning for visual saliency estimation in video, " Int. J. Comput. Vision, vol. 90, no. 2, pp. 150-165, May 2010.
58. J. Li, Y. Tian, and T. Huang, "Visual saliency with statistical priors, " Int. J. Comput. Vision, vol. 107, no. 3, pp. 239-253, May 2014.
59. S. Jones and L. Shao, "Content-based retrieval of human actions from realistic video databases, " Inform. Sci., vol. 236, pp. 56-65, Jul. 2013.
60. L. Zhang, X. Zhen, and L. Shao, "Learning object-To-class kernels for scene classification, " IEEE Trans. Image Process., vol. 23, no. 8, pp. 3241-3253, Aug. 2014.