Spatiotemporal saliency detection using textural contrast and its applications

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 24, Issue 4, 2014, Pages 646-659

  Kim, Wonjun ^a,b   Kim, Changick ^a  

^a Korea Advanced Institute of Science and Technology (KAIST)   (South Korea)

^b Samsung Advanced Institute of Technology (SAIT)   (South Korea)

Author keywords

Comparative evaluations; Computer vision applications; Human visual attention; Saliency detection; Textural contrast

Indexed keywords

NETWORKS (CIRCUITS); VIDEO SIGNAL PROCESSING;

COMPARATIVE EVALUATIONS; COMPUTER VISION APPLICATIONS; HUMAN VISUAL ATTENTION; SALIENCY DETECTION; TEXTURAL CONTRAST;

SECURITY SYSTEMS;

EID: 84897978322     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2013.2290579     Document Type: Article

References (53)

1. R. VanRullen, "Visual saliency and spike timing in the ventral visual pathway," J. Physiol., vol. 97, no. 2, pp. 365-377, 2003.
2. O. S. Packer and D. M. Dacey, "Synergistic center-surround receptive field model of monkey H1 horizontal cells," J. Vision, vol. 5, no. 11, pp. 1038-1054, 2005. (Pubitemid 43078791)
3. R. Achanta and S. Süsstrunk, "Saliency detection for content-aware image resizing," in Proc. IEEE Int. Conf. Image Process., Nov. 2009, pp. 1005-1008.
4. S. Goferman, L. Z. Manor, and A. Tal, "Context-aware saliency detection," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., Jun. 2010, pp. 2376-2383.
5. W. Kim, C. Jung, and C. Kim, "Spatiotemporal saliency detection and its applications in static and dynamic scenes," IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 4, pp. 446-456, Apr. 2011.
6. Y. Fu, J. Cheng, Z. Li, and H. Lu, "Saliency cuts: An automatic approach to object segmentation," in Proc. IEEE Int. Conf. Pattern Recognit., Dec. 2008, pp. 1-4.
7. T. Avraham and M. Lindenbaum, "Esaliency (extended saliency): Meaningful attention using stochastic image modeling," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 4, pp. 693-708, Apr. 2010.
8. O. L. Meur and J.-C. Chevet, "Relevance of a feed-forward model of visual attention for goal-oriented and free-viewing tasks," IEEE Trans. Image Process., vol. 19, no. 11, pp. 2801-2813, Nov. 2010.
9. C. Guo and L. Zhang, "A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression," IEEE Trans. Image Process., vol. 19, no. 1, pp. 185-198, Jan. 2010.
10. S. Marat, M. Guironnet, and D. Pellerin, "Video summarization using a visual attention model," in Proc. Eur. Signal Process. Conf., 2007, pp. 1784-1788.
11. A. Ninassi, O. L. Meur, P. L. Callet, and D. Barba, "Does where you gaze on an image affect your perception of quality? Applying to image quality metric," in Proc. IEEE Int. Conf. Image Process., vol. 2. Oct. 2007, pp. 169-172.
12. X. Feng, T. Liu, D. Yang, and Y. Wang, "Saliency inspired full-reference quality metric for packet-loss-impaired video," IEEE Trans. Broadcast., vol. 57, no. 1, pp. 81-88, Mar. 2011.
13. H. Liu and I. Heynderickx, "Visual attention in object image quality assessment: Based on eyetracking data," IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 7, pp. 971-982, Jul. 2011.
14. A. Toet, "Computational versus psychophysical bottom-up image saliency: A comparative evaluation study," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 11, pp. 2131-2146, Nov. 2011.
15. A. Oliva, A. Torralba, M. Castelhano, and J. Henderson, "Top-down control of visual attention in object detection," in Proc. IEEE Int. Conf. Image Process., vol. 1. Sep. 2003, pp. 253-256.
16. H. Li and K. N. Ngan, "Saliency model-based face segmentation and 646 tracking in head-and-shoulder video sequences," J. Visual Commun. Image Represent., vol. 19, no. 5, pp. 320-333, Jul. 2008.
17. D. Gao, V. Mahadevan, and N. Vasconcelos, "On the plausibility of the discriminant center-surround hypothesis for visual saliency," J. Vision, vol. 8, no. 7, pp. 1-18, 2008.
18. W. Kim and C. Kim, "Saliency detection via textural contrast," Opt. Lett., vol. 37, no. 9, pp. 1550-1552, May 2012.
19. L. Itti, C. Koch, and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254-1259, Nov. 1998. (Pubitemid 128742001)
20. Y. F. Ma and H. J. Zhang, "Contrast-based image attention analysis by using fuzzy growing," in Proc. ACM Int. Conf. Multimedia, 2003, pp. 374-381.
21. J. Harel, C. Koch, and P. Perona, "Graph-based visual saliency," in Proc. NIPS, vol. 19. 2007, pp. 545-552.
22. R. Achanta, S. Hemami, F. Estrada, and S. Süsstrunk, "Frequency-tuned salient region detection," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., Jun. 2009, pp. 1597-1604.
23. N. D. B. Bruce and J. K. Tsotsos, "Saliency, attention, and visual search: An information theoretic approach," J. Vision, vol. 9, no. 3, pp. 1-24, 2009.
24. H. Seo and P. Milanfar, "Static and space-time visual saliency detection by self-resemblance," J. Vision, vol. 8, no. 12, pp. 1-27, 2009.
25. Y. Xu, Y. Zhao, C. Jin, J. Qu, L. Liu, and X. Sun, "Salient target detection based on pseudo-Wigner-Ville distribution and Renyi entropy," Opt. Lett., vol. 35, no. 4, pp. 475-477, 2010.
26. T. Liu, Z. Yuan, J. Sun, J. Wang, N. Zheng, X. Tang, et al., "Learning to detect a salient object," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 2, pp. 353-367, Feb. 2011.
27. M.-M. Cheng, G.-X. Zhang, N. J. Mitra, X. Huang, and S.-M. Hu, "Global contrast based salient region detection," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., Jun. 2011, pp. 409-416.
28. M. Z. Aziz and B. Mertsching, "Fast and robust generation of feature maps for region-based visual attention," IEEE Trans. Image Process., vol. 17, no. 5, pp. 633-644, May 2008. (Pubitemid 351597156)
29. F. Perazzi, P. Krahenbuhl, Y. Pritch, and A. Hornung, "Saliency filters: Contrast based filtering for salient region detection," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., Jun. 2012, pp. 733-740.
30. J. Tünnermann and B. Mertsching, "Region-based artificial visual attention in space and time," Cognit. Comput., vol. 5, no. 2, pp. 1-19, Jun. 2013.
31. J. Tünnermann and B. Mertsching, "Continuous region-based processing of spatiotemporal saliency," in Proc. VISAPP, 2012, pp. 230-239.
32. M. Wischnewski, A. Belardinelli, W. X. Schneider, and J. J. Steil, "Where to look next? Combining static and dynamic proto-objects in a TVA-based model of visual attention," Cognit. Comput., vol. 2, no. 4, pp. 326-343, 2010.
33. D. Comaniciu and P. Meer, "Mean shift: A robust appraoch toward feature space analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 5, pp. 603-619, May 2002. (Pubitemid 34550429)
34. P. F. Felzenszwalb and D. P. Huttenlocher, "Efficient graph-based image segmentation," Int. J. Comput. Vision, vol. 59, no. 2, pp. 167-181, Sep. 2004.
35. X. Hou and L. Zhang, "Saliency detection: A spectral residual approach," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., Jun. 2007, pp. 1-8.
36. P. Bian and L. Zhang, "Biological plausibility of spectral domain approach for spatiotemporal visual saliency advances in neuro-information processing," in Proc. NIPS, 2009, pp. 251-258.
37. A. B. Chan, V. Mahadevan, and N. Vasconcelos, "Generalized Stauffer- Grimson background subtraction for dynamic scenes," Mach. Vision Applicat., vol. 22, no. 5, pp. 751-766, 2011.
38. L. Wolf, M. Guttmann, and D. Cohen-Or, "Non-homogeneous contentdriven video retargeting," in Proc. IEEE Int. Conf. Comput. Vision, Oct. 2007, pp. 1-6.
39. M. Everingham, L. V. Gool, C. Williams, J. Winn, and A. Zisserman, "The PASCAL visual object classes (VOC) challenge," Int. J. Comput. Vision, vol. 88, no. 2, pp. 303-338, 2010.
40. R. Vezzania and R. Cucchiara. (2009, Oct. 10). Video Surveillance Online Repository (ViSOR): An integrated Framework [Online]. Available: http://imagelab.ing.unimore.it/visor/index.asp
41. S. Avidan and A. Shamir, "Seam carving for content-aware image resizing," ACM Trans. Graph., vol. 26, no. 3, pp. 267-276, 2007.
42. M. Rubinstein, A. Shamir, and S. Avidan, "Improved seam carving for video retargeting," ACM Trans. Graphics, vol. 27, no. 3, pp. 1-8, 2008.
43. S. Cho, H. Choi, Y. Matsushita, and S. Lee, "Image retargeting using importance diffusion," in Proc. IEEE Int. Conf. Image Process., Nov. 2009, pp. 997-980.
44. F. Liu and M. Gleicher, "Automatic image retargeting with fisheye-view warping," in Proc. ACM Annu. Symp. User Interface Softw. Technol., 2005, pp. 153-162. (Pubitemid 44531234)
45. W. Kim and C. Kim, "A texture-aware salient edge model for image retargeting," IEEE Signal Process. Lett., vol. 18, no. 11, pp. 631-634, Nov. 2011.
46. Y. Boykov and G. Funka-Lea, "Graph cuts and efficient N-D image segmentation," Int. J. Comput. Vision, vol. 70, no. 2, pp. 109-131, 2006. (Pubitemid 44127252)
47. C. Rother, V. Kolmogorov, and A. Blake, "Grab cut-interactive foreground extraction using iterated graph cuts," in Proc. ACM SIGGRAPH, 2004, pp. 309-314.
48. A. B. Chan, V. Mahadevan, and N. Vasconcelos, "Generalized Stauffer- Grimson background subtraction for dynamic scenes," Mach. Vision Applicat., vol. 22, no. 5, pp. 751-766, 2011.
49. V. Mahadevan and N. Vasconcelos, "Background subtraction in highly dynamic scenes," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., Jun. 2008, pp. 1-6.
50. L. Li, W. Huang, I. Y.-H. Gu, and Q. Tian, "Statistical modeling of complex backgrounds for foreground object detection," IEEE Trans. Image Process., vol. 13, no. 11, pp. 1459-1472, Nov. 2004.
51. C. Stauffer and W. E. L. Grimson, "Adaptive background mixture models for real-time tracking," in Proc. IEEE Int. Conf. Comput. Vision Pattern Recognit., vol. 2. Jun. 1999, pp. 246-252.
52. G. Dalley, J. Migdal, and W. Grimson, "Background subtraction for temporally irregular dynamic textures," in Proc. IEEE Workshop Applicat. Comput. Vision, Jan. 2008, pp. 1-7.
53. S. Zhang, H. Yao, and S. Liu, "Dynamic background modeling and subtraction using spatio-temporal local binary patterns," in Proc. IEEE Int. Conf. Image Process., Oct. 2008, pp. 1556-1559.