Saliency prediction on stereoscopic videos

IEEE Transactions on Image Processing

Volumn 23, Issue 4, 2014, Pages 1476-1490

  Kim, Haksub ^a   Lee, Sanghoon ^a   Bovik, Alan Conrad ^b  

^a Yonsei University   (South Korea)

^b The University of Texas at Austin   (United States)

Author keywords

eye tracker; human visual system; saliency energy; saliency strength; scene classification; Stereoscopic

Indexed keywords

EYE TRACKERS; HUMAN VISUAL SYSTEM; SALIENCY ENERGY; SALIENCY STRENGTH; SCENE CLASSIFICATION; STEREOSCOPIC;

IMAGE SEGMENTATION; MATHEMATICAL MODELS; THREE DIMENSIONAL COMPUTER GRAPHICS;

THREE DIMENSIONAL;

ADULT; AUTOMATED PATTERN RECOGNITION; DEPTH PERCEPTION; HUMAN; MOTION; PROCEDURES; THEORETICAL MODEL; THREE DIMENSIONAL IMAGING; VIDEORECORDING; YOUNG ADULT;

ADULT; DEPTH PERCEPTION; HUMANS; IMAGING, THREE-DIMENSIONAL; MODELS, THEORETICAL; MOTION; PATTERN RECOGNITION, AUTOMATED; VIDEO RECORDING; YOUNG ADULT;

EID: 84897678080     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2014.2303640     Document Type: Article

References (47)

1. J. Lee and T. Ebrahimi, "Perceptual video compression: A survey," IEEE J. Sel. Topics Signal Process., vol. 6, no. 6, pp. 684-697, Oct. 2012.
2. Q. Huynh-Thu, M. Barkowsky, and P. L. Callet, "The importance of visual attention in improving the 3D-TV viewing experience: Overview and new perspectives," IEEE Trans. Broadcast., vol. 51, no. 2, pp. 421-431, Jun. 2011.
3. L. Itti and C. Koch, "Computational modeling of visual attention," Nature Rev. Neuro Sci., vol. 2, no. 3, pp. 194-203, 2001.
4. L. Itti and C. Koch, "A model of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254-1259, Nov. 1998. (Pubitemid 128742001)
5. N. Bruce and J. K. Tsotsos, "Saliency, attention, and visual search: An information theoretic approach," J. Vis., vol. 9, no. 3, pp. 1-24, 2009.
6. Y. Zhai and M. Shah, "Visual attention detection in video sequences using spatiotemporal cues," in Proc. 14th Annu. ACM Int. Conf. Multimedia, 2006, pp. 815-824. (Pubitemid 47113641)
7. L. Itti and P. Baldi, "Bayesian surprise attracts human attention," Vis. Res., vol. 49, no. 10, pp. 1295-1306, 2009.
8. S. Marat, T. H. Phuoc, L. Granjon, N. Guyader, D. Pellerin, and A. Guerin-Dugue, "Modelling spatio-temporal saliency to predict gaze direction for short videos," Int. J. Comput. Vis., vol. 82, no. 3, pp. 231-243, 2009.
9. L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, "SUN: A Bayesian framework for saliency using natural statistics," J. Vis., vol. 8, no. 7, pp. 1-20, 2008.
10. H. J. Seo and P. Milanfar, "Static and space-time visual saliency detection by self-resemblance," J. Vis., vol. 9, no. 12, pp. 1-27, 2009.
11. J. Harel, C. Koch, and P. Perona, "Graph-based visual saliency," in Proc. Adv. Neural Inf. Process. Syst., 2006, pp. 545-552.
12. U. Rajashekar, I. van der Linde, A. C. Bovik, and L. K. Cormack, "GAFFE: A gaze-attentive fixation finding engine," IEEE Trans. Image Process., vol. 17, no. 4, pp. 564-573, Apr. 2008. (Pubitemid 351493879)
13. P. Reinagel and A. M. Zador, "The effect of gaze on natural scene statistics," in Proc. NICW, Snowbird, UT, USA, 1997, pp. 1-15.
14. Y. Liu, L. K. Cormack, and A. C. Bovik, "Dichotomy between luminance and disparity features at binocular fixations," J. Vis., vol. 10, no. 12 pp. 1-17, 2010.
15. N. Bruce and J. K. Tsotsos, "An attentional framework for stereo vision," in Proc. Can. Conf. Comput. Robot Vis., 2005, pp. 88-95.
16. Y. Zhang, G. Jiang, M. Yu, and K. Chen, "Stereoscopic visual attention model for 3D video," Advances in Multimedia Modeling (ser. Lecture Notes in Computer Science). Berlin, Germany: Springer-Verlag, pp. 314-324, 2010.
17. Y. Ma, X. Hua, L. Lu, and H. Zhang, "A generic framework of user attention model and its application in video summarization," IEEE Trans. Multimedia, vol. 7, no. 5, pp. 907-919, Oct. 2005. (Pubitemid 41452519)
18. C. Siagian and L. Itti, "Rapid biologically-inspired scene classification using features shared with visual attention," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 2, pp. 300-312, Feb. 2007. (Pubitemid 46420889)
19. J. C. van Gemert, J. M. Geusebroek, C. J. Veenman, C. G. M. Snoek, and A. W. M. Smeulders, "Robust scene categorization by learning image statistics in context," in Proc. IEEE CVPR Workshop Semantic Learn. Appl. Multimedia, Jun. 2006, pp. 100-105.
20. A. Woods, T. Docherty, and R. Koch, "Image distortions in stereoscopic video systems," Proc. SPIE, vol. 1915, pp. 36-48, Jan. 1993.
21. D. Kim and K. Sohn, "Visual fatigue prediction for stereoscopic image," IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 2, pp. 231-236, Feb. 2011.
22. M. Lambooij, W. I. Jsselsteijn, M. Fortuin, and I. Heynderickx, "Visual discomfort and visual fatigue of stereoscopic displays: A review," J. Imag. Sci. Technol., vol. 53, no. 3, pp. 1-4, 2009.
23. T. Qhshima, H. Yamamoto, and H. Tamura, "Gaze-directed adaptive rendering for interacting with virtual space," in Proc. VRAIS, 1996, pp. 103-110.
24. M. J. Chen, L. K. Cormack, and A. C. Bovik, "No-reference quality assessment of natural stereopairs," IEEE Trans. Image Process., vol. 22, no. 9, pp. 3379-3391, Sep. 2013.
25. S. Daly, "Engineering observations from spatiovelocity and spatiotemporal visual models," Proc. SPIE, vol. 3299, pp. 180-191, Jul. 1998.
26. P. E. Hallett, Chapter 10 in Handbook of Perception and Human Performance. New York, NY. USA: Wiley, 1986.
27. L. Festinger, H. A. Sedgwick, and J. D. Holtzman, "Visual perception during smooth pursuit eye movements," J. Vis. Res., vol. 16, no. 12, pp. 1377-1386, 1976.
28. R. J. Leigh and D. S. Zee, The Neurology of Eye Movements. Oxford, U.K.: Oxford Univ. Press, 1999.
29. I. A. Rybak, V. I. Gusakova, A. V. Golovan, L. N. Podladchikova, and N. A. Shevtsova, "A model of attention-guided visual perception and recogniton," J. Vis. Res., vol. 38, nos. 15-16, pp. 2387-2400, 1998. (Pubitemid 28342379)
30. J. R. Wilson and S. M. Sherman, "Receptive-field characteristics of neurons in cat striate cortex: Changes with visual field eccentricity," J. Neurophysiol., vol. 29, no. 3, pp. 512-533, 1976.
31. D. Walther, U. Rutishauser, C. Koch, and P. Perona, "On the usefulness of attention for object recognition," in Proc. 8th Eur. Conf. Comput. Vis., 2004, pp. 96-103.
32. K. I. Beverley and D. Regan, "Visual perception of changing size: The effect of object size," J. Vis. Res., vol. 19, no. 10, pp. 1093-1104, 1979. (Pubitemid 10225235)
33. S. Lee, M. S. Pattichis, and A. C. Bovik, "Foveated video compression with optimal rate control," IEEE Trans. Image Process., vol. 10, no. 7, pp. 977-992, Jul. 2001. (Pubitemid 32650874)
34. S. Lee and A. C. Bovik, "Fast algorithms for foveated video processing," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 2, pp. 149-162, Feb. 2003.
35. S. Lee, M. S. Pattichis, and A. C. Bovik, "Foveated video quality assessment," IEEE Trans. Multimedia, vol. 4, no. 1, pp. 129-132, Mar. 2002. (Pubitemid 34291535)
36. W. S. Geisler and J. S. Perry, "A real-time foveated multi-resolution system for low-bandwidth video communication," Proc. SPIE, vol. 3299, pp. 294-305, 1998.
37. Smart Eye Pro-Remote 3D Eye Tracking for Research [Online]. Available: http://www.smarteye.se
38. Z. Chi and H. Yan, "Image segmentation using fuzzy rules derived from K-means clusters," J. Electron. Imaging, vol. 4, no. 2, pp. 199-206, 1995.
39. S. C. Zhu and A. Yuille, "Region competition: Unifying snakes, region growing and bayes/MDL for multiband image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, no. 9, pp. 884-900, Sep. 1996. (Pubitemid 126772831)
40. D. Sun, S. Roth, and M. J. Black, "Secrets of optical flow estimation and their principles," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun 2010, pp. 2432-2439.
41. M. Tanimoto, T. Fujii, K. Suzuki, N. Fukushima, and Y. Mori, "Reference softwares for depth estimation and view synthesis," in Proc. IEL, Apr. 2008, pp. 1-8.
42. (2012). Stereoscopic (3D Imaging) Database [Online]. Available: http://grouper.ieee.org/groups/3dhf/or ftp://165.132.126.47/
43. L. Goldmann, F. De Simone, and T. Ebrahimi, "A comprehensive database and subjective evaluation methodology for quality of experience in stereoscopic video," Proc. SPIE, vol. 7526, pp. 11-15, Jan. 2010.
44. A. Smolic, G. Tech, and H. Brust, "Report on generation of stereo video data base," Mobile3DTV Tech. Rep., vol. 2, no. 1, pp. 1-15, 2010.
45. G. Abdollahian, C. M. Taskiran, Z. Pizlo, and E. J. Delp, "Camera motion-based analysis of user generated video," IEEE Trans. Multimedia, vol. 12, no. 1, pp. 28-41, Jan. 2010.
46. K. T. Mullen, "The contrast sensitivity of human colour vision to redgreen and blue-yellow chromatic gratings," J. Physiol., vol. 359, no. 1, pp. 381-400, 1985. (Pubitemid 15152944)
47. B. W. Tatler, R. J. Baddeley, and I. D. Gilchrist, "Visual correlates of fixation selection: Effects of scale and time," Vis. Res., vol. 45, no. 5, pp. 643-659, 2005. (Pubitemid 40037974)