Visual attention in quality assessment

IEEE Signal Processing Magazine

Volumn 28, Issue 6, 2011, Pages 50-59

  Engelke, Ulrich ^a   Kaprykowsky, Hagen ^b   Zepernick, Hans Jurgen ^c   Ndjiki Nya, Patrick ^b  

^a PHILIPS RESEARCH LABORATORIES   (Netherlands)

^b FRAUNHOFER HEINRICH HERTZ INSTITUTE   (Germany)

^c BLEKINGE INSTITUTE OF TECHNOLOGY   (Sweden)

Author keywords

[No Author keywords available]

Indexed keywords

IMAGE PROCESSING; VIDEO SIGNAL PROCESSING;

COGNITIVE PROCESSING; FOCUS OF ATTENTION; HUMAN VISUAL SYSTEMS; IMAGE AND VIDEO PROCESSING; PERCEPTUAL QUALITY METRICS; PERCEPTUAL QUALITY MODELS; QUALITY ASSESSMENT; VISUAL INFORMATION;

BEHAVIORAL RESEARCH;

EID: 85032777922     PISSN: 10535888     EISSN: None     Source Type: Journal    
DOI: 10.1109/MSP.2011.942473     Document Type: Article

References (70)

1. J. Wolfe, "Visual attention," in Seeing, K. K. D. Valois, Ed. San Diego, CA: Academic, 2000, pp. 335-386.
2. R. Barland and A. Saadane, "Blind quality metric using a perceptual importance map for JPEG-2000 compressed images," in Proc. IEEE Int. Conf. Image Processing, Oct. 2006, pp. 2941-2944.
3. A. Ninassi, O. L. Meur, P. Le Callet, and D. Barba, "Does where you gaze on an image affect your perception of quality? Applying visual attention to image quality metric," in Proc. IEEE Int. Conf. Image Processing, Oct. 2007, vol. 2, pp. 169-172.
4. N. G. Sadaka, L. J. Karam, R. Ferzli, and G. P. Abousleman, "A no reference perceptual image sharpness metric based on saliency weighted foveal pooling," in Proc. IEEE Int. Conf. Image Processing, Oct. 2008, pp. 369-372.
5. A. K. Moorthy and A. C. Bovik, "Visual importance pooling for image quality assessment," IEEE J. Select. Topics Signal Processing, vol. 3, no. 2, pp. 193-201, 2009.
6. I. Gkioulekas, G. Evangelopoulos, and P. Maragos, "Spatial Bayesian surprise for image saliency and quality assessment," in Proc. IEEE Int. Conf. Image Processing, Sept. 2010, pp. 1081-1084.
7. H. Liu and I. Heynderickx, "Visual attention in objective image quality assessment: Based on eye-tracking data," IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 7, pp. 971-982, July 2011.
8. E. C. Larson, C. Vu, and D. M. Chandler, "Can visual fixation patterns improve image fidelity assessment?" in Proc. IEEE Int. Conf. Image Processing, Oct. 2008, pp. 2572-2575.
9. U. Engelke and H.-J. Zepernick, "A framework for optimal region of interest-based quality assessment in wireless imaging," J. Electron. Imaging (Special Section on Image Quality), vol. 19, no. 1, 2010, pp. 1-13.
10. A. Cavallaro and S. Winkler, "Segmentation-driven perceptual quality metrics," in Proc. IEEE Int. Conf. Image Processing, Oct. 2004, vol. 5, pp. 3543-3546. (Pubitemid 40820840)
11. Z. Lu, W. Lin, X. Yang, E. Ong, and S. Yao, "Modeling visual attention's modu-latory after effects on visual sensitivity and quality evaluation," IEEE Trans. Image Processing, vol. 14, no. 11, pp. 1928-1942, 2005. (Pubitemid 41655695)
12. J. You, A. Perkis, M. Hannuksela, and M. Gabbouj, "Perceptual quality assessment based on visual attention analysis," in Proc. ACM Int. Conf. Multimedia, Oct. 2009, pp. 561-564.
13. Q. Ma, L. Zhang, and B. Wang, "New strategy for image and video quality assessment," J. Electron. Imaging (Special Section on Image Quality), vol. 19, no. 1, 2010, pp. 1-14.
14. U. Engelke, M. Barkowsky, P. Le Callet, and H.-J. Zepernick, "Modelling sa-liency awareness for objective video quality assessment," in Proc. Int. Workshop Quality Multimedia Experience, June 2010, pp. 212-217.
15. U. Engelke, R. Pepion, P. Le Callet, and H.-J. Zepernick, "Linking distortion perception and visual saliency in H.264/AVC coded video containing packet loss," in Proc. SPIE/IEEE Int. Conf. Visual Communications and Image Processing, July 2010.
16. J. You, J. Korhonen, and A. Perkis, "Attention modeling for video quality assessment: Balancing global quality and local quality," in Proc. IEEE Int. Conf. Multimedia and Expo, July 2010, pp. 914-919.
17. O. Le Meur, A. Ninassi, P. Le Callet, and D. Barba, "Overt visual attention for free-viewing and quality assessment tasks: Impact of the regions of interest on a video quality metric," Signal Process. Image Commun., vol. 25, no. 7, pp. 547-558, 2010.
18. X. Gao, N. Liu, W. Lu, D. Tao, and X. Li, "Spatio-temporal salience based video quality assessment," in Proc. IEEE Int. Conf. Systems, Man and Cypernetics, Oct. 2010, pp. 1501-1505.
19. X. Feng, T. Liu, D. Yang, and Y. Wang, "Saliency inspired full reference quality metrics for packet-loss-impaired video," IEEE Trans. Broadcast., vol. 57, no. 1, pp. 81-88, Mar. 2011. ́
20. D. Culibrk, M. Mirković, V. Zlokolica, M. Pokrić, V. Crnojević, and D. Kukolj, "Salient motion features for video quality assessment," IEEE Trans. Image Processing, vol. 20, no. 4, pp. 948-958, Apr. 2011.
21. D. Burr, M. C. Morrone, and J. Ross, "Selective suppression of the magnocel-lular visual pathway during saccadic eye movements," Nature, vol. 371, no. 6497, p p. 511-513, 19 94. (Pubitemid 24306607)
22. E. Kowler, "Eye movements: The past 25 years," Vision Res., vol. 51, no. 13, pp. 1457-1483, 2011.
23. M. Carrasco, "Visual attention: The past 25 years," Vision Res., vol. 51, no. 13, pp. 1484-1525, 2011.
24. A. T. Smith, K. D. Singh, and M. W. Greenlee, "Attentional suppression of activity in the human visual cortex," Neuroreport, vol. 11, no. 2, pp. 271-278, 2000.
25. J. M. Wolfe and T. S. Horowitz, "What attributes guide the deployment of visual attention and how do they do it?" Nature Rev. Neurosci., vol. 5, pp. 1-7, June 2004.
26. A. Torralba, A. Oliva, M. Castelhano, and J. Henderson, "Contextual guidance of eye movements and attention in real-world scenes: The role of global features on object search," Psychol. Rev., vol. 113, no. 4, pp. 766-786, 2006. (Pubitemid 44629917)
27. B. Khurana and E. Kowler, "Shared attentional control of smooth eye movement and perception," Vision Res., vol. 27, no. 9, pp. 1603-1618, 1987. (Pubitemid 17145836)
28. L. Itti and P. Baldi, "Bayesian surprise attracts human attention," Vision Res., vol. 49, no. 10, pp. 1295-1306, 2009.
29. V. Mahadevan and N. Vasconcelos, "Spatiotemporal saliency in dynamic scenes," IEEE Trans. Pattern Anal. Machine Intell., vol. 32, no. 1, pp. 171-177, Jan. 2010.
30. B. J. Murphy, "Pattern thresholds for moving and stationary gratings during smooth eye movement," Vision Res., vol. 18, no. 5, pp. 521-530, 1978. (Pubitemid 8349757)
31. S. Treue, "Visual attention: The where, what, how and why of saliency," Curr. Opin. Neurobiol., vol. 13, no. 4, pp. 428-432, 2003. (Pubitemid 37088232)
32. M. Maggs. (2007). Colouring pencils. [Online]. Available: http://commons. wikimedia.org/wiki/File:Colouring pencils.jpg
33. A. M. Treisman and G. Gelade, "A feature-integration theory of attention," Cogn. Psychol., vol. 12, no. 1, pp. 97-136, 1980.
34. J. M. Wolfe, K. R. Cave, and S. L. Franzel, "Guided search: An alternative to the feature integration model for visual search," J. Exp. Psychol. Hum. Percept. Perform., vol. 15, no. 3, pp. 419-433, 1989.
35. C. Koch and S. Ullman, "Shifts in selection in visual attention: Towards the underlying neural circuitry," Hum. Neurobiol., vol. 4, no. 4, pp. 219-227, 1985. (Pubitemid 16176695)
36. H. J. Seo and P. Milanfar, "Static and space-time visual saliency detection by self-resemblance," J. Vis., vol. 9, no. 12:15, pp. 1-27, 2009.
37. N. D. B. Bruce and J. K. Tsotsos, "Saliency, attention, and visual search: An information theoretic approach," J. Vis., vol. 9, no. 3:5, pp. 1-24, 2009.
38. L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, "SUN: A Bayesian framework for saliency using natural statistics," J. Vis., vol. 8, no. 7:32, pp. 1-20, 2008.
39. L. Itti, C. Koch, and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Anal. Machine Intell., vol. 20, no. 11, pp. 1254-1259, 1998. (Pubitemid 128742001)
40. O. Le Meur, P. Le Callet, D. Barba, and D. Thoreau, "A coherent computational approach to model bottom-up visual attention," IEEE Trans. Pattern Anal. Machine Intell., vol. 28, no. 5, pp. 802-817, 2006.
41. U. Rajashekar, I. van der Linde, A. C. Bovik, and L. K. Cormack, "GAFFE: A gaze-attentive fixation finding engine," IEEE Trans. Image Processing, vol. 17, no. 4, pp. 564-573, 2008. (Pubitemid 351493879)
42. R. Achanta, S. Hemami, F. Estrada, and S. Süsstrunk, "Frequency-tuned salient region detection," in Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition, June 2009, pp. 1597-1604.
43. L. Itti and C. Koch, "Feature combination strategies for saliency-based visual attention systems," J. Electron. Imaging, vol. 10, no. 1, pp. 161-169, 2001. (Pubitemid 32279366)
44. H. C. Nothdurft, "Salience from feature contrast: Additivity across dimensions," Vision Res., vol. 40, no. 10-12, pp. 1183-1201, 2000.
45. A. Toet, "Computational versus psychophysical bottom-up image saliency: A comparative evaluation study," IEEE Trans. Pattern Anal. Machine Intell., vol. 99, 2011.
46. D. M. Levi, "Crowding-An essential bottleneck for object recognition: A mini review," Vision Res., vol. 48, no. 5, pp. 635-654, 2008.
47. S. P. McKee and K. Nakayama, "The detection of motion in the peripheral visual field," Vision Res., vol. 24, no. 1, pp. 25-32, 1984. (Pubitemid 14221098)
48. W. Osberger and A. M. Rohaly, "Automatic detection of regions of interest in complex video sequences," in Proc. IS&T/SPIE Human Vision and Electronic Imaging VI, Jan. 2001, vol. 4299, pp. 361-372. (Pubitemid 32934416)
49. Y. Zhai and M. Shah, "Visual attention detection in video sequences using spatiotemporal cues," in Proc. ACM Int. Conf. Multimedia, Oct. 2006, pp. 815-824. (Pubitemid 47113641)
50. C. Guo, Q. Ma, and L. Zhang, "Spatio-temporal saliency detection using phase spectrum of quaternion Fourier transform," in Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition, June 2008, pp. 1-8.
51. F. Navalpakkam and L. Itti, "Modeling the influence of task on attention," Vision Res., vol. 45, no. 2, pp. 205-231, 2005. (Pubitemid 39575753)
52. Y. F. Ma, X. S. Hua, L. Lu, and H. J. Zhang, "A generic framework of user attention model and its application in video summarization," IEEE Trans. Multimedia, vol. 7, no. 5, pp. 907-919, Oct. 2005. (Pubitemid 41452519)
53. C. Kanan, M. H. Tong, L. Zhang, and G. W. Cottrell, "SUN: Top-down saliency using natural statistics," Vis. Cogn., vol. 17, no. 6, pp. 979-1003, 2009.
54. S. Frintrop, E. Rome, and H. I. Christensen, "Computational visual attention systems and their cognitive foundations: A survey," ACM Trans. Appl. Percept., vol. 7, no. 1, pp. 1-46, 2010.
55. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Processing, vol. 13, no. 4, pp. 600-612, 2004.
56. D. Walther and C. Koch, "Modeling attention to salient proto-objects," Neural Netw., vol. 19, no. 9, pp. 1395-1407, 2006. (Pubitemid 44708603)
57. H. R. Sheikh and A. C. Bovik, "Image information and visual quality," IEEE Trans. Image Processing, vol. 15, no. 2, pp. 430-444, Feb. 2006. (Pubitemid 43126168)
58. H. R. Wu and M. Yuen, "A generalized block-edge impairment metric for video coding," IEEE Signal Processing Lett., vol. 4, no. 11, pp. 317-320, Nov. 1997. (Pubitemid 127554948)
59. D. M. Chandler and S. S. Hemami, "VSNR: A wavelet-based visual signal-to-noise ratio for natural images," IEEE Trans. Image Processing, vol. 16, no. 9, pp. 2284-2298, Sept. 2007. (Pubitemid 47316675)
60. M. Barkowsky, J. Bialkowski, B. Eskofier, R. Bitto, and A. Kaup, "Temporal trajectory aware video quality measure," IEEE J. Select. Topics Signal Processing, vol. 3, no. 2, pp. 266-279, 2009.
61. U. Engelke and H.-J. Zepernick, "Psychophysical assessment of perceived interest in natural images: The ROI-D database," in Proc. SPIE/IEEE Int. Conf. Visual Communications and Image Processing, Dec. 2011.
62. U. Engelke, A. J. Maeder, and H.-J. Zepernick, "Visual attention modelling for subjective image quality databases," in Proc. IEEE Int. Workshop Multimedia Signal Processing, Oct. 2009, pp. 1-6.
63. R. J. Peters, A. Iyer, L. Itti, and C. Koch, "Components of bottom-up gaze allocation in natural images," Vision Res., vol. 45, no. 18, pp. 2397-2416, Aug. 2005. (Pubitemid 40848284)
64. C. M. Masciocchi, S. Mihalas, D. Parkhurst, and E. Niebur, "Everyone knows what is interesting: Salient locations which should be fixated," J. Vision, vol. 9, no. 11:25, pp. 1-22, 2009.
65. J. Wang, D. M. Chandler, and P. Le Callet, "Quantifying the relationship between visual salience and visual importance," in Proc. IS&T/SPIE Human Vision and Electronic Imaging XV, Jan. 2010, vol. 7527.
66. M. S. Castelhano, M. L. Mack, and J. M. Henderson, "Viewing task influences eye movement control during active scene perception," J. Vision, vol. 9, no. 3:6, pp. 1-15, 2009.
67. C. T. Vu, E. C. Larson, and D. M. Chandler, "Visual fixation patterns when judging image quality: Effects of distortion type, amount, and subject experience," in Proc. IEEE Southwest Symp. Image Analysis and Interpretation, Mar. 2008, pp. 73-76.
68. U. Engelke, "Modelling perceptual quality and visual saliency for image and video communications," Ph.D. dissertation, Blekinge Inst. Technol., Karlskrona, Sweden, 2010.
69. O. Le Meur, A. Ninassi, P. Le Callet, and D. Barba, "Do video coding impairments disturb the visual attention deployment?" Signal Process. Image Commun., vol. 25, no. 8, pp. 597-609, 2010.
70. J. S. Lee, F. de Simone, and T. Ebrahimi, "Influence of audio-visual attention on perceived quality of standard definition multimedia content," in Proc. Int. Workshop Quality Multimedia Experience, July 2009, pp. 13-18.