Attention driven foveated video quality assessment

IEEE Transactions on Image Processing

Volumn 23, Issue 1, 2014, Pages 200-213

  You, Junyong ^a   Ebrahimi, Touradj ^b   Perkis, Andrew ^c  

^a CHRISTIAN MICHELSEN RESEARCH AS   (Norway)

^b EPFL   (Switzerland)

^c NORWEGIAN UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Norway)

Author keywords

Fixation prediction; Foveal imaging model; Video attention model; Video quality assessment; Visual perception

Indexed keywords

ATTENTION MODEL; CONTRAST SENSITIVITY; CONTRAST SENSITIVITY FUNCTION; FOVEAL IMAGING; HUMAN VISUAL SYSTEM; VIDEO QUALITY ASSESSMENT; VIDEO QUALITY DATABASE; VISUAL PERCEPTION;

EYE MOVEMENTS; FORECASTING;

VIDEO SIGNAL PROCESSING;

ALGORITHM; ARTICLE; ATTENTION; AUTOMATED PATTERN RECOGNITION; BIOMIMETICS; COMPUTER ASSISTED DIAGNOSIS; HUMAN; IMAGE ENHANCEMENT; METHODOLOGY; PATTERN RECOGNITION; PHYSIOLOGY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; VIDEORECORDING;

ALGORITHMS; ATTENTION; BIOMIMETICS; HUMANS; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; PATTERN RECOGNITION, AUTOMATED; PATTERN RECOGNITION, VISUAL; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; VIDEO RECORDING;

EID: 84888378342     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2013.2287611     Document Type: Article

References (58)

1. A. Pica, M. Isnardi, and J. Lubin, "HVS based perceptual video encoders," in Digital Video Image Quality and Perceptual Coding, H. R. Wu and K. R. Zhao, Eds. Boca Raton, FL, USA: CRC Press, 2006, ch. 12, pp. 337-360.
2. J. You, U. Reiter, M. M. Hannuksela, M. Gabbouj, and A. Perkis, "Perceptual-based objective quality metrics for audio-visual services-A survey," Signal Process. Image Commun., vol. 25, no. 7, pp. 482-501, 2010.
3. "Subjective video quality assessment methods for multimedia applications," International Telecommunication Union, Geneva, Switzerland, Recommendation ITU-T P.910, 1999.
4. S. Winkler, Digital Video Quality: Vision Models and Metrics. New York, NY, USA: Wiley, 2005.
5. M. Pinson and S. Wolf, "A new standardized method for objectively measuring video quality," IEEE Trans. Broadcast., vol. 50, no. 3, pp. 312-322, Sep. 2004.
6. K. Seshadrinathan, and A. C. Bovik, "Motion tuned spatio-temporal quality assessment of natural videos," IEEE Trans. Image Process., vol. 19, no. 2, pp. 335-350, Feb. 2010.
7. U. Engelke, and H.-J. Zepernick, "Perceptual-based quality metrics for image and video services: A survey," in Proc. 3rd EuroNGI Conf. Next Generat. Internet Netw., May 2007, pp. 190-197. (Pubitemid 47439987)
8. S. S. Hemami and A. R. Reibman, "No-reference image and video quality estimation: Applications and human-motivated design," Signal Process. Image Commun., vol. 25, no. 7, pp. 469-481, 2010.
9. C. A. Burbeck and D. H. Kelly, "Spatiotemporal characteristics of visual mechanisms: Excitatory-inhibitory model," J. Opt. Soc. Amer., vol. 70, no. 9, pp. 1121-1126, 1980.
10. W. S. Geisler and J. S. Perry, "A real-time foveated multi-resolution system for low-bandwidth video communication," Proc. SPIE, vol. 3299, pp. 294-305, Jan. 1998.
11. Z. Wang, A. C. Bovik, L. Lu, and J. Kouloheris, "Foveated wavelet image quality index," Proc. SPIE, vol. 4472, pp. 42-52, Jul./Aug. 2001. (Pubitemid 35152814)
12. S. Lee, M. S. Pattichis, and A. C. Bovik, "Foveated video quality assessment," IEEE Trans. Multimedia, vol. 4, no. 1, pp. 129-132, Mar. 2002. (Pubitemid 34291535)
13. D. H. Kelly, "Motion and vision II. Stabilized spatio-temporal threshold surface," J. Opt. Soc. Amer., vol. 69, no. 10, pp. 1340-1349, Oct. 1979.
14. S. Rimac-Drlje, M. Vranješ, and D. Žagar, "Foveated mean squared error-A novel video quality metric," Multimedia Tools Appl., vol. 49, no. 3, pp. 425-445, Sep. 2010.
15. W. James, The Principles of Psychology. Cambridge, MA, USA: Harvard Univ. Press, 1890.
16. A. Treisman and G. Gelade, "A feature-integration theory of attention," Cognit. Psychol., vol. 12, no. 1, pp. 97-136, 1980.
17. A. Borji and L. Itti, "State-of-the-art in visual attention modeling," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 1, pp. 185-207, Jan. 2013.
18. L. Itti and C. Koch, "Computational modeling of visual attention," Nature Rev. Neurosci., vol. 2, no. 3, pp. 194-203, Mar. 2001.
19. J. Harel, C. Koch, and P. Perona, "Graph-based visual saliency," in Advances in Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2007, pp. 681-688.
20. O. Le Meur, P. Le Callet, and D. Barba, "Predicting visual fixations on video based on low-level visual features," Vis. Res., vol. 47, no. 19, pp. 2483-2498, Sep. 2007. (Pubitemid 47376844)
21. E. Vig, M. Dorr, T. Martinetz, and E. Barth, "Intrinsic dimensionality predicts the saliency of natural dynamic scenes," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 6, pp. 1080-1091, Jun. 2012.
22. F. Baluch and L. Itti, "Mechanisms of top-down attention," Trends Neurosci., vol. 34, pp. 210-224, Mar. 2011.
23. C. T. Vu, E. C. Larson, and D. M. Chandler, "Visual fixation pattern when judging image quality: Effects of distortion type, amount, and subject experience," in Proc. IEEE Southwest Symp. Image Anal. Interpret., Mar. 2008, pp. 73-76.
24. A. Ninassi, O. Le Meur, P. Le Callet, D. Barba, and A. Tirel, "Task impact on the visual attention in subjective image quality assessment," in Proc. Eur. Conf. Signal Process., Florence, Italy, 2006, pp. 1-5.
25. M. S. Gide, and L. J. Karam, "Comparative evaluation of visual saliency models for quality assessment task," in Proc. 7th Int. Workshop VPQM, Jan. 2012. [Online]. Available: http://enpub.fulton.asu.edu/resp/vpqm/vpqm12/Papers/vpqm12-p16.pdf
26. U. Engelke, H. Kaprykowsky, H. Zepernick, and P. Ndjiki-Nya, "Video attention in quality assessment," IEEE Signal Process. Mag., vol. 28, no. 6, pp. 50-59, Nov. 2011.
27. J. You, "Attention driven visual QoE: Mechanism and methodologies," in Proc. IEEE China Summit, Int. Conf. Signal Inf. Process., Beijing, China, Jul. 2013, pp. 466-470.
28. O. Le Meur, A. Ninassi, P. Le Callet, and D. Barba, "Do video coding impairments disturb the visual attention deployment?" Signal Process. Image Commun., vol. 25, no. 8, pp. 597-609, 2010.
29. U. Engelke, "Modeling perceptual quality and visual saliency for image and video communications," Ph.D. dissertation, Dept. Electr. Eng., Blekinge Institute of Technology, Karlskrona, Sweden, 2010.
30. L. Itti and P. Baldi, "Bayesian surprise attracts human attention," Vis. Res., vol. 49, no. 10, pp. 1295-1306, 2009.
31. J. You, J. Korhonen, A. Perkis, and T. Ebrahimi, "Balancing attended and global stimuli in perceived video quality assessment," IEEE Trans. Multimedia, vol. 13, no. 6, pp. 1269-1265, Dec. 2011.
32. W. Wang, C. Chen, Y. Wang, T. Jiang, F. Fang, and Y. Yao, "Simulating human saccadic scanpaths on natural images," in Proc. IEEE Int. Conf. CVPR, Jun. 2011, pp. 441-448.
33. U. Rajashekar, I. V. D. Linde, A. C. Bovik, and L. K. Cormack, "GAFFE: A gaze-attentive fixation finding engine," IEEE Trans. Image Process., vol. 17, no. 4, pp. 564-573, Apr. 2008. (Pubitemid 351493879)
34. J. You, A. Perkis, M. M. Hannuksela, and M. Gabbouj, "Perceptual quality assessment based on visual attention analysis," in Proc. ACM Multimedia, Beijing, China, Oct. 2009, pp. 561-564.
35. Y.-F. Ma, X.-S. Hua, L. Lu, and H.-J. Zhang, "A generic framework of user attention model and its application in video summarization," IEEE Trans. Multimedia, vol. 7, no. 5, pp. 907-919, Oct. 2005. (Pubitemid 41452519)
36. (2009). IRCCyN/IVC Eyetracker SD 2009 12 Database [Online]. Available: http://www.irccyn.ec-nantes.fr/spip.php?article550&lang=en
37. S. Ramanathan, H. Katti, N. Sebe, M. Kankanhalli, and T.-S. Chua, "An eye fixation database for saliency detection in images," in Proc. Eur. Conf. Comput. Vis., Sep. 2010, pp. 30-43.
38. T. Fawcett, "An introduction to ROC analysis," Pattern Recognit. Lett., vol. 27, no. 8, pp. 861-874, Jun. 2006.
39. B. D. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," in Proc. Int. Conf. Artif. Intell., vol. 2. San Francisco, CA, USA, 1981, pp. 674-679. (Pubitemid 14478744)
40. S. L. Phung, A. Bouzerdoum, and D. Chai, "Skin segmentation using color pixel classification: Analysis and comparison," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 1, pp. 148-154, Jan. 2005. (Pubitemid 40195906)
41. X. K. Yang, W. S. Ling, Z. K. Lub, E. P. Ongb, and S. S. Yaob, "Just noticeable distortion model and its applications in video coding," Signal Process. Image Commun., vol. 20, no. 7, pp. 662-680, 2005. (Pubitemid 41023703)
42. P. J. Burt and E. H. Adelson, "The Laplacian pyramid as a compact image code," IEEE Trans. Commun., vol. 31, no. 4, pp. 532-540, Apr. 1983.
43. A. C. Schütz, D. I. Braun, D. Kerzel, and K. R. Gegenfurtner, "Improved visual sensitivity during smooth pursuit eye movements," Nature Neurosci., vol. 11, no. 10, pp. 1211-1216, Oct. 2008.
44. S. Daly, "Engineering observations from spatiovelocity and spatiotemporal visual models," Proc. SPIE, vol. 3299, pp. 180-191, Jan. 1998.
45. M. P. Eckert and G. Buchsbaum, "The significance of eye movements and image acceleration for coding television image sequences," in Digital Images and Human Vision, A. B. Watson, Ed., Cambridge, MA, USA: MIT Press, 1993, pp. 89-98.
46. F. Pestilli and M. Carrasco, "Attention enhances contrast sensitivity at cued and impairs it at uncued locations," Vis. Res., vol. 45, no. 14, pp. 1867-1875, Jun. 2005. (Pubitemid 40432076)
47. I. Motoyoshi, "Attentional modulation of temporal contrast sensitivity in human vision," PLoS ONE, vol. 6, no. 4, p. e19303, 2011. doi: 10.1371/journal.pone.0019303.
48. H. E. Pashler, The Psychology of Attention. Cambridge, MA, USA: MIT Press, 1998.
49. A. B. Watson, G. Y. Yang, J. A. Solomon, and J. Villasenor, "Visibility of wavelet quantization noise," IEEE Trans. Image Process., vol. 6, no. 8, pp. 1164-1175, Aug. 1997. (Pubitemid 127768411)
50. A. C. Schütz, D. I. Braun, and K. R. Gegenfurtner, "Eve movements and perception: A selective review," J. Vis., vol. 11, no. 5, pp. 1-30, Sep. 2011.
51. S. Li, L, Ma, and K. N. Ngan, "Full-reference video quality assessment by decoupling detail losses and additive impairments," IEEE. Trans. Circuits Syst. Video Technol., vol. 22, no. 7, pp. 1100-1112, Jul. 2012.
52. J. You, J. Korhonen, and A. Perkis, "Attention modeling for video quality assessment: Balancing global quality and local quality," in Proc. IEEE Int. Conf. Multimedia Expo, Singapore, Jul. 2010, pp. 914-919.
53. A. Ninassi, O. Le Meur, P. Le Callet, and D. Barba, "Considering temporal variations of spatial visual distortions in video quality assessment," IEEE J. Sel. Topics Signal Process., vol. 3, no. 2, pp. 253-265, Apr. 2009.
54. M. Zink, O. Künzel, J. Schmitt, and R. Steinmetz, "Subjective impression of variations in layer encoded videos," Lecture Notes Comput. Sci., vol. 2707. pp. 137-154, Jan. 2003.
55. J. You, T. Ebrahimi, and A. Perkis, "Modeling motion visual perception for video quality assessment," in Proc. ACM Int. Conf. Multimedia, Scottsdale, AZ, USA, 2011, pp. 1293-1296.
56. K. Seshadrinathan, R. Soundararajan, A. C. Bovik, and L. L. Cormack, "Study of subjective and objective quality assessment of video," IEEE Trans. Image Process., vol. 19, no. 6, pp. 1427-1441, Jun. 2010.
57. (2010, Jun.). Report on the Validation of Video Quality Models for High Definition Video Content [Online]. Available: http://www.its.bldrdoc.gov/vqeg/projects/hdtv/hdtv.aspx
58. M. Pinson and S. Wolf, "An objective method for combining multiple subjective data sets," Proc. SPIE, vol. 5150, pp. 583-592, Jun. 2003.