Video quality assessment based on measuring perceptual noise from spatial and temporal perspectives

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 21, Issue 12, 2011, Pages 1890-1902

  Zhao, Yin ^a   Yu, Lu ^a   Chen, Zhenzhong ^b   Zhu, Ce ^b  

^a ZHEJIANG UNIVERSITY   (China)

^b NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

Human visual system; just noticeable difference; perceptual quality index; temporal noise; video quality assessment

Indexed keywords

HUMAN VISUAL SYSTEM; JUST-NOTICEABLE DIFFERENCE; PERCEPTUAL QUALITY INDEX; TEMPORAL NOISE; VIDEO QUALITY ASSESSMENT;

ERROR DETECTION; IMAGE PROCESSING; QUALITY ASSURANCE; VIDEO SIGNAL PROCESSING;

RATING;

EID: 83255166542     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2011.2157189     Document Type: Review

References (67)

1. H. R. Wu and K. R. Rao, Digital Video Image Quality and Perceptual Coding. Boca Raton, FL: CRC Press, 2006, chs. 4, 9, 14.
2. Methodology for the Subjective Assessment of the Quality of Television Pictures, ITU-R Rec. BT.500-11, International Telecommunication Union, Geneva, Switzerland, 2002.
3. M. H. Pinson and S. Wolf, "A new standardized method for objectively measuring video quality," IEEE Trans. Broadcast., vol. 50, no. 3, pp. 312-322, Sep. 2004.
4. A. P. Hekstra, J. G. Beerends, D. Ledermann, F. E. de Caluwe, S. Kohler, R. H. Koenen, S. Rihs, M. Ehrsam, and D. Schlauss, "PVQM: A perceptual video quality measure," Signal Process. Image Commun., vol. 17, no. 10, pp. 781-798, Nov. 2002. (Pubitemid 35282146)
5. A. A. Webster, C. T. Jones, M. H. Pinson, S. D. Voran, and S. Wolf, "An objective video quality assessment system based on human perception," in Proc. 4th SPIE Hum. Vis. Visual Process. Digit. Display, Feb. 1993, pp. 15-26.
6. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, Apr. 2004.
7. H. R. Sheikh and A. C. Bovik, "Image information and visual quality," IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006. (Pubitemid 43126168)
8. D. M. Chandler and S. S. Hemami, "VSNR: A wavelet-based visual signal-to-noise ratio for natural images," IEEE Trans. Image Process., vol. 16, no. 9, pp. 2284-2298, Sep. 2007. (Pubitemid 47316675)
9. K. Seshadrinathan and A. C. Bovik, "Motion tuned spatio-temporal quality assessment of natural videos," IEEE Trans. Image Process., vol. 19, no. 2, pp. 335-350, Feb. 2010.
10. A. Ninassi, O. Le Meur, P. Le Callet, and D. Barba, "Considering temporal variations of spatial visual distortions in video quality assessment," IEEE J. Sel. Top. Signal Process., vol. 3, no. 2, pp. 253-265, Apr. 2009.
11. M. Barkowsky, J. Bialkowski, B. Eskofier, R. Bitto, and A. Kaup, "Temporal trajectory aware video quality measure," IEEE J. Sel. Top. Signal Process., vol. 3, no. 2, pp. 266-279, Apr. 2009.
12. Y. Zhao and L. Yu, "Evaluating video quality with temporal noise," in Proc. IEEE ICME, Jul. 2010, pp. 708-712.
13. Z. Wang, L. Lu, and A. C. Bovik, "Video quality assessment based on structural distortion measurement," Signal Process. Image Commun., vol. 19, no. 2, pp. 121-132, Feb. 2004.
14. Z. Wang and Q. Li, "Video quality assessment using a statistical model of human visual speed perception," J. Opt. Soc. Am. A Opt. Image Sci. Vis., vol. 24, no. 12, pp. B61-B69, Dec. 2007. (Pubitemid 351325125)
15. H. Marmolin, "Subjective MSE measures," IEEE Trans. Syst. Man Cybern., vol. 16, no. 3, pp. 486-489, May 1986.
16. K. T. Tan, M. Ghanbari, and D. E. Pearson, "An objective measurement tool for MPEG video quality," Signal Process. Image Commun., vol. 70, no. 3, pp. 279-294, Nov. 1998. (Pubitemid 128418940)
17. M. A. Masry and S. S. Hemami, "A metric for continuous quality evaluation of compressed video with severe distortions," Signal Process. Image Commun., vol. 19, no. 2, pp. 133-146, Feb. 2004.
18. N. Jayant, J. Johnston, and R. Safranek, "Signal compression based on models of human perception," Proc. IEEE, vol. 81, no. 10, pp. 1385-1422, Oct. 1993.
19. K. Seshadrinathan, R. Soundararajan, A. C. Bovik, and L. K. Cormack, "Study of subjective and objective quality assessment of video," IEEE Trans. Image Process., vol. 19, no. 16, pp. 1427-1441, Jun. 2010.
20. S. Winkler, "Issues in vision modeling for perceptual video quality assessment," Signal Process., vol. 78, no. 2, pp. 231-252, Oct. 1999. (Pubitemid 32078185)
21. A. Duchowski, Eye Tracking Methodology: Theory and Practice, 2nd ed. London, U.K.: Springer, 2007, chs. 2-3.
22. V. Virsu, R. Näsänen, and K. Osmoviita, "Cortical magnification and peripheral vision," J. Opt. Soc. Am. A, vol. 4, pp. 1568-1578, Aug. 1987.
23. S. L. Sally and R. Gurnsey, "Foveal and extra-foveal orientation discrimination," Exp. Brain Res., vol. 183, no. 3, pp. 351-360, Jul. 2007. (Pubitemid 47568659)
24. P. Mäkelä, J. Rovamo, and D. Whitaker, "The effects of eccentricity and stimulus magnification on simultaneous performance in position and movement acuity tasks," Vis. Res., vol. 37, pp. 1261-1270, Apr. 1997. (Pubitemid 27202161)
25. C. Ware, Information Visualization: Perception for Design, 2nd ed. San Francisco, CA: Morgan Kaufmann, 2004, chs. 1, 2, 11.
26. O. Le Meur, P. Le Callet, and D. Barba, "Predicting visual fixations on video based on low-level visual features," Vis. Res., vol. 47, no. 19, pp. 2483-2498, Sep. 2007. (Pubitemid 47376844)
27. C. Rashbass, "The relationship between saccadic and smooth tracking eye movements," J. Physiol., vol. 159, no. 2, pp. 326-338, 1961.
28. L. Itti and C. Koch, "Computational modeling of visual attention," Nature Rev. Neurosci., vol. 2, no. 3, pp. 194-203, Mar. 2001.
29. D. Parkhurst, K. Law, and E. Niebur, "Modeling the role of salience in the allocation of overt visual attention," Vis. Res., vol. 42, no. 1, pp. 107-123, Jan. 2002. (Pubitemid 34135927)
30. Z. Lu, W. Lin, X. Yang, E. Ong, and S. Yao, "Modeling visual attention's modulatory aftereffects on visual sensitivity and quality evaluation," IEEE Trans. Image Process., vol. 14, no. 11, pp. 1928-1942, Nov. 2005. (Pubitemid 41655695)
31. A. K. Moorthy and A. C. Bovik, "Visual importance pooling for image quality assessment," IEEE J. Sel. Top. Signal Process., vol. 3, no. 2, pp. 193-201, Apr. 2009.
32. O. Le Meur, A. Ninassi, P. Le Callet, and D. Barba, "Overt visual attention for free-viewing and quality assessment tasks: Impact of the regions of interest on a video quality metric," Signal Process. Image Commun., vol. 25, no. 7, pp. 547-558, Aug. 2010.
33. L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, "SUN: A Bayesian framework for saliency using natural statistics," J. Vis., vol. 8, no. 7, pp. 1-20, 2008.
34. O. Le Meur, P. Le Callet, D. Barba, and D. Thoreau, "A coherent computational approach to model bottom-up visual attention," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 5, pp. 802-817, May 2006.
35. B. W. Tatler, "The central fixation bias in scene viewing: Selecting an optimal viewing position independently of motor biases and image feature distributions," J. Vis., vol. 7, no. 14, pp. 1-17, 2007.
36. V. Manahilov, W. A. Simpson, and J. Calvert, "Why is second-order vision less efficient than first-order vision?" Vis. Res., vol. 45, no. 21, pp. 2759-2772, Oct. 2005. (Pubitemid 41140208)
37. A. J. Schofield and M. A. Georgeson, "The temporal properties of first and second-order vision," Vis. Res., vol. 40, pp. 2475-2487, Aug. 2000. (Pubitemid 30466284)
38. D. H. Kelly, "Visual responses to time-dependent stimuli: I. Amplitude sensitivity measurements," J. Opt. Soc. Am., vol. 51, no. 4, pp. 422-429, 1961.
39. B. G. Breitmeyer, "Visual masking: Past accomplishments, present status, future developments," Adv. Cognitive Psychol., vol. 3, nos. 1-2, pp. 9-20, 2007.
40. G. E. Legge and J. M. Foley, "Contrast masking in human vision," J. Opt. Soc. Am., vol. 70, no. 12, pp. 1458-1471, Dec. 1980.
41. N. Li, S. Desmet, A. Deknuydt, and L. V. Eycken, "Motion adaptive quantization in transform coding for exploiting motion masking effect," in Proc. SPIE Visual Commun. Image Process., Nov. 1992, pp. 1116-1123. (Pubitemid 23624936)
42. J. H. D. M. Westerink and K. Teunissen, "Perceived sharpness in complex moving images," Displays, vol. 16, no. 2, pp. 89-97, 1995.
43. R. Marois and J. Ivanoff, "Capacity limits of information processing in the brain," Trends Cognitive Sci., vol. 9, pp. 296-305, Jun. 2005. (Pubitemid 40733682)
44. G. Houghton and S. P. Tipper, "Inhibitory mechanisms of neural and cognitive control: Applications to selective attention and sequential action," Brain Cognition, vol. 30, no. 1, pp. 20-43, Feb. 1996. (Pubitemid 126684872)
45. C. H. Chou and C. W. Chen, "A perceptually optimized 3-D subband codec for video communication over wireless channels," IEEE Trans. Circuits Syst. Video Technol., vol. 6, no. 2, pp. 143-156, Apr. 1996.
46. V. Kayargadde and J. B. Martens, "An objective measure for perceived noise," Signal Process., vol. 49, no. 3, pp. 187-206, Mar. 1996. (Pubitemid 126349426)
47. S. Winkler, "Quality metric design: A closer look," in Proc. 5th SPIE Hum. Vis. Electron. Imaging, Jun. 2000, pp. 37-44.
48. Z. Wang and X. Shang, "Spatial pooling strategies for perceptual image quality assessment," in Proc. IEEE ICIP, Oct. 2006, pp. 2945-2948.
49. R. Aldridge, J. Davidoff, M. Ghanbari, D. Hands, and D. Pearson, "Measurement of scene-dependent quality variations in digitally coded television pictures," IEE Proc. Vis. Image Signal Process., vol. 142, no. 3, pp. 149-154, Jun. 1995.
50. D. S. Hands and S. E. Avons, "Recency and duration neglect in subjective assessment of television picture quality," Appl. Cognitive Psychol., vol. 15, no. 6, pp. 639-657, 2001. (Pubitemid 33673463)
51. J. R. Anderson and M. Matessa, "A production system theory of serial memory," Psychol. Rev., vol. 104, no. 4, pp. 728-748, 1997.
52. A. E. Burgess and H. Ghandeharian, "Visual signal detection: II. Signallocation identification," J. Opt. Soc. Am. A, vol. 1, no. 8, pp. 906-910, Aug. 1984. (Pubitemid 14630540)
53. D. Walther and C. Koch, "Modeling attention to salient Proto-objects," Neural Netw., vol. 19, no. 9, pp. 1395-1407, Nov. 2006. (Pubitemid 44708603)
54. O. Le Meur, A. Ninassi, P. Le Callet, and D. Barba, "Do video coding impairments disturb the visual attention deployment?" Signal Process. Image Commun., vol. 25, no. 8, pp. 597-609, Sep. 2010.
55. Z. Chen and C. Guillemot, "Perceptually-friendly H.264/AVC video coding based on foveated just-noticeable-distortion model," IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 806-819, Jun. 2010.
56. P. C. Teo and D. J. Heeger, "Perceptual image distortion," in Proc. 5th SPIE Hum. Vis. Visual Process. Digit. Display, May 1994, pp. 127-139.
57. A. B. Watson and J. A. Solomon, "Model of visual contrast gain control and pattern masking," J. Opt. Soc. Am. A, vol. 14, no. 9, pp. 2379-2391, Sep. 1997. (Pubitemid 127567280)
58. R. W. Bowen, J. Pola, and L. Matin, "Visual persistence: Effects of flash, luminance, duration and energy," Vis. Res., vol. 14, no. 4, pp. 295-303, Apr. 1974.
59. G. E. Meyer and W. M. Maguire, "Spatial frequency and the mediation of short-term visual storage," Science, vol. 198, pp. 524-525, Nov. 1977. (Pubitemid 8188142)
60. J. J. Mezrich, "The duration of visual persistence," Vis. Res., vol. 24, no. 6, pp. 631-632, 1984.
61. W. J. Tam, L. B. Stelmach, L. Wang, D. Lauzon, and P. Gray, "Visual masking at video scene cuts," in Proc. 6th SPIE Hum. Vis. Visual Process. Digit. Display, Apr. 1995, pp. 111-119.
62. G. Buchsbaum, "An analytical derivation of visual nonlinearity," IEEE Trans. Biomed. Eng., vol. 27, no. 5, pp. 237-242, May 1980. (Pubitemid 10045962)
63. F. de Simone, M. Tagliasacchi, M. Naccari, S. Tubaro, and T. Ebrahimi, "H.264/AVC video database for the evaluation of quality metrics," in Proc. IEEE ICASSP, Mar. 2010, pp. 2430-2433.
64. (2000). Final Report from the Video Quality Experts Group on the Validation of Objective Quality Metrics for Video Quality Assessment [Online]. Available: http://www.its.bldrdoc.gov/vqeg/projects/frtv?phaseI
65. Z. Wang, E. P. Simoncelli, A. C. Bovik, and M. Matthews, "Multiscale structural similarity for image quality assessment," in Proc. IEEE Asilomar Conf. Signals Syst. Comput., Nov. 2003, pp. 1398-1402.
66. H. R. Sheikh and A. C. Bovik, "A visual information fidelity approach to video quality assessment," in Proc. 1st Int. Conf. Video Process. Quality Metrics Consumer Electron., Jan. 2005, pp. 23-25.
67. B. D. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," in Proc. Int. Joint Conf. Artif. Intell., 1981, pp. 121-130.