Motion tuned spatio-temporal quality assessment of natural videos

IEEE Transactions on Image Processing

Volumn 19, Issue 2, 2010, Pages 335-350

  Seshadrinathan, Kalpana ^a   Bovik, Alan Conrad ^b  

^a INTEL CORPORATION   (United States)

^b The University of Texas at Austin   (United States)

Author keywords

Motion perception; Perceptual quality; Quality assessment; Temporal distortion; Video quality; Visual perception

Indexed keywords

MOTION PERCEPTION; PERCEPTUAL QUALITY; QUALITY ASSESSMENT; TEMPORAL DISTORTIONS; VIDEO QUALITY; VISUAL PERCEPTION;

ALGORITHMS; EQUIPMENT TESTING; MOTION PICTURES; VISION;

QUALITY CONTROL;

ALGORITHM; ARTICLE; HUMAN; IMAGE PROCESSING; METHODOLOGY; MOVEMENT (PHYSIOLOGY); PHYSIOLOGY; VIDEORECORDING; VISION;

ALGORITHMS; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MOVEMENT; VIDEO RECORDING; VISUAL PERCEPTION;

EID: 76849101279     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2009.2034992     Document Type: Article

References (85)

1. Z. Wang and A. C. Bovik, Image Quality Assessment. New York: Morgan and Claypool, 2006.
2. S. Winkler, Digital Video Quality. New York: Wiley, 2005.
3. C. J. van den Branden Lambrecht, D. M. Costantini, G. L. Sicuranza, and M.Kunt, "Quality assessment of motion rendition in video coding," IEEE Trans. Pattern Anal. Mach. Intell., vol.9, no.5, pp. 766-782, May 1999.
4. Z. Wang, L. Lu, and A. C. Bovik, "Video quality assessment based on structural distortion measurement," Signal Process.: Image Commun., vol.19, no.2, pp. 121-132, Feb. 2004.
5. Z.Wang and Q. Li, "Video quality assessment using a statistical model of human visual speed perception," J. Opt. Soc. Amer. A, vol.24, no.12, pp. B61-B69, Dec. 2007.
6. A. Ninassi, O. Le Meur, P. Le Callet, and D. Barba, "Considering temporal variations of spatial visual distortions in video quality assessment," IEEE J. Sel. Topics Signal Process., vol.3, no.2, pp. 253-265, 2009.
7. M. Barkowsky, J. Bialkowski, B. Eskofier, R. Bitto, and A. Kaup, "Temporal trajectory aware video quality measure," IEEE J. Sel. Topics Signal Process., vol.3, no.2, pp. 266-279, 2009.
8. A. B.Watson, J. Ahumada, and A. J., "Model of human visual-motion sensing," J. Opt. Soc. Amer. A, vol.2, no.2, pp. 322-342, 1985.
9. K. Seshadrinathan and A. C. Bovik, "A structural similarity metric for video based on motion models," presented at the IEEE Int. Conf. Acoustics, Speech, and Signal Processing, 2007.
10. B. A. Wandell, Foundations of Vision. Sunderland, MA: Sinauer, 1995.
11. L. Itti and C. Koch, "Computational modelling of visual attention," Nature Rev. Neurosci., vol.2, no.3, pp. 194-203, 2001.
12. Z.Wang and A. Bovik, "A universal image quality index," IEEE Signal Process. Lett., vol.9, no.3, pp. 81-84, 2002.
13. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Process., vol.13, no.4, pp. 600-612, Apr. 2004.
14. H. R. Sheikh and A. C. Bovik, "Image information and visual quality," IEEE Trans. Image Process., vol.15, no.2, pp. 430-444, Feb. 2006.
15. H. R. Sheikh, A. C. Bovik, and G. de Veciana, "An information fidelity criterion for image quality assessment using natural scene statistics," IEEE Trans. Image Process., vol.14, no.12, pp. 2117-2128, Dec. 2005.
16. H. R. Sheikh and A. C. Bovik, "An evaluation of recent full reference image quality assessment algorithms," IEEE Trans. Image Process., vol.15, no.11, pp. 3440-3451, Nov. 2006.
17. K. Seshadrinathan and A. C. Bovik, "An information theoretic video quality metric based on motion models," presented at the 3rd Int.Workshop Video Processing and Quality Metrics for Consumer Electronics, 2007.
18. B. Girod, "What's wrong with mean-squared error," in Digital Images and Human Vision, A. B. Watson, Ed. Cambridge, MA: MIT Press, 1993, pp. 207-220.
19. K. Seshadrinathan and A. C. Bovik, "Video quality assessment," in The Essential Guide to Video Processing, A. C. Bovik, Ed. New York: Elsevier, 2009.
20. J. M. Libert, C. P. Fenimore, and P. Roitman, "Simulation of graded video impairment by weighted summation: Validation of the methodology," Proc. SPIE, vol.3845, no.1, pp. 254-265, Nov. 1999.
21. C. Lee and O. Kwon, "Objective measurements of video quality using the wavelet transform," Opt. Eng., vol.42, no.1, pp. 265-272, Jan. 2003.
22. C. Taylor and S. Dey, "Run-time allocation of buffer resources for maximizing video clip quality in a wireless last-hop system," presented at the IEEE Int. Conf. Communications, 2004.
23. J. Mannos and D. Sakrison, "The effects of a visual fidelity criterion of the encoding of images," IEEE Trans. Inf. Theory, vol.20, no.4, pp. 525-536, Apr. 1974.
24. J. Lubin, "The use of psychophysical data and models in the analysis of display system performance," in Digital Images and Human Vision, A. B. Watson, Ed. Cambridge, MA: MIT Press, 1993, pp. 163-178.
25. P. C. Teo and D. J. Heeger, "Perceptual image distortion," presented at the IEEE Int. Conf. Image Processing, 1994.
26. S. Daly, "The visible difference predictor: An algorithm for the assessment of image fidelity," in Digital Images and Human Vision, A. B. Watson, Ed. Cambridge, MA: MIT Press, 1993, pp. 176-206.
27. D. M. Chandler, K. H. Lim, and S. S. Hemami, "Effects of spatial correlations and global precedence on the visual fidelity of distorted images," Proc. SPIE, vol.6057, no.1, p. 60570F, Feb. 2006.
28. K. Seshadrinathan, T. N. Pappas, R. J. Safranek, J. Chen, Z. Wang, H. R. Sheikh, and A. C. Bovik, "Image quality assessment," in The Essential Guide to Image Processing, A. C. Bovik, Ed. New York: Elsevier, 2008.
29. G. E. Legge, "Sustained and transient mechanisms in human vision: Temporal and spatial properties," Vis. Res., vol.18, no.1, pp. 69-81, 1978.
30. J. J. Kulikowski and D. J. Tolhurst, "Psychophysical evidence for sustained and transient detectors in human vision," J. Physiol., vol.232, no.1, pp. 149-162, Jul. 1973.
31. C. J. van den Branden Lambrecht and O. Verscheure, "Perceptual quality measure using a spatiotemporal model of the human visual system," Proc. SPIE, vol.2668, no.1, pp. 450-461, Mar. 1996.
32. S. Winkler, "Perceptual distortion metric for digital color video," in Proc. SPIE, May 1999, vol. 3644, no. 1, pp. 175-184.
33. A. B. Watson, J. Hu, and J. F. McGowan, III, "Digital video quality metric based on human vision," J. Electron. Imag., vol.10, no.1, pp. 20-29, Jan. 2001.
34. M. Masry, S. S. Hemami, and Y. Sermadevi, "A scalable wavelet-based video distortion metric and applications," IEEE Trans. Circuits Syst. Video Technol., vol.16, no.2, pp. 260-273, Feb. 2006.
35. J. A. Movshon and W. T. Newsome, "Visual response properties of striate cortical neurons projecting to area MT in Macaque monkeys," J. Neurosci., vol.16, no.23, pp. 7733-7741, 1996.
36. R. T. Born and D. C. Bradley, "Structure and function of visual area MT," Annu. Rev. Neurosci., vol.28, pp. 157-189, 2005.
37. M. A. Smith, N. J. Majaj, and J. A. Movshon, "Dynamics of motion signaling by neurons in macaque area MT," Nature Neurosci., vol.8, no.2, pp. 220-228, Feb. 2005.
38. J. A. Perrone, "A visual motion sensor based on the properties of V1 and MT neurons," Vis. Res., vol.44, no.15, pp. 1733-1755, Jul. 2004.
39. N. J. Priebe, S. G. Lisberger, and J. A. Movshon, "Tuning for spatiotemporal frequency and speed in directionally selective neurons of macaque striate cortex," J. Neurosci., vol.26, no.11, pp. 2941-2950, Mar. 2006.
40. J. Nachmias and R. V. Sansbury, "Grating contrast: Discrimination may be better than detection," Vis. Res., vol.14, no.10, pp. 1039-1042, Oct. 1974.
41. G. Legge and J. Foley, "Contrast masking in human vision," J. Opt. Soc. Amer. A, vol.70, no.12, pp. 1458-1471, Dec. 1980.
42. M. H. Pinson and S.Wolf, "A new standardized method for objectively measuring video quality," IEEE Trans. Broadcast., vol.50, no.3, pp. 312-322, Sep. 2004.
43. A. P. Hekstra, J. G. Beerends, D. Ledermann, F. E. de Caluwe, S. Kohler, R. H. Koenen, S. Rihs, M. Ehrsam, and D. Schlauss, "PVQM-A perceptual video quality measure," Signal Process.: Image Commun., vol.17, pp. 781-798, 2002.
44. International Telecommunications Union, Objective Perceptual Multimedia Video Quality Measurement in the Presence of a Full Reference, 2008, ITU-T Rec. J. 247, Tech. Rep..
45. NTT, (2008) NTT News Release [Online]. Available: http://www. ntt.co.jp/news/news08e/0808/080825a.html
46. Opticom [Online]. Available: http://www.opticom.de/technology/ pevq-video-quality-testing.html
47. M. Malkowski and D. Claben, "Performance of video telephony services inUMTSusing live measurements and network emulation," Wireless Personal Commun., vol.1, pp. 19-32, 2008.
48. M. Barkowsky, J. Bialkowski, R. Bitto, and A. Kaup, "Temporal registration using 3D phase correlation and a maximum likelihood approach in the perceptual evaluation of video quality," presented at the IEEE Workshop Multimedia Signal Processing, 2007.
49. M. Yuen and H. R. Wu, "A survey of hybrid MC/DPCM/DCT video coding distortions," Signal Process., vol.70, no.3, pp. 247-278, Nov. 1998.
50. K. Seshadrinathan and A. C. Bovik, "Motion-based perceptual quality assessment of video," presented at the SPIE-Human Vision and Electronic Imaging, 2009.
51. J. A. Movshon, I. D. Thompson, and D. J. Tolhurst, "Spatial summation in the receptive fields of simple cells in the cat's striate cortex," J. Physiol., vol.283, pp. 53-77, Oct. 1978.
52. J. G. Daugman, "Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters," J. Opt. Soc. Amer. A, vol.2, no.7, pp. 1160-1169, 1985.
53. A. C. Bovik, M. Clark, and W. S. Geisler, "Multichannel texture analysis using localized spatial filters," IEEE Trans. Pattern Anal. Mach. Intell., vol.12, no.1, pp. 55-73, Jan. 1990.
54. D. J. Tolhurst and J. A. Movshon, "Spatial and temporal contrast sensitivity of striate cortical neurones," Nature, vol.257, no.5528, pp. 674-675, Oct. 1975.
55. S. M. Friend and C. L. Baker, "Spatio-temporal frequency separability in area 18 neurons of the cat," Vis. Res., vol.33, no.13, pp. 1765-1771, Sep. 1993.
56. M. C. Morrone, M. D. Stefano, and D. C. Burr, "Spatial and temporal properties of neurons of the lateral suprasylvian cortex of the cat," J. Neurophysiol., vol.56, no.4, pp. 969-986, Oct. 1986.
57. D. J. Heeger, "Optical flowusing spatiotemporal filters," Int. J. Comput. Vis., vol.1, no.4, pp. 279-302, 1987.
58. E. P. Simoncelli and D. J. Heeger, "A model of neuronal responses in visual area MT," Vis. Res., vol.38, no.5, pp. 743-761, Mar. 1998.
59. E. H. Adelson and J. R. Bergen, "Spatiotemporal energy models for the perception of motion," J. Opt. Soc. Amer. A, vol.2, no.2, pp. 284-299, Feb. 1985.
60. D. Fleet and A. Jepson, "Computation of component image velocity from local phase information," Int. J. Comput. Vis., vol.5, no.1, pp. 77-104, 1990.
61. H. R. Sheikh and A. C. Bovik, "A visual information fidelity approach to video quality assessment," presented at the 1st Int. Workshop Video Processing and Quality Metrics for Consumer Electronics, 2005.
62. R. Fox, "Visual masking," in Handbook of Sensory Physiology. VIII. Perception, R. Held, H. W. Leibowitz, and H. L. Teuber, Eds. New York: Springer-Verlag, 1978.
63. J. Foley, "Human luminance pattern-vision mechanisms: Masking experiments require a new model," J. Opt. Soc. Amer. A, vol.11, no.6, pp. 1710-1719, Jun. 1994.
64. A. Watson and J. Solomon, "Model of visual contrast gain control and pattern masking," J. Opt. Soc. Amer. A, vol.14, no.9, pp. 2379-2391, Sep. 1997.
65. D. J. Heeger, "Normalization of cell responses in cat striate cortex," Vis. Neurosci., vol.9, no.2, pp. 181-197, Aug. 1992.
66. D. G. Albrecht and W. S. Geisler, "Motion selectivity and the contrastresponse function of simple cells in the visual cortex," Vis. Neurosci., vol.7, no.6, pp. 531-546, Dec. 1991.
67. W. S. Geisler and D. G. Albrecht, "Cortical neurons: Isolation of contrast gain control," Vis. Res., vol.32, no.8, pp. 1409-1410, Aug. 1992.
68. O. Schwartz and E. P. Simoncelli, "Natural signal statistics and sensory gain control," Nature Neurosci., vol.4, no.8, pp. 819-825, Aug. 2001.
69. K. Seshadrinathan and A. C. Bovik, "Unifying analysis of full reference image quality assessment," presented at the IEEE Int. Conf. Image Processing, 2008.
70. D. J. Field, "Relations between the statistics of natural images and the response properties of cortical cells," J. Opt. Soc. Amer. A, vol.4, no.12, pp. 2379-2394, Dec. 1987.
71. J. A. Perrone and A. Thiele, "Speed skills: Measuring the visual speed analyzing properties of primate MT neurons," Nature Neurosci., vol.4, no.5, pp. 526-532, May 2001.
72. N. C. Rust, V. Mante, E. P. Simoncelli, and J. A. Movshon, "How MT cells analyze the motion of visual patterns," Nature Neurosci., vol.9, no.11, pp. 1421-1431, Nov. 2006.
73. (2000) Final Report From the Video Quality Experts Group on the Validation of Objective Quality Metrics for Video Quality Assessment [Online]. Available: http://www.its.bldrdoc.gov/vqeg/projects/frtv-phaseI
74. A. Webster, "Progress and future plans for VQEG," presented at the ETSI STQ Workshop Multimedia Quality of Service 2008 [Online]. Available: http://www.portal.etsi.org/docbox/Workshop/ 2008/2008-06-STQWORKSHOP/VQEG-Ar thurWebster.pdf
75. S. H. Dai and M. O. Wang, Reliability Analysis in Engineering Applications. New York: Van Nostrand Reinhold, 1993.
76. E. P. Simoncelli, "Statistical modeling of photographic images," in Handbook of Image and Video Processing, A. C. Bovik, Ed., 2nd ed. New York: Academic, 2005.
77. (2003) LIVE Image Quality Assessment Database [Online]. Available: http://www.live.ece.utexas.edu/research/quality/subjective.htm
78. A. K. Moorthy and A. C. Bovik, "Perceptually significant spatial pooling strategies for image quality assessment," presented at the SPIE-Human Vision and Electronic Imaging, 2009.
79. U. Rajashekar, I. Van Der Linde, A. C. Bovik, and L. K. Cormack, "GAFFE: A gaze-attentive fixation finding engine," IEEE Trans. Image Process., vol.17, no.4, pp. 564-573, Apr. 2008.
80. A. K. Moorthy and A. C. Bovik, "Visual importance pooling for image quality assessment," IEEE J. Sel. Topics Signal Process., vol.3, no.2, pp. 193-201, 2009.
81. S. S. Channappayya, A. C. Bovik, C. Caramanis, and R. W. Heath, Jr., "Design of linear equalizers optimized for the structural similarity index," IEEE Trans. Image Process., vol.17, no.6, pp. 857-872, Jun. 2008.
82. J. L. Barron, D. J. Fleet, and S. S. Beauchemin, "Performance of optical flow techniques," Int. J. Comput. Vis., vol.12, no.1, pp. 43-77, Feb. 1994.
83. E. P. Simoncelli, "Distributed Analysis and Representation of Visual Motion," Ph.D. dissertation, MIT, Cambridge, MA, 1993.
84. P. Anandan, "A computational framework and an algorithm for the measurement of visual motion," Int. J. Comput. Vis., vol.2, no.3, pp. 283-310, Jan. 1989.
85. B. D. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," presented at the Int. Joint Conf. Artificial Intelligence, Vancouver, BC, Canada, 1981.