Perceptual video coding based on SSIM-inspired divisive normalization

IEEE Transactions on Image Processing

Volumn 22, Issue 4, 2013, Pages 1418-1429

  Wang, Shiqi ^a   Rehman, Abdul ^b   Wang, Zhou ^b   Ma, Siwei ^a   Gao, Wen ^a  

^a PEKING UNIVERSITY   (China)

^b UNIVERSITY OF WATERLOO   (Canada)

Author keywords

Divisive normalization; H.264 AVC coding; perceptual video coding; rate distortion optimization; structural similarity (SSIM) index

Indexed keywords

DIVISIVE NORMALIZATION; H.264/AVC CODING; PERCEPTUAL VIDEO CODING; RATE-DISTORTION OPTIMIZATION; STRUCTURAL SIMILARITY INDICES (SSIM);

DISCRETE COSINE TRANSFORMS; IMAGE CODING; MOTION PICTURE EXPERTS GROUP STANDARDS;

VIDEO SIGNAL PROCESSING;

EID: 84874054971     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2012.2231090     Document Type: Article

References (59)

1. B. Girod, "What's wrong with mean-squared error," in Digital Images Human Vision. Cambridge, MA: MIT Press, 1993, pp. 207-220.
2. Z. Wang and A. C. Bovik, Modern Image Quality Assessment (Syntheses Lectures on Image, Video and Multimedia Processing). San Rafael, CA: Morgan Claypool, Mar. 2006.
3. Z. Wang and A. Bovik, "Mean squared error: Love it or leave it? A new look at signal fidelity measures," IEEE Signal Process. Mag., vol. 26, no. 1, pp. 98-117, Jan. 2009.
4. C.-W. Tang, C.-H. Chen, Y.-H. Yu, and C.-J. Tsai, "Visual sensitivity guided bit allocation for video coding," IEEE Trans. Multimedia, vol. 8, no. 1, pp. 11-18, Feb. 2006. (Pubitemid 43146126)
5. C.-W. Tang, "Spatial temporal visual considerations for effcient video coding," IEEE Trans. Multimedia, vol. 9, no. 2, pp. 231-238, Jan. 2007. (Pubitemid 46188196)
6. C. Sun, H.-J. Wang, and H. Li, "Macroblock-level rate-distortion optimization with perceptual adjustment for video coding," in Proc. IEEE Data Compress. Conf., Mar. 2008, p. 546.
7. F. Pan, Y. Sun, Z. Lu, and A. Kassim, "Complexity-based rate distortion optimization with perceptual tuning for scalable video coding," in Proc. Int. Conf. Image Process., Sep. 2005, pp. 37-40. (Pubitemid 44479411)
8. Z. Chen and C. Guillemot, "Perceptually-friendly H.264/AVC video coding based on foveated just-noticeable-distortion model," IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 806-819, Jun. 2010.
9. J. Chen, J. Zheng, and Y. He, "Macroblock-level adaptive frequency weighting for perceptual video coding," IEEE Trans. Consum. Electron., vol. 53, no. 2, pp. 775-781, May 2007. (Pubitemid 47215100)
10. A. Tanizawa and T. Chujoh, "Adaptive quantization matrix selection," Toshiba, Geneva, Switzerland, Tech. Rep. T05-SG16-060403-D- 0266, Apr. 2006.
11. T. Suzuki, P. Kuhn, and Y. Yagasaki, "Quantization tools for high quality video," ISO/IEC, Washington, DC, Tech. Rep. MPEG ITU-T VCEG JVT-B067, Jan. 2002.
12. T. Suzuki, K. Sato, and Y. Yagasaki, "Weighting matrix for JVT codec," ISO/IEC, Washington, DC, Tech. Rep. MPEG ITU-T VCEG JVT-C053, May 2002.
13. J. Malo, J. Gutierrez, I. Epifanio, F. Ferri, and J. M. Artigas, "Perceptual feedback in multigrid motion estimation using an improved DCT quantization," IEEE Trans. Image Process., vol. 10, no. 10, pp. 1411-1427, Oct. 2001. (Pubitemid 32981816)
14. Z. Wang, L. Lu, and A. Bovik, "Foveation scalable video coding with automatic fixation selection," IEEE Trans. Image Process., vol. 12, no. 2, pp. 243-254, Feb. 2003.
15. T. Pappas, T. Michel, and R. Hinds, "Supra-threshold perceptual image coding," in Proc. IEEE Int. Conf. Image Process., Sep. 1996, pp. 237-240.
16. W. Zeng, S. Daly, and S. Lei, "An overview of the visual optimization tools in JPEG 2000," Signal Process. Image Commun., vol. 17, no. 1, pp. 85-104, Jan. 2001. (Pubitemid 33142066)
17. D. Chandler and S. Hemami, "Dynamic contrast-based quantization for lossy wavelet image compression," IEEE Trans. Image Process., vol. 14, no. 4, pp. 397-410, Apr. 2005. (Pubitemid 40535895)
18. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, Apr. 2004.
19. B. Aswathappa and K. R. Rao, "Rate-distortion optimization using structural information in H.264 strictly intra-frame encoder," in Proc. South East. Symp. Syst. Theory, Apr. 2010, pp. 367-370.
20. Z. Mai, C. Yang, L. Po, and S. Xie, "A new rate-distortion optimization using structural information in H.264 I-frame encoder," in Proc. ACIVS, 2005, pp. 435-441.
21. Z. Mai, C. Yang, and S. Xie, "Improved best prediction mode(s) selection methods based on structural similarity in H.264 I-frame encoder," in Proc. IEEE Int. Conf. Syst. Man Cybern., May 2005, pp. 2673-2678. (Pubitemid 41674791)
22. Z. Mai, C. Yang, K. Kuang, and L. Po, "A novel motion estimation method based on structural similarity for H.264 inter prediction," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., vol. 2. Feb. 2006, pp. 913-916.
23. C. Yang, R. Leung, L. Po, and Z. Mai, "An SSIM-optimal H.264/AVC inter frame encoder," in Proc. IEEE Int. Conf. Intell. Comput. Intell. Syst., vol. 4. Sep. 2009, pp. 291-295.
24. C. Yang, H. Wang, and L. Po, "Improved inter prediction based on structural similarity in H.264," in Proc. IEEE Int. Conf. Signal Process. Commun., vol. 2. Aug. 2007, pp. 340-343.
25. Y. H. Huang, T. S. Ou, P. Y. Su, and H. H. Chen, "Perceptual rate-distortion optimization using structural similarity index as quality metric," IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 11, pp. 1614-1624, Nov. 2010.
26. H. Chen, Y. Huang, P. Su, and T. Ou, "Improving video coding quality by perceptual rate-distortion optimization," in Proc. IEEE Int. Conf. Multimedia Expo, Jul. 2010, pp. 1287-1292.
27. P. Su, Y. Huang, T. Ou, and H. Chen, "Predictive Lagrange multiplier selection for perceptual-based rate-distortion optimization," in Proc. 5th Int. Workshop Video Process. Qual. Metrics Consum. Electron., Jan. 2010, pp. 1-6.
28. Y. Huang, T. Ou, and H. H. Chen, "Perceptual-based coding mode decision," in Proc. IEEE Int. Symp. Circuits Syst., May 2010, pp. 393-396.
29. T. Ou, Y. Huang, and H. Chen, "A perceptual-based approach to bit allocation for H.264 encoder," Proc. SPIE Visual Commun. Image Process., vol. 7744, p. 77441B, Jul. 2010.
30. S. Wang, A. Rehman, Z. Wang, S. Ma, and W. Gao, "Rate-SSIM optimization for video coding," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., May 2011, pp. 833-836.
31. S. Wang, A. Rehman, Z. Wang, S. Ma, and W. Gao, "SSIM-motivated rate distortion optimization for video coding," IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 4, pp. 516-529, Apr. 2012.
32. D. Brunet, E. R. Vrscay, and Z. Wang, "On the mathematical properties of the structural similarity index," IEEE Trans. Image Process., vol. 21, no. 4, pp. 1488-1499, Apr. 2012.
33. M. J. Wainwright and E. P. Simoncelli, "Scale mixtures of Gaussians and the statistics of natural images," in Advances in Neural Information Processing Systems, vol. 12. Cambridge, MA: MIT Press, 2000, pp. 855-861.
34. S. Lyu and E. P. Simoncelli, "Statistically and perceptually motivated nonlinear image representation," Proc. SPIE, vol. 6492, pp. 67-91, Jan. 2007.
35. J. Foley, "Human luminance pattern-vision mechanisms: Masking experiments require a new model," J. Opt. Soc. Amer., vol. 11, no. 6, pp. 1710-1719, 1994.
36. A. B. Watson and J. A. Solomon, "Model of visual contrast gain control and pattern masking," J. Opt. Soc. Amer., vol. 14, no. 9, pp. 2379-2391, 1997. (Pubitemid 127567280)
37. O. Schwartz and E. P. Simoncelli, "Natural signal statistics and sensory gain control," Nature Neurosci., vol. 4, no. 8, pp. 819-825, Aug. 2001. (Pubitemid 32702514)
38. D. J. Heeger, "Normalization of cell responses in cat striate cortex," Vis. Neural Sci., vol. 9, no. 2, pp. 181-198, 1992.
39. E. P. Simoncelli and D. J. Heeger, "A model of neuronal responses in visual area MT," Vis. Res., vol. 38, pp. 743-761, Mar. 1998. (Pubitemid 28081676)
40. Q. Li and Z. Wang, "Reduced-reference image quality assessment using divisive normalization-based image representation," IEEE J. Sel. Topics Signal Process., vol. 3, no. 2, pp. 202-211, Apr. 2009.
41. A. Rehman and Z. Wang, "Reduced-reference image quality assessment by structural similarity estimation," IEEE Trans. Image Process., vol. 21, no. 8, pp. 3378-3389, Aug. 2012.
42. J. Malo, I. Epifanio, R. Navarro, and E. P. Simoncelli, "Non-linear image representation for efficient perceptual coding," IEEE Trans. Image Process., vol. 15, no. 1, pp. 68-80, Jan. 2006. (Pubitemid 43051456)
43. S. Wang, A. Rehman, Z. Wang, S. Ma, and W. Gao, "SSIM-inspired divisive normalization for perceptual video coding," in Proc. IEEE Int. Conf. Image Process., Sep. 2011, pp. 1657-1660.
44. J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli, "Image denoising using scale mixtures of Gaussians in the wavelet domain," IEEE Trans. Image Process., vol. 12, no. 11, pp. 1338-1351, Nov. 2003.
45. M. J. Wainwright and E. P. Simoncelli, "Scale mixtures of Gaussians and the statistics of natural images," in Advances in Neural Information Processing Systems, vol. 12. Cambridge, MA: MIT Press, 2000, pp. 855-861.
46. S. Channappayya, A. C. Bovik, and J. R. W. Heathh, "Rate bounds on SSIM index of quantized images," IEEE Trans. Image Process., vol. 17, no. 9, pp. 1624-1639, Sep. 2008.
47. J. Foley, "Human luminance pattern-vision mechanisms: Masking experiments require a new model," J. Opt. Soc. Amer., vol. 11, no. 6, pp. 1710-1719, 1994.
48. A. B. Watson and J. A. Solomon, "Model of visual contrast gain control and pattern masking," J. Opt. Soc. Amer., vol. 14, no. 9, pp. 2379-2391, 1997. (Pubitemid 127567280)
49. E. P. Simoncelli and D. J. Heeger, "A model of neuronal responses in visual area MT," Vis. Res., vol. 38, pp. 743-761, Mar. 1998. (Pubitemid 28081676)
50. E. C. Larson and D. M. Chandler, "Most apparent distortion: Fullreference image quality assessment and the role of strategy," J. Electron. Imag., vol. 19, no. 1, p. 011006, Jan. 2010.
51. Z. Wang and Q. Li, "Information content weighting for perceptual image quality assessment," IEEE Trans. Image Process., vol. 20, no. 5, pp. 1185-1198, May 2011.
52. X. Li, N. Oertel, A. Hutter, and A. Kaup, "Laplace distribution based Lagrangian rate distortion optimization for hybrid video coding," IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 2, pp. 193-205, Feb. 2009.
53. X. Zhao, J. Sun, S. Ma, and W. Gao, "Novel statistical modeling, analysis and implementation of rate-distortion estimation for H.264/AVC coders," IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 5, pp. 647-660, May 2010.
54. E. Y. Lam and J. W. Goodman, "A mathematical analysis of the DCT coefficient distributions for images," IEEE Trans. Image Process., vol. 9, no. 10, pp. 1661-1666, Oct. 2000.
55. Joint Video Team (JVT) Reference Software. (2010) [Online]. Available: http://iphome.hhi.de/suehring/tml/download/old-jm
56. Z. Wang, L. Lu, and A. C. Bovik, "Video quality assessment based on structural distortion measurement," Signal Process., Image Commun., vol. 19, pp. 121-132, Feb. 2004.
57. G. Bjontegaard, "Calculation of average PSNR difference between RD curves," in Proc. 13th Meeting ITU-T Q.6/SG16 VCEG, Austin, TX, Apr. 2001.
58. S. Winkler, "Analysis of public image and video database for quality assessment," IEEE J. Sel. Topics Signal Process., vol. 6, no. 6, pp. 616-625, Oct. 2012.
59. Recommendation 500-10: Methodology for the Subjective Assessment of the Quality of Television Pictures, ITU-R Standard Rec. BT.500, 2012.