SSIM-motivated rate-distortion optimization for video coding

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 22, Issue 4, 2012, Pages 516-529

  Wang, Shiqi ^a,b   Rehman, Abdul ^b   Wang, Zhou ^b   Ma, Siwei ^a   Gao, Wen ^a  

^a PEKING UNIVERSITY   (China)

^b UNIVERSITY OF WATERLOO   (Canada)

Author keywords

H.264 AVC coding; Lagrange multiplier; rate distortion optimization; reduced reference image quality assessment; structural similarity (SSIM) index

Indexed keywords

CODING SCHEME; ESTIMATION ALGORITHM; H.264/AVC; IMAGE AND VIDEO CODING; MACROBLOCK-LEVEL; MEAN SQUARED ERROR; MOTION INFORMATION; PERCEPTUAL UNCERTAINTY; RATE MODELS; RATE-DISTORTION OPTIMIZATION; REDUCED REFERENCE; REDUCED-REFERENCE IMAGE QUALITY ASSESSMENTS; SELECTION METHODS; SIDE INFORMATION; STRUCTURAL SIMILARITY; THEORETICAL APPROACH; VISUAL QUALITIES;

IMAGE QUALITY; INFORMATION THEORY; LAGRANGE MULTIPLIERS; MOTION PICTURE EXPERTS GROUP STANDARDS;

VIDEO SIGNAL PROCESSING;

EID: 84859727815     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2011.2168269     Document Type: Article

References (50)

1. G. J. Sullivan and T. Wiegand, "Rate-distortion optimization for video compression," IEEE Signal Process. Mag., vol. 15, no. 6, pp. 74-90, Nov. 1998.
2. B. Girod, "Whats wrong with mean-squared error," in Digital Images and Human Vision. Cambridge, MA: MIT Press, 1993, pp. 207-220.
3. X. Zhao, L. Zhang, S. Ma, and W. Gao, "Rate-distortion optimized transform for intra-frame coding," in Proc. IEEE Int. Conf. Acou., Speech Signal Process., Mar. 2010, pp. 1414-1417.
4. M. Karczewicz, Y. Ye, and I. Chong, "Rate distortion optimized quantization," document VCEG-AH21, ITU-T Q.6/SG16 VCEG, Antalya, Turkey, Jan. 2008.
5. E. H. Yang and X. Yu, "Rate distortion optimization for H.264 interframe video coding: A general framework and algorithms," IEEE Trans. Image Process., vol. 16, no. 7, pp. 1774-1784, Jul. 2007. (Pubitemid 47018875)
6. E. H. Yang and X. Yu, "Soft decision quantization for H.264 with main profile compatibility," IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 1, pp. 122-127, Jan. 2009.
7. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, Apr. 2004.
8. H. Sheikh and A. C. Bovik, "Image information and visual quality," IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006. (Pubitemid 43126168)
9. D. Chandler and S. Hemami, "VSNR: A wavelet-based visual signalto-noise ratio for natural images," IEEE Trans. Image Process., vol. 16, no. 9, pp. 2284-2298, Sep. 2007. (Pubitemid 47316675)
10. Z. Wang and A. Bovik, "Mean squared error: Love it or leave it? A new look at signal fidelity measures," IEEE Signal Process. Mag., vol. 26, no. 1, pp. 98-117, Jan. 2009.
11. Advanced Video Coding, ITU-T Rec. H.264 and ISO/IEC 14496-10 (MPEG-4 Part 10), 2010.
12. T. Wiegand, H. Schwarz, A. Joch, F. Kossentini, and G. J. Sullian, "Rateconstrained coder control and comparison of video coding standards," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 688-703, Jul. 2003.
13. T. Wiegand and B. Girod, "Lagrange multiplier selection in hybrid video coder control," in Proc. Int. Conf. Image Process., 2001, pp. 542-545. (Pubitemid 33058039)
14. Z. Wang and Q. Li, "Video quality assessment using a statistical model of human visual speed perception," J. Optic. Soc. Am. A, vol. 24, pp. B61-B69, Dec. 2007. (Pubitemid 351325125)
15. L. Chen and I. Garbacea, "Adaptive a; estimation in Lagrangian ratedistortion optimization for video coding," Proc. SPIE, vol. 6077, pp. 1-8, Jan. 2006.
16. Z. He and S. Mitra, "Optimum bit allocation and accurate rate control for video coding via rho-domain source modeling," IEEE Trans. Circuits Syst. Video Technol., vol. 12, no. 10, pp. 840-849, Oct. 2002. (Pubitemid 35468408)
17. X. Li, N. Oertel, A. Hutter, and A. Kaup, "Laplace distribution based Lagrangian rate distortion optimization for hybrid video coding," IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 2, pp. 193-205, Feb. 2009.
18. M. Jiang and N. Ling, "On Lagrange multiplier and quantizer adjustment for H.264 frame-layer video rate control," IEEE Trans. Circuits Syst. Video Technol., vol. 16, no. 5, pp. 663-669, May 2006.
19. M. Wang and B. Yan, "Lagrangian multiplier based joint three-layer rate control for H.264/AVC," IEEE Signal Process. Lett., vol. 16, no. 8, pp. 679-682, Aug. 2009.
20. J. Zhang, X. Yi, N. Ling, and W. Shang, "Context adaptive Lagrange multiplier (CALM) for rate-distortion optimal motion estimation in video coding," IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 820-828, Jun. 2010.
21. J. Zhang, X. Yi, N. Ling, and W. Shang, "Context adaptive lagrange multiplier (CALM) for motion estimation in JM-improvement," document JVT-T046, Joint Video Team (JVT) of ISO/IEC MPEG ITU-T VCEG, Jul. 2006.
22. X. Yang, W. Lin, Z. Lu, E. Ong, and S. Yao, "Motion-compensated residue pre-processing in video coding based on just-noticeable distortion profile," IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 6, pp. 742-752, Jun. 2005.
23. X. Yang, W. Lin, Z. Lu, E. Ong, and S. Yao, "Just noticeable distortion model and its applications in video coding," Signal Process.: Image Commun., vol. 22, pp. 662-680, Aug. 2005. (Pubitemid 41023703)
24. Z. Chen and C. Guillemot, "Perceptually-friendly H.264/AVC video coding based on foveated just-noticeable-distortion model," IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 806-819, Jun. 2010.
25. C. Sun, H.-J. Wang, and H. Li, "Macroblock-level rate-distortion optimization with perceptual adjustment for video coding," in Proc. IEEE DCC, Mar. 2008, p. 546.
26. F. Pan, Y. Sun, Z. Lu, and A. Kassim, "Complexity-based rate distortion optimization with perceptual tuning for scalable video coding," in Proc. Int. Conf. Image Process., Sep. 2005, pp. 37-40. (Pubitemid 44479411)
27. A. Brooks, X. Zhao, and T. Pappas, "Structural similarity quality metrics in a coding context: Exploring the space of realistic distortions," IEEE Trans. Image Process., vol. 17, no. 8, pp. 121-132, Aug. 2008.
28. B. Aswathappa and K. R. Rao, "Rate-distortion optimization using structural information in H.264 strictly intra-frame encoder," in Proc. South Eastern Symp. Syst. Theory, 2010, pp. 367-370.
29. Z. Mai, C. Yang, L. Po, and S. Xie, "A new rate-distortion optimization using structural information in H.264 I-frame encoder," in Proc. ACIVS, 2005, pp. 435-441.
30. Z. Mai, C. Yang, and S. Xie, "Improved best prediction mode(s) selection methods based on structural similarity in H.264 I-frame encoder," in Proc. IEEE Int. Conf. Syst., Man Cybern., Oct. 2005, pp. 2673-2678. (Pubitemid 41674791)
31. Z. Mai, C. Yang, K. Kuang, and L. Po, "A novel motion estimation method based on structural similarity for H.264 inter prediction," in Proc. IEEE Int. Conf. Acou., Speech, Signal Process., vol. 2. May 2006, pp. 913-916.
32. C. Yang, R. Leung, L. Po, and Z. Mai, "An SSIM-optimal H.264/AVC inter frame encoder," in Proc. IEEE Int. Conf. Intell. Comput. Intell. Syst., vol. 4. Jun. 2009, pp. 291-295.
33. C. Yang, H. Wang, and L. Po, "Improved inter prediction based on structural similarity in H.264," in Proc. IEEE Int. Conf. Signal Process. Commun., vol. 2. Nov. 2007, pp. 340-343.
34. Y. H. Huang, T. S. Ou, P. Y. Su, and H. Chen, "Perceptual rate-distortion optimization using structural similarity index as quality metric," IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 11, pp. 1614-1624, Nov. 2010.
35. H. Chen, Y. Huang, P. Su, and T. Ou, "Improving video coding quality by perceptual rate-distortion optimization," in Proc. IEEE Int. Conf. Multimedia Exp., Jul. 2010, pp. 1287-1292.
36. P. Su, Y. Huang, T. Ou, and H. Chen, "Predictive Lagrange multiplier selection for perceptual-based rate-distortion optimization," in Proc. 5th Int. Workshop Video Process. Qual. Metrics Consum. Electron., Jan. 2010.
37. Y. Huang, T. Ou, and H. Chen, "Perceptual-based coding mode decision," in Proc. IEEE Int. Symp. Circuits Syst., May 2010, pp. 393-396.
38. T. Ou, Y. Huang, and H. Chen, "A perceptual-based approach to bit allocation for H.264 encoder," Proc. SPIE: Vis. Commun. Image Process., vol. 7744, pp. 1-10, Jul. 2010.
39. Z. Wang, L. Lu, and A. C. Bovik, "Video quality assessment based on structural distortion measurement," Signal Process.: Image Commun., vol. 19, pp. 121-132, Feb. 2004.
40. J. Sun, W. Gao, D. Zhao, and Q. Huang, "Statistical model, analysis and approximation of rate-distortion function in MPEG-4 FGS videos," IEEE Trans. Circuits Syst. Video Technol., vol. 16, no. 4, pp. 535-539, Apr. 2006.
41. Y. Altunbasak and N. Kamaci, "An analysis of the DCT coeffcient distribution with the H.264 video coder," in Proc. IEEE ICASSP, vol. 3. May 2004, pp. 177-180.
42. A. Rehman and Z. Wang, "Reduced-reference SSIM estimation," in Proc. Int. Conf. Image Process., Sep. 2010, pp. 289-292.
43. S. Channappayya, A. C. Bovik, and J. R. W. Heathh, "Rate bounds on SSIM index of quantized images," IEEE Trans. Image Process., vol. 17, no. 9, pp. 1624-1639, Sep. 2008.
44. D. Kwon, M. Shen, and C. Kuo, "Rate control for H.264 video with enhanced rate and distortion models," IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 5, pp. 517-529, May 2007.
45. A. A. Stocker and E. P. Simoncelli, "Noise characteristics and prior expectations in human visual speed perception," Nat. Neurosci., vol. 9, no. 4, pp. 578-585, Apr. 2006.
46. T. Vlachos, "Simple method for estimation of global motion paramenters using spares translational motion vector fields," Electron. Lett., vol. 34, no. 1, pp. 90-91, Jan. 1998.
47. S. Wang, S. Ma, and W. Gao, "SSIM based perceptual distortion rate optimization coding," Proc. SPIE: Vis. Commun. Image Process., vol. 7744, pp. 1-10, Jul. 2010.
48. H. Gish and J. Pierce, "Asymptotically efficient quantizing," IEEE Trans. Inform. Theory, vol. 14, no. 5, pp. 676-683, Oct. 1968.
49. Joint Video Team (JVT) Reference Software [Online]. Available: http://iphome.hhi.de/suehring/tml/download/old-jm
50. G. Bjøntegaard, "Calculation of average PSNR difference between RD curves," document ITU-T Q.6/SG16, 13th VCEG-M33 Meeting, Austin, TX, Apr. 2001.