Perceptually-friendly H.264/AVC video coding based on foveated just-noticeable-distortion model

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 20, Issue 6, 2010, Pages 806-819

  Chen, Zhenzhong ^a,b   Guillemot, Christine ^c  

^a CAMPUS DE BEAULIEU   (France)

^b NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^c INRIA   (France)

Author keywords

Bit allocation; Foveation model; H.264 advanced video coding (AVC); Human visual system (HVS); Just noticeabledistortion (JND); Video coding

Indexed keywords

ADVANCED VIDEO CODING; BIT ALLOCATION; FOVEATION MODEL; HUMAN VISUAL SYSTEMS; VIDEO CODING;

IMAGE COMPRESSION; LAGRANGE MULTIPLIERS; MOTION ESTIMATION; MOTION PICTURE EXPERTS GROUP STANDARDS; OPTIMIZATION; VIDEO SIGNAL PROCESSING; VISIBILITY; VISUAL COMMUNICATION;

IMAGE CODING;

EID: 77953091169     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2010.2045912     Document Type: Article

References (69)

1. Information Technology: Digital Compression and Coding of Continuous Tone Still Images, ISO/IEC IS 10918-1 | ITU-T Rec. T.81, 1994.
2. Information Technology: JPEG 2000 Image Coding System, ISO/IEC 15444-15451, 2000.
3. Video Codec for Audiovisual Services at p×64kb/s, ITU-T Rec. H.261, version 1, 1990; version 2, 1993.
4. Video Coding for Low-Bitrate Communication, ITU-T Rec. H.263, version 1, 1995; version 2, 1998; version 3, 2000.
5. Coding of Moving Pictures and Associated Audio Fordigital Storage Media at up to About 1.5 Mb/s, Part 2: Visual, ISO/IEC 11172-11182:1991, 1991.
6. Information Technology: Generic Coding of Moving Pictures and Associated Audio, Part 2: Visual, ISO/IEC 13818-13822:1994, 1994.
7. Information Technology: Coding of Audio/Visual Objects, Part 2: Visual, ISO/IEC 14496-2:1999, version 1, 1999; version 2, 2000; version 3, 2004.
8. Draft ITU-T Recommendation and Final Draft International Standard of Joint Video Specification (ITU-T Rec. H.264|ISO/IEC 14496-14510 AVC), document JVT-G050.doc, Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, 2003.
9. A. N. Netravali and B. Prasada, "Adaptive quantization of picture signals using spatial masking," Proc. IEEE, vol.65, no.4, pp. 536-548, Apr. 1977.
10. N. Jayant, J. Johnston, and R. Safranek, "Signal compression based on models of human perception," Proc. IEEE, vol.81, no.10, pp. 1385-1422, Oct. 1993.
11. K. N. Ngan, K. S. Leong, and H. Singh, "Adaptive cosine transform coding of images in perceptual domain," IEEE Trans. Acoust., Speech, Signal Process., vol.37, no.11, pp. 1743-1749, Nov. 1989.
12. R. J. Safranek and J. D. Johnston, "A perceptually tuned sub-band image coder with image dependent quantization and post-quantization data compression," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., Pacific Grove, CA, May 1989, pp. 1945-1948.
13. A. B. Watson, G. Y. Yang, J. A. Solomon, and J. Villasenor, "Visibility of wavelet quantization noise," IEEE Trans. Image Process., vol.6, no.8, pp. 1164-1175, Aug. 1997.
14. S. S. Hemami, "Visual sensitivity considerations for sub-band coding," in Proc. Asilomar Conf. Signals, Sytems, Comput., Pacific Grove, CA, Nov. 1997.
15. I. S. Höntsch and L. J. Karam, "Locally adaptive perceptual image coding," IEEE Trans. Image Process., vol.9, no.9, pp. 1472-1483, Sep. 2000.
16. J. Malo, J. Gutierrez, I. Epifanio, F. Ferri, and J. M. Artigas, "Perceptual feedback in multigrid motion estimation using an improved DCT quantization," IEEE Trans. Image Process., vol.10, no.10, pp. 1411-1427, Oct. 2001.
17. I. S. Höntsch and L. J. Karam, "Adaptive image coding with perceptual distortion control," IEEE Trans. Image Process., vol.11, no.3, pp. 213-222, Mar. 2002.
18. M. Gaubatz, S. Kwan, B. Chern, D. M. Chandler, and S. S. Hemami, "Spatially-adaptive wavelet image compression via structural masking," in Proc. IEEE Int. Conf. Image Process., Atlanta, GA, Oct. 2005, pp. 1897-1900.
19. Z. Wang, Q. Li, and X. Shang, "Perceptual image coding based on a maximum of minimal structural similarity criterion," in Proc. IEEE Int. Conf. Image Process., San Antonio, TX, Sep. 2007, pp. 121-124.
20. C.-W. Tang, C.-H. Chen, Y.-H. Yu, and C.-J. Tsai, "Visual sensitivity guided bit allocation for video coding," IEEE Trans. Multimedia, vol.8, no.1, pp. 11-18, Feb. 2006.
21. C.-W. Tang, "Spatial temporal visual considerations for efficient video coding," IEEE Trans. Multimedia, vol.9, no.2, pp. 231-238, Jan. 2007.
22. D. H. Kelly, "Motion and vision II: Stabilized spatio-temporal surface," J. Opt. Soc. Am., vol.69, pp. 1340-1349, Oct. 1979.
23. C.-H. Chou and Y.-C. Li, "A perceptually tuned sub-band image coder based on the measure of just-noticeable-distortion profile," IEEE Trans. Circuits Syst. Video Technol., vol.5, no.6, pp. 467-476, Dec. 1995.
24. C.-H. Chou and C.-W. Chen, "A perceptually optimized 3-D sub-band codec for video communication over wireless channels," IEEE Trans. Circuits Syst. Video Technol., vol.6, no.2, pp. 143-156, Apr. 1996.
25. X. Yang, W. Lin, Z. Lu, E. P. Ong, and S. Yao, "Motion-compensated residue preprocessing in video coding based on just-noticeable-distortion profile," IEEE Trans. Circuits Syst. Video Technol., vol.15, no.6, pp. 742-752, Jun. 2005.
26. W. Lin, "Computational models for just-noticeable difference," in Digital Video Image Quality and Perceptual Coding, H. R. Wu and K. R. Rao, Eds. Boca Raton, FL: CRC Press, 2006.
27. X. Zhang, W. Lin, and P. Xue, "Just-noticeable difference estimation with pixels in images," J. Visual Commun. Image Represent., vol.19, pp. 30-41, Jan. 2008.
28. B. Wandell, Foundations of Vision, Sunderland, MA: Sinauer Associates, Inc., 1995.
29. R. S. Wallace, P.-W. Ong, B. B. Bederson, and E. L. Schwartz, "Spacevariant image processing," Int. J. Comput. Vis., vol.13, pp. 71-90, Jan. 1994.
30. T. Kuyel, W. Geisler, and J. Ghosh, "Retinally reconstructed images: Digital images having a resolution match with the human eyes," IEEE Trans. Syst, Man, Cybern. A, vol.29, no.2, pp. 235-243, Mar. 1999.
31. Z. Wang and A. C. Bovik, "Embeded foveation image coding," IEEE Trans. Image Process., vol.10, no.10, pp. 1397-1410, Oct. 2001.
32. Z. Wang, L. Lu, and A. C. Bovik, "Foveation scalable video coding with automatic fixation selection," IEEE Trans. Image Process., vol.12, no.2, pp. 243-254, Feb. 2003.
33. C.-C. Ho, J.-L. Wu, and W.-H. Cheng, "A practical foveation-based rate-shaping mechanism for MPEG videos," IEEE Trans. Circuits Syst. Video Technol., vol.15, no.11, pp. 1365-1372, Nov. 2005.
34. Z. Chen, J. Han, and K. N. Ngan, "Dynamic bit allocation for multiple video objects coding," IEEE Trans. Multimedia, vol.8, no.6, pp. 1117-1124, Dec. 2006.
35. P. L. Silsbee, A. C. Bovik, and D. Chen, "Visual pattern image sequence coding," IEEE Trans. Circuits Syst. Video Technol., vol.3, no.4, pp. 291-301, Aug. 1993.
36. E. Nguyen, C. Labit, and J. M. Odobez, "A ROI approach for hybrid image sequence coding," in Proc. IEEE Int. Conf. Image Process., Austin, TX, Nov. 1994, pp. 245-249.
37. P. Cicconi and H. Nicolas, "Efficient region-based motion estimation and symmetry oriented segmentation for image sequence coding," IEEE Trans. Circuits Syst. Video Technol., vol.4, no.3, pp. 357-364, Jun. 1994.
38. P. Salembier, L. Torres, F. G. Meyer, and C. Gu, "Region-based video coding using mathematical morphology," Proc. IEEE, vol.83, no.6, pp. 843-857, Jun. 1995.
39. H. Sanderson and G. Crebbin, "Image segmentation for compression of images and image sequences," in Proc. IEE-Vision, Image Signal Process., vol.142. Feb. 1995, pp. 15-21.
40. X. Yang and K. Ramchandran, "A low-complexity region-based video coder using backward morphological motion field segmentation," IEEE Trans. Image Process., vol.8, no.3, pp. 332-345, Mar. 1999.
41. H. P. Zhang and F. Bossen, "Region-based coding of motion fields for low-bitrate video compression," in Proc. IEEE Int. Conf. Image Process., Singapore, Oct. 2004, pp. 1117-1120.
42. X. Yang, W. Lin, Z. Lu, X. Lin, S. Rahardja, E. P. Ong, and S. Yao, "Rate control for videophone using local perceptual cues," IEEE Trans. Circuits Syst. Video Technol., vol.15, no.4, pp. 496-507, Apr. 2005.
43. L. Itti, C. Koch, and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Anal. Machine Intell., vol.20, no.11, pp. 1254-1259, Nov. 1998.
44. L. Itti and C. Koch, "Computational modeling of visual attention," Nature Rev. Neurosci., vol.2, pp. 194-203, Mar. 2001.
45. O. Le Meur, P. Le Callet, and D. Barba, "A coherent computational approach to model the bottom-up visual attention," IEEE Trans. Pattern Anal. Machine Intell., vol.28, no.5, pp. 802-817, May 2006.
46. L. Itti, "Automatic foveation for video compression using a neurobio-logical model of visual attention," IEEE Trans. Image Process., vol.13, no.10, pp. 1304-1318, Oct. 2004.
47. A. P. Bradley and F. W. M. Stentiford, "Visual attention for region of interest coding in JPEG 2000," J. Visual Commun. Image Represent., vol.14, pp. 232-250, Sep. 2003.
48. A. N. Netravali and B. G. Haskell, Digital Pictures: Representation and Compression, New York: Plenum, 1988.
49. A. B. Watson, "DCTune: A technique for visual optimization of DCT quantization matrices for individual images," Soc. for Info. Display Dig. Tech. Papers, vol.XXIV, pp. 946-949, 1993.
50. J. Lubin, "A visual system discrimination model for imaging system design and evaluation," in Vision Models for Target Detection and Recognition, E. Peli, Ed. River Edge, NJ: World Scientific, 1995, pp. 245-283.
51. W. S. Geisler and J. S. Perry, "A real-time foveated multisolution system for low-bandwidth video communication," in Proc. SPIE, vol.3299. Jul. 1998, pp. 294-305.
52. Y. Liu, Z. G. Li, and Y. C. Soh, "Region-of-interest-based resource allocation for conversational video communication of H.264/AVC," IEEE Trans. Circuits Syst. Video Technol., vol.18, no.1, pp. 134-139, Jan. 2008.
53. J. Ribas-Corbera and S. Lei, "Rate control in DCT video coding for low-delay communications," IEEE Trans. Circuits Syst. Video Technol., vol.9, no.1, pp. 172-185, Feb. 1999.
54. Z. G. Li, W. Gao, F. Pan, S. W. Ma, K. P. Lim, G. N. Feng, X. Lin, S. Rahardja, H. Q. Lu, and Y. Lu, "Adaptive rate control for H.264," J. Visual Commun. Image Represent., vol.17, pp. 376-406, Apr. 2006.
55. N. Kamaci, Y. Altunbasak, and R. M. Mersereau, "Frame bit allocation for the H.264/AVC video coder via Cauchy-density-based rate and distortion models," IEEE Trans. Circuits Syst. Video Technol., vol.15, no.8, pp. 994-1006, Aug. 2005.
56. Z. Chen and K. N. Ngan, "Toward rate-distortion tradeoff in real-time color video coding," IEEE Trans. Circuits Syst. Video Technol., vol.17, no.2, pp. 158-167, Feb. 2007.
57. T. Wiegand, H. Schwarz, A. Joch, F. Kossentini, and G. J. Sullivan, "Rate-constrained coder control and comparison of video coding standards," IEEE Trans. Circuits Syst. Video Technol., vol.13, no.7, pp. 688-703, Jul. 2003.
58. Methodology for the Subjective Assessment of the Quality of Television Pictures, ITU-R BT.500-11, 2002.
59. Y. Jia, W. Lin, and A. A. Kassim, "Estimating just-noticeable distortion for video," IEEE Trans. Circuits Syst. Video Technol., vol.16, no.7, pp. 820-829, Jul. 2006.
60. KTA Software 1.4 [Online]. Available: http://iphome.hhi.de/suehring/tml/ download/KTA/
61. A. B. Watson, J. Hu, and J. F. McGowan, III, "Digital video quality metric based on human vision," J. Electron. Imaging, vol.10, pp. 20-29, Jan. 2001.
62. A. B. Watson and J. Malo, "Video quality measures based on the standard spatial observer," in Proc. IEEE Int. Conf. Image Process., Rochester, NY, Sep. 2002, pp. III-41-III-44.
63. Z. Wang, A. C. Bovik, and E. P. Simoncelli, "Objective video quality assessment," in The Handbook of Video Databases Design and Applications, B. Furht and O. Marqure, Eds. Boca Raton, FL: CRC Press, Sep. 2003, pp. 1041-1078.
64. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Process., vol.13, no.4, pp. 600-612, Apr. 2004.
65. M. G. Ramos and S. S. Hemami, "Suprathreshold wavelet coefficient quantization in complex stimuli: Psychophysical evaluation and analysis," J. Opt. Soc. Am. A, vol.18, pp. 2385-2397, Oct. 2001.
66. D. M. Chandler and S. S. Hemami, "Suprathreshold image compression based on contrast allocation and global precedence," in Proc. 8th Soc. Photo-Optic. Instrum. Eng. Human Vision Electron. Imaging, 2003, pp. 73-86.
67. T. N. Pappas, T. A. Michel, and R. O. Hinds, "Supra-threshold perceptual image coding," in Proc. IEEE Int. Conf. Image Process., Lausanne, Switzerland, Sep. 1996, pp. 237-240.
68. C. F. Harris and J. W. Modestino, "Entropy-constrained sub-band coding of images using a perceptual distortion criteria," in Proc. IEEE Int. Symp. Informat. Theory, San Antonio, TX, Jan. 1993, p. 279.
69. A. B. Watson, "DCT quantization matrics visually optimized for individual images," in Proc. 4th Human Vision, Visual Process., Digital Display, Bellingham, WA, 1993.