Saliency-aware video compression

IEEE Transactions on Image Processing

Volumn 23, Issue 1, 2014, Pages 19-33

  Hadizadeh, Hadi ^a   Bajic, Ivan V ^a  

^a SIMON FRASER UNIVERSITY   (Canada)

Author keywords

DCT; global motion estimation; Video compression; visual saliency

Indexed keywords

COMPRESSION METHODS; DCT; GLOBAL MOTION ESTIMATION; PERCEPTUAL VIDEO CODING; REGION OF INTEREST; STATE OF THE ART; VISUAL QUALITIES; VISUAL SALIENCY;

IMAGE CODING; IMAGE COMPRESSION; VISUAL COMMUNICATION;

VIDEO SIGNAL PROCESSING;

ALGORITHM; ARTICLE; ARTIFACT; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; IMAGE ENHANCEMENT; INFORMATION PROCESSING; METHODOLOGY; PHOTOGRAPHY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; SIGNAL PROCESSING; VIDEORECORDING;

ALGORITHMS; ARTIFACTS; DATA COMPRESSION; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; PATTERN RECOGNITION, AUTOMATED; PHOTOGRAPHY; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; SIGNAL PROCESSING, COMPUTER-ASSISTED; VIDEO RECORDING;

EID: 84888246981     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2013.2282897     Document Type: Article

References (62)

1. M. A. Robertson and R. L. Stevenson, "DCT quantization noise in compressed images," IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 1, pp. 27-38, Jan. 2005. (Pubitemid 40135875)
2. A. Leontaris, P. C. Cosman, and A. R. Reibman, "Quality evaluation of motion-compensated edge artifacts in compressed video," IEEE Trans. Image Process., vol. 16, no. 11, pp. 943-956, Apr. 2007. (Pubitemid 46542385)
3. L. Itti, C. Koch, and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254-1259, Nov. 1998. (Pubitemid 128742001)
4. L. Itti, "Automatic foveation for video compression using a neurobiological model of visual attention," IEEE Trans. Image Process., vol. 13, no. 10, pp. 1304-1318, Oct. 2004.
5. Z. Chen, N. K. Ngan, and W. Lin, "Perceptual video coding: Challenges and approaches," in Proc. IEEE ICME, Jul. 2010, pp. 784-789.
6. Z. Li, S. Qin, and L. Itti, "Visual attention guided bit allocation in video compression," Image and Vision Computing, vol. 29, no. 1, pp. 1-14, 2011.
7. Y. Liu, Z. Li, and Y. C. Soh, "Region-of-interest based resource allocation for conversational video communications of H.264/AVC," IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 1, pp. 134-139, Jan. 2008.
8. S. Daly, "The visible difference predictor: An algorithm for the assessment of image fidelity," in Digital Images and Human Vision, A. B. Watson, Ed. Cambridge, MA, USA: MIT Press, 1993, pp. 179-206.
9. E. C. Larson and D. M. Chandler, "Most apparent distortion: Fullreference image quality assessment and the role of strategy," J. Electron. Imag., vol. 19, no. 1, pp. 011006.1-011006.21, 2010.
10. H. Hadizadeh and I. V. Bajíc, "Saliency-preserving video compression," in Proc. IEEE ICME, Jul. 2011, pp. 1-6.
11. I. E. Richardson, The H.264 Advanced Video Compression Standard. New York, NY, USA: Wiley, 2010.
12. L. Itti, "Automatic foveation for video compression using a neurobiological model of visual attention," IEEE Trans. Image Process., vol. 13, no. 10, pp. 1304-1318, Oct. 2004.
13. T. Wiegand and B. Girod, "Lagrange multiplier selection in hybrid video coder control," in Proc. IEEE Int. Conf. Image Process., vol. 3. Oct. 2001, pp. 542-545. (Pubitemid 33058039)
14. G. Sullivan and T. Wiegand, "Rate-distortion optimization for video compression," IEEE Signal Process. Mag., vol. 15, no. 6, pp. 74-90, Nov. 1998.
15. The H.264/AVC JM Reference Software ver. 16.1 [Online] Available: http://iphome.hhi.de/suehring/tml/
16. X. Li, N. Oertel, A. Hutter, and A. Kaup, "Laplace distribution based Lagrangian rate distortion optimization for hybrid video coding," IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 2, pp. 193-205, Feb. 2009.
17. X. Li, N. Oertel, A. Hutter, and A. Kaup, "Advanced lagrange multiplier selection for hybrid video coding," in Proc. IEEE ICME, Jul. 2007, pp. 364-367.
18. L. Chen and I. Garbacea, "Adaptive estimation in Lagrangian ratedistortion optimization for video coding," Proc. SPIE, vol. 6077, pp. 60772B-1-60772B-8, Jan. 2006.
19. J. Zhang, X. Yi, N. Ling, and W. Shang, "Context adaptive Lagrange multiplier CALM) for rate-distortion optimal motion estimation in video coding," IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 820-828, Jun. 2010.
20. M. Jiang and N. Ling, "On Lagrange multiplier and quantizer adjustment for H.264 frame-layer video rate control," IEEE Trans. Circuits Syst. Video Technol., vol. 16, no. 5, pp. 663-669, May 2006.
21. M. Wang and B. Yan, "Lagrangian multiplier based joint three-layer rate control for H.264/AVC," IEEE Signal Process. Lett., vol. 16, no. 8, pp. 679-682, Aug. 2009.
22. X. Yang, W. Lin, Z. Lu, E. Ong, and S. Yao, "Motion-compensated residue pre-processing in video coding based on just-noticeable distortion profile," IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 6, pp. 742-752, Jun. 2005.
23. Z. Chen and C. Guillemot, "Perceptually-friendly H.264/AVC video coding based on foveated just-noticeable-distortion model," IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 806-819, Jun. 2010.
24. Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, Apr. 2004.
25. C. Yang, R. Leung, L. Po, and Z. Mai, "An SSIM-optimal H.264/AVC inter frame encoder," in Proc. IEEE Int. Conf. Intell. Comput. Intell. Syst., vol. 4. Nov. 2009, pp. 291-295.
26. C. Yang, H. Wang, and L. Po, "Improved inter prediction based on structural similarity in H.264.," in Proc. IEEE Int. Conf. Signal Process. Commun., vol. 2. Nov. 2007, pp. 340-343.
27. Y. H. Huang, T. S. Ou, P. Y. Su, and H. Chen, "Perceptual ratedistortion optimization using structural similarity index as quality metric," IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 11, pp. 1614-1624, Nov. 2010.
28. S. Wang, A. Rehman, Z. Wang, S. Ma, and W. Gao, "SSIM-motivated rate-distortion optimization for video coding," IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 4, pp. 516-529, Apr. 2012.
29. X. Wang, L. Su, Q. Huang, and C. Liu, "Visual perception based Lagrangian rate distortion optimization for video coding," in Proc. IEEE Int. Conf. Image Process., Sep. 2011, pp. 1653-1656.
30. S. Wang, A. Rehman, Z. Wang, S. Ma, and W. Gao, "Perceptual video coding based on SSIM-inspired divisive normalization," IEEE Trans. Image Process., vol. 22, no. 4, pp. 1418-1429, Apr. 2013.
31. C. Yeo, H. L. Tan, and Y. H. Tan, "On rate distortion optimization using SSIM," in Proc. IEEE ICASSP, Mar. 2012, pp. 833-836.
32. X. Yang, W. Lin, Z. Lu, X. Lin, S. Rahardja, E. P. Ong, and S. Yao, "Rate control for videophone using local perceptual cues," IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 4, pp. 496-507, Apr. 2005.
33. V. A. Mateescu, H. Hadizadeh, and I. V. Bajíc, "Evaluation of several visual saliency models in terms of gaze prediction accuracy on video," in Proc. IEEE Globecom Workshops, Dec. 2012, pp. 1304-1308.
34. H. Hadizadeh, I. V. Bajíc, and G. Cheung, "Video error concealment using a computation-efï'cient low saliency prior," IEEE Trans. Multimedia, to appear.
35. R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, "Frequency- tuned salient region detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Miami Beach, FL, USA, Jun. 2009, pp. 1597-1604.
36. A. Torralba and A. Oliva, "Statistics of natural image categories," Netw., Comput. Neural Syst., vol. 14, no. 3, pp. 391-412, 2003. (Pubitemid 38011142)
37. H. Hadizadeh, "Visual saliency in video compression and transmission," Ph.D. dissertation, School Eng. Sci., Simon Fraser Univ., Apr. 2013.
38. L. Itti, "Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes," Vis. Cognit., vol. 12, no. 6, pp. 1093-1123, 2005. (Pubitemid 41283179)
39. Z. Wang and Q. Li, "Video quality assessment using a statistical model of human visual speed perception," J. Opt. Soc. Amer. A, vol. 24, no. 2, pp. B61-B69, Dec. 2007. (Pubitemid 351325125)
40. O. Le Meur, P. Le Callet, and D. Barba, "Predicting visual fixations on video based on low-level visual features," Vis. Res., vol. 47, no. 19, pp. 2483-2498, Sep. 2007. (Pubitemid 47376844)
41. Y.-M. Chen and I. V. Bajíc, "Motion vector outlier rejection cascade for global motion estimation," IEEE Signal Process. Lett., vol. 17, no. 2, pp. 197-200, Feb. 2010.
42. Y. Fang, Z. Chen, W. Lin, and C.-W. Lin, "Saliency detection in the compressed domain for adaptive image retargeting," IEEE Trans. Image Process., vol. 9, no. 21, pp. 3888-3901, Sep. 2012.
43. C. Chamaret, J. C. Chevet, and O. Le Meur, "Spatio-temporal combination of saliency maps and eye-tracking assessment of different strategies," in Proc. IEEE Int. Conf. Image Process., Sep. 2010, pp. 1077-1080.
44. Z. G. Li, W. Gao, F. Pan, S. W. Ma, K. P. Lim, G. N. Feng, X. Lin, S. Rahardja, H. Q. Lu, and Y. Lu, "Adaptive rate control for H.264," J. Vis. Commun. Image Represent., vol. 17, pp. 376-406, Apr. 2006. (Pubitemid 43326253)
45. N. Kamaci, Y. Altunbasak, and R. M. Mersereau, "Frame bit allocation for the H.264/AVC video coder via Cauchy-density-based rate and distortion models," IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 8, pp. 994-1006, Aug. 2005. (Pubitemid 41343370)
46. Z. Chen and K. N. Ngan, "Toward rate-distortion tradeoff in real-time color video coding," IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 2, pp. 158-167, Feb. 2007. (Pubitemid 46379664)
47. C. Yeo, Y. Han Tan, Z. Li, and S. Rahardja, "Mode-dependent transforms for coding directional intra prediction residuals," IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 4, pp. 545-554, Apr. 2012.
48. I.-M. Pao and M.-T. Sun, "Modeling DCT coefficients for fast video encoding," IEEE Trans. Circuits Syst. Video Technol., vol. 9, no. 4, pp. 608-616, Jun. 1999.
49. A. K. Jain, Fundamentals of Digital Image Processing. Englewood Cliffs, NJ, USA: Prentice-Hall, 1989.
50. B. Widrow and I. Kollar, Quantization Noise: Roundoff Error in Digital Computation, Signal Processing, Control, and Communications. Cambridge, U.K.: Cambridge Univ. Press, 2008.
51. H. Hadizadeh, M. J. Enriquez, and I. V. Bajíc, "Eye-tracking database for a set of standard video sequences," IEEE Trans. Image Process., vol. 21, no. 2, pp. 898-903, Feb. 2012.
52. iLab Neuromorphic Vision C++ Toolkit ver. 3.1 [Online] Available: http://ilab.usc.edu/toolkit/
53. M. Narwaria, L. Weisi, and L. Anmin, "Low-complexity video quality assessment using temporal quality variations," IEEE Trans. Multimedia, vol. 14, no. 3, pp. 525-535, Mar. 2012.
54. M. Narwaria and L.Weisi, "Machine learning based modeling of spatial and temporal factors for video quality assessment," in Proc. IEEE ICIP, Sep. 2011, pp. 2513-2516.
55. J. T. McClave and T. Sincich, Statistics, 9th ed. Upper Saddle River, NJ, USA: Prentice-Hall, 2003.
56. G. Bjontegaard, "Calculation of average PSNR differences between RD curves," in 13th VCEG Meeting,VCEG-M33, Apr. 2001.
57. M. Pinson and S. Wolf, "A new standardized method for objectively measuring video quality," IEEE Trans. Broadcast., vol. 50, no. 3, pp. 312-322, Jun. 2004.
58. S. Chikkerur, V. Sundaram, M. Reisslein, and L. J. Karam, "Objective video quality assessment methods: A classification, review, and performance comparison," IEEE Trans. Broadcast., vol. 57, no. 2, pp. 165-182, Jun. 2011.
59. T.-S. Ou, Y.-H. Huang, and H. H. Chen, "SSIM-based perceptual rate control for video coding," IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 5, pp. 682-691, May 2011.
60. M. Taylor and C. Creelman, "PEST: Efficient estimates on probability functions," J. Acoust. Soc. Amer., vol. 41, no. 4A, pp. 782-787, Jan. 1967.
61. "Recommendation BT.500-8: Methodology for the subjective assessment of the quality of television pictures," ITU, San Jose, CA, USA, Tech. Rep. ITU-R BT.500-11, 1998.
62. D. J. Sheskin, Handbook of Parametric and Nonparametric Statistical Procedures. London, U.K.: Chapman & Hall, 2007. Canada