Subjective quality evaluation of foveated video coding using audio-visual focus of attention

IEEE Journal on Selected Topics in Signal Processing

Volumn 5, Issue 7, 2011, Pages 1322-1331

  Lee, Jong Seok ^a,b   De Simone, Francesca ^b   Ebrahimi, Touradj ^b  

^a EPFL   (Switzerland)

^b Yonsei University   (South Korea)

Author keywords

Audio visual focus of attention; content dependence; foveated coding; H.264 AVC; memory effect; quality of experience; subjective quality assessment

Indexed keywords

CONTENT DEPENDENCE; FOCUS OF ATTENTION; FOVEATED CODING; H.264/AVC; MEMORY EFFECTS; QUALITY OF EXPERIENCE; SUBJECTIVE QUALITY ASSESSMENTS;

EXPERIMENTS; MOTION PICTURE EXPERTS GROUP STANDARDS; QUALITY OF SERVICE;

QUALITY CONTROL;

EID: 80054802141     PISSN: 19324553     EISSN: None     Source Type: Journal    
DOI: 10.1109/JSTSP.2011.2165199     Document Type: Article

References (43)

1. Z. Wang, L. Lu, and A. C. Bovic, "Foveation scalable video coding with automatic fixation selection," IEEE Trans. Image Process., vol. 12, no. 2, pp. 243-254, Feb. 2003.
2. L. Itti, "Automatic foveation for video compression using a neurobiological model of visual attention," IEEE Trans. Image Process., vol. 13, no. 10, pp. 1304-1318, Oct. 2004.
3. A. Cavallaro, O. Steiger, and T. Ebrahimi, "Semantic video analysis for adaptive content delivery and automatic description," IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 10, pp. 1200-1209,Oct. 2005. (Pubitemid 41640558)
4. C.-W. Tang, "Spatiotemporal visual considerations for video coding," IEEE Trans. Multimedia, vol. 9, no. 2, pp. 231-238, Feb. 2007. (Pubitemid 46188196)
5. G. Boccignone, A. Marcelli, P. Napoletano, G. D. Fiore, G. Iacovoni, and S. Morsa, "Bayesian integration of face and low-level cues for foveated video coding," IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 12, pp. 1727-1740, Dec. 2008.
6. Z. Chen and C. Guillemot, "Perceptually-friendly H.264/AVC video coding based on foveated just-noticeable distortion model," IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 6, pp. 806-819, Jun. 2010.
7. Q. Chen, G. Zhai, X. Yang, and W. Zhang, "Application of scalable visual sensitivity profile in image and video coding," in Proc. IEEE Int. Symp. Circuits Syst., Seattle, WA, May 2008, pp. 268-271.
8. M.-C. Chi, C.-H. Yeh, and M.-J. Chen, "Robust region-of-interest determination based on user attention model through visual rhythm analysis," IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 7, pp. 1025-1038, Jul. 2010.
9. J.-S. Lee and T. Ebrahimi, "Efficient video coding in H.264/AVC by using audio-visual information," in Proc. Int. Conf. Multimedia Signal Process., Rio de Janeiro, Brazil, Oct. 2009, pp. 1-6.
10. J. Driver and C. Spence, "Attention and the crossmodal construction of space," Trends in Cognitive Sci., vol. 2, no. 7, pp. 254-262, Jul. 1998. (Pubitemid 28394243)
11. C. Spence, "Crossmodal spatial attention," Ann. New York Acad. Sci., vol. 1191, pp. 182-200, Mar. 2010.
12. J.-S. Lee, F. D. Simone, and T. Ebrahimi, "Efficient video coding based on audio-visual focus of attention," J. Vis. Commun. Image R., 2010, doi: 10.1016/j.jvcir.2010.11.002.
13. C. Spence and J.Driver, "Audiovisual links in exogenous covert spatial orienting," Percept. Psychophys., vol. 59, no. 1, pp. 1-22, Jan. 1997. (Pubitemid 127456363)
14. C. Spence and J. Driver, "Audiovisual links in endogenous covert spatial attention," J. Exper. Psychol.: Human Percept. Perf., vol. 22, no. 4, pp. 1005-1030, Aug. 1996. (Pubitemid 126418438)
15. D. J. Tellinghuisen and E. J. Nowak, "The inability to ignore auditory distractors as a function of visual task perceptual load," Percept. Psychophys., vol. 65, no. 5, pp. 817-828, 2003. (Pubitemid 37544856)
16. V. Mazza, M. Turatto, M. Rossi, and C. Umilta, "How automatic are audiovisual links in exogenous spatial attention?," Neuropsychologia, vol. 45, no. 3, pp. 514-522, 2007. (Pubitemid 44894459)
17. M. G. Martini and C. T. E. R. Hewage, "Flexible macroblock ordering for context-aware ultrasound video transmission over mobile WiMAX," Int. J. Telemed. Appl., vol. 2010, pp. 1-14, 2010.
18. W. S. Geisler and J. S. Perry, "A real-time foveated multiresolution system for low-bandwidth video communication," in Proc. SPIE, San Jose, CA, Jan. 1998, vol. 3299, pp. 294-305.
19. C. Guo and L. Zhang, "Anovelmultiresolution spatiotemporal saliency detection model and its applications in image and video compression," IEEE Trans. Image Process., vol. 19, no. 1, pp. 185-198, Jan. 2010.
20. Y. Liu, Z. G. Li, andY. C. Soh, "Region-of-interest based resource allocation for conversational video communication of H.264/AVC," IEEE Trans. Circuits Syst. Video Technol., vol. 18, no. 1, pp. 134-139, Jan. 2008.
21. M.-C. Chi, M.-J. Chen, C.-H. Yeh, and J.-A. Jhu, "Region-of-interest video coding based on rate and distortion variations for H.263 ," Signal Process.: Image Commun., vol. 23, pp. 127-142, 2008.
22. D. Chai and K. N. Ngan, "Face segmentation using skin-color map in videophone applications," IEEE Trans. Circuits Syst. Video Technol., vol. 9, no. 4, pp. 551-934, Jun. 1999.
23. N. Doulamis, A. Doulamis, D. Kalogeras, and S. Kollias, "Low bit-rate coding of image sequences using adaptive regions of interest," IEEE Trans. Circuits Syst. Video Technol., vol. 8, no. 8, pp. 928-934, Dec. 1998. (Pubitemid 128743744)
24. Y. Sun, I. Ahmad, D. Li, and Y.-Q. Zhang, "Region-based rate control and bit allocation for wireless video transmission," IEEE Trans. Multimedia, vol. 8, no. 1, pp. 1-10, Feb. 2006.
25. H. R. Sheikh, B. L. Evans, and A. C. Bovik, "Real-time foveation techniques for low bit rate video coding," Real-Time Imaging, vol. 9, pp. 27-40, 2003.
26. Z. Li, S. Qin, and L. Itti, "Visual attention guided bit allocation in video compression," Image Vis. Comput., vol. 29, no. 1, pp. 1-14, Jan. 2011.
27. S.Winkler and P.Mohandas, "The evolution of video quality measurement: From PSNR to hybrid metrics," IEEE Trans. Broadcast., vol. 54, no. 3, pp. 660-668, Sep. 2008.
28. F. D. Simone, L. Goldmann, J.-S. Lee, and T. Ebrahimi, "Towards high efficiency video coding: Subjective evaluation of potential coding technologies," J. Vis. Commun. Image R., 2011, doi: 10.1016/j.jvcir. 2011.01.008.
29. Methodology for the Subjective Assessment of the Quality of Television Pictures, Rec. ITU-R BT.500-11, 2002, Geneva, Switzerland.
30. D. Gatica-Perez, G. Lathoud, J.-M. Odobez, and I.McCowan, "Audiovisual probabilistic tracking of multiple speakers in meetings," IEEE Trans. Audio Speech Lang. Process., vol. 15, no. 2, pp. 601-616, Feb. 2007.
31. P. Perez, J. Vermaak, and A. Blake, "Data fusion for visual tracking with particles," Proc. IEEE, vol. 92, no. 3, pp. 495-513, Mar. 2004. (Pubitemid 40890756)
32. V. Pavlovic, A. Garg, J. M. Rehg, and T. S. Huang, "Multimodal speaker detection using error feedback dynamic Bayesian networks," in Proc. Int. Conf. Comput. Vis. Pattern Recognit., Hilton Head Island, SC, 2000, pp. 34-41.
33. P. Besson, V. Popovici, J.-M. Vesin, J.-P. Thiran, and M. Kunt, "Extraction of audio features specific to speech production for multimodal speaker detection," IEEE Trans. Multimedia, vol. 10, no. 1, pp. 63-73, Jan. 2008.
34. D. R. Hardoon, S. Szedmak, and J. Shawe-Taylor, "Canonical correlation analysis: An overview with application to learning methods," Neural Comput., vol. 16, no. 12, pp. 2639-2664, 2004. (Pubitemid 39604012)
35. J.-S. Lee, F. D. Simone, and T. Ebrahimi, "Video coding based on audio-visual attention," in Proc. Int. Conf. Multimedia Expo, New York, Jun. 2009, pp. 57-60.
36. E. Kidron, Y. Y. Schechner, and M. Eland, "Cross-modal localization via sparsity," IEEE Trans. Signal Process., vol. 55, no. 4, pp. 1390-1404, Apr. 2007. (Pubitemid 46563162)
37. M. Nyström and K. Holmqvist, "Effect of compressed offline foveated video on viewing behavior and subjective quality," ACM Trans. Multimedia Comput., Commun., Applicat., vol. 6, no. 1, pp. 1-14, Feb. 2010.
38. H.264/AVC JM Reference Software, 2008 [Online]. Available: http:// iphome.hhi.de/suehring/tml/
39. S. Onat, K. Libertus, and P. König, "Integrating audiovisual information for the control of overt attention," J. Vis., vol. 7, no. 10, pp. 1-16, Jul. 2007.
40. A. L. Yarbus, Eye Movements and Vision. New York: Plenum, 1976.
41. W. Einhäuser, U. Rutishauser, and C. Koch, "Task-demands can immediately reverse the effects of sensory-driven saliency in complex visual stimuli," J. Vis., vol. 8, no. 2, pp. 1-19, Feb. 2008.
42. R. Carmi and L. Itti, "The role of memory in guiding attention during natural vision," J. Vis., vol. 6, no. 9, pp. 898-914, Aug. 2006.
43. O. L. Meur, A. Ninassi, P. L. Callet, and D. Barba, "Do video coding impairments disturb the visual attention deployment?," Signal Process.: Image Commun., vol. 25, no. 8, pp. 597-609, Sep. 2010.