Efficient video coding based on audio-visual focus of attention

Journal of Visual Communication and Image Representation

Volumn 22, Issue 8, 2011, Pages 704-711

  Lee, Jong Seok ^a   De Simone, Francesca ^a   Ebrahimi, Touradj ^a  

^a EPFL   (Switzerland)

Author keywords

Audio visual focus of attention; Audio visual source localization; Canonical correlation analysis; Flexible macroblock ordering (FMO); H.264 AVC; Quality of experience; Subjective quality assessment; Video coding

Indexed keywords

AUDIO-VISUAL; CANONICAL CORRELATION ANALYSIS; FLEXIBLE MACROBLOCK ORDERING; FOCUS OF ATTENTION; H.264/AVC; QUALITY OF EXPERIENCE; SUBJECTIVE QUALITY ASSESSMENTS;

IMAGE CODING; MOTION PICTURE EXPERTS GROUP STANDARDS; QUALITY OF SERVICE; VISUAL COMMUNICATION;

AUDIO ACOUSTICS;

EID: 80053958796     PISSN: 10473203     EISSN: 10959076     Source Type: Journal    
DOI: 10.1016/j.jvcir.2010.11.002     Document Type: Article

References (31)

1. L. Itti Automatic foveation for video compression using a neurobiological model of visual attention IEEE Trans. Image Process. 13 2004 1304 1318
2. G. Boccignone, A. Marcelli, P. Napoletano, G.D. Fiore, G. Iacovoni, and S. Morsa Bayesian integration of face and low-level cues for foveated video coding IEEE Trans. Circuits Syst. Video Technol. 18 2008 1727 1740
3. Z. Wang, L. Lu, and A.C. Bovic Foveation scalable video coding with automatic fixation selection IEEE Trans. Image Process. 12 2003 243 254
4. Q. Chen, G. Zhai, X. Yang, W. Zhang, Application of scalable visual sensitivity profile in image and video coding, in: Proc. IEEE Int. Symposium on Circuits and Systems, Seattle, USA, 2008, pp. 268-271.
5. C.-W. Tang Spatiotemporal visual considerations for video coding IEEE Trans. Multimedia 9 2007 231 238 (Pubitemid 46188196)
6. A. Cavallaro, O. Steiger, and T. Ebrahimi Semantic video analysis for adaptive content delivery and automatic description IEEE Trans. Circuits Syst. Video Technol. 15 2005 1200 1209 (Pubitemid 41640558)
7. J. Driver, and C. Spence Attention and the crossmodal construction of space Trends Cogn. Sci. 2 1998 254 262 (Pubitemid 28394243)
8. J.J. McDonald, W.A. Teder-Salejarvi, F.D. Russo, and S.A. Hillyard Neural substrates of perceptual enhancement by cross-modal spatial attention J. Cogn. Neurosci. 15 2003 10 19 (Pubitemid 36106259)
9. C. Spence, and J. Driver Audiovisual links in exogenous covert spatial orienting Percept. Psychophys. 59 1997 1 22 (Pubitemid 127456363)
10. C. Spence, and J. Driver Audiovisual links in endogenous covert spatial attention J. Exp. Psychol.: Human Percept. Perform. 22 1996 1005 1030 (Pubitemid 126418438)
11. D.J. Tellinghuisen, and E.J. Nowak The inability to ignore auditory distractors as a function of visual task perceptual load Percept. Psychophys. 65 2003 817 828 (Pubitemid 37544856)
12. V. Mazza, M. Turatto, M. Rossi, and C. Umilta How automatic are audiovisual links in exogenous spatial attention? Neuropsychologia 45 2007 514 522 (Pubitemid 44894459)
13. J. Vroomen, and B. de Gelder Perceptual effects of cross-modal stimulation: ventriloquism and the freezing phenomenon Handbook of Multisensory Processes 2004 MIT Press 141 150
14. J. Vroomen, and B. de Gelder Sound enhances visual perception: cross-modal effects of auditory organisation on vision J. Exp. Psychol.: Human Percept. Perform. 26 2000 1583 1590
15. L.A. Ross, D. Saint-Amour, V.M. Leavitt, D.C. Javitt, and J.J. Foxe Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments Cereb. Cortex 17 2007 1147 1153 (Pubitemid 46598855)
16. H. McGurk, and J. MacDonald Hearing lips and seeing voices Nature 264 1976 746 748
17. R. Sharma, V.I. Pavlović, and T.S. Huang Toward multimodal human-computer interface Proc. IEEE 86 1998 853 869 (Pubitemid 128720228)
18. E. Macaluso, and J. Driver Multisensory spatial interactions: a window onto functional integration in the human brain Trends Neurosci. 28 2005 264 271 (Pubitemid 41556911)
19. J.-S. Lee, T. Ebrahimi, Efficient video coding in H.264/AVC by using audio-visual information, in: Proc. Int. Conf. Multimedia Signal Processing, Rio de Janeiro, Brazil, 2009, pp. 1-6.
20. H. Zhou, M. Taj, and A. Cavallaro Target detection and tracking with heterogeneous sensors IEEE J. Sel. Top. Sign. Proc. 2 2008 503 513
21. D. Gatica-Perez, G. Lathoud, J.-M. Odobez, and I. McCowan Audiovisual probabilistic tracking of multiple speakers in meetings IEEE Trans. Audio Speech Lang. Proc. 15 2007 601 616
22. P. Perez, J. Vermaak, and A. Blake Data fusion for visual tracking with particles Proc. IEEE 92 2004 495 513 (Pubitemid 40890756)
23. V. Pavlović, A. Garg, J.M. Rehg, T.S. Huang, Multimodal speaker detection using error feedback dynamic Bayesian networks, in: Proc. Int. Conf. Computer Vision and Pattern Recognition, Hilton Head Island, SC, USA, 2000, pp. 34-41.
24. R. Cutler, L. Davis, Look who's talking: speaker detection using video and audio correlation, in: Proc. Int. Conf. Multimedia and Expo, New York, NY, USA, 2000, pp. 1589-1592. (Pubitemid 33059248)
25. P. Besson, V. Popovici, J.-M. Vesin, J.-P. Thiran, and M. Kunt Extraction of audio features specific to speech production for multimodal speaker detection IEEE Trans. Multimedia 10 2008 63 73
26. D.R. Hardoon, S. Szedmak, and J. Shawe-Taylor Canonical correlation analysis: an overview with application to learning methods Neural Comput. 16 2004 2639 2664 (Pubitemid 39604012)
27. J.-S. Lee, F. De Simone, T. Ebrahimi, Video coding based on audio-visual attention, in: Proc. Int. Conf. Multimedia and Expo, New York, NY, USA, 2009, pp. 57-60.
28. E. Kidron, Y.Y. Schechner, and M. Eland Cross-modal localization via sparsity IEEE Trans. Signal Process. 55 2007 1390 1404 (Pubitemid 46563162)
29. Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG, H.264/AVC JM Reference Software, 2008, .
30. Recommendation ITU-R BT.500-11, Methodology for the Subjective Assessment of the Quality of Television Pictures, 2002.
31. J.-S. Lee, F. De Simone, T. Ebrahimi, Influence of audio-visual attention on perceived quality of standard definition multimedia content, in: Proc. Int. Workshop on Quality of Multimedia Experience, San Diego, CA, USA, 2009, pp. 13-19.