The importance of visual attention in improving the 3D-TV viewing experience: Overview and new perspectives

IEEE Transactions on Broadcasting

Volumn 57, Issue 2 PART 2, 2011, Pages 421-431

  Huynh Thu, Quan ^a   Barkowsky, Marcus ^b   Le Callet, Patrick ^b  

^a Technicolor Research and Innovation   (France)

^b UNIVERSITÉ DE NANTES   (France)

Author keywords

3D TV; Human factors; perception; video quality; visual attention

Indexed keywords

3D-TV; HUMAN FACTORS; PERCEPTION; VIDEO QUALITY; VISUAL ATTENTION;

HUMAN ENGINEERING; TELEVISION BROADCASTING;

THREE DIMENSIONAL;

EID: 79957605661     PISSN: 00189316     EISSN: None     Source Type: Journal    
DOI: 10.1109/TBC.2011.2128250     Document Type: Article

References (61)

1. 3D@Home Consortium, "3D theater releases," 2011 [Online]. Available: http://www.3dathome.org/experience-theater.aspx
2. S. Jolly, M. Armstrong, and R. Salmon, "Three-dimensional television a broadcaster's perspective," in Proc. SPIE Conf. Stereoscopic Displays Appl. XX, Jan. 2009, vol. 7237.
3. ISO/IEC JTC1/SC29/WG11 (MPEG) N10357 Video and Requirements Group, "Vision on 3D video," Feb. 2009.
4. ITU-R Study Group 6, "Digital three-dimensional (3D) TV broadcasting," Question ITU-R 128/6, 2008.
5. ITU-T Study Group 9, "Objective and subjective methods for evaluating perceptual audiovisual quality in multimedia services within the terms of Study Group 9," Question 12/9, 2009.
6. J. E. Cutting and P. M. Vishton, "Perceiving layout and knowing distances: The integration, relative potency, and contextual use of different information about depth," in Handbook of Perception and Cognition, W. Epstein and S. Rogers, Eds. San Diego: Academic Press, 1995, vol. 5, Perception of space and motion, pp. 69-117.
7. K. Hamburger, T. Hansen, and K. R. Gegenfurtner, "Geometric-optical illusions at isoluminance," Vis. Res., vol. 47, no. 26, pp. 3276-3285, Dec. 2007. (Pubitemid 350102620)
8. E. Svarverud, S. J. Gilson, and A. Glennerster, "Cue combination for 3D location judgements," J. Vis., vol. 10, no. 1, pp. 1-13, Jan. 2010.
9. G. P. Bingham, A. Bradley, M. Bailey, and R. Vinner, "Accommodation, occlusion, and disparity matching are used to guide reaching: A comparison of actual versus virtual environments," J. Exp. Psych.: Human Percept. Perform., vol. 27, no. 6, pp. 1314-1334, Dec. 2001. (Pubitemid 33597715)
10. Y. Okada, K. Ukai, J. S. Wolffsohn, B. Gilmartin, A. Iijima, and T. Bando, "Target spatial frequency determines the response to conflicting defocus-and convergence-driven accommodative stimuli," Vis. Res., vol. 46, no. 4, pp. 475-484, Feb. 2006. (Pubitemid 41797421)
11. D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, "Vergence accommodation con?icts hinder visual performance and cause visual fatigue," J. Vis., vol. 8, no. 3, pp. 1-30, 2008.
12. W. Chen, J. Fournier, M. Barkowsky, and P. Le Callet, "New requirements of subjective video quality assessment methodologies for 3D-TV," in Proc. 5th Int. Workshop Video Process. Quality Metrics (VPQM), Scottsdale, AZ, Jan. 2010.
13. D. A. Palmer, "Measurement of the horizontal extent of Panum's area by a method of constant stimuli," Optica Acta: Int. J. Opt., vol. 8, no. 2, pp. 151-159, 1961.
14. D. E. Mitchell, "Retinal disparity and diplopia," Vis. Res., vol. 6, no. 7-8, pp. 441-451, Aug. 1966.
15. I. P. Howard and B. J. Rogers, "Binocular fusion and rivalry," in Seeing in Depth, 2002, vol. 1, pp. 271-315, Basic mechanisms. I. Porteous.
16. L. M. Wilcox, J. M. Harris, and S. P. McKee, "The role of binocular stereopsis in monoptic depth perception," Vis. Res., vol. 47, no. 18, pp. 2367-2377, Aug. 2007. (Pubitemid 47239061)
17. A. L. Yarbus, Eye Movements and Vision. New-York: Plenum Press, 1967.
18. M. I. Posner, "Orienting of attention," Quart. J. Exp. Psych., vol. 32, no. 1, pp. 3-25, 1980.
19. A. M. Treisman and G. Gelade, "Feature integration theory of attention," Cognitive Psych., vol. 12, pp. 97-136, Jan. 1980.
20. C. Koch and S. Ullman, "Shifts in selection in visual attention: Towards the underlying neural circuitry," Human Neurobiology, vol. 4, no. 4, pp. 219-227, 1985. (Pubitemid 16176695)
21. O. Le Meur and P. Le Callet, "What we see is most likely to be what matters: Visual attention and applications," in Proc. IEEE Int. Conf. Image Process., Nov. 2009, pp. 3085-3088.
22. L. Itti, C. Koch, and E. Niebur, "Model of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254-1259, Nov. 1998. (Pubitemid 128742001)
23. O. Le Meur, P. Le Callet, D. Barba, and D. Thoreau, "A coherent computational approach to model bottom-up visual attention," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 5, pp. 802-819, May 2006.
24. N. D. B. Bruce and J. K. Tsotsos, "Saliency based on information maximization," Adv. Neural Inf. Process. Syst., vol. 18, pp. 155-162, Jun. 2006.
25. O. Le Meur, P. Le Callet, and D. Barba, "Predicting visual ?xations on video based on low-level visual features," Vis. Res., vol. 47, 2007.
26. X. Hou and L. Zhang, "Saliency detection: A spectral residual approach," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2007, vol. 17, pp. 1-8, no. 22.
27. L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, "Sun: A Bayesian framework for saliency using natural statistics," J. Vis., vol. 8, no. 7, pp. 1-20, 2008.
28. L. Itti and P. F. Baldi, "Bayesian surprise attracts human attention," Vis. Res., vol. 49, no. 10, pp. 1295-1306, May 2009.
29. L. Jansen, S. Onat, and P. König, "Influence of disparity on ?xation and saccades in free viewing of natural scenes," J. Vis., vol. 9, no. 1, pp. 1-19, Jan. 2009.
30. D. A.Wismeijer, C. J. Erkelens, R. van Ee, and M.Wexler, "Depth cue combination in spontaneous eye movements," J. Vis., vol. 10, no. 6, pp. 1-15, Jun. 2010.
31. J. Häkkinen, T. Kawai, J. Takatalo, R. Mitsuya, and G. Nyman, "What do people look at when they watch stereoscopic movies?," in Proc. SPIE Conf. Stereoscopic Displays Appl. XXI, San Jose, CA, Jan. 2010, vol. 7524.
32. C. Ramasamy, D. House, A. Duchowski, and B. Daugherty, "Using eye tracking to analyze stereoscopic filmmaking," in Proc. SIGGRAPH 2009: Posters, 2010.
33. Q. Huynh-Thu and L. Schiatti, "Examination of 3D visual attention in stereoscopic video content," in Proc. SPIE Conf. Human Vis. Electron. Imaging XVI, San Francisco, CA, Jan. 2010.
34. A. Maki, J. O. Eklundh, and P. Nordlund, "A computational model of depth-based attention," in Proc. Int. Conf. Pattern Recog.,Vienna, Aug. 1996, vol. 4.
35. A. Maki, P. Nordlund, and J. O. Eklundh, "Attentional scene segmentation: Integrating depth and motion from phase," Comput. Vis. Image Understanding, vol. 78, pp. 351-373, 2000.
36. N. Ouerhani and H. Hügli, "Computing visual attention from scene depth," in Proc. Int. Conf. Pattern Recog., 2000, pp. 375-378.
37. N. D. B. Bruce and J. K. Tsotsos, "An attentional framework for stereo vision," in Proc. 2nd Canadian Conf. Comput. Robot Vis., May 2005, pp. 88-95.
38. J. K. Tsotsos, S. M. Culhane, W. Y. K. Wai, Y. Lai, N. Davis, and F. Nuflo, "Modeling visual attention via selective tuning," Artif. Intell., vol. 78, no. 1-2, pp. 507-545, 1995.
39. Y. Zhang, G. Jiang, M. Yu, and K. Chen, "Stereoscopic visual attention model for 3D video," in Advances in Multimedia Modeling, ser. Lecture Notes in Computer Science. Berlin: Springer, 2010, vol. 5916, pp. 314-324.
40. K. Essig, M. Pomplun, and H. Ritter, "A neural network for 3D gaze recording with binocular eye trackers," Int. J. Parallel, Emergent Distributed Syst., vol. 21, no. 2, pp. 79-95, Apr. 2006. (Pubitemid 43065595)
41. Y.-M. Kwon, K.-W. Jeon, J. Ki, Q. M. Shahab, S. Jo, and S.-K. Kim, "3D gaze estimation and interaction to stereo display," Int. J. Virtual Reality, vol. 5, no. 3, pp. 41-45, Sep. 2006.
42. J. Chen, Y. Tong, W. Gray, and Q. Ji, "A robust 3D eye gaze tracking system using noise reduction," in Proc. Symp. Eye Tracking Res. Appl. (ETRA), Mar. 2008, vol. 1, pp. 189-196.
43. J. Kim, A. Baik, Y. J. Jung, and D. Park, "2D-to-3D conversion by using visual attention analysis," in Proc. SPIE Conf. Stereoscopic Displays Appl. XXI, San Jose, CA, Jan. 2010, vol. 7524.
44. C. Chamaret and O. Le Meur, "Attention-based video reframing: Validation using eye-tracking," in Proc. Int. Conf. Pattern Recognition, Dec. 2008, pp. 1-4.
45. X. Fan, X. Xie, W.-Y. Ma, H.-J. Zhang, and H.-Q. Zhou, "Visual attention based image browsing on mobile devices," in Proc. Int. Conf. Multimedia Expo., 2003, vol. 2, pp. 53-56.
46. O. Le Meur, S. Cloarec, and P. Guillotel, "Automatic content repurposing for mobile applications," SMPTE Motion Imaging J., vol. 119, no. 1, pp. 30-34, Feb. 2010.
47. C. Chamaret, S. Godeffroy, P. Lopez, and O. Le Meur, "Adaptive 3D rendering based on region-of-interest," in Proc. SPIE Conf. Stereoscopic Displays Appl. XXI, San Jose, CA, Jan. 2010, vol. 7524.
48. ITU-T Study Group 16, "Advanced video coding for generic audiovisual services," ITU-T Recommendation H.264, Mar. 2005.
49. Z. Li and L. Itti, "Visual attention guided video compression," in Proc. Vis. Sci. Soc. Annu. Meeting, Naples, May 2008.
50. M. Barkowsky, K. Wang, R. Cousseau, K. Brunnström, R. Olsson, and P. Le Callet, "Subjective quality assessment of error concealment strategies for 3D-TV in the presence of asymmetric transmission errors," in Proc. Packet Video Workshop, Hong Kong, Dec. 2010.
51. J. Carreira, L. Pinto, N. Rodrigues, S. Faria, and P. Assuncao, "Subjective assessment of frame loss concealment methods in 3D video," in Proc. Picture Coding Symp., Dec. 2010, pp. 182-185.
52. S. Pastoor, "Human factors of 3D displays in advanced image communications," Displays, vol. 14, no. 3, pp. 150-157, Jul. 1993.
53. S. Nagata, "The binocular fusion of human vision on stereoscopic displays-? eld of view and environment effects," Ergonomics, vol. 39, no. 11, pp. 1273-1284, Nov. 1996. (Pubitemid 26370937)
54. M. Wöpking, "Viewing comfort with stereoscopic pictures: An experiment study on subjective effects of disparity magnitude and depth of focus," J. Soc. Inf. Display, vol. 3, no. 3, pp. 101-103, 1995.
55. J. J. Semmlow and D. Heerema, "The role of accommodative convergence at the limit of fusional vergence," Invest. Ophthalmology Vis. Sci., vol. 18, pp. 970-976, 1979. (Pubitemid 9245147)
56. K. Talmi and J. Liu, "Eye and gaze tracking for visually controlled interactive stereoscopic displays," Signal Process. Image Commun., vol. 14, no. 10, pp. 799-810, Aug. 1999.
57. M. Lambooij, W. I'Jsselsteijn, M. Fortuin, and I. Heynderickx, "Visual discomfort and visual fatigue of stereoscopic displays: A review," J. Imaging Sci. Technol., vol. 53, no. 3, pp. 1-14, May 2009.
58. G. Mather and D. R. R. Smith, "Depth cue integration: stereopsis and image blur," Vis. Res., vol. 40, no. 25, pp. 3501-3506, Jan. 2000.
59. S. Hillaire, A. Lécuyer, R. Cozot, and G. Casiez, "Depth-of-?eld blur effects for ?rst-person navigation in virtual environments," IEEE Comput. Graphics Appl., vol. 28, no. 6, pp. 47-55, Nov. 2008.
60. S. Hillaire, A. Lécuyer, R. Cozot, and G. Casiez, "Using an eye-tracking system to improve depth-of-?eld blur effects and camera motions in virtual environments," in Proc. IEEE Virtual Reality, Reno, NV, Mar. 2008, pp. 47-51.
61. S. Hillaire, A. Lécuyer, G. Breton, and T. R. Corte, "Gaze behavior and visual attention model when turning in virtual environments," in Proc. ACM Symp. Virtual Reality Softw. Technol., Nov. 2009, pp. 43-50