Stereoscopic 3D visual discomfort prediction: A dynamic accommodation and vergence interaction model

IEEE Transactions on Image Processing

Volumn 25, Issue 2, 2016, Pages 615-629

  Oh, Heeseok ^a   Lee, Sanghoon ^a   Bovik, Alan Conrad ^b  

^a Yonsei University   (South Korea)

^b The University of Texas at Austin   (United States)

Author keywords

Accommodation vergence mismatch; Dynamic accommodation and vergence interaction model; Stereoscopic 3D; Visual discomfort prediction

Indexed keywords

BINOCULARS; BINS; CRYSTALLINE MATERIALS; DYNAMICS; EYE MOVEMENTS; FORECASTING; LENSES; OPTICAL SYSTEMS; STEREO IMAGE PROCESSING;

ACCURATE PREDICTION; HUMAN VISUAL SYSTEM; MASSIVELY PARALLEL PROCESSING; PANUM'S FUSIONAL AREA; QUANTITATIVE MODELS; SUBJECTIVE ASSESSMENTS; VERGENCES; VISUAL DISCOMFORT;

PHYSIOLOGICAL MODELS;

ADULT; BIOLOGICAL MODEL; DEPTH PERCEPTION; EYE MOVEMENT; HUMAN; PHYSIOLOGY; PROCEDURES; STATISTICAL MODEL; THREE DIMENSIONAL IMAGING; YOUNG ADULT;

ADULT; DEPTH PERCEPTION; EYE MOVEMENTS; HUMANS; IMAGING, THREE-DIMENSIONAL; MODELS, BIOLOGICAL; MODELS, STATISTICAL; YOUNG ADULT;

EID: 85009437523     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2015.2506340     Document Type: Article

References (65)

1. C. Richardt, L. Swirski, I. P. Davies, and N. A. Dodgson, "Predicting stereoscopic viewing comfort using a coherence-based computational model, " in Proc. Int. Symp. Comput. Aesthetics Graph. Visualizat., Imag., Vancouver, BC, Canada, Aug. 2011, pp. 97-104.
2. F. L. Kooi and A. Toet, "Visual comfort of binocular and 3D displays, " Displays, vol. 25, nos. 2-3, pp. 99-108, Aug. 2004.
3. H. Sohn, Y. J. Jung, S.-I. Lee, and Y. M. Ro, "Predicting visual discomfort using object size and disparity information in stereoscopic images, " IEEE Trans. Broadcast., vol. 59, no. 1, pp. 28-37, Mar. 2013.
4. M. T. M. Lambooij, W. A. Ijsselsteijn, and I. Heynderickx, "Visual discomfort in stereoscopic displays: A review, " Proc. SPIE, vol. 6490, p. 64900I, Feb. 2007.
5. D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, "Vergence-accommodation conflicts hinder visual performance and cause visual fatigue, " J. Vis., vol. 8, no. 3, pp. 1-30, Mar. 2008.
6. S. Yano, M. Emoto, and T. Mitsuhashi, "Two factors in visual fatigue caused by stereoscopic HDTV images, " Displays, vol. 25, no. 4, pp. 141-150, Nov. 2004.
7. W. J. Tam, F. Speranza, S. Yano, K. Shimono, and H. Ono, "Stereoscopic 3D-TV: Visual comfort, " IEEE Trans. Broadcast., vol. 57, no. 2, pp. 335-346, Jun. 2011.
8. K. Ukai and P. A. Howarth, "Visual fatigue caused by viewing stereoscopic motion images: Background, theories, and observations, " Displays, vol. 29, no. 2, pp. 106-116, Mar. 2010.
9. M. Emoto, T. Niida, and F. Okano, "Repeated vergence adaptation causes the decline of visual functions in watching stereoscopic television, " J. Display Technol., vol. 1, no. 2, pp. 328-340, Dec. 2005.
10. S. Yang and J. E. Sheedy, "Effects of vergence and accommodative responses on viewer's comfort in viewing 3D stimuli, " in Proc. SPIE, vol. 7863, p. 78630Q, Jan. 2012.
11. Y. Okada, K. Ukai, J. S. Wolffsohn, B. Gilmartin, A. Iijima, and T. Bando, "Target spatial frequency determines the response to con-flicting defocus-and convergence-driven accommodative stimuli, " Vis. Res., vol. 46, pp. 475-484, Feb. 2005.
12. V. V. Krishnan and L. Stark, "A heuristic model for the human vergence eye movement system, " IEEE Trans. Biomed. Eng., vol. BME-24, no. 1, pp. 44-49, Jan. 1977.
13. A. S. Eadie, L. S. Gray, P. Carlin, and M. Mon-Williams, "Modelling adaptation effects in vergence and accommodation after exposure to a simulated virtual reality stimulus, " Ophthalmic Physiol. Opt., vol. 20, no. 3, pp. 242-251, 2000.
14. C. J. Erkelens, "A dual visual-local feedback model of the vergence eye movement system, " J. Vis., vol. 11, no. 10, pp. 1-14, 2011.
15. Y.-F. Chen, Y.-Y. Lee, T. Chen, J. L. Semmlow, and T. L. Alvarez, "Behaviors, models, and clinical applications of vergence eye movements, " J. Med. Biol. Eng., vol. 30, no. 1, pp. 1-16, 2010.
16. S. Yano, S. Ide, T. Mitsuhashi, and H. Thwaites, "A study of visual fatigue and visual comfort for 3D HDTV/HDTV images, " Displays, vol. 23, no. 4, pp. 191-201, Sep. 2002.
17. Y Nojiri, H. Yamanoue, A. Hanazato, and F. Okano, "Measurement of parallax distribution and its application to the analysis of visual comfort for stereoscopic HDTV, " Proc. SPIE, vol. 5006, pp. 195-205, May 2003.
18. J. Choi, D. Kim, S. Choi, and S. Sohn, "Visual fatigue modeling and analysis for stereoscopic video, " Opt. Eng., vol. 51, no. 1, pp. 017206-1-017206-11, Feb. 2012.
19. D. Kim and K. Sohn, "Visual fatigue prediction for stereoscopic image, " IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 2, pp. 231-236, Feb. 2011.
20. F. Qi, X. Fan, D. Zhao, T. Jiang, and J. Zhang, "A visual comfort assessment metric for stereoscopic images, " in Proc. IEEE Int. Conf. Image Process. (ICIP), Quebec City, QC, Canada, Sep. 2015.
21. J. Park, S. Lee, and A. C. Bovik, "3D visual discomfort prediction: Vergence, foveation, and the physiological optics of accommodation, " IEEEJ. Sel. Topics Signal Process., vol. 8, no. 3, pp. 415-427, Jun. 2014.
22. J. Park, H. Oh, S. Lee, and A. C. Bovik, "3D visual discomfort predictor: Analysis of disparity and neural activity statistics, " IEEE Trans. Image Process., vol. 24, no. 3, pp. 1101-1114, Mar. 2015.
23. C. M. Schor, "A dynamic model of cross-coupling between accommodation and convergence: Simulations of step and frequency responses, " Optometry Vis. Sci, vol. 69, no. 4, pp. 258-269, 1992.
24. A. P. Pentland, "A new sense for depth of field, " IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-9, no. 4, pp. 523-531, Jul. 1987.
25. V. De Silva, A. Fernando, S. Worrall, H. K. Arachchi, and A. Kondoz, "Sensitivity analysis of the human visual system for depth cues in stereoscopic 3D displays, " IEEE Trans. Multimedia, vol. 13, no. 3, pp. 498-506, Jun. 2011.
26. J. E. Cutting and P. M. Vishton, "Perceiving layout and knowing distances: The integration, relative potency, and contextual use of different information about depth, " in Perception of Space and Motion, W. Epstein and S. Rogers, Eds. San Diego, CA, USA: Academic, 1995, pp. 69-117.
27. C. M. Schor, "The relationship between fusional vergence eye movements and fixation disparity, " Vis. Res., vol. 19, no. 12, pp. 1359-1367, 1979.
28. C. M. Schor, "Adaptive regulation of accommodative vergence and vergence accommodation, " Amer. J. Optometry Physiol. Opt., vol. 63, no. 8, pp. 587-609, 1986.
29. C. M. Schor and T. K. Tsuetaki, "Fatigue of accommodation and vergence modifies their mutual interactions, " Invest. Ophthalmolo. Vis. Sci., vol. 28, no. 8, pp. 1250-1259, 1987.
30. D. T. McRuer and D. Graham, "Human pilot dynamics in compensatory system, " Tech. Rep. AFFDL-TR-65-15, 1965.
31. A. S. Eadie and P. J. Carlin, "Evolution of control system models of ocular accommodation, vergence and their interaction, " Med. Biol. Eng. Comput., vol. 33, pp. 517-524, Jul. 1995.
32. J. Semmlow and P. Wetzel, "Dynamic contributions of the components of binocular vergence, " J. Opt. Soc. Amer. A, vol. 69, no. 5, pp. 639-645, 1979.
33. J. A. Bullinaria and P. M. Riddell, "Learning and evolution of control systems, " Neural Netw. World, vol. 10, no. 4, pp. 535-544, 2000.
34. W. Jaschinski, A. Svede, and S. Jainta, "Relation between fixation disparity and the asymmetry between convergent and divergent disparity step responses, " Vis. Res., vol. 48, no. 2, pp. 253-263, Jan. 2008.
35. B. C. Jiang, "A method for correction of CA/C ratio based on linear model of accommodation and vergence, " Optometry Vis. Sci., vol. 71, no. 3, pp. 192-198, 1994.
36. J. Maxwell, J. Tong, and C. M. Schor, "The first and second order dynamics of accommodative convergence and disparity convergence, " Vis. Res., vol. 50, no. 17, pp. 1728-1739, Aug. 2010.
37. B. C. Jiang, "Accommodative vergence is driven by the phasic com-ponent of the accommodative controller, " Vis. Res., vol. 36, no. 1, pp. 97-102, Jan. 1996.
38. "Subjective assessment of stereoscopic television pictures, " ITU-R, Tech. Rep. BT.1438.
39. "Methodology for the subjective assessment of the quality of television pictures, " ITU-R, Tech. Rep. BT.500-7.
40. R. T. Held, E. A. Cooper, and M. S. Banks, "Blur and disparity are complementary cues to depth, " Current Biol., vol. 22, pp. 426-431, Mar. 2012.
41. D. M. Hoffman and M. S. Banks, "Focus information is used to interpret binocular images, " J. Vis., vol. 10, no. 5, May 2010.
42. T. Shibata, J. Kim, D. M. Hoffman, and M. S. Banks, "The zone of comfort: Predicting visual discomfort with stereo displays, " J. Vis., vol. 11, no. 8, Jul. 2011.
43. H.-L. Liou and N. A. Brennan, "Anatomically accurate, finite model eye for optical modeling, " J. Opt. Soc. Amer. A, vol. 14, no. 8, pp. 1684-1695, 1997.
44. J. Wang, M. Barkowsky, V. Ricordel, and P. Le Callet, "Quantifying how the combination of blur and disparity affects the perceived depth, " Proc. SPIE, vol. 7865, pp. 78650K-1-78650K-10, Feb. 2011.
45. I. P. Howard, Seeing in Depth: Basic Mechanisms, vol. 1. Toronto, ON, Canada: Porteous Publishing, 2002.
46. T. Ohshima, H. Yamamoto, and H. Tamura, "Gaze-directed adaptive rendering for interacting with virtual space, " in Proc. IEEE Virtual Reality Annu. Int. Symp. (VRAIS), Santa Clara, CA, USA, Mar./Apr. 1996, pp. 103-110 and 267.
47. J. Li, M. Barkowsky, and P. Le Callet, "The influence of relative disparity and planar motion velocity on visual discomfort of stereoscopic videos, " in Proc. 3rd Int. Workshop Quality Multimedia Exper., Sep. 2011, pp. 155-160.
48. J. Park, K. Seshadrinathan, S. Lee, and A. C. Bovik, "VQPooling: Video quality pooling adaptive to perceptual distortion severity, " IEEE Trans. Image Process., vol. 22, no. 2, pp. 610-620, Feb. 2013.
49. T. Fukushima, M. Torii, K. Ukai, J. S. Wolffsohn, and B. Gilmartin, "The relationship between CA/C ratio and individual differences in dynamic accommodative responses while viewing stereoscopic images, " J. Vis., vol. 9, no. 13, pp. 1-13, Dec. 2009.
50. A. C. Bovik, "Automatic prediction of perceptual image and video quality, " Proc. IEEE, vol. 101, no. 9, pp. 2008-2024, Sep. 2013.
51. IEEE Standards Association Stereoscopic (3D Imaging) Database. [Online] Available: http://grouper.ieee.org/group/3dhf
52. C.-C. Chang and C.-J. Lin. (2001). LIBSVM: A Library for Sup-port Vector Machines. [Online]. Available: http://www.csie.ntu.edu.tw/cjlin/libsvm/
53. K. Seshadrinathan and A. C. Bovik, "Motion tuned spatio-temporal quality assessment of natural videos, " IEEE Trans. Image Process., vol. 19, no. 2, pp. 335-350, Feb. 2010.
54. A. K. Moorthy and A. C. Bovik, "Visual importance pooling for image quality assessment, " IEEE J. Sel. Topics Signal Process., vol. 3, no. 2, pp. 193-201, Apr. 2009.
55. A. C. Bovik, The Essential Guide to Video Processing. San Francisco, CA, USA: Academic, 2009.
56. T. Kim, J. Kang, S. Lee, and A. C. Bovik, "Multimodal interactive continuous scoring of subjective 3D video quality of experience, " IEEE Trans. Multimedia, vol. 16, no. 2, pp. 387-402, Feb. 2014.
57. H. Kim, S. Lee, and A. C. Bovik, "Saliency prediction on stereoscopic videos, " IEEE Trans. Image Process., vol. 23, no. 4, pp. 1476-1490, Apr. 2014.
58. K. Lee, A. K. Moorthy, S. Lee, and A. C. Bovik, "3D visual activity assessment based on natural scene statistics, " IEEE Trans. Image Process., vol. 23, no. 1, pp. 450-465, Jan. 2014.
59. K. Lee and S. Lee, "3D perception based quality pooling: Stereopsis, binocular rivalry, and binocular suppression, " IEEE J. Sel. Topics Signal Process., vol. 9, no. 3, pp. 533-545, Apr. 2015.
60. M. Tanimoto, T. Fujii, K. Suzuki, N. Fukushima, and Y. Mori, "Reference softwares for depth estimation and view synthesis, " in Proc. IEL, Apr. 2008, pp. 1-8.
61. S.-P. Du, B. Masia, S.-M. Hu, and D. Gutierrez, "A metric of visual comfort for stereoscopic motion, " ACM Trans. Graph., vol. 32, no. 6, Nov. 2013.
62. Y Bi and J. Zhou, "A visual comfort metric for stereoscopic 3D video based on SMDE approach, " in Proc. Int. Conf. Signal Process., Commun. Comput. (ISPCC), Hong Kong, Aug. 2012, pp. 72-77.
63. K. Seshadrinathan and A. C. Bovik, "A structural similarity metric for video based on motion models, " in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Honolulu, HI, USA, Apr. 2007, pp. I-869-I-872.
64. M. A. Saad, A. C. Bovik, and C. Charrier, "Blind prediction of natural video quality, " IEEE Trans. Image Process., vol. 23, no. 3, pp. 1352-1365, Mar. 2014.
65. A. K. Moorthy, L. K. Choi, A. C. Bovik, and G. de Veciana, "Video quality assessment on mobile devices: Subjective, behavioral and objective studies, " IEEE J. Sel. Topics Signal Process., vol. 6, no. 6, pp. 652-671, Oct. 2012.