3D visual discomfort predictor: Analysis of horizontal disparity and neural activity statistics

IEEE Transactions on Image Processing

Volumn 24, Issue 3, 2015, Pages 1101-1114

  Park, Jincheol ^a   Oh, Heeseok ^a   Lee, Sanghoon ^a   Bovik, Alan Conrad ^b  

^a Yonsei University   (South Korea)

^b The University of Texas at Austin   (United States)

Author keywords

accommodation vergence conflict; middle temporal neural activity; S3D; stereoscopic 3D viewing; vergence; visual discomfort assessment

Indexed keywords

BINOCULAR VISION; BINOCULARS; BRAIN; EYE MOVEMENTS; NEURONS; STEREO IMAGE PROCESSING;

HORIZONTAL DISPARITY; NEURAL ACTIVITY; PREDICTION ALGORITHMS; SENSORIMOTOR CONTROL; STATISTICAL FRAMEWORK; STEREOSCOPIC 3D; VERGENCES; VISUAL DISCOMFORT;

THREE DIMENSIONAL DISPLAYS;

ADULT; ADVERSE EFFECTS; BIOLOGICAL MODEL; BRAIN; HUMAN; PATHOPHYSIOLOGY; PHOTOSTIMULATION; PHYSIOLOGY; THREE DIMENSIONAL IMAGING; VISION; VISUAL DISORDER; YOUNG ADULT;

ADULT; BRAIN; HUMANS; IMAGING, THREE-DIMENSIONAL; MODELS, NEUROLOGICAL; PHOTIC STIMULATION; VISION DISORDERS; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84923167035     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2014.2383327     Document Type: Article

References (78)

1. T. W. Dillon and H. H. Emurian, "Some factors affecting reports of visual fatigue resulting from use of a VDU," Comput. Human Behaviour, Vol. 12, No. 1, pp. 49-59, 1996.
2. M. Emoto, T. Niida, and F. Okana, "Repeated vergence adaptation causes the decline of visual functions in watching stereoscopic television," J. Display Technol., Vol. 1, No. 2, pp. 328-340, 2005.
3. J. S. Cooper, C. R. Burns, S. A. Cotter, K. M. Daum, J. M. Griffin, and M. M. Scheiman, "Optometric clinical practice guideline care of the patient with accommodative and vergence dysfunction," Amer. Optometric Assoc., St. Louis, MO, USA, Tech. Rep., 2001.
4. D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, "Vergence-accommodation conflicts hinder visual performance and cause visual fatigue," J. Vis., Vol. 8, No. 3, pp. 1-30, Mar. 2008.
5. T. Fukushima, M. Torii, K. Ukai, J. S. Wolffsohn, and B. Gilmartin, "The relationship between CA/C ratio and individual differences in dynamic accommodative responses while viewing stereoscopic images," J. Vis., Vol. 9, No. 13, pp. 1-13, Dec. 2008.
6. T. Shibata, J. Kim, D. M. Hoffman, and M. S. Banks, "The zone of comfort: Predicting visual discomfort with stereo displays," J. Vis., Vol. 11, No. 8, Jul. 2011, Art. ID 11.
7. L. M. J. Meesters, W. A. IJsselsteijn, and P. J. H. Seuntiens, "A survey of perceptual evaluations and requirements of three-dimensional TV," IEEE Trans. Circuits Syst. Video Technol., Vol. 14, No. 3, pp. 381-391, Mar. 2004.
8. F. L. Kooi and A. Toet, "Visual comfort of binocular and 3D displays," Displays, Vol. 25, nos. 2-3, pp. 99-108, 2004.
9. C. W. Tyler, L. T. Likova, K. Atanassov, V. Ramachandra, and S. Goma, "3D discomfort from vertical and torsional disparities in natural images," Proc. SPIE, Vol. 8291, pp. 82910Q-1-82910Q-9, Feb. 2012.
10. F. Liu, Y. Niu, and H. Jin, "Keystone correction for stereoscopic cinematography," in Proc. IEEE Workshop 3D Cinematograph., Jun. 2012, pp. 1-7.
11. Y. Jung, H. Sohn, S.-I. Lee, and Y. Ro, "Visual comfort improvement in stereoscopic 3D displays using perceptually plausible assessment metric of visual comfort," IEEE Trans. Consum. Electron., Vol. 60, No. 1, pp. 1-9, Apr. 2014.
12. M. Lambooij, W. IJsselsteijn, M. Fortuin, and I. Heynderickx, "Visual discomfort and visual fatigue of stereoscopic displays: A review," J. Imag. Sci. Technol., Vol. 53, No. 3, pp. 30201-1-30201-14, May 2009.
13. T. Bando, A. Iijima, and S. Yano, "Visual fatigue caused by stereoscopic images and the search for the requirement to prevent them: A review," Displays, Vol. 33, No. 2, pp. 76-83, Apr. 2012.
14. M. A. Saad, A. C. Bovik, and C. Charrier, "Blind image quality assessment: A natural scene statistics approach in the DCT domain," IEEE Trans. Image Process., Vol. 21, No. 8, pp. 3339-3352, Aug. 2012.
15. A. C. Bovik, "Automatic prediction of perceptual image and video quality," Proc. IEEE, Vol. 101, No. 9, pp. 2008-2024, Sep. 2013.
16. S. Ide, H. Yamanoue, M. Okui, and F. Okano, M. Bitou, and N. Terashima, "Parallax distribution for ease of viewing in stereoscopic HDTV," Proc. SPIE, Vol. 4660, pp. 38-45, May 2002.
17. J. Park, K. Seshadrinathan, S. Lee, and A. C. Bovik, "Video quality pooling adaptive to perceptual distortion severity," IEEE Trans. Image Process., Vol. 22, No. 2, pp. 610-620, Feb. 2013.
18. A. K. Moorthy and A. C. Bovik, "Visual importance pooling for image quality assessment," IEEE J. Sel. Topics Signal Process., Vol. 3, No. 2, pp. 193-201, Apr. 2009.
19. Y. Nojiri, H. Yamanoue, S. Ide, S. Yano, and F. Okana, "Parallax distribution and visual comfort on stereoscopic HDTV," in Proc. IBC, 2006, pp. 373-380.
20. Y. Nojiri, H. Yamanoue, A. Hanazato, and F. Okano, "Measurement of parallax distribution and its application to the analysis of visual comfort for stereoscopic HDTV," Proc. SPIE, Vol. 5006, pp. 195-205, May 2003.
21. J. Choi, D. Kim, S. Choi, and K. Sohn, "Visual fatigue modeling and analysis for stereoscopic video," Opt. Eng., Vol. 51, No. 1, pp. 017206-1-017206-11, Jan. 2010.
22. D. Kim and K. Sohn, "Visual fatigue prediction for stereoscopic image," IEEE Trans. Circuits Syst. Video Technol., Vol. 21, No. 2, pp. 231-236, Feb. 2011.
23. M. Wopking, "Viewing comfort with stereoscopic pictures: An experimental study on the subjective effects of disparity magnitude and depth of focus," J. Soc. Inf. Display, Vol. 3, No. 3, pp. 101-103, Dec. 1995.
24. Y. Nojiri, H. Yamanoue, A. Hanazato, M. Emoto, and F. Okano, "Visual comfort/discomfort and visual fatigue caused by stereoscopic HDTV viewing," Proc. SPIE, Vol. 5291, pp. 303-313, Jan. 2004.
25. Y.-Y. Yeh and L. D. Silverstein, "Limits of fusion and depth judgment in stereoscopic color displays," Human Factors, Vol. 32, No. 1, pp. 45-60, Feb. 1990.
26. S. Yano, S. Ide, T. Mitsuhashi, and H. Thwaites, "A study of visual fatigue and visual comfort for 3D HDTV/HDTV images," Displays, Vol. 23, No. 4, pp. 191-201, Jun. 2002.
27. S. Yano, M. Emoto, and T. Mitsuhashi, "Two factors in visual fatigue caused by stereoscopic HDTV images," Displays, Vol. 25, No. 4, pp. 141-150, Oct. 2004.
28. M. Emoto, Y. Nojiri, and F. Okano, "Changes in fusional vergence limit and its hysteresis after viewing stereoscopic TV," Displays, Vol. 25, nos. 2-3, pp. 67-76, Aug. 2004.
29. M.-J. Chen, D.-K. Kwon, L. K. Cormack, and A. C. Bovik, "Optimizing 3D image display using the stereoacuity function," in Proc. IEEE Int. Conf. Image Process., Orlando, FL, USA, Sep. 2012, pp. 617-620.
30. F. Speranza and L. M. Wilcox, "Viewing stereoscopic images comfortably: The effects of whole-field vertical disparity," Proc. SPIE, Vol. 4660, pp. 18-25, May 2002.
31. W. A. IJsselsteijn, H. de Ridder, and J. Vliegen, "Subjective evaluation of stereoscopic images: Effects of camera parameters and display duration," IEEE Trans. Circuits Syst. Video Technol., Vol. 10, No. 2, pp. 225-233, Mar. 2000.
32. A. J. Woods, T. Docherty, and R. Koch, "Image distortions in stereoscopic video systems," Proc. SPIE, Vol. 1915, pp. 36-48, Sep. 1993.
33. B. T. Backus, D. J. Fleet, A. J. Parker, and D. J. Heeger, "Human cortical activity correlates with stereoscopic depth perception," J. Neurophysiol., Vol. 86, No. 4, pp. 2054-2068, Oct. 2001.
34. P. Neri, "A stereoscopic look at visual cortex," J. Neurophysiol., Vol. 93, No. 4, pp. 1823-1826, 2005.
35. G. C. DeAngelis and T. Uka, "Coding of horizontal disparity and velocity by MT neurons in the alert macaque," J. Neurophysiol., Vol. 89, No. 2, pp. 1094-1111, 2003.
36. A. W. Roe, A. J. Parker, R. T. Born, and G. C. DeAngelis, "Disparity channels in early vision," J. Neurosci., Vol. 27, No. 44, pp. 11820-11831, Oct. 2007.
37. L. R. Squire, Ed., Fundamental Neuroscience. San Diego, CA, USA: Academic, 1999.
38. L. M. Martinez and J.-M. Alonso, "Complex receptive fields in primary visual cortex," Neuroscientist, Vol. 9, No. 5, pp. 317-331, Oct. 2003.
39. J. Read, "Early computational processing in binocular vision and depth perception," Progr. Biophys. Molecular Biol., Vol. 87, No. 1, pp. 77-108, 2005.
40. B. G. Cumming and G. C. DeAngelis, "The physiology of stereopsis," Annu. Rev. Neurosci., Vol. 24, No. 1, pp. 203-238, 2001.
41. G. C. DeAngelis, B. G. Cumming, and W. T. Newsome, "Cortical area MT and the perception of stereoscopic depth," Nature, Vol. 394, No. 6694, pp. 677-680, Aug. 1998.
42. L. G. Ungerleider and M. Mishkin, Analysis of Visual Behavior. Cambridge, MA, USA: MIT Press, 1982.
43. P. Janssen, R. Vogels, and G. A. Orban, "Three-dimensional shape coding in inferior temporal cortex," Neuron, Vol. 27, No. 2, pp. 385-397, Aug. 2000.
44. K. Seshadrinathan and A. C. Bovik, "Motion tuned spatio-temporal quality assessment of natural videos," IEEE Trans. Image Process., Vol. 19, No. 2, pp. 335-350, Feb. 2010.
45. J. D. Nguyenkim and G. C. DeAngelis, "Disparity-based coding of three-dimensional surface orientation by macaque middle temporal neurons," J. Neurosci., Vol. 23, No. 18, pp. 7117-7128, Aug. 2003.
46. R. T. Born and D. C. Bradley, "Structure and function of visual area MT," Annu. Rev. Neurosci., Vol. 28, pp. 157-189, Mar. 2005.
47. Y. Liu, A. C. Bovik, and L. K. Cormack, "Disparity statistics in natural scenes," J. Vis., Vol. 8, No. 11, Aug. 2008, Art. ID 19.
48. J. P. Roy, H. Komatsu, and R. H. Wurtz, "Disparity sensitivity of neurons in monkey extrastriate area MST," J. Neurosci., Vol. 12, No. 7, pp. 2478-2492, 1992.
49. A. Takemura, Y. Inoue, K. Kawano, C. Quaia, and F. A. Miles, "Singleunit activity in cortical area MST associated with disparity-vergence eye movements: Evidence for population coding," J. Neurophysiol., Vol. 85, No. 5, pp. 2245-2266, 2001.
50. T. Akao, M. J. Mustari, J. Fukushima, S. Kurkin, and K. Fukushima, "Discharge characteristics of pursuit neurons in MST during vergence eye movements," J. Neurophysiol., Vol. 93, No. 5, pp. 2415-2434, May 2005.
51. A. Wong, Eye Movement Disorders. London, U.K.: Oxford Univ. Press, 2007.
52. P. D. R. Gamlin, "Neural mechanisms for the control of vergence eye movements," Ann. New York Acad. Sci., Vol. 956, No. 1, pp. 264-272, Apr. 2002.
53. U. Buttner and J. A. Buttner-Ennever, "Present concepts of oculomotor organization," Progr. Brain Res., Vol. 151, pp. 1-42, 2006.
54. U. Schwarz, "Neuroophthalmology: A brief vademecum," Eur. J. Radiol., Vol. 49, No. 1, pp. 31-63, 2004.
55. J. Park, H. Oh, and S. Lee. (2012). IEEE-SA Stereo Image Database. [Online]. Available: http://grouper.ieee.org/groups/3dhf/
56. P. D. R. Gamlin, "Subcortical neural circuits for ocular accommodation and vergence in primates," Ophthalmic Physiol. Opt., Vol. 19, No. 2, pp. 81-89, 1999.
57. E. R. Kandel, J. H. Schwartz, and T. M. Jessel, Eds., Principles of Neural Science. New York, NY, USA: Elsevier, 1990.
58. R. S. Zemel, P. Dayan, and A. Pouget, "Probabilistic interpretation of population codes," Neural Comput., Vol. 10, No. 2, pp. 403-430, 1998.
59. T. D. Sanger, "Neural population codes," Current Opinion Neurobiol. Vol. 13, No. 2, pp. 238-249, 2003.
60. A. C. Bovik, M. Clark, and W. S. Geisler, "Multichannel texture analysis using localized spatial filters," IEEE Trans. Pattern Anal. Mach. Intell., Vol. 12, No. 1, pp. 55-73, Jan. 1990.
61. M. Clark and A. C. Bovik, "Experiments in segmenting texton patterns using localized spatial filters," Pattern Recognit., Vol. 22, No. 6, pp. 707-717, 1989.
62. S. Wu, S. Amari, and H. Nakahara, "Population coding and decoding in a neural field: A computational study," Neural Comput., Vol. 14, No. 5, pp. 999-1026, 2002.
63. L. Paninski, J. Pillow, and J. Lewi, "Statistical models for neural encoding, decoding, and optimal stimulus design," Progr. Brain Res., Vol. 165, pp. 493-507, Aug. 2007.
64. L. Goldmann, F. De Simone, and T. Ebrahimi, "Impact of acquisition distortion on the quality of stereoscopic images," in Proc. Int. Workshop Video Process. Quality Metrics Consum. Electron. (VPQM), 2010.
65. F. Zilly, J. Kluger, and P. Kauff, "Production rules for stereo acquisition," Proc. IEEE, Vol. 99, No. 4, pp. 590-606, Apr. 2011.
66. D. Sun, S. Roth, and M. J. Black, "Secrets of optical flow estimation and their principles," in Proc. IEEE Comput. Vis. Pattern Recognit. (CVPR), Jun. 2010, pp. 2432-2439.
67. D. Sun, S. Roth, and M. Black. (2010). Optical Flow Software. [Online]. Available: http://www.cs.brown.edu/~black/
68. D. Scharstein and R. Szeliski, "A taxonomy and evaluation of dense twoframe stereo correspondence algorithms," Int. J. Comput. Vis., Vol. 47, No. 1, pp. 7-42, Apr. 2002.
69. D. Scharstein and R. Szeliski. Middlebury Stereo Evaluation-Version 2. [Online]. Available: http://vision.middlebury.edu/stereo/
70. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge, U.K.: Cambridge Univ. Press, 2000.
71. "Methodology for the subjective assessment of the quality of television pictures," ITU-R, Geneva, Switzerland, Tech. Rep. BT.500-11, 2002.
72. "Subjective assessment of stereoscopic television pictures," ITU-R, Geneva, Switzerland, Tech. Rep. BT.1438, 2000.
73. B. Scholkopf, A. J. Smola, R. C. Williamson, and P. L. Bartlett, "New support vector algorithms," Neural Comput., Vol. 12, No. 5, pp. 1207-1245, 2000.
74. C. J. C. Burges, "A tutorial on support vector machines for pattern recognition," Data Mining Knowl. Discovery, Vol. 2, No. 2, pp. 121-167, 1998.
75. C. Chang and C. Lin. (2001). LIBSVM: A Library for Support Vector Machines. [Online]. Available: http://www.csie.ntu.edu.tw/cjlin/libsvm/
76. Y. Liu, L. K. Cormack, and A. C. Bovik, "Statistical modeling of 3-D natural scenes with application to Bayesian stereopsis," IEEE Trans. Image Process., Vol. 20, No. 9, pp. 2515-2530, Sep. 2011.
77. C.-C. Su, L. K. Cormack, and A. C. Bovik, "Color and depth priors in natural images," IEEE Trans. Image Process., Vol. 22, No. 6, pp. 2259-2274, Jun. 2013.
78. K. Umeda, S. Tanabe, and I. Fujida, "Representation of stereoscopic depth based on relative disparity in macaque area V4," J. Neurophysiol., Vol. 98, No. 1, pp. 241-252, 2007.