3D Media and the Human Visual System

Emerging Technologies for 3D Video: Creation, Coding, Transmission and Rendering

Volumn , Issue , 2013, Pages 349-376

  Watt, Simon J ^a   MacKenzie, Kevin J ^a  

^a BANGOR UNIVERSITY   (United Kingdom)

Author keywords

Binocular vision; Human factors; Human vision; Stereo 3D (S3D); Stereoscopic displays; User issues

Indexed keywords

EID: 84886853984     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9781118583593.ch18     Document Type: Chapter

References (114)

1. Akeley, K., Watt, S. J., Girshick, A. R. and Banks, M. S. (2004) A stereo display prototype with multiple focal distances. ACM Trans. Graphic., 23, 1084-1813.
2. Alais, D. and Burr, D. (2004) The ventriloquist effect results from near-optimal cross-modal integration, Curr. Biol., 14, 257-262.
3. Allison, R. S. (2007) An analysis of the influence of vertical disparities arising in toed-in stereoscopic cameras. Imaging Sci. Technol., 51, 317-327.
4. Alpern, M. (1958) Variability of accommodation during steady state fixation at various levels of illuminance. J. Opt. Soc. Am., 48, 193-197.
5. Ardito, M. (1994) Studies of the influence of display size and picture brightness on the preferred viewing distance for HDTV programs. SMPTE J., 103, 517-522.
6. ATSC (2011) Advanced Television Systems Committee (ATSC) report on 3D digital television, ATSC Planning Team 1 Interim Report, http://www.atsc.org/PT1/PT-1-Interim-Report.pdf.
7. Backus, B. T., Banks, M. S., van Ee, R. and Crowell, J. A. (1999) Horizontal and vertical disparity, eye position, and stereoscopic slant perception. Vision Res., 39, 1143-1170.
8. Banks, M. S., Gepshtein, S. and Landy, M. S. (2004) Why is spatial stereoresolution so low? J. Neurosci., 24, 2077-2089.
9. Banks, M. S., Held, R. T. and Girshick, A. R. (2009) Perception of 3-D layout in stereo displays. Inform. Disp., 25, 12-16.
10. Banks, M. S., Read, J. R., Allison, R. S. and Watt, S. J. (2012) Stereoscopy and the human visual system. SMPTE Motion Imag. J., 121 (4), 24-43.
11. Benzeroual, K., Wilcox, L. M., Kazimi, A. and Allison, R. S. (2011) On the distinction between perceived & predicted depth in S3D films. Proceedings of the International Conference on 3D Imaging.
12. Benzeroual, K., Allison, R., and Wilcox, L. M. (2012) 3D display matters: compensating for the perceptual effects of S3D display scaling, in 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE, pp. 45-52.
13. Blake, A. and Bülthoff, H. H. (1990) Does the brain know the physics of specular reflections? Nature, 343, 165-168.
14. Blakemore, C. (1970) Range and scope of binocular depth discrimination in man. J. Physiol., 211, 599-622.
15. Bresciani, J. P., Dammeier, F. and Ernst, M. O. (2006) Vision and touch are automatically integrated for the perception of sequences of events. J. Vision, 6 (5), 554-564.
16. Burian, H. M. (1943) Influence of prolonged wearing of meridional size lenses on spatial localization. Arch. Ophthamol., 30, 645-668.
17. Burr, D. C. and Ross, J. (1979) How does binocular delay give information about depth. Vision Res., 19, 523-532.
18. Burt, P. and Julesz, B. (1980) A disparity gradient limit for binocular fusion. Science, 208, 615-617.
19. Campbell, F. W. (1957) The depth of field of the human eye. Opt. Acta, 4, 157-164.
20. Charman, W. N. and Whitefoot, H. (1977) Pupil diameter and the depth-of-field of the human eye as measured by laser speckle. Opt. Acta, 24, 1211-1216.
21. Clark, J. J. and Yuille, A. L. (1990) Data Fusion for Sensory Information Processing Systems, Kluwer, Boston, MA.
22. Cormack, L. K., Stevenson, S. B. and Schor, C. M. (1991) Interocular correlation, luminance contrast and cyclopean processing. Vision Res., 31, 2195-2207.
23. Devernay, F. and Beardsley, P. (2010) Stereoscopic cinema, in Image and Geometry Processing for 3-D Cinematography (eds R. Ronfard and G. Taubin), Geometry and Computing, Vol. 5, Springer, Berlin, pp. 11-51.
24. Dodgson, N. (2004) Variation and extrema of human interpupillary distance, in Stereoscopic Displays and Virtual Reality Systems XI (eds A. J.Woods, J. O. Merritt, S. A. Benton and M. T. Bolas), Proceedings of SPIE, Vol. 5291, SPIE, Bellingham,WA, pp. 36-46.
25. Emoto, M., Niida, R., and Okano, F. (2005) Repeated vergence adaptation causes the decline of visual functions in watching stereoscopic television. J. Disp. Technol., 1, 328-340.
26. Ernst, M. O. (2006) A Bayesian view on multimodal cue integration, in Human Body Perception From The Inside Out (eds G. Knoblich, I.M. Thornton, M. Grosjean and M. Shiffrar), Oxford University Press, New York, NY.
27. Ernst, M. O. (2007) Learning to integrate arbitrary signals from vision and touch. J. Vision, 7 (7), 1-14.
28. Ernst, M. O. and Banks, M. S. (2002) Humans integrate visual and haptic information in a statistically optimal fashion. Nature, 415, 429-433.
29. Fender, D. H. (1964) Control mechanisms of the eye. Sci. Am., 211, 24-33.
30. Fincham, E. F. and Walton, J. (1957) The reciprocal actions of accommodation and convergence. J. Physiol., 137, 488-508.
31. French, J.W. (1921) The interocular distance. Trans. Opt. Soc., 23, 44-55.
32. Fry, G. A. (1939) Further experiments on the accommodation-convergence relationship. Am. J. Optom. Arch. Am. Acad. Optom., 16, 325-336.
33. Gârding, J., Porrill, J., Mayhew, J. E. W. and Frisby, J. P. (1995) Stereopsis, vertical disparity and relief transformations. Vision Res., 35, 703-722.
34. Gardner, B. R. (2012) Dynamic floating window: new creative tool for three-dimensional movies. J. Electron. Imag., 21 (1), 011009.
35. Gepshtein, S., Burge, J., Ernst, M. and Banks, M. S. (2005) The combination of vision and touch depends on spatial proximity. J. Vision, 5, 1013-1023.
36. Girschick, A. R. and Banks, M. S. (2009) Probabilistic combination of slant information: weighted averaging and robustness as optimal percepts. J. Vision, 9 (9), 1-20.
37. Greenwald, H. S. and Knill, D. C. (2009) Cue integration outside central fixation: a study of grasping in depth. J. Vision, 9 (2), 1-16.
38. Häkkinen, J., Pölönen, M., Takatalo, J. and Nyman, G. (2006) Simulator sickness in virtual gaming: a comparison of stereoscopic and non-stereoscopic situations, in MobileHCI '06 Proceedings of the 8th Conference on Human- Computer Interaction with Mobile Devices and Services, ACM, New York, NY, pp. 227-230.
39. Heath, G. G. (1956) Components of accommodation. Am. J. Optom., 33, 569-579.
40. Held, R.T., Cooper, E., O'Brien, J. and Banks, M.S. (2010) Using blur to affect perceived distance and size. ACM Trans. Graphic., 29 (2), article 19.
41. Held, R. T., Cooper, E. A. and Banks, M. S. (2012) Blur and disparity are complementary cues to depth. Curr. Biol., 22, 1-6.
42. Hillis, J. M., Watt, S. J., Landy, M. S. and Banks, M. S. (2004) Slant from texture and disparity cue: optimal cue combination. J. Vision, 4, 967-992.
43. Hoffman, D. M., Girshick, A. R., Akeley, K. and Banks, M. S. (2008) Vergence-accommodation conflicts hinder visual performance and cause visual fatigue. J. Vision, 8 (3), 1-30.
44. Hoffman, D. M., Karasev, V. I. and Banks, M. S. (2011) Temporal presentation protocols in stereoscopic displays: flicker visibility, perceived motion, and perceived depth. J. Soc. Inf. Disp., 19 (3), 255-281.
45. Howard, I. P. and Rogers, B. J. (2002) Seeing in Depth, University of Toronto Press, Toronto, ON.
46. Julesz, B. (1960) Binocular depth perception of computer-generated patterns. Bell Labs Tech. J., 39, 1125-1162.
47. Kane, D., Held, R. T. and Banks, M. S. (2012) Visual discomfort with stereo 3D displays when the head is not upright, in Stereoscopic Displays and Applications XXIII (eds A. J. Woods, N. S. Holliman and G. E. Favalora), Proceedings of the SPIE, Vol. 8288, SPIE, Bellingham, WA, p. 828814.
48. Keefe, B. D., Hibbard, P. B. and Watt, S. J. (2011) Depth-cue integration in grasp programming: no evidence for a binocular specialism. Neuropsychogia, 49, 1249-1257.
49. Kim, J., Kane, D. and Banks, M. S. (2012) Visual discomfort and the temporal properties of the vergence-accommodation conflict, in Stereoscopic Displays and Applications XXIII (eds A. J. Woods, N. S. Holliman and G. E. Favalora), Proceedings of the SPIE, Vol. 8288, SPIE, Bellingham,WA, p. 828811.
50. Knill, D. C. (1998) Surface orientation from texture: ideal observers, generic observers and the information content of texture cues. Vision Res., 38, 1655-1682.
51. Knill, D. C. (2007) Robust cue integration: a Bayesian model and evidence from cue-conflict studies with stereoscopic and figure cues to slant. J. Vision, 7 (7), 1-24.
52. Knill, D. C. and Pouget, A. (2004) The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci., 27, 712-719.
53. Knill, D. C. and Saunders, J. (2003) Do humans optimally integrate stereo and texture information for judgments of surface slant? Vision Res., 43, 2539-2558.
54. Kooi, F. L. and Toet, A. (2004) Visual comfort of binocular 3D displays. Displays, 25, 99-108.
55. Körding, K. P., Beierhold, U., Ma, W. J. et al. (2007) Causal inference in multisensory perception. PLoS ONE, 2, e943.
56. Kotulak, J. C. and Schor, C.M. (1986) A computational model of the error detector of human visual accommodation. Biol. Cybern., 54, 189-194.
57. Kulikowski, J. J. (1978) Limit of single vision depends on contour sharpness. Nature, 276, 126-127.
58. Lambooij, M., Ijsselsteijn,W., Fortuin, M. and Heynderickx, I. (2009) Visual discomfort and visual fatigue of stereoscopic displays: a review. J. Imaging Sci. Technol., 53 (3), 1-14.
59. Landis, C. (1954) Determinants of the critical flicker-fusion threshold. Physiol. Rev., 34, 259-286.
60. Landy, M. S., Maloney, L. T., Johnston, E. B. and Young, M. (1995) Measurement and modeling of depth cue combination: in defense of weak fusion. Vision Res., 35, 389-412.
61. Legge, G. E. and Gu, Y. (1989) Stereopsis and contrast. Vision Res., 29, 989-1004.
62. Liu, S. and Hua, H. (2009) Time-multiplexed dual-focal plane head-mounted display with a liquid lens. Opt. Lett., 34 (11), 1642-1644.
63. Longuet-Higgins, H. C. (1981) A computer algorithm for reconstructing a scene from two projections. Nature, 293, 133-135.
64. Love, G. D., Hoffman, D. M., Hands, P. J. W. et al. (2009) High-speed switchable lens enables the development of a volumetric stereoscopic display. Opt. Express, 17, 15716-15725.
65. MacKenzie, K. J., Hoffman, D. M. and Watt, S. J. (2010) Accommodation to multiple-focal-plane displays: implications for improving stereoscopic displays and for accommodation control. J. Vision, 10 (8), 1-20.
66. MacKenzie, K. J., Dickson, R. A. and Watt, S. J. (2012) Vergence and accommodation to multiple-image-plane stereoscopic displays: "real world" responses with practical image-plane separations? J. Electron. Imaging, 21 (1), 011002.
67. Manny, R. E. and Banks, M. S. (1984) A model of steady-state accommodation: II. Effects of luminance. Invest. Ophthalmol. Vis. Sci., 25, 182.
68. Martens, T. G. and Ogle, K. N. (1959) Observations on accommodative convergence; especially its nonlinear relationships. Am. J. Otolaryngol., 47, 455-462.
69. Mendiburu, B. (2009) 3D Movie Making: Stereoscopic Digital Cinema from Script to Screen, Focal Press, Burlington, MA.
70. Miles, F. A., Judge, S. J. and Optican, L. M. (1987) Optically induced changes in the couplings between vergence and accommodation. J. Neurosci., 7, 2576-2589.
71. Miles, H. C., Pop, S. R.,Watt, S. J. et al. (2012) A review of virtual environments for training in ball sports. Comput. Graph., 36 (6), 714-726.
72. Mon-Williams, M., Treisilan, J. R. and Roberts, A. (2000) Vergence provides veridical depth perception from horizontal retinal image disparities. Exp. Brain Res., 133, 407-413.
73. Morgan, M. J. (1979) Perception of continuity in stroboscopic motion: a temporal frequency analysis. Vision Res., 19, 523-532.
74. Nienborg, H., Bridge, H., Parker, A. J. and Cumming, B. G. (2004) Receptive field size in V1 neurons limits acuity for perceiving disparity modulation. J. Neurosci., 24, 2065-2076.
75. Ogle, K. N. (1932) An analytical treatment of the longtitudnal horopter, its measurement and application and application to related phenomena, especially to the relative size and shape of the ocular images. J. Opt. Soc. Am., 22, 665-728.
76. Ogle, K. N. (1938) Induced size effect. I. A new phenomenon in binocular space perception associated with the relative sizes of the images of the two eyes. Arch. Ophthalmol., 20, 604-623.
77. O'Leary, A. and Wallach, H. (1980) Familiar size and linear perspective as distance cues in stereoscopic depth constancy. Percept. Psychophys., 27, 131-135.
78. OruSc, I., Maloney, L. T. and Landy, M. S. (2003) Weighted linear cue combination with possibly error. Vision Res., 43, 2451-2468.
79. Owens, D. A. (1980) A comparison of accommodative responses and contrast sensitivity for sinusoidal gratings. Vision Res., 20, 159-167.
80. Peli, E. (1998) The visual effects of head-mounted display (HMD) are not distinguishable from those of desk-top computer display. Vision Res., 38, 2053-2066.
81. Percival, A. S. (1920) The Prescribing of Spectacles, J. Wright, Bristol.
82. Ravikumar, S., Akeley, K. and Banks, M. S. (2011) Creating effective focus cues in multi-plane 3D displays. Opt. Express, 19 (21), 20940-20952.
83. Roach, N.W., Heron, J. and McGraw, P. V. (2006) Resolving multisensory conflict: a strategy for balancing the costs and benefits of audio-visual integration. Proc. R. Soc. Lond. B Biol. Sci., 273, 2159-2168.
84. Rogers, B. J. and Bradshaw, M. F. (1993) Vertical disparities, differential perspective and binocular stereopsis. Nature, 361, 253-255.
85. Rogers, B. J. and Bradshaw, M. F. (1995) Disparity scaling and the perception of frontoparallel surfaces. Perception, 24, 155-179.
86. Rolland, J. P., Goon, A. A. and Krueger, M. W. (1999) Dynamic focusing in head-mounted displays, in Stereoscopic Displays and Virtual Reality Systems VI, (eds J. O. Merritt, M. T. Bolas and S. S. Fisher), Proceedings of the SPIE, Vol. 3639, SPIE, Bellingham,WA, pp. 463-470.
87. Rovamo, J. and Raminen, A. (1988) Critical flicker frequency as a function of stimulus area and luminance at various eccentricities in human cone vision: a revision of Granit-Harper and Ferry-Porter laws. Vision Res., 28, 785-790.
88. Rushton, S. K. and Riddell, P. M. (1999) Developing visual systems and exposure to virtual reality and stereo displays: some concerns and speculations about the demands on accommodation and vergence. Appl. Ergon., 30, 69-78.
89. Saladin, J. J. and Sheedy, J. E. (1978) A population study of relationships between fixation disparity, hetereophoria and vergences. Am. J. Optom. Phys. Opt., 55, 744-750.
90. Schor, C.M.,Wood, I. and Ogawa, J. (1984) Binocular sensory fusion is limited by spatial resolution. Vision Res., 24, 661-665.
91. Schor, C.M., Heckman, T. and Tyler, C.W. (1989) Binocular fusion limits are independent of target contrast, luminance gradient and component phase. Vision Res., 29, 821-835.
92. Sedgwick, H.A. (1986) Space perception, in Handbook of Perception and Human Performance, Vol. 1, Sensory Processes and Perception (eds K. R. Boff, L. Kaufman and J. P. Thomas), John Wiley & Sons, Inc., New York, NY.
93. Shams, L., Ma, W. J. and Beierholm, U. (2005) Sound-induced flash illusion as an optimal percept. Neuroreport, 16, 1923-1927.
94. Shibata, T., Kim, J., Hoffman, D. M. and Banks, M. S. (2011) The zone of comfort: predicting visual discomfort with stereo displays. J. Vision, 11 (8), 1-29.
95. Takahashi, C., Diedrichsen, J. and Watt, S. J. (2009) Integration of vision and haptics during tool use. J. Vision, 9, 1-15.
96. Toates, F. M. (1972) Accommodation function of the human eye. Physiol. Rev., 52, 828-863.
97. Tsirilin, I., Wilcox, L. M. and Allison, R. S. (2011) The effect of crosstalk on the perceived depth from disparity and monocular occlusions. IEEE Trans. Broadcast., 57, 445-453.
98. Tyler C.W. (1973) Stereoscopic vision: cortical limitations and a disparity scaling effect. Science, 181, 276-278.
99. Tyler, C. W. (1974) Depth perception in disparity gratings. Nature, 251, 140-142.
100. Tyler, C. W. and Julesz, B. (1978) Binocular cross-correlation in time and space. Vision Res., 18, 101-105.
101. Ukai, K. (2007) Visual fatigue caused by viewing stereoscopic images and mechanism of accommodation. Proceedings of the First International Symposium on Universal Communication, Kyoto, Vol. 1, 176-179.
102. Ukai, K. and Howarth, P. A. (2008) Visual fatigue caused by viewing stereoscopic motion images: background, theories and observations. Displays, 29, 106-116.
103. Vishwanath, D. and Blaser, E. (2010) Retinal blur and the perception of egocentric distance. J. Vision, 10 (10), 1-16.
104. Vishwanath, D., Girshick, A. and Banks, M. S. (2005) Why pictures look right when viewed from the wrong place. Nat. Neurosci., 8, 1401-1410.
105. Wann, J. P. and Mon-Williams, M. (1997) Health issues with virtual reality displays: what we do know and what we don't. ACM SIGGRAPH Comput. Graph., 31, 53-57.
106. Watt, S. J., Akeley, K., Ernst, M. O. and Banks, M. S. (2005a) Focus cues affect perceived depth. J. Vision, 5, 834-886.
107. Watt, S. J., Akeley, K., Girshick, A. R. and Banks, M. S. (2005b) Achieving near-correct focus cues in a 3-D display using multiple image planes, in Human Vision and Electronic Imaging X (eds B. E. Rogowitz, T. N. Pappas and S. J. Daly), Proceedings of SPIE, Vol. 5666, SPIE, Bellingham,WA, pp. 393-401.
108. Watt, S. J., MacKenzie, K. J. and Ryan, L. (2012) Real-world stereoscopic performance in multiple-focal-plane displays: how far apart should the image planes be? in Stereoscopic Displays and Applications XXIII (eds A. J. Woods, N. S. Holliman and G. E. Favalora), Proceedings of the SPIE, Vol. 8288, SPIE, Bellingham, WA, p. 82881E.
109. Wendt, G., Faul, F. and Mausfeld, R. (2008) Highlight disparity contributes to the authenticity and strength of perceived glossiness. J. Vision, 8 (1): 14, 1-10.
110. Woods, A. J., Docherty, T. and Koch, R. (1993) Image distortions in stereoscopic video systems, in Stereoscopic Displays and Applications IV, (eds J. O. Merritt and S. S. Fisher), Proceedings of the SPIE, Vol. 1915, SPIE, Bellingham,WA, p. 36-48.
111. Wöpking, M. (1995) Viewing comfort with stereoscopic pictures: an experimental study on the subjective effects of disparity magnitude and depth of focus. J. Soc. Inf. Disp., 3, 101-103.
112. Yang, S., Schlieski, T., Selmins, B. et al. (2011) Individual differences and seating positions affect immersion and symptoms in stereoscopic 3D viewing. Technical Report, Vision Performance Institute, Pacific University.
113. Yano, S., Emoto, M. and Mitsuhashi, T. (2004) Two factors in visual fatigue caused by stereoscopic HDTV imges. Displays, 25, 141-150.
114. Yuille, A. and Bülthoff, H. (1996) Bayesian decision theory and psychophysics, in Perception as Bayesian Inference (eds D. C. Knill and W. Richards), Cambridge University Press, Cambridge, UK.