Perceptual issues in stereoscopic signal processing

IEEE Transactions on Broadcasting

Volumn 57, Issue 2 PART 2, 2011, Pages 347-361

  Daly, Scott J ^a   Held, Robert T ^b   Hoffman, David M ^c  

^a DOLBY LABORATORIES   (United States)

^b Soda Hall   (United States)

^c MEDIATEK INC   (Taiwan)

Author keywords

Distortion; perceptual; stereoscopic

Indexed keywords

DEPTH RANGE; DISPLAY SURFACE; DISTORTION; MATCHING PARAMETERS; PERCEPTUAL; PERCEPTUAL DISTORTION; STEREOSCOPIC; STEREOSCOPIC CAMERA; TWO-DIMENSIONAL IMAGES; VIDEO IMAGERY; VIEWING CONDITIONS;

CAMERAS; DISPLAY DEVICES; GEOMETRICAL OPTICS; SIGNAL PROCESSING; TWO DIMENSIONAL;

THREE DIMENSIONAL;

EID: 79957596544     PISSN: 00189316     EISSN: None     Source Type: Journal    
DOI: 10.1109/TBC.2011.2127630     Document Type: Article

References (90)

1. D. Strohmeier, S. Jumisko-Pyykko, and K. Kunze, "New, lively and exciting or just artificial, straining, and distracting? A sensory profiling approach to understand mobile 3D audiovisual quality," VPQM, 2010.
2. P. Seuntiens, "Visual experiences of 3DTV," PhD Thesis, Eindhoven University of Technology and Philips Research, Eindhoven, 2006, Ch. 4.
3. W. J. Tam, "Psychovisual aspects of viewing stereoscopic video sequences," Proc. SPIE, vol. 3295, pp. 226-235, 1998. (Pubitemid 35004015)
4. W. J. Tam, F. Speranza, S. Yano, K. Shimono, and H. Ono, "3D-TV visual comfort," IEEE Trans Broadcast Eng., 2011.
5. N. S. Holliman, N. A. Dodgson, G. E. Favalora, and L. Pockett, "Threedimensional displays:Areviewand applications analysis," IEEE Trans. Broadcast., Jun. 2011.
6. R. Kaptein and I. Heynderickx, "Effect of crosstalk in multi-view autostereoscopic 3D displays on perceived image quality," in SID Digest, 2007, pp. 1220-1223. (Pubitemid 47266428)
7. L. Lipton, "High resolution immersion view," Proc. SPIE, vol. 2177, p. 132-, 1994.
8. "Planar-model SD2620W," [Online]. Available: http://www. planar3d.com/entry-level-3d/
9. C.-Y. Chen et al., "3D technology development and human factor," in SID 10 Digest, 2010, pp. 518-521.
10. X. Feng and S. Daly, "Vision-based strategy to reduce the perceived color misregistration of image capturing devices," Proc. IEEE: Special Issue on Applied Visual Models, vol. 90, no. 1, pp. 18-27, Jan. 2002. (Pubitemid 43779311)
11. Y. Yeh and L. D. Silverstein, "Limits of fusion and depth judgment in stereoscopic color displays," Human Factors, vol. 32, no. 1, pp. 45-60, 1990. (Pubitemid 20298907)
12. K. C. Huang et al., "A study of how crosstalk affects stereopsis in stereoscopic displays," SPIE, vol. 5006, pp. 247-253, 2003.
13. Pastoor, "Human factors of 3D images: Results of recent research," in Proc. IDW5, 1995, vol. 3D-7, pp. 69-72.
14. A. Hanazato, M. Okui, and I. Yuyama, "Subjective evaluation of cross talk disturbance in stereoscopic displays," in SID Digest, 2000, pp. 288-291. (Pubitemid 35452047)
15. F. Kooi and M. Lucassen, "Visual comfort of binocular and 3D displays," in HVEI VI SPIE Proc, 2001, p. 4299.
16. P. Seuntiens, L. Meesters, and W. Ijsselsteijn, "Perceptual attributes of crosstalk in 3D images," Displays, vol. V26, pp. 177-183, 2005. (Pubitemid 41455563)
17. P. J. B. Seuntiens et al., "Viewing experience and naturalness of 3D images," Proc. SPIE, vol. 5006, 2003.
18. L. Chen et al., "Investigation of crosstalk in a 2-view 3D display," in SID Digest, 2008, pp. 1138-1141.
19. C.-Y. Chiang, K.-T. Chen, Y.-C. Chang, and Y.-P. Huang, "Effect of crosstalk for stereoscopic 3D-Dynamic moving images," in SID Symp. Digest, 2009, vol. 40, no. 1, pp. 808-811.
20. S. Pala, R. Stevens, and P. Surman, "Optical crosstalk and visual comfort of stereoscopic display used in a real-time application," Proc. SPIE, vol. 6940, p. 11.1-11.1, 2007.
21. J. S. Lipscomb and W. L.Wooten, "Reducing crosstalk between stereoscopic views," SPIE, vol. 2177, pp. 92-96, 1994.
22. J. Konrad, B. Lacotte, and E. Dubois, "Cancellation of image crosstalk in time-sequential displays of stereoscopic video," IEEE Trans. Image Process., vol. 9, no. 5, p. 897, May 2000.
23. T. Tsai, C.-W. Chen, C.-H. Shih, and W.-M. Huang, "An image processing method for the elimination of ghost image and improvement of the image quality in stereoscopic display," in SID IDW, 2009.
24. M. Barkowski, P. Campisi, P. LeCallet, and V. Rizzo, "Crosstalk measurement and mitigation for autostereoscopic displays," SPIE Electron. Imaging Conf.. 3D Image Process. Appl., 2010.
25. L. Kerofsky, Y. Yoshida, S. Deshpande, and J. Speigle, "Crosstalk in 3D-TV: Adaptive cancellation and perceptual validation," in SID IDW, 2010.
26. D. Vishwanath, A. R. Girshick, and M. S. Banks, "Why pictures look right when viewed from the wrong place," Nature Neurosci., vol. 8, no. 10, pp. 1401-1410, 2005. (Pubitemid 41630430)
27. A. J. Woods, T. Docherty, and R. Koch, "Image distortions in stereoscopic video systems," in Proc. SPIE Stereoscopic Displays Appl. IV, 1993, vol. 1915, pp. 36-47.
28. G. Jones, D. Lee, D. Ezra, and N. Holliman, "Controlling perceived depth in stereoscopic images," in Proc. SPIE Stereoscopic Displays Virtual Reality Syst. VIII , 2001, p. 4297.
29. R. T. Held and M. S. Banks, "Misperceptions in stereoscopic displays: A vision science perspective," in Proc. Appl. Perception Graphics, Vis. (APGV). ACM, 2008, pp. 23-32.
30. L. B. Stelmach, W. J. Tam, F. Speranza, R. Renaud, and T. Martin, "Improving the visual comfort of stereoscopic images," Proc. SPIE, vol. 5006, 2003.
31. F. Kooi and A. Toet, "Visual comfort of binocular 3D displays," Displays, vol. 25, pp. 99-108, 2004.
32. R. T. Held, "Software," 2009 [Online]. Available: http://www.eecs. berkeley.edu/~rheld/Robert-T-Held/Software.html
33. D. Hoffman, A. Girshick, K. Akeley, and M. Banks, "Vergence- accommodation conflicts hinder visual performance and cause visual fatigue," J. Vis., vol. 8, no. 3, pp. 33, 1-33, 30, 2008.
34. D. L. MacAdams, "Stereoscopic perceptions of size shape distance and direction," SMPTE J., vol. 62, pp. 271-289, 1954.
35. H. Yamanoue, M. Okui, and F. Okano, "Geometrical analysis of puppet-theatre and cardboard effects in stereoscopic HDTV images," IEEE Trans. Circuits Syst. Video Technol., vol. 16, no. 6, pp. 744-752, 2006. (Pubitemid 44109547)
36. K. Masaoka et al., "Spatial distortion prediction system for stereoscopic images," J. Electron. Imaging, vol. 15, no. 1, p. 013002, 2006.
37. R. T. Held, E. A. Cooper, J. F. O'Brien, and M. S. Banks, "Using blur to affect perceived distance and size," ACM Trans. Graph., vol. 29, 2, Apr. 2010.
38. H. Yamanoue, M. Okui, and I. Yuyama, "A study on the relationship between shooting conditions and cardboard effect of stereoscopic images," IEEE Trans. Circuits Syst. Video Technol., vol. 10, no. 3, pp. 411-416, 2000. (Pubitemid 30594832)
39. W. J. Tam, K. Shimono, and S. Yano, "Perceived size of targets displayed stereoscopically," Stereo Displays Appl. XII, 2001.
40. K. Shimono, W. J. Tam, C. Vasquez, F. Speranzo, and R. Renaud, "Removing the cardboard effect in stereoscopic images using smoothed depth maps," SPIE, vol. 7524 -48, 2009.
41. S. Kishi, S. H. Kim, T. Shibata, T. Kawai, J. Hakkinen, J. Takatalo, and G. Nyman, "Scalable 3D image conversion and ergonomic evaluation," Proc. SPIE, vol. 6803, 2008.
42. G. Sun and N. Holliman, "Evaluating methods for controlling depth perception in stereoscopic cinematography," SDA, Proc. SPIE, vol. 7237, 2009.
43. S. Poulakos, A. Smolic, and M. Gross, "Non-linear disparity mapping for stereoscopic 3D," ACM Trans. Graphics (Siggraph), 2010.
44. H. Pan, C. Yuan, and S. Daly, "Stereoscopic 3D content depth tuning guided by human visual models," in Proc. SPIE Stereoscopic Displays Appl. XXII, Electron. Imaging, 2011, vol. 7863.
45. P. F. P. Meiers, "Parallactic depth-dependent pixel shifts,," U.S. Patent 5 929 869, Jul. 27, 1999.
46. L.Wang, H. Jin, R. Yang, and M. Gong, "Stereoscopic inpainting: joint color and depth completion from stereo images," in IEEE CVPR, 2008.
47. O. J.Woodsford, J. D. Reid, P. H. S. Torr, and W. Fitzgibbon, "On new view synthesis using multiview stereo," in BMVC, 2007.
48. R. K. Gunnewiek, R. Beretty, B. Barenbrug, and J. Magalhaes, "Coherent spatial and temporal occlusion generation," Proc. SPIE, vol. 7237, 2009.
49. J. M. Harris, "Monocular zones in stereoscopic scenes: A useful source of information for human binocular vision," Proc. SPIE, vol. 7524, 2010.
50. Y. J. Jeong, Y. Kwak, Y. J. Yung, and D. Park, "Depth-image -based rendering (DIBR) using disocclusion area restoration," in SID Digest, 2009, pp. 119-122.
51. L. Zhang and W. J. Tam, "Stereoscopic image generation based on depth images for 3DTV," IEEE Trans. Broadcast., vol. 52, no. 2, 2005.
52. C. Vasquez and W. J. Tam, "3D-TV: Coding of dislocations for 2D ? depth representation of multi-view images," in IASTED Int. Conf. Comput. Graphics Imaging, 2008.
53. W. J. Tam, G. Alain, L. Zhang, T. Martin, and R. Renaud, "Smoothing depth maps for improved stereoscopic image quality," in Proc SPIE3-D TV, Video, Display III, 2004, vol. 5599, pp. 162-172. (Pubitemid 40512677)
54. C. W. Tyler, "Stereoscopic depth movement: Two eyes less sensitive than one," Science, vol. 174, pp. 958-961, 1971.
55. C. W. Tyler, C.M. Schor, and N. J. Coletta, "Spatiotemporal limitations of vernier and stereoscopic alignment acuity," in Proc. Int. Soc. Opt. Eng., 1992, vol. 1669, pp. 112-121.
56. B. N. Vlaskamp, G. Yoon, and M. S. Banks, "Neural and optical constraints on stereoacuity," Perception 37 ECVP Abstract Supplement, p. 2, 2008.
57. F. W. Campbell and D. G. Green, "Optical and retinal factors affecting visual resolution," J. Physiology, vol. 181, pp. 576-593, 1965.
58. G. C. DeAngelis, I. Ohzawa, and R. D. Freeman, "Depth is encoded in the visual cortex by a specialized receptive field structure," Nature, vol. 352, no. 11, pp. 156-159, 1992. (Pubitemid 21896719)
59. R. von der Heydt, H. Zhou, and H. S. Friedman, "Representation of stereoscopic edges in monkey visual cortex," Vis. Res., vol. 40, pp. 1955-1967, 2000. (Pubitemid 30319287)
60. F. Allenmark and J. Read, "Spatial stereoresolution," J. Vis., vol. 9, no. 8, p. 262, 2009.
61. M. G. Perkins, "Data compression of stereopairs," IEEE Commun., vol. 40, no. 4, pp. 684-696, 1992.
62. W. D. Reynolds and R. V. Kenyon, "The wavelet transform and the suppression theory of binocular vision for stereo image compression," IEEE ICIP, vol. 1, pp. 557-560, 1996.
63. P. J. B. Seuntiens et al., "Viewing experience and naturalness of 3D images," Proc SPIE, vol. 5006, 2003, 17.
64. Gorley and N. Holliman, "Stereoscopic image quality metrics and compression," in SPIE P.W. Stereoscopic Displays Virtual Reality Syst. XIX, 2008, vol. 6803.
65. A. Vetro, A. M. Tourapis, K. Müller, and T. Chen, "3D-TV content storage and transmission," IEEE Trans. Broadcast., Jun. 2011.
66. A. Berthold, ATR Research laboratories technical report, "The influence of blur on the perceived quality and sensation of depth of 2D and stereo images," Kyoto, Japan, TR-H-232, 1997.
67. J. Y. Chen, Z. Liwei, and S. Q. Ding, "The effect of JPEG coding scheme on the perceived quality of 3D images," in SID Symp., 1998, vol. 29, pp. 1211-1214.
68. L. B. Stelmach and W. J. Tam, "Stereoscopic image coding: Effects of disparate image quality in left-and right-eye views," Signal Process.: Image Commun., vol. 14, pp. 111-117, 1998. (Pubitemid 128410665)
69. L. B. Stelmach, W. J. Tam, and D. V. Meegan, "Stereo image quality: Effects of spatiotemporal resolution," IEEE Trans. Circuits Syst. Video Technol., vol. 10, pp. 188-193, 1999.
70. D. V. Meegan, L. Stelmach, and W. J. Tam, "Unequal weighting of monocular inputs in binocular combination : Implications for compression for stereoscopic imagery," J. Exp. Psy.: Appl., vol. 7, no. 2, pp. 143-153, 2001. (Pubitemid 33595855)
71. P. J. B. Seuntiens et al., "Viewing experience and naturalness of 3D images," Proc SPIE, vol. 5006, 2003.
72. C.Vazquez, W. J. Tam, and F. Speranza, "Stereoscopic imaging : filling disoccluded areas in depth-image-based rendering," in Proc. SPIE 3-D TV, Video, Display V.,2006, vol. 6392, p. 63920D.
73. D. C. Burr and J. Ross, "How does binocular delay give information about depth?," Vis. Res., vol. 19, pp. 523-532, 1979. (Pubitemid 9160784)
74. A. B.Watson, A. J. Ahumada, and J. E. Farrell, "Window of visibility: A psychophysical theory of fidelity in time-sampled visual motion displays," JOSA A: Optics Image Sci., vol. 3, pp. 300-307, 1986.
75. D. H. Kelly, "Motion and Vision. II. Stabilized spatio-temporal threshold surface," J. Opt. Soc. Am., vol. 69, no. 10, pp. 1340-1349, 1979.
76. B. Girod, "Motion compensation: Visual aspects, accuracy, and limitations," in Motion Analysis and Image Sequence Processing, M. Sezan, Ed. Lambrecht: Kluwer, 1993, pp. 125-152.
77. S. Daly, "Engineering observations from spatiovelocity and spatiotemporal visual models," in Vision Models and Applications to Image and Video Processing. Lambrecht: Kluwer, 2001.
78. J. Laird, M. Rosen, J. Pelz, E. Montag, and S. Daly, "Spatio- velocity CSF as a function of retinal velocity using unstabilized stimuli," in HVEI, Proc. SPIE, 2006, vol. 6057, pp. 605705:1-605705:11.
79. C. R. Cavonius, "Binocular interaction in flicker," Quart. J. Exp. Psychol., vol. 31, pp. 273-280, 1979.
80. C. Landis, "Determinants of the critical flicker-fusion threshold," Physiological Rev., vol. 34, pp. 259-286, 1954.
81. D. M. Hoffman, I. V. Karasev, and M. S. Banks, "Stereo presentation protocols: How they affect flicker visibility, perceived motion, and perceived depth," J. Soc. Inf. Display (in press 2011).
82. T. Fujine, Y. Kikuchi, M. Sugino, and Y. Yoshida, "Real-life in-home viewing conditions for FPDs and statistical characteristics of broadcast video signal," in Proc. AM-FPD, 2006, p. 85-.
83. L. Zhang, C. Vazquez, and S. Knorr, "3D-TV content creation: Automatic 2D to 3D video conversion," IEEE Trans. Broadcast., Jun. 2011.
84. A. Benoit, P. LeCallet, P. Campisi, and R. Cousseau, "Quality assessment of stereo images," Eurasip J. Image Video Process., vol. 2008, 2008, article ID 659024.
85. B. W. Keelan, Handbook of Image Quality. Boca Raton, FL: CRC Press, 2002.
86. H. de Ridder, "Minkowski-metrics as a combination rule for digitalimage coding impairments," in SPIE HVEI III, 1992, vol. 1666, pp. 16-26.
87. S. Endrikhovski, E. Jin, M. E. Miller, and R. W. Ford, "Predicting individual fusional range from optometric data," in Proc. SPIE Stereoscopic Displays Virtual Reality Syst. XII, 2005, vol. 5664.
88. R. Patterson, "Human factors of 3-D displays," JSID, vol. 15/11, pp. 861-872, 2007.
89. R. Patterson, "Human factors of stereo displays : An update," J. SID, vol. 17/12, pp. 987-996, 2009.
90. W. J. Tam, A. S. Yee, J. Ferriera, S. Tariq, and F. Speranza, "Stereoscopic image rendering based on depth maps created from blur and edge information," in Proc. SPIE Stereo Depth Appl., 2005, vol. 5664, pp. 104-115. (Pubitemid 40949104)