3D Visual activity assessment based on natural scene statistics

IEEE Transactions on Image Processing

Volumn 23, Issue 1, 2014, Pages 450-465

  Lee, Kwanghyun ^a   Moorthy, Anush Krishna ^b   Lee, Sanghoon ^a   Bovik, Alan Conrad ^b  

^a Yonsei University   (South Korea)

^b The University of Texas at Austin   (United States)

Author keywords

3D coordinate transform; 3D visual activity (3DVA); human visual system (HVS); stereoscopic video; visual natural scene statistic (visual NSS)

Indexed keywords

3D COORDINATES; HUMAN VISUAL SYSTEMS; NATURAL SCENE STATISTICS; STEREOSCOPIC VIDEO; VISUAL ACTIVITY;

DISPLAY DEVICES; PROBABILITY DISTRIBUTIONS; THREE DIMENSIONAL; VIRTUAL REALITY;

THREE DIMENSIONAL COMPUTER GRAPHICS;

ARTICLE; AUTOMATED PATTERN RECOGNITION; BIOMIMETICS; COMPUTER ASSISTED DIAGNOSIS; COMPUTER SIMULATION; DEPTH PERCEPTION; HUMAN; IMAGE ENHANCEMENT; METHODOLOGY; PHYSIOLOGY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; STATISTICAL ANALYSIS; STATISTICAL MODEL; THREE DIMENSIONAL IMAGING;

BIOMIMETICS; COMPUTER SIMULATION; DATA INTERPRETATION, STATISTICAL; HUMANS; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; IMAGING, THREE-DIMENSIONAL; MODELS, STATISTICAL; PATTERN RECOGNITION, AUTOMATED; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; SPACE PERCEPTION;

EID: 84890922933     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2013.2290592     Document Type: Article

References (74)

1. E. Lantz, "Future directions in visual display systems," ACM SIGGRAPH Comput. Graph., vol. 31, no. 2, pp. 38-42, May 1997. (Pubitemid 127575004)
2. L. Steinbach, "3D or not 3D is that a question," Museum J., vol. 54, no. 1, pp. 41-54, Jan. 2011.
3. M. Sonka, V. Hlavac, and R. Boyle, Image Processing, Analysis and Machine Vision, 2nd ed. Pacific Grove, CA, USA: Brooks/Cole, 1999.
4. C.-W. Ngo, T.-C. Pong, and H.-J. Zhang, "Motion analysis and segmentation through spatio-temporal slices processing," IEEE Trans. Image Process., vol. 12, no. 3, pp. 341-355, Mar. 2003.
5. H. Greenspan, C. H. Anderson, and S. Akber, "Image enhancement by nonlinear extrapolation in frequency space," IEEE Trans. Image Process., vol. 9, no. 6, pp. 1035-1048, Jun. 2000.
6. T. Bose, F. Meyer, and M. Q. Chen, Digital Signal and Image Processing. New York, NY, USA: Wiley, 2004.
7. S. Lee, M. S. Pattichis, and A. C. Bovik, "Foveated video compression with optimal rate control," IEEE Trans. Image Process., vol. 10, no. 7, pp. 977-992, Jul. 2001. (Pubitemid 32650874)
8. S. Lee, M. S. Pattichis, and A. C. Bovik, "Foveated video quality assessment," IEEE Trans. Multimedia, vol. 4, no. 1, pp. 129-132, Mar. 2002. (Pubitemid 34291535)
9. S. Lee and A. C. Bovik, "Fast algorithms for foveated video processing," IEEE Trans. Circuit Syst. Video Technol., vol. 13, no. 2, pp. 149-162, Feb. 2003.
10. H. Lee and S. Lee, "Visual entropy gain for wavelet image coding," IEEE Signal Process. Lett., vol. 13, no. 9, pp. 553-556, Sep. 2006. (Pubitemid 44275813)
11. H. Ha, T. Oh, and S. Lee, "Macroblock-based frequency selective weighting for visual scalable video coding of H.264/AVC," IEEE Trans. Broadcast., vol. 55, no. 3, pp. 559-568, Sep. 2009.
12. H. Ha, J. Park, S. Lee, and A. C. Bovik, "Perceptually scalable extension of H.264," IEEE Trans. Circuit Syst. Video Technol., vol. 21, no. 11, pp. 1667-1678, Nov. 2011.
13. C. J. van den Branden Lambrecht and M. Kunt, "Characterization of human visual sensitivity for video imaging applications," Signal Process., vol. 67, pp. 255-269, May 1996.
14. U. Jang, H. Lee, and S. Lee, "Optimal carrier loading control for the enhancement of visual quality over OFDMA cellular networks," IEEE Trans. Multimedia, vol. 10, no. 6, pp. 1181-1196, Oct. 2008.
15. J. Park, H. Lee, S. Lee, and A. C. Bovik, "Optimal channel adaptation of scalable video over a multi-carrier based multi-cell environment," IEEE Trans. Multimedia, vol. 11, no. 6, pp. 1062-1071, Oct. 2009.
16. F. Yang, S.Wan, Y. Chang, and H. R. Wu, "A novel objective noreference metric for digital video quality assessment," IEEE Signal Process. Lett., vol. 12, no. 10, pp. 685-688, Oct. 2005. (Pubitemid 41448575)
17. ITU-T Recommendation BT.500-10, Methodology for the Subjective Assessment of the Quality of Television Pictures, Geneva, The Switzerland: International Telecommunication Union, 2000.
18. J. Park, K. Seshadrinathan, S. Lee, and A. C. Bovik, "VQPooling: Video quality pooling adaptive to perceptual distortion severity," IEEE Trans. Image Process., vol. 22, no. 2, pp. 610-620, Feb. 2013.
19. A. Boev, M. Poikela, A. Gotchev, and A. Aksay, "Modeling of the stereoscopic HVS," MOBILE3DTV, Tech. Rep. D5.3, Jul. 2009.
20. H. R. Sheikh, A. C. Bovik, and G. de Veciana, "An information fidelity criterion for image quality assessment using natural scene statistics," IEEE Trans. Image Process., vol. 14, no. 12, pp. 2117-2128, Dec. 2005. (Pubitemid 41763727)
21. H. R. Sheikh, A. C. Bovik, and L. K. Cormack, "No-reference quality assessment using natural scene statistics: JPEG2000," IEEE Trans. Image Process., vol. 14, no. 11, pp. 1918-1927, Nov. 2005. (Pubitemid 41646081)
22. Y. Liu, L. K. Cormack, and A. C. Bovik, "Statistical modeling of 3D natural scenes with application to Bayesian stereopsis," IEEE Trans. Image Process., vol. 20, no. 9, pp. 2515-2530, Sep. 2011.
23. S. Osindero, M. Welling, and G. E. Hinton, "Topographic product models applied to natural scene statistics," Neural Comput., vol. 18, no. 2, pp. 381-414, 2006. (Pubitemid 43543842)
24. R. W. Buccigrossi and E. P. Simoncelli, "Image compression via joint statistical characterization in the wavelet domain," IEEE Trans. Image Process., vol. 8, no. 12, pp. 1688-1701, Dec. 1999. (Pubitemid 30527916)
25. O. Schwartz and E. P. Simoncelli, "Natural signal statistics and sensory gain control," Nature, Neurosci., vol. 4, pp. 819-825, Aug. 2001. (Pubitemid 32702514)
26. D. L. Ruderman, "The statistics of natural images," Netw. Comput. Neural Syst., vol. 5, no. 4, pp. 517-548, 1994.
27. E. P. Simoncelli and B. A. Olshausen, "Natural image statistics and neural representation," Ann. Rev. Neurosci., vol. 24, no. 1, pp. 1193-1216, 2001. (Pubitemid 32695258)
28. W. S. Geisler, "Visual perception and the statistical properties of natural scenes," Ann. Rev. Psychol., vol. 59, no. 1, pp. 167-192, Jan. 2008.
29. G. F. Poggio and B. Fischer, "Binocular interaction and depth sensitivity in striate and prestriate cortex of behaving rhesus monkey," J. Neurophysiol., vol. 40, no. 6, pp. 1392-1405, Nov. 1977. (Pubitemid 8233147)
30. S. Prince, B. G. Cumming, and A. J. Parker, "Range and mechanism of encoding of horizontal disparity in macaque V1," J. Neurophysiol., vol. 87, no. 1, pp. 209-221, Jan. 2002. (Pubitemid 34669943)
31. S. J. D. Prince, A. D. Pointon, B. G. Cumming, and A. J. Parker, "Quantitative analysis of the responses of V1 neurons to horizontal disparity in dynamic random-dot stereograms," J. Neurophysiol., vol. 87, no. 1, pp. 191-208, Jan. 2002. (Pubitemid 34669942)
32. G. C. DeAngelis and T. Uka, "Coding of horizontal disparity and velocity by MT neurons in the alert macaque," J. Neurophysiol., vol. 89, no. 2, pp. 1094-1111, Feb. 2003. (Pubitemid 36206154)
33. M. Lambooij, M. Fortuin, I. Heynderickx, and W. I. Jsselsteijin, "Visual discomfort and visual fatigue of stereoscopic displays: A review," J. Imag. Sci. Technol., vol. 53, no. 3, pp. 1-14, May 2009.
34. S. Pastoor, COST230 Final Report Working Group 1 Psychooptics. Roma, Italy: Fondazinoe Ugo Bordoni, 1998, pp. 17-32.
35. C. W. G. Clifford, "Perceptual adaptation: Motion parallels orientation," Trends Cognit. Sci., vol. 6, no. 3, pp. 136-143, Mar. 2002. (Pubitemid 34171577)
36. F. Speranza W. J. Tam, R. Renaud, and N. Hur, "Effect of disparity and motion on visual comfort of stereoscopic images," Proc. SPIE, vol. 6055, pp. 94-103, Jan. 2006.
37. A. Wilkins, I. N. Smith, A. Tait, C. Mcmanus, S. D. Sala, A. Tilley, et al., "A neurological basis for visual discomfort," Brain, vol. 107, no. 4, pp. 989-1017, 1984. (Pubitemid 15182093)
38. L. Leroy, P. Fuchs, and G. Moreau, "Visual fatigue Reduction for immersive stereoscopic displays by disparity, content, and focuspoint adapted blur," IEEE Trans. Ind. Electron., vol. 59, no. 10, pp. 3998-4004, Oct. 2012.
39. S. M. Haigh, L. Barningham, M. Berntsen, L. V. Coutts, E. S. T. Hobbs, J. Irabor, et al., "Discomfort and the cortical haemodynamic response to coloured gratings," Vis. Res., vol. 89, pp. 47-53, Aug. 2013.
40. I. Juricevic, L. Land, A. J. Wilkins, and M. A. Webster, "Visual discomfort and natural image statistics," Perception, vol. 39, no. 7, pp. 884-899, 2010.
41. M. Iwasaki and H. Inomata, "Relation between superficial capillaries and foveal structures in the human retina," Invest. Ophthalmol. Vis. Sci., vol. 27, no. 12, pp. 1698-1705, 1986. (Pubitemid 17207841)
42. H. Kim, S. Lee, and A. C. Bovik, "Saliency prediction on stereoscopic videos," IEEE Trans. Image Process., to be published.
43. Y. Liu, L. K. Cormack, and A. C. Bovik, "Dichotomy between luminance and disparity features at binocular fixations," J. Vision, vol. 10, no. 12 pp. 1-17, 2010.
44. IEEE Standards Association Stereoscopic Database [Online]. Available: http://grouper.ieee.org/groups/3dhf/ (2008)
45. J. G. Robson and N. Graham, "Probability summation and regional variation in contrast sensitivity across the visual field," Vis. Res., vol. 21, no. 3, pp. 409-418, 1981. (Pubitemid 11143371)
46. M. S. Banks, A. B. Sekuler, and S. J. Anderson, "Peripheral spatial vision: Limited imposed by optics, photoreceptors and receptor pooling," J. Opt. Soc. Amer., vol. 8, no. 11, pp. 1775-1787, 1991.
47. W. S. Geisler and J. S. Perry, "A real-time foveated multiresolution system for low bandwidth video communication," Proc. SPIE, vol. 3299, pp. 1-8, Jul. 1998.
48. Middlebury Stereo [Online]. Available: http://vision.middlebury.edu/ stereo/data/ (2012)
49. A. Coltekin, "Foveation for 3D visualization and stereo imaging," Ph.D. dissertation, Dept. Sci. Technol., Helsinki Univ. Technol., Helsinki, Finland, 2006.
50. T. Ohshima, H. Yamamoto, and H. Tamura, "Gaze-directed adaptive rendering for interacting with virtual space," in Proc. IEEE VRAIS, Apr. 1996, pp. 103-110.
51. M. Tanimoto, T. Fujii, K. Suzuki, N. Fukushima, and Y. Mori, "Reference softwares for depth estimation and view synthesis," New York, NY, USA: Springer-Verlag, Apr. 2008.
52. K.-J. Oh, S. Yea, and Y.-S. Ho, "Hole filling method using depth based in-ainting for view synthesis in free viewpoint television and 3D video," in Proc. Picture Coding Symp., May 2009, pp. 1-4.
53. A. C. Bovik, M. Clark, and W. S. Geisler, "Multichannel texture analysis using localized spatial filters," IEEE Trans. Pattern Anal. Mach. Intell., vol. 12, no. 1, pp. 55-73, Jan. 1990. (Pubitemid 20653599)
54. L. Zhang, L. Zhang, D. Zhang, and H. Zhu, "Ensemble of local and global information for finger-knuckle-rint recognition," Pattern Recognit., vol. 44, no. 9, pp. 1990-1998, Sep. 2011.
55. R. Gao and W. F. Bischof, "Detection of linear structures in remotesensed images," in Image Analysis and Recognition. New York, NY, USA: Springer-Verlag, 2009, pp. 896-905.
56. E. P. Simoncelli and W. T. Freeman, "The steerable pyramid: A flexible architecture for multi-scale derivative computation," in Proc. IEEE Int. Conf. Image Process., vol. 3. pp. 444-447, Oct. 1995.
57. D. J. Heeger, "Normalization of cell responses in cat striate cortex," Vis. Neurosci., vol. 9, no. 2, pp. 181-198, 1992.
58. Z. Wang and A. C. Bovik, "Reduced and no reference visual quality assessment-The natural scene statistic model approach," IEEE Signal Process. Mag., vol. 29, no. 6, pp. 29-40, Nov. 2011.
59. M. J. Wainwright and E. P. Simoncelli, "Scale mixtures of Gaussians and the statistics of natural images," Adv. Neural Inf. Process. Syst., vol. 12, no. 1, pp. 855-861, 2000.
60. M. J. Wainwright, O. Schwartz, and E. P. Simoncelli, "Natural image statistics and divisive normalization: Modeling nonlinearities and adaptation in cortical neurons," in Probabilistic Models of the Brain: Perception and Neural Function. Cambridge, MA, USA: MIT Press, Feb. 2002, pp. 203-222.
61. J. Huang, A. Lee, and D. Mumford, "Statistics of range images," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., vol. 1. Jun. 2000, pp. 324-331.
62. Y. Liu, L. K. Cormack, and A. C. Bovik, "Statistical modeling of 3D natural scenes with application to Bayesian stereopsis," IEEE Trans. Image Process., vol. 20, no. 9, pp. 2515-2530. Sep. 2011.
63. K. Sharifi and A. Leon-Garcia, "Estimation of shape parameter for generalized Gaussian distributions in subband decompositions of video," IEEE Trans. Circuits Syst. Video Technol., vol. 5, no. 1, pp. 52-56, Feb. 1995.
64. A. Jaina, K. Nandakumara, and A. Ross, "Score normalization in multimodal biometric systems," Pattern Recognit., vol. 38, no. 12, pp. 2270-2285, Dec. 2005. (Pubitemid 41336698)
65. R. Cappelli, D. Maio, and D. Maltoni, "Combining fingerprint classifiers," in Proc. 1st Int. Workshop MCS, Jun. 2000, pp. 351-361.
66. Image Safety - Reducing the Incidence of Undesirable Biomedical Effects Caused by Visual Image Sequences, Geneva, The Switzerland, ISO, 2005.
67. T. Kim, J. Kang, S. Lee, and A. C. Bovik, "Multimodal interactive continuous scoring of subjective 3D video quality of experience," IEEE Trans. Multimedia, to be published.
68. ITU, "Methodology for the subjective assessment of the quality of television pictures," ITU-R, Tech. Rep. BT.500-13, 2012.
69. K. Seshadrinathan, R. Soundararajan, A. C. Bovik, and L. K. Cormack, "Study of subjective and objective quality assessment of video," IEEE Trans. Image Process., vol. 19, no. 6, pp. 1427-1441, Jun. 2010.
70. J. M. Bland and D. G. Altman, "Statistical methods for assessing agreement between two methods of clinical measurement," Lancet, vol. 327, no. 8476, pp. 307-310, Feb. 1986. (Pubitemid 16134762)
71. B. Scholkopf, A. Smola, R. Williamson, and P. Bartlett, "New support vector algorithms," Neural Computat., vol. 12, no. 5, pp. 1207-1245, 2000.
72. C. Burges, "A tutorial on support vector machines for pattern recognition," Data Mining Knowl. Discov., vol. 2, no. 2, pp. 121-167, 1998. (Pubitemid 128695475)
73. A. K. Moorthy and A. C. Bovik, "Blind image quality assessment: From natural scene statistics to perceptual quality," IEEE Trans. Image Process., vol. 20, no. 12, pp. 3350-3364, Dec. 2011.
74. M. A. Saad, A. C. Bovik, and C. Charrier, "Blind image quality assessment: A natural scene statistics approach in the DCT domain," IEEE Trans. Image Process., vol. 20, no. 12, pp. 3350-3364, Mar. 2012.