Towards the Perceptual Quality Evaluation of Compressed Light Field Images

IEEE Transactions on Broadcasting

Volumn 63, Issue 3, 2017, Pages 507-522

  Paudyal, Pradip ^a   Battisti, Federica ^a   Sjöström, Mårten ^b   Olsson, Roger ^b   Carli, Marco ^a  

^a ROMA TRE UNIVERSITY   (Italy)

^b MID SWEDEN UNIVERSITY   (Sweden)

Author keywords

dataset; light field image; perceptual quality; quality metrics; Quality of experience

Indexed keywords

QUALITY CONTROL; QUALITY OF SERVICE; STATISTICAL TESTS;

DATASET; LIGHT FIELDS; PERCEPTUAL QUALITY; QUALITY METRICS; QUALITY OF EXPERIENCE (QOE);

IMAGE QUALITY;

EID: 85029911746     PISSN: 00189316     EISSN: None     Source Type: Journal    
DOI: 10.1109/TBC.2017.2704430     Document Type: Article

References (92)

1. C. Conti, L. D. Soares, and P. Nunes, "HEVC-based 3D holoscopic video coding using self-similarity compensated prediction," J. Signal Process. Image Commun., vol. 42, pp. 59-78, Mar. 2016.
2. G. Lippmann, "Sur la théorie de la photographie des couleurs simples et composées par la méthode interférentielle," J. Phys. Theor. Appl., vol. 3, no. 1, pp. 97-107, 1984.
3. L. Gabriel, "Epreuves reversibles photographies integrals," J. Comptes Rendus Academie des Sci., vol. 146, no. 8, pp. 446-451, 1908.
4. H. E. Ives, "Parallax panoramagram," U.S. Patent 1 918 705, Jul. 18, 1933.
5. E. H. Adelson and J. Y. A. Wang, "Single lens stereo with a plenoptic camera," IEEE Trans. Pattern Anal. Mach. Intell., vol. 14, no. 2, pp. 99-106, Feb. 1992.
6. R. Ng et al., "Light field photography with a hand-held plenoptic camera," Comput. Sci. Tech. Rep., vol. 2, no. 11, pp. 1-11, 2005.
7. A. Lumsdaine and T. Georgiev, "The focused plenoptic camera," in Proc. IEEE Int. Conf. Comput. Photography (ICCP), San Francisco, CA, USA, 2009, pp. 1-8.
8. T. Georgiev and A. Lumsdaine, "Depth of field in plenoptic cameras," in Proc. Eurograph., 2009, pp. 1-4.
9. I. Montilla, M. Puga, J. P. Lüke, J. G. Marichal-Hernandez, and J. M. Rodriguez-Ramos, "Design and laboratory results of a plenoptic objective: From 2D to 3D with a standard camera," J. Display Technol., vol. 11, no. 1, pp. 73-78, Jan. 2015.
10. T. Ebrahimi, S. Foessel, F. Pereira, and P. Schelkens, "JPEG pleno: Toward an efficient representation of visual reality," IEEE MultiMedia, vol. 23, no. 4, pp. 14-20, Oct./Dec. 2016.
11. Z. Wang, "Objective image quality assessment: Facing the real-world challenges," in Proc. Image Qual. Syst. Perform. (Keynote Speech Paper), 2016, pp. 1-6.
12. K. Gu et al., "Analysis of distortion distribution for pooling in image quality prediction," IEEE Trans. Broadcast., vol. 62, no. 2, pp. 446-456, Jun. 2016.
13. P. Paudyal, F. Battisti, and M. Carli, "Impact of video content and transmission impairments on quality of experience," Multimedia Tools Appl., vol. 75, no. 23, pp. 16461-16485, 2016.
14. V. Vaish and A. Adams. (2008). The (New) Stanford Light Field Archive. [Online]. Available: http://lightfield.stanford.edu/
15. (2016). Synthetic Light Field Archive. [Online]. Available: http://web.media.mit.edu/~gordonw/SyntheticLightFields/
16. S. Wanner, S. Meister, and B. Goldlüecke, "Datasets and benchmarks for densely sampled 4D light fields," in Proc. Annu. Workshop Vis. Model. Vis. (VMV), 2013, pp. 225-226.
17. A. Ghasemi, N. Afonso, and M. Vetterli, "LCAV-31: A dataset for light field object recognition," in Proc. IS T SPIE Electron. Imag., San Francisco, CA, USA, 2014, Art. no. 902014.
18. A. Mousnier, E. Vural, and C. Guillemot, "Partial light field tomographic reconstruction from a fixed-camera focal stack," Comput. Res. Repository, vol. 1503.01903, 2015.
19. N. Li, J. Ye, Y. Ji, H. Ling, and J. Yu, "Saliency detection on light field," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Columbus, OH, USA, 2014, pp. 2806-2813.
20. R. Raghavendra, K. B. Raja, and C. Busch, "Exploring the usefulness of light field cameras for biometrics: An empirical study on face and iris recognition," IEEE Trans. Inf. Forensics Security, vol. 11, no. 5, pp. 922-936, May 2016.
21. M. Rerabek, L. Yuan, L. A. Authier, and T. Ebrahimi, "[ISO/IEC JTC 1/SC 29/WG1 contribution] EPFL light-field image dataset," Multimedia Signal Process. Group, École Polytechnique Fédérale de Lausanne, Lausanne, Switzerland, Tech. Rep. ISO/IEC JTC 1/SC 29/WG1, 2015.
22. P. Paudyal, R. Olsson, M. Sjöström, F. Battisti, and M. Carli, "SMART: A light field image quality dataset," in Proc. ACM Multimedia Syst. Conf., Klagenfurt, Austria, 2016, pp. 1-6.
23. B. W. Keelan and H. Urabe, "ISO 20462: A psychophysical image quality measurement standard," in Proc. Electron. Imag. 2004 SPIE, 2003, pp. 181-189.
24. "Subjective video quality assessment methods for multimedia applications," ITU-T Rec. P.910, Geneva, Switzerland, 2008.
25. "Methodology for the subjective assessment of the quality of television pictures," ITU-R Rec. BT.500, Geneva, Switzerland, 2012.
26. X. Dong, D. Qionghan, and X. Wenli, "Data compression of light field using wavelet packet," in Proc. IEEE Int. Conf. Multimedia Expo. (ICME), vol. 2. Taipei, Taiwan, 2004, pp. 1071-1074.
27. C. Perra, "Lossless plenoptic image compression using adaptive block differential prediction," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP), South Brisbane, QLD, Australia, 2015, pp. 1231-1234.
28. F. Dai, J. Zhang, Y. Ma, and Y. Zhang, "Lenselet image compression scheme based on subaperture images streaming," in Proc. IEEE Int. Conf. Image Process. (ICIP), Québec City, QC, Canada, 2015, pp. 4733-4737.
29. Y. Li, M. Sjöström, R. Olsson, and U. Jennehag, "Coding of focused plenoptic contents by displacement intra prediction," IEEE Trans. Circuits Syst. Video Technol., vol. 26, no. 7, pp. 1308-1319, Jul. 2016.
30. Y. Li, M. Sjöström, R. Olsson, and U. Jennehag, "Scalable coding of plenoptic images by using a sparse set and disparities," IEEE Trans. Image Process., vol. 25, no. 1, pp. 80-91, Jan. 2016.
31. N. Dimitrova et al., "Applications of video-content analysis and retrieval," IEEE Multimedia, vol. 9, no. 3, pp. 42-55, Jul./Sep. 2002.
32. S. Winkler, "Analysis of public image and video databases for quality assessment," IEEE J. Sel. Topics Signal Process., vol. 6, no. 6, pp. 616-625, Oct. 2012.
33. O. Hadar, R. Shmueli, R. Huber, and M. Huber, "Effects of compression parameters on the perceived quality of video stream over a lossy Internet protocol network," Opt. Eng., vol. 45, no. 8, 2006, Art. no. 087003.
34. R. Shmueli, O. Hadar, R. Huber, M. Maltz, and M. Huber, "Effects of an encoding scheme on perceived video quality transmitted over lossy Internet protocol networks," IEEE Trans. Broadcast., vol. 54, no. 3, pp. 628-640, Sep. 2008.
35. K. Gu, G. Zhai, X. Yang, and W. Zhang, "Hybrid no-reference quality metric for singly and multiply distorted images," IEEE Trans. Broadcast., vol. 60, no. 3, pp. 555-567, Sep. 2014.
36. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, Apr. 2004.
37. F. Shao, W. Lin, S. Wang, G. Jiang, and M. Yu, "Blind image quality assessment for stereoscopic images using binocular guided quality lookup and visual codebook," IEEE Trans. Broadcast., vol. 61, no. 2, pp. 154-165, Jun. 2015.
38. T. Zhao, J. Wang, Z. Wang, and C. W. Chen, "SSIM-based coarsegrain scalable video coding," IEEE Trans. Broadcast., vol. 61, no. 2, pp. 210-221, Jun. 2015.
39. P. Paudyal, F. Battisti, and M. Carli, "Evaluation of the effects of transmission impairments on perceived video quality by exploiting retrieved dataset," J. Electron. Imag., vol. 26, no. 2, 2017, Art. no. 023003.
40. N. Ponomarenko et al., "Image database TID2013: Peculiarities, results and perspectives," Signal Process. Image Commun., vol. 30, pp. 57-77, Jan. 2015.
41. L. Liu, B. Liu, H. Huang, and A. C. Bovik, "No-reference image quality assessment based on spatial and spectral entropies," Signal Process. Image Commun., vol. 29, no. 8, pp. 856-863, 2014.
42. I. Viola et al., "Objective and subjective evaluation of light field image compression algorithms," in Proc. Picture Coding Symp. (PCS), Nuremberg, Germany, 2016, pp. 1-5.
43. C. Faloutsos et al., "Efficient and effective querying by image content," J. Intell. Inf. Syst., vol. 3, nos. 3-4, pp. 231-262, 1994.
44. D. Hasler and S. E. Suesstrunk, "Measuring colorfulness in natural images," Electron. Imag., vol. 5007, pp. 87-95, Jun. 2003.
45. W. Lin and C.-C. J. Kuo, "Perceptual visual quality metrics: A survey," J. Vis. Commun. Image Represent., vol. 22, no. 4, pp. 297-312, 2011.
46. T. E. Bishop, S. Zanetti, and P. Favaro, "Light field superresolution," in Proc. IEEE Int. Conf. Comput. Photography (ICCP), San Francisco, CA, USA, 2009, pp. 1-9.
47. R. Szeliski, S. Avidan, and P. Anandan, "Layer extraction from multiple images containing reflections and transparency," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., vol. 1. 2000, pp. 246-253.
48. Q. Wang, H. Lin, Y. Ma, S. B. Kang, and J. Yu, "Automatic layer separation using light field imaging," Comput. Res. Repository, vol. 1506.04721, 2015.
49. A. Levin and Y. Weiss, "User assisted separation of reflections from a single image using a sparsity prior," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 9, pp. 1647-1654, Sep. 2007.
50. Objective Perceptual Assessment of Video Quality: Full Reference Television, ITU, Geneva, Switzerland, 2004.
51. F. Bossen, Common Test Conditions and Software Reference Configurations, document JCTVC-F900, Joint Collaborative Team Video Coding (JCT-VC), Dresden, Germany, 2011.
52. (2016). High Efficiency Video Coding (HEVC): HM Software. [Online]. Available: https://hevc.hhi.fraunhofer.de/
53. K. Seshadrinathan, R. Soundararajan, A. C. Bovik, and L. K. Cormack, "Study of subjective and objective quality assessment of video," IEEE Trans. Image Process., vol. 19, no. 6, pp. 1427-1441, Jun. 2010.
54. D. Grois, D. Marpe, A. Mulayoff, B. Itzhaky, and O. Hadar, "Performance comparison of H.265/MPEG-HEVC, VP9, and H.264/MPEG-AVC encoders," in Proc. Picture Coding Symp. (PCS), San Jose, CA, USA, 2013, pp. 394-397.
55. G. D. Donald. (2016). Light Field Toolbox for MATLAB (V 0.4). [Online]. Available: http://www.mathworks.com/ matlabcentral/fileexchange/49683-light-field-toolbox-v0-4
56. D. G. Dansereau, O. Pizarro, and S. B. Williams, "Linear volumetric focus for light field cameras," ACM Trans. Graph., vol. 34, no. 2, pp. 1-15, 2015.
57. D. Liu et al., "Pseudo-sequence-based light field image compression," in Proc. IEEE Int. Conf. Multimedia Expo Workshops (ICMEW), Seattle, WA, USA, 2016, pp. 1-4.
58. R. Monteiro et al., "Light field HEVC-based image coding using locally linear embedding and self-similarity compensated prediction," in Proc. IEEE Int. Conf. Multimedia Expo Workshops (ICMEW), Seattle, WA, USA, 2016, pp. 1-4.
59. H. Common, "Test conditions and software reference configurations," ITU-T/ISO/IEC Joint Collaborative Team Video Coding (JCT-VC), Tech. Rep. JCTVC-L1100, Geneve, 2013.
60. A. Jones, I. McDowall, H. Yamada, M. Bolas, and P. Debevec, "Rendering for an interactive 360 light field display," ACM Trans. Graph., vol. 26, no. 3, p. 40, 2007.
61. D. G. Dansereau, O. Pizarro, and S. B. Williams, "Decoding, calibration and rectification for lenselet-based plenoptic cameras," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Portland, OR, USA, 2013, pp. 1027-1034.
62. Y. J. Jeong et al., "Efficient light-field rendering using depth maps for 100-mpixel multi-projection 3D display," IEEE J. Display Technol., vol. 11, no. 10, pp. 792-799, Oct. 2015.
63. T. Georgiev and A. Lumsdaine, "Focused plenoptic camera and rendering," J. Electron. Imag., vol. 19, no. 2, 2010, Art. no. 021106.
64. K. Tsukida and M. R. Gupta, "How to analyze paired comparison data," Dept. Elect. Eng., Univ. Washington, Seattle, WA, USA, Tech. Rep. UWEETR-2011-0004, 2011.
65. J. Korhonen, N. Burini, J. You, and E. Nadernejad, "How to evaluate objective video quality metrics reliably," in Proc. 4th Int. Workshop Qual. Multimedia Exp. (QoMEX), 2012, pp. 57-62.
66. L. Krasula, K. Fliegel, P. Le Callet, and M. Klíma, "Using full-reference image quality metrics for automatic image sharpening," in Proc. SPIE Photon. Europe, Brussels, Belgium, 2014, Art. no. 913807.
67. R. A. Bradley and M. E. Terry, "Rank analysis of incomplete block designs: I. The method of paired comparisons," Biometrika, vol. 39, nos. 3-4, pp. 324-345, 1952.
68. J. C. Handley, "Comparative analysis of Bradley-Terry and Thurstone-Mosteller paired comparison models for image quality assessment," in Proc. PICS, 2001, pp. 108-112.
69. F. Wickelmaier and C. Schmid, "A MATLAB function to estimate choice model parameters from paired-comparison data," Behav. Res. Methods Instrum. Comput., vol. 36, no. 1, pp. 29-40, 2004.
70. L. Xu et al., "Pairwise comparison and rank learning for image quality assessment," Displays, vol. 44, pp. 21-26, Sep. 2016.
71. R. K. Mantiuk, A. Tomaszewska, and R. Mantiuk, "Comparison of four subjective methods for image quality assessment," in Computer Graphics Forum. Chichester, U.K.: Wiley, 2012, pp. 2478-2491.
72. M. Piegel, Schaum's Outline of Theory and Problems of Statistics. New York, NY, USA: Schaum, 1961.
73. R. McGill, J. W. Tukey, and W. A. Larsen, "Variations of box plots," Amer. Stat., vol. 32, no. 1, pp. 12-16, 1978.
74. G. Barnard, "A new test for 2 × 2 tables," Nature, vol. 156, p. 177, 1945.
75. J. Li, M. Barkowsky, and P. Le Callet, "Subjective assessment methodology for preference of experience in 3DTV," in Proc. IEEE 11th IVMSP Workshop, Seoul, South Korea, 2013, pp. 1-4.
76. I. Jolliffe, Principal Component Analysis. New York, NY, USA: Springer-Verlag, 2002.
77. P. Hanhart, E. Bosc, P. Le Callet, and T. Ebrahimi, "Free-viewpoint video sequences: A new challenge for objective quality metrics," in Proc. IEEE Int. Workshop Multimedia Signal Process. (MMSP), Jakarta, Indonesia, 2014, pp. 1-6.
78. S. Le, J. Josse, and F. Husson, "FactoMineR: An R package for multivariate analysis," J. Stat. Softw., vol. 25, no. 1, pp. 1-18, 2008.
79. K. Brunnström et al., "QUALINET white paper on definitions of quality of experience," in Proc. 5th Meeting Eur. Netw. Qual. Exp. Multimedia Syst. Services (COST Action IC), Novi Sad, Serbia, 2013.
80. M. Varela et al., "QoE-Defining a user-centric concept for service quality," in Multimedia Quality of Experience (QoE): Current Status and Future Requirements. Hoboken, NJ, USA: Wiley, 2015, pp. 1-5.
81. A. Norkin et al., "HEVC deblocking filter," IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 12, pp. 1746-1754, Dec. 2012.
82. S. Saha, M. Tahtali, A. Lambert, and M. Pickering, "Perceptual dissimilarity: A measure to quantify the degradation of medical images," in Proc. Int. Conf. Digit. Image Comput. Tech. Appl. (DICTA), Fremantle, WA, Australia, 2012, pp. 1-6.
83. S. Saha, M. Tahtali, A. Lambert, and M. Pickering, "Perceptual dissimilarity metric: A full reference objective image quality measure to quantify the degradation of perceptual image quality," in Proc. IEEE Int. Symp. Signal Process. Inf. Technol., Athens, Greece, 2013, pp. 327-332.
84. M. Gaubatz and S. Hemami. (2016). MeTriX MuX Visual Quality Assessment Package. [Online]. Available: http://foulard, ece. cornell. Edu/gaubatz/metrix-mux
85. T. Mitsa and K. L. Varkur, "Evaluation of contrast sensitivity functions for the formulation of quality measures incorporated in halftoning algorithms," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process. (ICASSP), vol. 5. Minneapolis, MN, USA, 1993, pp. 301-304.
86. K. Egiazarian et al., "New full-reference quality metrics based on HVS," in Proc. 2nd Int. Workshop Video Process. Qual. Metrics, vol. 4. 2006, p. 4.
87. N. Ponomarenko et al., "On between-coefficient contrast masking of DCT basis functions," in Proc. 3rd Int. Workshop Video Process. Qual. Metrics, vol. 4. Scottsdale, AZ, USA, 2007, p. 4.
88. D. M. Chandler and S. S. Hemami, "VSNR: A wavelet-based visual signal-to-noise ratio for natural images," IEEE Trans. Image Process., vol. 16, no. 9, pp. 2284-2298, Sep. 2007.
89. Z. Wang, E. P. Simoncelli, and A. C. Bovik, "Multiscale structural similarity for image quality assessment," in Proc. ASILOMAR Conf. Signals Syst. Comput., Pacific Grove, CA, USA, 2003, pp. 1398-1402.
90. H. R. Sheikh and A. C. Bovik, "Image information and visual quality," IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006.
91. Z. Wang and A. C. Bovik, "A universal image quality index," IEEE Signal Process. Lett., vol. 9, no. 3, pp. 81-84, Mar. 2002.
92. P. Y. Chen and P. M. Popovich, Correlation: Parametric and Nonparametric Measures. Thousand Oaks, CA, USA: SAGE, 2002, pp. 137-139.