New measurements reveal weaknesses of image quality metrics in evaluating graphics artifacts

ACM Transactions on Graphics

Volumn 31, Issue 6, 2012, Pages

  Čadík, Martin ^a   Herzog, Robert ^a   Mantiuk, Rafał ^b   Myszkowski, Karol ^a   Seidel, Hans Peter ^a  

^a MAX PLANCK INSTITUTE FOR INFORMATICS   (Germany)

^b BANGOR UNIVERSITY   (United Kingdom)

Author keywords

Global illumination; Image quality metrics (IQM); Noticeable and objectionable distortions; Perceptual experiments

Indexed keywords

CONTRAST CHANGES; DATA SETS; DISTORTION MAPS; GLOBAL ILLUMINATION; HIGH QUALITY; IMAGE QUALITY METRICS; IMAGE REGIONS; MULTISCALES; NO REFERENCES; PRESENCE/ABSENCE; QUALITY METRICS; REFERENCE IMAGE; RELIABLE DETECTION; SPATIAL LOCALIZATION;

COMPUTER GRAPHICS;

IMAGE QUALITY;

EID: 84870200955     PISSN: 07300301     EISSN: 15577368     Source Type: Journal    
DOI: 10.1145/2366145.2366166     Document Type: Conference Paper

References (36)

1. BALDI, P., BRUNAK, S., CHAUVIN, Y., ANDERSEN, C. A. F., AND NIELSEN, H. 2000. Assessing the accuracy of prediction algorithms for classification: an overview. Bioinformatics 16, 5 (May), 412-424.
2. BOLIN, M., AND MEYER, G. 1998. A perceptually based adaptive sampling algorithm. In Proc. of SIGGRAPH, 299-310.
3. ČADÍK, M., AYDIN, T. O., MYSZKOWSKI, K., AND SEIDEL, H.- P. 2011. On evaluation of video quality metrics: an HDR dataset for computer graphics applications. In SPIE HVEI XVI.
4. CHANDLER, D., AND HEMAMI, S. 2007. VSNR: a wavelet-based visual signal-to-noise ratio for natural images. IEEE Trans. on Image Processing 16, 9, 2284-2298.
5. DALY, S. 1993. The Visible Differences Predictor: An algorithm for the assessment of image fidelity. In Digital Images and Human Vision, MIT Press, 179-206.
6. DRAGO, F., MYSZKOWSKI, K., ANNEN, T., AND N.CHIBA. 2003. Adaptive logarithmic mapping for displaying high contrast scenes. Proc. of Eurographics 22, 3, 419-426.
7. FATTAL, R., LISCHINSKI, D., AND WERMAN, M. 2002. Gradient domain high dynamic range compression. In Proc. of SIGGRAPH, 249-256.
8. HACHISUKA, T., OGAKI, S., AND JENSEN, H. W. 2008. Progressive photon mapping. In Proc. of SIGGRAPH Asia, 130:1-130:8.
9. HERZOG, R., MYSZKOWSKI, K., AND SEIDEL, H.-P. 2009. Anisotropic radiance-cache splatting for efficiently computing high-quality GI with lightcuts. Proc. of Eurographics, 259-268.
10. HERZOG, R., ČADÍK, M., AYDIN, T. O., KIM, K. I., MYSZKOWSKI, K., AND SEIDEL, H.-P. 2012. NoRM: noreference image quality metric for realistic image synthesis. Computer Graphics Forum 31, 2, 545-554.
11. HOWELL, D. C. 2007. Statistical Methods for Psychology, 6th edition ed. Thomas Wadsworth. ITU-R-BT.500-11, 2002. Methodology for the subjective assessment of the quality of television pictures. ITU-T-P.910. 2008. Subjective audiovisual quality assessment methods for multimedia applications. Tech. rep.
12. JENSEN, H. W. 2001. Realistic Image Synthesis Using Photon Mapping. AK, Peters.
13. KELLER, A. 1997. Instant radiosity. In Proc. of SIGGRAPH, 49-56.
14. KŘIVÁNEK, J., GAUTRON, P., PATTANAIK, S., AND BOUATOUCH, K. 2005. Radiance caching for efficient global illumination computation. IEEE TVCG 11, 5, 550-561.
15. LARSON, E. C., AND CHANDLER, D. M. 2010. Most apparent distortion: full-reference image quality assessment and the role of strategy. J. Electron. Imaging 19, 1, 011006:1-21.
16. LEDDA, P., CHALMERS, A., TROSCIANKO, T., AND SEETZEN, H. 2005. Evaluation of tone mapping operators using a high dynamic range display. Proc. of SIGGRAPH 24, 3, 640-648.
17. LIN, W., AND KUO, C.-C. J. 2011. Perceptual visual quality metrics: A survey. JVCIR, 297-312.
18. LUBIN, J. 1995. Vision Models for Target Detection and Recognition. World Scientific, ch. A Visual Discrimination Model for Imaging System Design and Evaluation, 245-283.
19. MANTIUK, R., DALY, S., MYSZKOWSKI, K., AND SEIDEL, H.- P. 2005. Predicting visible differences in high dynamic range images-model and its calibration. In SPIE HVEI X.
20. MANTIUK, R., MYSZKOWSKI, K., AND SEIDEL, H.-P. 2006. A perceptual framework for contrast processing of high dynamic range images. ACM Trans. on Applied Perception 3, 3, 286-308.
21. MANTIUK, R., DALY, S., AND KEROFSKY, L. 2008. Display adaptive tone mapping. In Proc. of SIGGRAPH, vol. 27(3), #68.
22. MANTIUK, R., KIM, K. J., REMPEL, A. G., AND HEIDRICH, W. 2011. HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions. In Proc. of SIGGRAPH, #40.
23. PEDERSEN, M., AND HARDEBERG, JON, Y. 2011. Full-Reference Image Quality Metrics: Classification and Evaluation. Foundations and Trends in Computer Graphics and Vision 7, 1, 1-80.
24. PONOMARENKO, N., LUKIN, V., ZELENSKY, A., EGIAZARIAN, K., CARLI, M., AND BATTISTI, F. 2009. TID2008-A database for evaluation of full-reference visual quality assessment metrics. Advances of Modern Radioelectronics 10, 30-45.
25. RAMANARAYANAN, G., FERWERDA, J., WALTER, B., AND BALA, K. 2007. Visual equivalence: towards a new standard for image fidelity. In Proc. of SIGGRAPH, #76.
26. RAMASUBRAMANIAN, M., PATTANAIK, S., AND GREENBERG, D. 1999. A perceptually based physical error metric for realistic image synthesis. In Proc. of SIGGRAPH, 73-82.
27. REINHARD, E., STARK, M. M., SHIRLEY, P., AND FERWERDA, J. A. 2002. Photographic tone reproduction for digital images. In Proc. of SIGGRAPH, 267-276.
28. RUSHMEIER, H., WARD, G., PIATKO, C., SANDERS, P., AND RUST, B. 1995. Comparing real and synthetic images: some ideas about metrics. In Rendering Techniques '95, 82-91.
29. SALKIND, N., Ed. 2007. Encyclopedia of measurement and statistics. A Sage reference publication. SAGE Thousand Oaks.
30. SHEIKH, H., SABIR, M., AND BOVIK, A. 2006. A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. on Image Processing 15, 11, 3440-3451.
31. WALTER, B., FERNANDEZ, S., ARBREE, A., BALA, K., DONIKIAN, M., AND GREENBERG, D. 2005. Lightcuts: A scalable approach to illumination. Proc. of SIGGRAPH, 1098-1107.
32. WANG, Z., AND BOVIK, A. C. 2006. Modern Image Quality Assessment. Morgan & Claypool Publishers.
33. WANG, Z., SIMONCELLI, E. P., AND BOVIK, A. C. 2003. Multiscale structural similarity for image quality assessment. In Proc. IEEE Asilomar Conf. on Signals, Systems & Comp., 1398-1402.
34. WARD, G. J., RUBINSTEIN, F. M., AND CLEAR, R. D. 1988. A ray tracing solution for diffuse interreflection. In Proc. of SIGGRAPH, 85-92.
35. WU, H., AND RAO, K. 2005. Digital Video Image Quality and Perceptual Coding. CRC Press.
36. ZHANG, X., AND WANDELL, B. A. 1998. Color image fidelity metrics evaluated using image distortion maps. Signal Proc. 70, 3, 201-214.