Comparison of four subjective methods for image quality assessment

Computer Graphics Forum

Volumn 31, Issue 8, 2012, Pages 2478-2491

  Mantiuk, Rafał K ^a   Tomaszewska, Anna ^b   Mantiuk, Radosław ^b  

^a BANGOR UNIVERSITY   (United Kingdom)

^b WEST POMERANIAN UNIVERSITY OF TECHNOLOGY   (Poland)

Author keywords

double stimulus; image quality; pairwise comparison; quality metrics; ranking; similarity judgements; single stimulus; subjective metrics; user studies

Indexed keywords

COMPUTER GRAPHICS; DATA HANDLING; INFORMATION ANALYSIS; QUALITY CONTROL; STANDARDS;

DOUBLE STIMULUS; PAIR-WISE COMPARISON; QUALITY METRICS; RANKING; SIMILARITY JUDGEMENTS; SINGLE STIMULUS; SUBJECTIVE METRICS; USER STUDY;

IMAGE QUALITY;

EID: 84874930812     PISSN: 01677055     EISSN: 14678659     Source Type: Journal    
DOI: 10.1111/j.1467-8659.2012.03188.x     Document Type: Article

References (26)

1. Cunningham D., Wallraven C.,: Experimental Design: From User Studies to Psychophysics, vol. 2011. Taylor & Francis, London, UK, 2011.
2. De Ridder H., Majoor G.,: Numerical category scaling: An efficient method for assessing digital image coding impairments. In Proceedings of SPIE (San Jose, CA, USA, 1990), vol. 1249, p. 65.
3. Engeldrum P.,: Psychometric Scaling: A Toolkit for Imaging Systems Development. Imcotek Press, Winchester, MA, USA, 2000.
4. Ferwerda, J. A.,: Psychophysics 101: How to run perception experiments in computer graphics. In Proceedings of ACM SIGGRAPH 2008 Classes (Los Angeles, CA, USA, 2008), ACM, pp. 1-60.
5. Gulliksen H., Tucker L.,: A general procedure for obtaining paired comparisons from multiple rank orders. Psychometrika 26 (1961), 173-184.
6. Howell D. C.,: Statistical Methods for Psychology (6th edition). Thomas Wadsworth, Belmont, CA, USA, 2007.
7. ITU-R.R ec.BT.500-11: Methodology for the subjective assessment of the quality for television pictures, 2002.
8. ITU-T.R ec.P.910: Subjective audiovisual quality assessment methods for multimedia applications, 2008.
9. Keelan B. W.,: ISO 20462: A psychophysical image quality measurement standard. In Proceedings of the SPIE (San Jose, CA, USA, 2003), vol. 5294, pp. 181-189.
10. Ledda P., Chalmers A., Troscianko T., Seetzen H.,: Evaluation of tone mapping operators using a high dynamic range display. ACM Transactions on Graphics 24, 3 (2005), 640-648.
11. Mantiuk R., Mantiuk R., Tomaszewska A., Heidrich W.,: Color correction for tone mapping. Computer Graphics Forum 28, 2 (2009), 193-202.
12. Pellacini F., Ferwerda J. A., Greenberg D. P.,: Toward a psychophysically-based light reflection model for image synthesis. Proceedings of SIGGRAPH (New Orleans, LA, USA, 2000), vol. 2000, pp. 55-64.
13. Pedersen M., HardebergJon Y.,: Full-reference image quality metrics: Classification and evaluation. Foundations and Trends in Computer Graphics and Vision 7, 1 (2011), 1-80.
14. Ponomarenko N., Lukin V., Zelensky A., Egiazarian K., Carli M., Battisti F.,: TID2008-A database for evaluation of full-reference visual quality assessment metrics. Advances of Modern Radioelectronics 10 (2009), 30-45.
15. Rubinstein M., Gutierrez D., Sorkine O., Shamir A.,: A comparative study of image retargeting. ACM Transactions on Graphics 29, 6 (2010), 160.
16. Redi J., Liu H., Alers H., Zunino R., Heynderickx I.,: Comparing subjective image quality measurement methods for the creation of public databases. In Proceedings of the SPIE (San Jose, CA, USA, 2010), vol. 7529, pp. 752903-1-752903-11.
17. Silverstein D., Farrell J.,: Efficient method for paired comparison. Journal of Electronic Imaging 10 (2001), 394-398.
18. Sheikh H., Sabir M., Bovik A.,: A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Transactions on Image Processing 15, 11 (2006), 3441-3452.
19. Sundstedt V., Whitton M., Bloj M.,: The whys, how tos and pitfals of user studies. In Proceedings of ACM SIGGRAPH 2009 Courses (New Orleans, LA, USA, 2009).
20. Tominaga T., Hayashi T., Okamoto J., Takahashi A.,: Performance comparisons of subjective quality assessment methods for mobile video. In Proceedings of the International Workshop on Quality of Multimedia Experience (QoMEX) (Trondheim, Norway, 2010), pp. 82-87.
21. Torgerson W. S.,: Theory and Methods of Scaling. Wiley, New York, NY, USA, 1985.
22. Van Dijk A. M., Martens J. B., Watson A. B.,: Quality assessment of coded images using numerical category scaling. In Proceedings of the SPIE (San Jose, CA, USA, 1995), vol. 2451, pp. 90-101.
23. ČAdík M., Wimmer M., Neumann L., Artusi A.,: Evaluation of HDR tone mapping methods using essential perceptual attributes. Computers & Graphics 32, 3 (2008), 330-349.
24. Wang Z., Bovik A. C.,: Modern Image Quality Assessment. Morgan & Claypool Publishers, 2006.
25. Winkler, S.,: On the properties of subjective ratings in video quality experiments. In Proceedings of the 1st International Workshop on Quality of Multimedia Experience (QoMEX) (San Diego, CA, USA, 2009), pp. 139-144.
26. Yoshida A., Mantiuk R., Myszkowski K., Seidel H. -P.,: Analysis of reproducing real-world appearance on displays of varying dynamic range. Computer Graphics Forum 25, 3 (2006), 415-426.