Crowdsourcing subjective image quality evaluation

Proceedings - International Conference on Image Processing, ICIP

Volumn , Issue , 2011, Pages 3097-3100

  Ribeiro, Flavio ^a   Florencio, Dinei ^b   Nascimento, Vitor ^a  

^a UNIVERSITY OF SÃO PAULO   (Brazil)

^b MICROSOFT RESEARCH   (United States)

Author keywords

crowdsourcing; mean opinion score; mechanical turk; MOS; quality assessment; subjective quality

Indexed keywords

CROWDSOURCING; MEAN OPINION SCORES; MECHANICAL TURKS; MOS; QUALITY ASSESSMENT; SUBJECTIVE QUALITY;

IMAGE QUALITY; INTERNET;

QUALITY CONTROL;

EID: 84856283152     PISSN: 15224880     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICIP.2011.6116320     Document Type: Conference Paper

References (14)

1. "Subjective video quality assessment methods for multimedia applications," ITU-T Recommendation P.910, Apr. 2008.
2. "Methods for subjective determination of transmission quality," ITU-T Recommendation P.800, Aug. 1996.
3. "Methodology for the subjective assessment of the quality of television pictures," ITU-R Recommendation BT.500-12, Sept. 2009.
4. Z.Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, 2004.
5. H.R. Sheikh and A.C. Bovik, "Image information and visual quality," IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, 2006.
6. D.M. Chandler and S.S. Hemami, "VSNR: A wavelet-based visual signal-to-noise ratio for natural images," IEEE Trans. Image Process., vol. 16, no. 9, pp. 2284-2298, 2007. (Pubitemid 47316675)
7. A.K. Moorthy and A.C. Bovik, "Perceptually significant spatial pooling techniques for image quality assessment," in Proc. of SPIE, 2009.
8. F. Ribeiro, D. Florencio, C. Zhang, and M. Seltzer, "CrowdMOS Standalone Tools," available at http://research.microsoft.com/crowdmos/.
9. K.T. Chen, C.C. Wu, Y.C. Chang, and C.L. Lei, "A crowdsourceable QoE evaluation framework for multimedia content," in Proc. ACM Multimedia 2009. ACM, 2009, pp. 491-500.
10. F. Ribeiro, D. Florencio, C. Zhang, and M. Seltzer, "CrowdMOS: an approach for crowdsourcing mean opinion score studies," in Proc. of ICASSP, 2011.
11. "Methods for the subjective assessment of small impairments in audio systems including multichannel sound systems," ITU-R Recommendation BS.1116-1, Oct. 1997.
12. B. Efron, R. Tibshirani, and R.J. Tibshirani, An introduction to the bootstrap, Chapman & Hall/CRC, 1993.
13. A.M. van Dijk, J.B. Martens, and A.B. Watson, "Quality assessment of coded images using numerical category scaling," in Proc. of SPIE, 1995.
14. H.R. Sheikh, Z. Wang, L. Cormack, and A.C. Bovik, "LIVE image quality assessment database," 2003, http://live.ece.utexas.edu/research/ quality/.