Objective video quality assessment-towards large scale video database enhanced model development

IEICE Transactions on Communications

Volumn E98B, Issue 1, 2015, Pages 2-11

  Barkowsky, Marcus ^a   Masala, Enrico ^a   Vanwallendael, Glenn ^b   Brunnstrom, Kjell ^d,e   Staelens, Nicolas ^c   Le Callet, Patrick ^a  

^a UNIVERSITÉ DE NANTES   (France)

^b POLITECNICO DI TORINO   (Italy)

^c GHENT UNIVERSITY   (Belgium)

^d ACREO AB   (Sweden)

^e MID SWEDEN UNIVERSITY   (Sweden)

Author keywords

Large scale database; Reproducible research; Video quality assessment

Indexed keywords

ALGORITHMS; BINARY SEQUENCES; SUBJECTIVE TESTING; VIDEO RECORDING; VIDEO SIGNAL PROCESSING;

LARGE-SCALE DATABASE; OBJECTIVE VIDEO QUALITY; OBJECTIVE VIDEO QUALITY MEASUREMENT; REPRODUCIBLE RESEARCH; VERIFICATION-AND-VALIDATION; VIDEO QUALITY ASSESSMENT; VIDEO QUALITY EXPERTS GROUPS; VIDEO QUALITY MEASUREMENT;

DATABASE SYSTEMS;

EID: 84924690334     PISSN: 09168516     EISSN: 17451345     Source Type: Journal    
DOI: 10.1587/transcom.E98.B.2     Document Type: Article

References (48)

1. F. Campbell, R. Carpenter, and J. Levinson, "Visibility of aperiodic patterns compared with that of sinusoidal gratings," The Journal of Physiology, vol.204, pp.283-298, 1969.
2. A.B. Watson and J. Ahumada, "A standard model for foveal detection of spatial contrast," Journal of Vision, vol.5, pp.717-740, 2006.
3. Q. Huynh-Thu and M. Ghanbari, "Impact of jitter and jerkiness on perceived video quality," Proc. Workshop on Video Processing and Quality Metrics, 2006.
4. J. Li, M. Barkowsky, and P. Le Callet, "The influence of relative disparity and planar motion velocity on visual discomfort of stereoscopic videos," International Workshop on Quality of Multimedia Experience, pp.155-160, 2011.
5. NTIA/ITS, "A3: Objective Video Quality Measurement Using a Peak-Signal-to-Noise-Ratio (PSNR) Full Reference Technique," ATIS T1.TR.PP.74-2001, 2001.
6. Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Process., vol.13, pp.600-612, 2004.
7. R. Dosselmann and X. Yang, "A comprehensive assessment of the structural similarity index," Signal, Image and Video Processing, vol.5, pp.81-91, 2011.
8. S. Chikkerur, V. Sundaram, M. Reisslein, and L. Karam, "Objective video quality assessment methods: A classification, review, and performance comparison," IEEE Trans. Broadcast., vol.57, pp.165-182, 2011.
9. M. Barkowsky, I. Sedano, K. Brunnström, M. Leszczuk, and N. Staelens, "Hybrid video quality prediction: Reviewing video quality measurement for widening application scope," Multimedia Tools and Applications, pp.1-21, 2014.
10. K. Fliegel and C. Timmerer, eds., "WG4 Databases White Paper v1.5: QUALINET Multimedia Database enabling QoE Evaluations and Benchmarking," http://dbq-wiki.multimediatech.cz, March 2013.
11. I.T.U. ITU-T, "Objective perceptual assessment of video quality: Full reference television," Technical papers and tutorials, 2004.
12. I.S.G. 9, "Final Report of VQEG's Multimedia Phase I Validation Test," TD 923, 2008.
13. G. Cermak, L. Thorpe, and M. Pinson, "Test plan for evaluation of video quality models for use with high definition TV content," Video Quality Experts Group (VQEG), http://www.its.bldrdoc.gov/media/5871/vqeg hdtv testplan v3 1.doc, 2009.
14. J. Berger, C. Lee, D. Hands, N. Staelens, Y. Dhondt, and M. Pinson, eds., "Hybrid perceptual bitstream test plan 2.11," Video Quality Experts Group (VQEG), http://www.its.bldrdoc.gov/media/36068/VQEG hybrid testplan v2 11.doc, 2012.
15. M. Pinson and S. Wolf, "An objective method for combining multiple subjective data sets," SPIE Video Communications and Image Processing Conference, pp.8-11, 2003.
16. Y. Pitrey, U. Engelke, M. Barkowsky, R. Pépion, and P. Le Callet, "Aligning subjective tests using a low cost common set," QoE for Multimedia Content Sharing, Lisbonne, Portugal, 2011.
17. M. Pinson, M. Barkowsky, and P. Le Callet, "Selecting scenes for 2D and 3D subjective video quality tests," EURASIP Journal on Image and Video Processing, vol.2013, p.1, 2013.
18. H. Schwarz, D. Marpe, and T. Wiegand, "Overview of the scalable video coding extension of the H.264/AVC standard," IEEE Trans. Circuits Syst. Video Technol., vol.17, no.9, pp.1103-1120, Sept. 2007.
19. J.-R. Ohm and G.J. Sullivan, "High efficiency video coding: The next frontier in video compression," IEEE Signal Process. Mag., vol.30, no.1, pp.152-158, Jan. 2013.
20. F. Battisti, M. Carli, and A. Neri, "No reference quality assessment for MPEG video delivery over IP," EURASIP Journal on Image and Video Processing, vol.2014, no.1, pp.1-19, 2014.
21. J. Song and F. Yang, "Real-time quality monitoring for networked H.264/AVC video streaming," Proc. 3rd International Conference on Multimedia Technology (ICMT 2013), ed. A.A. Farag, J. Yang, and F. Jiao, Lecture Notes in Electrical Engineering, vol.278, pp.237-245, Springer Berlin Heidelberg, 2014.
22. M. Alreshoodi and J. Woods, "Survey on QoE\QoS correlation models for multimedia services," International Journal of Distributed and Parallel Systems (IJDPS), vol.4, no.3, pp.53-72, May 2013.
23. N. Staelens, D. Deschrijver, E. Vladislavleva, B. Vermeulen, T. Dhaene, and P. Demeester, "Constructing a no-reference H.264/AVC bitstream-based video quality metric using genetic programming-based symbolic regression," IEEE Trans. Circuits Syst. Video Technol., vol.23, no.8, pp.1322-1333, Aug. 2013.
24. J. Joskowicz, R. Sotelo, and J. Lopez Arado, "Comparison of parametric models for video quality estimation: Towards a general model," IEEE International Symposium onBroadband Multimedia Systems and Broadcasting (BMSB), pp.1-7, June 2012.
25. S. Argyropoulos, M.N. Garcia, M. Salem, D. List, R. Schleicher, and A. Raake, "Objective no-reference prediction of saliency changes in the presence of packet loss," Seventh International Workshop on Video Processing and Quality Metrics for Consumer Electronics (VPQM-13), pp.6-11, Jan. 2013.
26. N. Staelens, G. Van Wallendael, K. Crombecq, N. Vercammen, J. De Cock, B. Vermeulen, R. Van De Walle, T. Dhaene, and P. Demeester, "No-reference bitstream-based visual quality impairment detection for high definition H.264/AVC encoded video sequences," IEEE Trans. Broadcast., vol.58, no.2, pp.187-199, June 2012.
27. A. Reibman, S. Kanumuri, V. Vaishampayan, and P. Cosman, "Visibility of individual packet losses in MPEG-2 video," International Conference on Image Processing (ICIP), pp.171-174, Oct. 2004.
28. A. Reibman, V. Vaishampayan, and Y. Sermadevi, "Quality monitoring of video over a packet network," IEEE Trans. Multimed., vol.6, no.2, pp.327-334, April 2004.
29. Y. Liang, J. Apostolopoulos, and B. Girod, "Analysis of packet loss for compressed video: Does burst-length matter?," IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), vol.5, pp.684-687, April 2003.
30. M.N. Garcia, S. Argyropoulos, N. Staelens, M. Naccari, M. Rios-Quintero, and A. Raake, "Video streaming: Advanced concepts, applications and methods," in Quality of Experience, ed. S. Möller and A. Raake, T-Labs Series in Telecommunication Services, pp.277-297, Springer International Publishing, 2014.
31. P. Ameigeiras, A. Azcona-Rivas, J. Navarro-Ortiz, J. Ramos-Munoz, and J. Lopez-Soler, "A simple model for predicting the number and duration of rebuffering events for YouTube flows," IEEE Commun. Lett., vol.16, no.2, pp.278-280, Feb. 2012.
32. A. Gouta, C. Hong, D. Hong, A.M. Kermarrec, and Y. Lelouedec, "Large scale analysis of HTTP adaptive streaming in mobile networks," IEEE 14th International Symposium and Workshops on a World of Wireless, Mobile and Multimedia Networks (WoWMoM), pp.1-10, June 2013.
33. M. Seufert, M. Slanina, S. Egger, and M. Kottkamp, "To pool or not to pool": A comparison of temporal pooling methods for HTTP adaptive video streaming," Fifth International Workshop on Quality of Multimedia Experience (QoMEX), pp.52-57, July 2013.
34. N. Staelens, S. Moens,W. Van Den Broeck, I.Mariën, B. Vermeulen, P. Lambert, R. Van De Walle, and P. Demeester, "Assessing quality of experience of IPTV and Video on Demand services in real-life environments," IEEE Trans. Broadcast., vol.56, no.4, pp.458-466, Dec. 2010.
35. CRAWDAD project, "A community resource for archiving wireless data at Dartmouth," http://crawdad.cs.dartmouth.edu, 2014.
36. M. Ellis, C. Perkins, and D.P. Pezaros, "End-to-end and networkinternal measurements on real-time traffic to residential users," Proc. ACM Multimedia Systems, pp.111-116, San Jose, CA, USA, Feb. 2011.
37. E. Elliot, "Estimates on error rates for codes on burst-noise channels," Bell Syst. Tech. J., vol.42, pp.1977-1997, Sept. 1963.
38. S.A. Khayam and H. Radha, "Markov-based modeling of wireless local area networks," Proc. ACMIntl.Workshop on Modeling Analysis and Simulation of Wireless and Mobile Systems (MSWIN), pp.100-107, 2003.
39. JCT-VC, "HEVC test model (HM) reference software, v. 12.1," Nov. 2013.
40. G.J. Sullivan, J.M. Boyce, Y. Chen, J.-R. Ohm, C.A. Segall, and A. Vetro, "Standardized extensions of High Efficiency Video Coding (HEVC)," IEEE J. Sel. Top. Signal Process., vol.7, pp.1001-1016, Dec. 2013.
41. A. Inc., "Best Practices for Creating and Deploying HTTP Live Streaming Media for the iPhone and iPad," TN2224, April 2014.
42. Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Process., vol.13, no.4, pp.600-612, April 2004.
43. H.R. Sheikh and A.C. Bovik, "Image information and visual quality," IEEE Trans. Image Process., vol.15, no.2, pp.430-444, Feb. 2006.
44. M. Pinson and S. Wolf, "A new standardized method for objectively measuring video quality," IEEE Trans. Broadcast., vol.50, no.3, pp.312-322, Sept. 2004.
45. A.P. Hekstraa, J.G. Beerends, D. Ledermann, F.E. de Caluwe, S. Kohler, R.H. Koenen, S. Rihs, M. Ehrsam, and D. Schlauss, "PVQM-A perceptual video quality measure," Signal Processing: Image Communication, vol.17, no.10, pp.781-798, Nov. 2002.
46. P. Hanhart and R. Hahling, "Video quality measurement tool (VQMT)," http://mmspg.epfl.ch/vqmt, Sept. 2013.
47. A. Reibman, "A strategy to jointly test image quality estimators subjectively," Image Processing (ICIP), 2012 19th IEEE International Conference on, pp.1501-1504, 2012.
48. M. Leszczuk, L. Janowski, and M. Barkowsky, "Freely available large-scale video quality assessment database in full-HD resolution with H.264 coding," Proc. IEEE Globecom 2013, p.1, Atlanta, Etats-Unis, 2013.