On evaluating video object segmentation quality: A perceptually driven objective metric

IEEE Journal on Selected Topics in Signal Processing

Volumn 3, Issue 2, 2009, Pages 319-335

  Gelasca, Elisa Drelie ^a   Ebrahimi, Touradj ^b  

^a University of California   (United States)

^b EPFL   (Switzerland)

Author keywords

Foreground background extraction; Mixed reality; Objective evaluation; Perceptual metric; Psychophysical tests; Segmentation; Subjective quality assessment; Video object; Video object compression; Video surveillance

Indexed keywords

APPLICATIONS; COLOR PHOTOGRAPHY; IMAGE CODING; IMAGE COMPRESSION; IMAGE SEGMENTATION; IMAGE SENSORS; MOTION COMPENSATION; MOTION ESTIMATION; MOTION PICTURE EXPERTS GROUP STANDARDS; SECURITY SYSTEMS; VIDEO AMPLIFIERS; VIDEO RECORDING; VIDEO SIGNAL PROCESSING; VIRTUAL REALITY; VISUAL COMMUNICATION;

FOREGROUND/BACKGROUND EXTRACTION; MIXED REALITY; OBJECTIVE EVALUATION; PERCEPTUAL METRIC; PSYCHOPHYSICAL TESTS; SEGMENTATION; SUBJECTIVE QUALITY ASSESSMENT; VIDEO OBJECT; VIDEO OBJECT COMPRESSION; VIDEO SURVEILLANCE;

QUALITY CONTROL;

EID: 63649160736     PISSN: 19324553     EISSN: None     Source Type: Journal    
DOI: 10.1109/JSTSP.2009.2015067     Document Type: Article

References (42)

1. M. Sonka, V. Hlavic, and R. Boyle, Image Processing, Analysis and Machine Vision, 2nd ed. Tampa, FL: Thomson, 1999.
2. T. N. Pappas, J. Chen, and D. Depalov, "Learning perception," OE Mag., vol. 5, pp. 18-20, Oct. 2005.
3. F. C. Pereira and T. Ebrahimi, The MPEG-4 Book. Upper Saddle River, NJ: Prentice-Hall, 2002.
4. C. Kim and J. Hwang, "Fast and automatic video object segmentation and tracking for content-based applications," IEEE Trans. Circuits Syst. Video Technol., vol. 12, no. 2, pp. 122-129, Feb. 2002.
5. A. R. J. François and G. G. Medioni, "Adaptive color background modeling for real-time segmentation of video streams," Int. Imag. Sci., Syst., Technol., pp. 227-232, 1999.
6. T. Horprasert, D. Harwood, and L. S. Davis, "A statistical approach for real-time robust background substraction and shadow detection," in Proc. IEEE ICCV Frame Rate Workshop, 1999, pp. 1-19.
7. J. Shen, "Motion detection in color image sequence and shadow elimination," in Vis. Commun. Image Process., Jan. 2004, pp. 731-740.
8. S. Jabri, Z. Duric, H.Wechsler, and A. Rosenfeld, "Detection and location of people in video images using adaptive fusion of color and edge information," in Int. Conf. Pattern Recognition (ICPR), Sep. 2000, pp. 627-630.
9. S. J. McKenna, S. Jabri, Z. Duric, A. Rosenfeld, and H. Wechsler, "Tracking groups of people," Comput.Vis. Image Understanding, vol. 80, pp. 42-56, 2000.
10. M. Farias, "No-reference and reduced reference video quality metrics: New contributions," Ph.D. dissertation, Univ. of Santa Barbara, CA, Aug. 2004.
11. S. E. Maxwell and H. Delaney, Designing Experiments and Analyzing Data: A Model Comparison Perspective. Mahwah, NJ: Lawrence Erlbaum Associates, 1999.
12. E. D. Gelasca, T. Ebrahimi, M. Farias, and S. Mitra, "Impact of topology changes in video segmentation evaluation," in Proc. IEEE Workshop Image Anal. Multimedia Interactive Services, Apr. 2004, CD-ROM.
13. E. D. Gelasca, T. Ebrahimi, M. Farias, M. Carli, and S. Mitra, "Towards perceptually driven segmentation evaluation metrics," in Proc. IEEE CVPR Workshop (Percept. Org. Comput. Vis.), Jun. 2004, p..
14. E. D. Gelasca, T. Ebrahimi, M. Farias, C. Marco, and S. Mitra, "Annoyance of spatio-temporal artifacts in segmentation quality assessment," in Proc. IEEE Int. Conf. Image Process., Oct. 2004, pp. 345-348.
15. A. Cavallaro, E. D. Gelasca, and T. Ebrahimi, "Objective evaluation of segmentation quality using spatio-temporal context," in Proc. IEEE Int. Conf. Image Process., Rochester, NY, Sep. 22-25, 2002, pp. 301-304.
16. E. Drelie, "Full-reference objective quality metrics for video watermarking, video segmentation and 3D model watermarking," Ph.D. dissertation, EPFL, Lausanne, Switzerland, 2005.
17. M. Heath, S. Sarkar, T. Sanocki, and K. Bowyer, "A robust visual method for assessing the relative performance of edge detection algorithms," Trans. Pattern Anal. Mach. Intell., vol. 19, no. 12, pp. 1338-1359, Dec. 1997.
18. D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," in Proc. ICCV-01 Vancouver, BC, Canada, Jul. 7-14, 2001, vol. 2, pp. 416-425.
19. "Compare Your Segmentation Algorithm to the Cost 211 qam," [Online]. Available: http://www.iva.cs.tut.fi/COST211/Call/Call.htm
20. P. Correia and F. Pereira, "Objective evaluation of video segmentation quality," IEEE Trans. Image Process., vol. 12, no. 2, pp. 186-200, Feb. 2003.
21. K. McKoen, R. Navarro-Prieto, E. Durucan, B. Duc, F. Ziliani, and T. Ebrahimi, "Evaluation of segmentation methods for surveillance applications," in Proc. EUSIPCO, Sep. 2000, pp. 1045-1048.
22. P. Villegas and X. Marichal, "Perceptually-weighted evaluation criteria for segmentation masks in video sequences," IEEE Trans. Image Process., vol. 13, no. 8, pp. 1092-1103, Aug. 2004.
23. M. Wollborn and R. Mech, "Refined procedure for objective evaluation of video object generation algorithms," in Proc. 43rd MPEG Meeting, Tokyo, Japan, 1998, ISO/IECJTCI/SC29/WG11 M3448.
24. R. Mech and F. Marques, "Objective evaluation criteria for 2-D shape estimation results of moving objects," EURASIP J. Appl. Signal Process., vol. 4, pp. 401-409, Apr. 2002.
25. C. Erdem and B. Sankur, "Performance evaluation metrics for object-based video segmentation," in Proc. X Eur. Signal Process. Conf., Tampere, Finland, 2000, vol. 2, pp. 917-920.
26. Y. J. Zhang, "A survey on evaluation methods for image segmentation," Pattern Recognition, vol. 29, pp. 1335-1346, 1996.
27. X. Marichal and P. Villegas, "Objective evaluation of segmentation masks in video sequences," in Proc. X Eur. Signal Process. Conf., Tampere, Finland, 2000, pp. 2139-2196.
28. J. Nascimento and J. S. Marques, "New performance evaluation metrics for object detection algorithms," in Proc. 6th Int. Workshop Perform. Eval. Tracking Surveill. (PETS, ECCV), Prague, Czech Republic, May 2004, pp. 7-14.
29. J. Black, T. Ellis, and P. Rosin, "A novel method for video tracking performance evaluation," in Proc. Joint IEEE Int. Workshop Vis. Surveill. Perform. Eval Tracking Surveill., Oct. 2003, pp. 125-132.
30. R. J. Oliveira, P. C. Ribeiro, J. S. Marques, and J. M. Lemos, "A video system for urban surveillance: Function integration and evaluation," in Proc. Int. Workshop Image Anal. Multimedia Interactive Syst., 2004, CD-ROM.
31. F. Oberti, E. Stringa, and G. Vernazza, "Performance evaluation criterion for characterizing video surveillance systems," Real Time Imaging, vol. 7, pp. 457-471, 2001.
32. A. Senior, A. Hampaput, Y. Tian, L. Brown, S. Pankanti, and R. Bolle, "Appearance models for occlusion handling," in Proc. 2nd IEEEWorkshop Perform. Eval. Tracking Surveill., 2000, CD-ROM.
33. "IEEE and 2005 and Winter Vision, Performance Evaluation of Tracking and Surveillance (PETS)." [Online]. Available: http:// pets2005.visualsurveillance.org/
34. "EU CAVIAR Project Caviar Test Case Scenarios." [Online]. Available: http://homepages.inf.ed.ac.uk/rbf/CAVIAR,
35. N. Otsu, "A threshold selection method from gray-level histograms," IEEE Trans. Syst., Man, Cybern., vol. SMC-9, no. 1, pp. 62-66, Jan. 1979.
36. "Subjective video quality assessment methods for multimedia applications recommendation," Int. Telecomm. Union, Geneva, Switzerland, 1996.
37. "Methodology for subjective assessment of the quality of television pictures recommendation," Int. Telecomm. Union, Geneva, Switzerland, BT.500-1, 2002,.
38. P. Milgram and H. Colquhoun, "A taxonomy of real and virtual world display integration," in Mixed Reality, Merging Real and Virtual Worlds, Y. Otha and H. Tamura, Eds. New York: Ohmsha-Springer, 1999.
39. X. Marichal, B. Macq, D. Douxchamps, and T. Umeda, "and art.live consortium the ART.LIVE architecture for mixed reality," in Proc. VRIC, Laval, France, Jun. 2002, pp. 19-21.
40. J. W. Senders, "Distribution of visual attention in static and dynamic displays," SPIE, Human Vis. Electron. Imaging II, vol. 3016, pp. 186-194, Feb. 1997.
41. R. Hamberg and H. de Ridder, "Time-varying image quality: Modeling the relation between instantaneous and overall quality," SMPTE J., vol. 108, pp. 802-811, 1999.
42. Y. Inazumi, Y. Horita, K. Kotani, and T. Murai, "Quality evaluation method considering time transition of coded quality," in Proc. ICIP Oct. 24-28, 1999, vol. 4, pp. 338-342.