Supervised Evaluation of Image Segmentation and Object Proposal Techniques

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 38, Issue 7, 2016, Pages 1465-1478

  Pont Tuset, Jordi ^a   Marques, Ferran ^a  

^a UNIVERSITAT POLITÈCNICA DE CATALUNYA   (Spain)

Author keywords

Image segmentation; meta measures; object proposals; supervised evaluation

Indexed keywords

ALGORITHMS;

GROUND TRUTH; META MEASURES; OBJECT PROPOSALS; PROPOSAL ALGORITHM; SEGMENTATION METHODS; SEGMENTATION RESULTS; STATE OF THE ART; SUPERVISED EVALUATION;

IMAGE SEGMENTATION;

EID: 84976520638     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2015.2481406     Document Type: Article

References (65)

1. P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2001, pp. 511-518.
2. C. Lampert, M. Blaschko, T. Hofmann, "Beyond sliding windows: Object localization by efficient subwindow search," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2008, pp. 1-8.
3. R. Girshick, J. Donahue, T. Darrell, J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2014.
4. B. Hariharan, P. Arbelez, R. Girshick, J. Malik, "Simultaneous detection and segmentation," in Proc. Eur. Conf. Comput. Vis., 2014, pp. 297-312.
5. P. Arbel-aez, J. Pont-Tuset, J. Barron, F. Marques, J. Malik, "Multiscale combinatorial grouping," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2014, pp. 328-335.
6. J. Carreira and C. Sminchisescu, "CPMC: Automatic object segmentation using constrained parametric min-cuts," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 7, pp. 1312-1328, Jul. 2012.
7. P. Arbel-aez, B. Hariharan, C. Gu, S. Gupta, L. Bourdev, J. Malik, "Semantic segmentation using regions and parts," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2012, pp. 3378-3385.
8. A. Ion, J. Carreira, C. Sminchisescu, "Image segmentation by figure-ground composition into maximal cliques," in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 2110-2117.
9. A. Levinshtein, C. Sminchisescu, S. Dickinson, "Optimal contour closure by superpixel grouping," in Proc. Eur. Conf. Comput. Vis., 2010, pp. 480-493.
10. P. Rantalankila, J. Kannala, E. Rahtu, "Generating object segmentation proposals using global and local search," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2014, pp. 2417-2424.
11. I. Endres and D. Hoiem, "Category-independent object proposals with diverse ranking," IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 2, pp. 222-234, Feb. 2014.
12. D. Hoiem, Y. Chodpathumwan, Q. Dai, "Diagnosing error in object detectors," in Proc. Eur. Conf. Comput. Vis., 2012, pp. 340-353.
13. D. Martin, "An empirical approach to grouping and segmentation," Ph.D. dissertation, EECS Dept., Univ. California, Berkeley, CA, USA, Aug. 2003.
14. P. Arbel-aez, M. Maire, C. C. Fowlkes, J. Malik, "Contour detection and hierarchical image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 5, pp. 898-916, May 2011.
15. R. Unnikrishnan, C. Pantofaru, M. Hebert, "Toward objective evaluation of image segmentation algorithms," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 6, pp. 929-944, Jun. 2007.
16. M. Meil-a, "Comparing clusterings: An axiomatic view," in Proc. 22nd Int. Conf. Mach. Learn., 2005, pp. 577-584.
17. J. Pont-Tuset and F.Marques, "Supervised assessment of segmentation hierarchies," in Proc. Eur. Conf. Comput.Vis., 2012., pp. 814-827.
18. A. Hoover, G. Jean-Baptiste, X. Jiang, P. J. Flynn, H. Bunke, D. B. Goldgof, K. Bowyer, D. W. Eggert, A. Fitzgibbon, R. B. Fisher, "An experimental comparison of range image segmentation algorithms," IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, no. 7, pp. 673-689, Jul. 1996.
19. C. Gu, J. Lim, P. Arbelaez, J. Malik, "Recognition using regions," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2009, pp. 1030-1037.
20. F. Li, J. Carreira, C. Sminchisescu, "Object recognition as ranking holistic figure-ground hypotheses," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2010, pp. 1712-1719.
21. J. S. Cardoso and L. Corte-Real, "Toward a generic evaluation of image segmentation," IEEE Trans. Image Process., vol. 14, no. 11, pp. 1773-1782, Nov. 2005.
22. X. Jiang, C. Marti, C. Irniger, H. Bunke, "Distance measures for image segmentation evaluation," EURASIP J. Appl. Signal Process., vol. 2006, pp. 1-10, 2006.
23. J. Pont-Tuset and F. Marques, "Measures and meta-measures for the supervised evaluation of image segmentation," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2013, pp. 2131-2138.
24. J. Czekanowski, "Zarys metod statystycznych w zastosowaniu do antropologii," Prace Towarzystwa Naukowego Warszawskiego, vol. 5, 1913.
25. L. R. Dice, "Measures of the amount of ecologic association between species," Ecology, vol. 26, no. 3, pp. pp. 297-302, 1945.
26. T. Sørensen, "A method of establishing groups of equal amplitude in plant sociology based on similarity of species and its application to analyses of the vegetation on Danish commons," Biologiske Skrifter, vol. 5, pp. 297-302, 1948.
27. S. Alpert, M. Galun, A. Brandt, R. Basri, "Image segmentation by probabilistic bottom-up aggregation and cue integration," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 2, pp. 315-327, Feb. 2012.
28. K. Smith, D. Gatica-Perez, J.-M. Odobez, S. Ba, "Evaluating multi-object tracking," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2005, pp. 36-43.
29. L. Leal-Taix-e, A. Milan, I. Reid, S. Roth, K. Schindler, "MOT challenge 2015: Towards a benchmark for multi-target tracking," arXiv:1504.01942, 2015.
30. K. H. Zou, S. K. Warfield, A. Bharatha, C. M. Tempany, M. R. Kaus, S. J. Haker, W. M. Wells, F. A. Jolesz, R. Kikinis, "Statistical validation of image segmentation quality based on a spatial overlap index," Acad. Radiol., vol. 11, no. 2, pp. 178-189, 2004.
31. P. Jaccard, "-Etude comparative de la distribution florale dans une portion des alpes et des jura," Bulletin de la Soci-et-e Vaudoise des Sciences Naturelles, vol. 37, pp. 547-579, 1901.
32. M. Everingham, L. V. Gool, C. K. I. Williams, J. Winn, A. Zisserman. (2010). The PASCAL visual object classes challenge 2010 Results. [Online]. Available: http://www.pascal-network. org/challenges/VOC/voc2010/workshop/index.html
33. K. McGuinness and N. E. O'Connor, "A comparative evaluation of interactive segmentation algorithms," Pattern Recog., vol. 43, pp. 434-444, 2010.
34. F. Ge, S. Wang, T. Liu, "Image-segmentation evaluation from the perspective of salient object extraction," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2006, pp. 1146-1153.
35. F. Ge, S. Wang, T. Liu, "New benchmark for image segmentation evaluation," Electron. Imaging, vol. 16, no. 3, pp. 033 011.1-16, 2007.
36. T. Malisiewicz and A. A. Efros, "Improving spatial support for objects via multiple segmentations," in Proc. Brit. Mach. Vis. Conf., 2007, pp. 55.1-55.10.
37. B. Russell, W. Freeman, A. Efros, J. Sivic, A. Zisserman, "Using multiple segmentations to discover objects and their extent in image collections," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2006, pp. 1605-1614.
38. A. Suinesiaputra, et al., "Left ventricular segmentation challenge from cardiac MRI: A collation study," in Proc. 2nd Int. Conf. Statist. Atlases Comput. Models Heart, 2012, pp. 88-97.
39. R. Margolin, L. Zelnik-Manor, A. Tal, "How to evaluate foreground maps" in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2014, pp. 248-255.
40. S. Belongie, J. Malik, J. Puzicha, "Shape matching and object recognition using shape contexts," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 4, pp. 509-522, Apr. 2002.
41. R. C. Veltkamp, "Shape matching: Similarity measures and algorithms," in Proc. Int. Conf. Shape Modeling Appl., 2001, pp. 188-197.
42. F. J. Estrada and A. D. Jepson, "Benchmarking image segmentation algorithms," Int. J. Comput. Vis., vol. 85, pp. 167-181, 2009.
43. A. Humayun, F. Li, J. M. Rehg, "RIGOR: Recycling inference in graph cuts for generating object regions," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2014, pp. 336-343.
44. J. R. R. Uijlings, K. E. A. V. d. Sande, T. Gevers, A. W. M. Smeulders, "Selective search for object recognition," Int. J. Comput. Vis., vol. 104, no. 2, pp. 154-171, 2013.
45. B. Alexe, T. Deselaers, V. Ferrari, "Measuring the objectness of image windows," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 11, pp. 2189-2202, Nov. 2012.
46. Q. Huang and B. Dom, "Quantitative methods of evaluating image segmentation," in Proc. Int. Conf. Image Process., 1995, pp. 53-56.
47. S. Dongen, "Performance criteria for graph clustering and Markov cluster experiments," Centrum voor Wiskunde en Informatica (CWI), Amsterdam, The Nederlands, Tech. Rep. INS-R0012, 2000.
48. W. Rand, "Objective criteria for the evaluation of clustering methods," J. Am. Statist. Assoc., vol. 66, no. 336, pp. 846-850, 1971.
49. G. Liu and R. Haralick, "Assignment problem in edge detection performance evaluation," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2000, pp. 26-31.
50. J. Pont-Tuset. (2014). Image segmentation evaluation and its application to object detection. Ph.D. dissertation, Universitat Polit-ecnica de Catalunya, UPC BarcelonaTech [Online]. Available: http://jponttuset.github.io
51. T. Kanungo, B. Dom, W. Niblack, D. Steele, "A fast algorithm for MDL-based multi-band image segmentation," IBM Res. Division, Tech. Rep. RJ 9754, 84640, 1994.
52. S. Nowozin, P. Gehler, C. Lampert, "On parameter learning in CRF-based approaches to object class image segmentation," in Proc. Eur. Conf. Comput. Vis., 2010, pp. 98-111.
53. D. Martin, C. Fowlkes, D. Tal, J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," in Proc. IEEE Int. Conf. Comput. Vis., 2001, pp. 416-423.
54. M. Everingham, H. Muller, B. Thomas, "Evaluating image segmentation algorithms using the Pareto front," in Proc. Eur. Conf. Comput. Vis., 2006, pp. 255-259.
55. R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, S. Susstrunk, "SLIC superpixels," EPFL, Lausanne, Switzerland, Tech. Rep. 149300, 2010.
56. R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, S. Susstrunk, "SLIC superpixels compared to state-of-the-art superpixel methods," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 11, pp. 2274-2282, Nov. 2012.
57. P. Krahenbuhl and V. Koltun, "Geodesic object proposals," in Proc. Eur. Conf. Comput. Vis., 2014, pp. 725-739.
58. Z. Ren and G. Shakhnarovich, "Image segmentation by cascaded region agglomeration," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2013, pp. 2011-2018.
59. P. F. Felzenszwalb and D. P. Huttenlocher, "Efficient graph-based image segmentation," Int. J. Comput. Vis., vol. 59, pp. 167-181, 2004.
60. J. Shi and J. Malik, "Normalized cuts and image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888-905, Aug. 2000.
61. D. Comaniciu and P. Meer, "Mean shift: A robust approach toward feature space analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 5, pp. 603-619, May 2002.
62. V. Vilaplana, F. Marques, P. Salembier, "Binary partition trees for object detection," IEEE Trans. Image Process., vol. 17, no. 11, pp. 2201-2216, Nov. 2008.
63. F. Calderero and F. Marques, "Region merging techniques using information theory statistical measures," IEEE Trans. Image Process., vol. 19, no. 6, pp. 1567-1586, Jun. 2010.
64. L. Najman and M. Schmitt, "Geodesic saliency of watershed contours and hierarchical segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, no. 12, pp. 1163-1173, Dec. 1996.
65. J. Pont-Tuset and F. Marques. (2013). SEISM: Supervised evaluation of image segmentation methods [Online]. Available: http://goo.gl/OLIXZV