Performance measures for object detection evaluation

Pattern Recognition Letters

Volumn 31, Issue 10, 2010, Pages 1128-1137

  Özdemir, Bahadir ^a   Aksoy, Selim ^a   Eckert, Sandra ^b   Pesaresi, Martino ^b   Ehrlich, Daniele ^b  

^a BILKENT UNIVERSITY   (Turkey)

^b JOINT RESEARCH CENTRE   (Italy)

Author keywords

Multi criteria ranking; Object detection; Object matching; Performance evaluation; Shape modeling

Indexed keywords

BUILDING DETECTION; MULTI-CRITERIA RANKING; OBJECT DETECTION; OBJECT DETECTION ALGORITHMS; OBJECT MATCHING; OBJECT SHAPE; PERFORMANCE INDICATORS; PERFORMANCE MEASURE; QUANTITATIVE EVALUATION; SHAPE MODELING; VISUAL INSPECTION;

BENCHMARKING; VISUAL COMMUNICATION;

OBJECT RECOGNITION;

EID: 77953173241     PISSN: 01678655     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patrec.2009.10.016     Document Type: Article

References (30)

1. Akcay H.G., and Aksoy S. Automatic detection of geospatial objects using multiple hierarchical segmentations. IEEE Trans. Geosci. Remote Sens. 46 7 (2008) 2097-2111
2. Aksoy, S., Ozdemir, B., Eckert, S., Kayitakire, F., Pesaresi, M., Aytekin, O., Borel, C.C., Cech, J., Christophe, E., Duzgun, S., Erener, A., Ertugay, K., Hussain, E., Inglada, J., Lefevre, S., Ok, O., San, D.K., Sara, R., Shan, J., Soman, J., Ulusoy, I., Witz, R., 2008. Performance evaluation of building detection and digital surface model extraction algorithms: Outcomes of the PRRS 2008 algorithm performance contest. In: Proc. 5th IAPR Workshop on Pattern Recognition in Remote Sensing, Tampa, Florida.
3. Aksoy, S., Ye, M., Schauf, M.L., Song, M., Wang, Y., Haralick, R.M., Parker, J.R., Pivovarov, J., Royko, D., Sun, C., Farneback, G., 2000. Algorithm performance contest. In: Proc. 15th IAPR Internat. Conf. on Pattern Recognition, vol. IV, Barcelona, Spain, pp. 870-876.
4. Alparone L., Wald L., Chanussot J., Thomas C., Gamba P., and Bruce L.M. Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data-fusion contest. IEEE Trans. Geosci. Remote Sens. 45 10 (2007) 3012-3021
5. Asuncion, A., Newman, D.J., 2007. UCI machine learning repository. .
6. Bruzzone, L., Persello, C., 2008. A novel protocol for accuracy assessment in classification of very high resolution multispectral and SAR images. In: Proc. IEEE Internat. Geoscience and Remote Sensing Symposium, Boston, Massachusetts.
7. In: Christensen H.I., and Phillips P.J. (Eds). Empirical Evaluation Methods in Computer Vision (2002), World Scientific Press, Singapore
8. Everingham, M., Gool, L.V., Williams, C.K.I., Winn, J., Zisserman, A., 2008. The PASCAL Visual Object Classes Challenge 2008 Results. .
9. Flynn P.J., Hoover A., and Phillips P.J. Special issue on empirical evaluation of computer vision algorithms. Computer Vision and Image Understanding 84 1 (2001) 1-4
10. Haralick R.M. Propagating covariance in computer vision. Internat. J. Pattern Recognition Artif. Intell. 10 5 (1996) 561-572
11. Hoover A., Jean-Baptiste G., Jiang X., Flynn P.J., Bunke H., Goldgof D.B., Bowyer K., Eggert D.W., Fitzgibbon A., and Fisher R.B. An experimental comparison of range image segmentation algorithms. IEEE Trans. Pattern Anal. Machine Intell. 18 7 (1996) 673-689
12. Huang, Q., Dom, B., 1995. Quantitative methods of evaluating image segmentation. In: IEEE Internat. Conf. on Image Processing, vol. 3, Washington, DC, pp. 53-56.
13. Jiang X., Marti C., Irniger C., and Bunke H. Distance measures for image segmentation evaluation. EURASIP J. Appl. Signal Process. (2006) 1-10 (Article ID 35909)
14. Levina, E., Bickel, P., 2001. The earth mover's distance is the mallows distance: Some insights from statistics. In: Proc. IEEE Internat. Conf. on Computer Vision, vol. 2. Vancouver, British Columbia, Canada, pp. 251-256.
15. Liu G., and Haralick R.M. Optimal matching problem in detection and recognition performance evaluation. Pattern Recognition 35 10 (2002) 2125-2139
16. Liu X., Kanungo T., and Haralick R.M. On the use of error propagation for statistical validation of computer vision software. IEEE Trans. Pattern Anal. Machine Intell. 27 10 (2005) 1603-1614
17. Mallows C.L. A note on asymptotic joint normality. Ann. Math. Statist. 43 2 (1972) 508-515
18. Mariano, V. Y., Min, J., Park, J.-H., Kasturi, R., Mihalcik, D., Li, H., Doermann, D., Drayer, T., 2002. Performance evaluation of object detection algorithms. In: Proc. 16th IAPR Internat. Conf. on Pattern Recognition, vol. 3, Quebec, Canada, pp. 965-969.
19. Martin D.R., Fowlkes C.C., and Malik J. Learning to detect natural image boundaries using local brightness, color, and texture cues. IEEE Trans. Pattern Anal. Machine Intell. 26 5 (2004) 530-549
20. Meur Y.L., Vignolle J.-M., and Chanussot J. Practical use of receiver operating characteristic analysis to assess the performances of defect detection algorithms. J. Electron. Imaging 17 3 (2008)
21. Munkres J. Algorithms for the assignment and transportation problems. J. Soc. Ind. Appl. Math. 5 1 (1957) 32-38
22. Ortiz A., and Oliver G. On the use of the overlapping area matrix for image segmentation evaluation: A survey and new performance measures. Pattern Recognition Lett. 27 16 (2006) 1916-1926
23. Pacifici F., Frate F.D., Emery W.J., Gamba P., and Chanussot J. Urban mapping using coarse SAR and optical data: Outcome of the 2007 GRSS data fusion contest. IEEE Geosci. Remote Sens. Lett. 5 3 (2008) 331-335
24. Patil G.P., and Taillie C. Multiple indicators, partially ordered sets, and linear extensions: Multi-criterion ranking and prioritization. Environ. Ecol. Statist. 11 2 (2004) 199-228
25. Phillips P.J., and Bowyer K.W. Empirical evaluation of computer vision algorithms. IEEE Trans. Pattern Anal. Machine Intell. 21 4 (1999) 289-290
26. Rubner Y., Tomasi C., and Guibas L.J. The earth mover's distance as a metric for image retrieval. Internat. J. Comput. Vision 40 2 (2000) 99-121
27. Smeaton, A.F., Over, P., Kraaij, W., 2006. Evaluation campaigns and TRECVid. In: Proc. 8th ACM Internat. Workshop on Multimedia Information Retrieval. Santa Barbara, California, pp. 321-330.
28. Thacker N.A., Clark A.F., Barron J.L., Beveridge J.R., Courtney P., Crum W.R., Ramesh V., and Clark C. Performance characterization in computer vision: A guide to best practices. Computer Vision and Image Understanding 109 3 (2008) 305-334
29. Wirth M., Fraschini M., Masek M., and Bruynooghe M. Performance evaluation in image processing. EURASIP J. Appl. Signal Process. (2006) 1-3 (Article ID 45742)
30. Zhang D., and Lu G. Review of shape representation and description techniques. Pattern Recognition 37 1 (2004) 1-19