Good practices for object-based accuracy assessment

Remote Sensing

Volumn 9, Issue 7, 2017, Pages

  Radoux, Julien ^a   Bogaert, Patrick ^a  

^a UNIVERSITÉ CATHOLIQUE DE LOUVAIN   (Belgium)

Author keywords

GEOBIA; Legend; Overall accuracy; Polygons; Quality assessment; Response design; Sampling design; Sampling unit; Spatial entities; Spatial regions

Indexed keywords

EARTH SCIENCES; REMOTE SENSING;

GEOBIA; LEGEND; OVERALL ACCURACIES; POLYGONS; QUALITY ASSESSMENT; RESPONSE DESIGNS; SAMPLING DESIGN; SAMPLING UNITS; SPATIAL ENTITIES; SPATIAL REGIONS;

QUALITY CONTROL;

EID: 85022338740     PISSN: None     EISSN: 20724292     Source Type: Journal    
DOI: 10.3390/rs9070646     Document Type: Article

References (65)

1. Stehman, S.V.; Wickham, J.D. Pixels, blocks of pixels, and polygons: Choosing a spatial unit for thematic accuracy assessment. Remote Sens. Environ. 2011, 115, 3044-3055
2. Castilla, C. We must pay more attention to rigor in accuracy assessment: Additional comment to The improvement of land cover classification by thermal remote sensing. Remote Sens. 2016, 288, 8-13
3. Strahler, A.H.; Boschetti, L.; Foody, G.M.; Friedl, M.A.; Hansen, M.C.; Herold, M.; Mayaux, P.; Morisette, J.T.; Stehman, S.V.; Woodcock, C.E. Global Land Cover Validation: Recommendations for Evaluation and Accuracy Assessment of Global Land Cover Maps. Available online: http://s3.amazonaws.com/academia. edu.documents/3461734/GOLD_25.pdf?AWSAccessKeyId=AKIAIWOWYYGZ2Y53UL3A&Expires=1497924032&Signature=CqruoCFfPw6KAFW%2FdTPdJ95ju40%3D&response-content-disposition=inline%3B%20filename%3DGlobal_land_cover_validation_Recommendat.pdf (accessed on 20 June 2017)
4. Olofsson, P.; Foody, G.M.; Herold, M.; Stehman, S.V.; Woodcock, C.E.; Wulder, M.A. Good practices for estimating area and assessing accuracy of land change. Remote Sens. Environ. 2014, 148, 42-57
5. Hernando Gallego, A.; Castilla Castellano, G.; Zang, C.; Mazumdar, D.; Macdermic, G. An Integrated Framework for Assessing the Accuracy of Geobia Landcover Products. Available online: http://oa.upm.es/21039/1/INVE_MEM_2012_129483.pdf (accessed on 20 June 2017)
6. Burrough, P.A. Natural Objects with Indeterminate Boundaries. In Geographic Objects with Indeterminate Boundaries; Taylor & Francis: London, UK, 1996; Volume 2, pp. 3-28
7. Bian, L. Object-oriented representation of environmental phenomena: Is everything best represented as an object? Ann. Assoc. Am. Geogr. 2007, 97, 267-281
8. Couclelis, H. Towards an operational typology of geographic entities with ill-defined boundaries. In Geographic Objects with Indeterminate Boundaries; Taylor & Francis: London, UK, 1996; pp. 45-55
9. Blaschke, T.; Hay, G.J.; Kelly, M.; Lang, S.; Hofmann, P.; Addink, E.; Feitosa, R.Q.; van der Meer, F.; van der Werff, H.; van Coillie, F.; et al. Geographic object-based image analysis-towards a new paradigm. ISPRS J. Photogramm. Remote Sens. 2014, 87, 180-191
10. Hay, G.J.; Castilla, G. Geographic Object-Based Image Analysis (GEOBIA): A new name for a new discipline. In Object-Based Image Analysis; Springer: Berlin/Heidelberg, Germany, 2008; pp. 75-89
11. Baatz, M.; Schäpe, A. Multiresolution Segmentation: An optimization approach for high quality multi-scale image segmentation. In Angewandte Geographische Informationsverarbeitung XII; Strobl, J., Blaschke, T., Griesebner, G., Eds.; Salzburg Geographical Materials: Salzburg, Germany, 2000; pp. 12-23
12. Vetrivel, A.; Kerle, N.; Gerke, M.; Nex, F.; Vosselman, G. Towards automated satellite image segmentation and classification for assessing disaster damage using data-specific features with incremental learning. In Proceedings of the GEOBIA 2016: Solutions and Synergies, Enschede, The Netherlands, 14-16 September 2016
13. Girshick, R.; Donahue, J.; Darrell, T.; Malik, J. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, 23-28 June 2014; pp. 580-587
14. Dean, A.; Smith, G. An evaluation of per-parcel land cover mapping using maximum likelihood class probabilities. Int. J. Remote Sens. 2003, 24, 2905-2920
15. Smith, G.M.; Morton, R.D. Real world objects in GEOBIA through the exploitation of existing digital cartography and image segmentation. Photogramm. Eng. Remote Sens. 2010, 76, 163-171
16. Castilla, G.; Hay, G. Image objects and geographic objects. In Object-Based Image Analysis; Springer: Berlin, Germany, 2008; pp. 91-110
17. Schöpfer, E.; Lang, S.; Albrecht, F. Object-fate analysis: Spatial relationships for the assessment of object transition and correspondence. In Object-Based Image Analysis; Springer: Berlin, Germany, 2008; pp. 785-801
18. Zhan, Q.; Molenaar, M.; Tempfli, K.; Shi, W. Quality assessment for geo-spatial objects derived from remotely sensed data. Int. J. Remote Sens. 2005, 26, 2953-2974
19. Dronova, I.; Gong, P.;Wang, L. Object-based analysis and change detection of major wetland cover types and their classification uncertainty during the low water period at Poyang Lake, China. Remote Sens. Environ. 2011, 115, 3220-3236
20. Whiteside, T.G.; Maier, S.W.; Boggs, G.S. Area-based and location-based validation of classified image objects. Int. J. Appl. Earth Obs. Geoinf. 2014, 28, 117-130
21. Lizarazo, I. Accuracy assessment of object-based image classification: Another STEP. Int. J. Remote Sens. 2014, 35, 6135-6156
22. Hernando, A.; Tiede, D.; Albrecht, F.; Lang, S. Spatial and thematic assessment of object-based forest stand delineation using an OFA-matrix. Int. J. Appl. Earth Obs. Geoinf. 2012, 19, 214-225
23. Congalton, R.; Green, K. A practical look at the sources of confusion in error matrix generation. Photogramm. Eng. Remote Sens. 1993, 59, 641-644
24. Stehman, S. Selecting and Interpreting Measures of Thematic Classification Accuracy. Remote Sens. Environ. 1997, 62, 77-89
25. Stehman, S.; Czaplewski, R. Design and analysis for thematic map accuracy assessment: Fundamental principles. Remote Sens. Environ. 1998, 64, 331-344
26. Lamarche, C.; Santoro, M.; Bontemps, S.; d'Andrimont, R.; Radoux, J.; Giustarini, L.; Brockmann, C.; Wevers, J.; Defourny, P.; Arino, O. Compilation and Validation of SAR and Optical Data Products for a Complete and Global Map of Inland/OceanWater Tailored to the Climate Modeling Community. Remote Sens. 2017, 9, doi:10.3390/rs9010036
27. Congalton, R.G.; Green, K. Assessing the Accuracy of Remotely Sensed Data: Principles and Practices; CRC Press: Boca Raton, FL, USA, 2008
28. Eikvil, L.; Aurdal, L.; Koren, H. Classification-based vehicle detection in high-resolution satellite images. ISPRS J. Photogramm. Remote Sens. 2009, 64, 65-72
29. Benz, U.C.; Hofmann, P.; Willhauck, G.; Lingenfelder, I.; Heynen, M. Multi-resolution, object-oriented fuzzy analysis of remote sensing data for GIS-ready information. ISPRS J. Photogramm. Remote Sens. 2004, 58, 239-258
30. Gougeon, F.A. Automatic individual tree crown delineation using a valley-following algorithm and rule-based system. In Proceedings of the International Forum on Automated Interpretation of High Spatial Resolution Digital Imagery for Forestry, Victoria, BC, Canada, 10-12 February 1998; pp. 11-23
31. Yang, Z.;Wang, T.; Skidmore, A.; de Leeuw, J.; Said, M.; Freer, J. Spotting East African Mammals in Open Savannah from Space. PLoS ONE 2014, 9, doi:org/10.1371/journal.pone.0115989
32. Van Coillie, F.; Van Camp, N.; De Wulf, R.; Bral, L.; Gautama, S. Segmentation quality evaluation for large scale mapping purposes in Flanders, Belgium. Available online: http://www.isprs.org/proceedings/XXXVIII/4-C7/pdf/VanCoillie_182.pdf (accessed on 20 June 2017)
33. Da Costa, J.P.; Michelet, F.; Germain, C.; Lavialle, O.; Grenier, G. Delineation of vine parcels by segmentation of high resolution remote sensed images. Precis. Agric. 2007, 8, 95-110
34. Montaghi, A.; Larsen, R.; Greve, M.H. Accuracy assessment measures for image segmentation goodness of the Land Parcel Identification System (LPIS) in Denmark. Remote Sens. Lett. 2013, 4, 946-955
35. Stuckens, J.; Coppin, P.; Bauer, M. Integrating contextual information with per-pixel classification for improved land cover classification. Remote Sens. Environ. 2000, 71, 282-296
36. Duro, D.C.; Franklin, S.E.; Dubé, M.G. A comparison of pixel-based and object-based image analysis with selected machine learning algorithms for the classification of agricultural landscapes using SPOT-5 HRG imagery. Remote Sens. Environ. 2012, 118, 259-272
37. De Wit, A.; Clevers, J. Efficiency and accuracy of per-field classification for operational crop mapping. Int. J. Remote Sens. 2004, 25, 4091-4112
38. Powers, D.M. Evaluation: From Precision, Recall and F-Measure to ROC, Informedness, Markedness and Correlation. Available online: https://www.researchgate.net/publication/276412348_Evaluation_From_precision_recall_and_F-measure_to_ROC_informedness_markedness_correlation (accessed on 20 June 2017)
39. Ke, Y.; Quackenbush, L.J. A review of methods for automatic individual tree-crown detection and delineation from passive remote sensing. Int. J. Remote Sens. 2011, 32, 4725-4747
40. Lamar, W.R.; McGraw, J.B.; Warner, T.A. Multitemporal censusing of a population of eastern hemlock (Tsuga canadensis L.) from remotely sensed imagery using an automated segmentation and reconciliation procedure. Remote Sens. Environ. 2005, 94, 133-143
41. Liu, C.; Frazier, P.; Kumar, L. Comparative assessment of the measures of thematic classification accuracy. Remote Sens. Environ. 2007, 107, 606-616
42. Radoux, J.; Bogaert, P. Accounting for the area of polygon sampling units for the prediction of primary accuracy assessment indices. Remote Sens. Environ. 2014, 142, 9-19
43. Castilla, G.; Hernando, A.; Zhang, C.; McDermid, G.J. The impact of object size on the thematic accuracy of landcover maps. Int. J. Remote Sens. 2014, 35, 1029-1037
44. Radoux, J.; Bogaert, P.; Fasbender, D.; Defourny, P. Thematic accuracy assessment of geographic object-based image classification. Int. J. Geogr. Inf. Sci. 2011, 25, 1365-8816
45. MacLean, M.G.; Congalton, R.G. Map accuracy assessment issues when using an object-oriented approach. In Proceedings of the Annual Conference of the American Society for Photogrammetry and Remote Sensing, Sacramento, CA, USA, 19-23 March 2012; pp. 1-5
46. Lang, S.; Langanke, T. Object-based mapping and object-relationship modeling for land use classes and habitats. Photogramm. Fernerkund. Geoinf. 2006, 2006, 5-18
47. Tiede, D.; Lang, S.; Albrecht, F.; Hölbling, D. Object-based class modeling for cadastre-constrained delineation of geo-objects. Photogramm. Eng. Remote Sens. 2010, 76, 193-202
48. Di Gregorio, A.; Jansen, L. Land Cover Classification System (LCCS): Classification Concepts and User Manual; Food and Agriculture Organization: Rome, Italy, 2000
49. Quintero, R.; Guzmán, G.; Menchaca-Mendez, R.; Torres, M.; Moreno-Ibarra, M. An ontology-driven approach for the extraction and description of geographic objects contained in raster spatial data. Expert Syst. Appl. 2012, 39, 9008-9020
50. Worboys, M.F.; Hearnshaw, H.M.; Maguire, D.J. Object-oriented data modelling for spatial databases. Int. J. Geogr. Inf. Syst. 1990, 4, 369-383
51. Radoux, J.; Defourny, P. Automated Image-to-Map Discrepancy Detection using Iterative Trimming. Photogramm. Eng. Remote Sens. 2010, 76, 173-181
52. Card, D.H. Using known map category marginal frequencies to improve estimates of thematic map accuracy. Photogramm. Eng. Remote Sens. 1982, 48, 431-439
53. Tiede, D.; Lang, S.; Hoffmann, C. Domain-specific class modelling for one-level representation of single trees. In Object-Based Image Analysis; Springer: Berlin, Germany, 2008; pp. 133-151
54. Mesner, N.; Oštir, K. Investigating the impact of spatial and spectral resolution of satellite images on segmentation quality. J. Appl. Remote Sens. 2014, 8, doi:10.1117/1.JRS.8.083696
55. Ma, L.; Cheng, L.; Li, M.; Liu, Y.; Ma, X. Training set size, scale, and features in geographic object-based image analysis of very high resolution unmanned aerial vehicle imagery. ISPRS J. Photogramm. Remote Sens. 2015, 102, 14-27
56. Grenier, M.; Labrecque, S.; Benoit, M.; Allard, M. Accuracy assessment method for wetland object-based classification. In Proceedings of the GEOBIA, Calgary, AB, Canada, 21 August 2008; pp. 285-289
57. Lang, S.; Zeil, P.; Kienberger, S.; Tiede, D. Geons-policy-relevant geo-objects for monitoring high-level indicators. In Proceedings of the GI Forum, Salzburg, Austria, 1-4 July 2008; Volume 8, pp. 180-185
58. Clinton, N.; Holt, A.; Scarborough, J.; Yan, L.; Gong, P. Accuracy assessment measures for object-based image segmentation goodness. Photogramm. Eng. Remote Sens. 2010, 76, 289-299
59. Van Coillie, F.M.; Gardin, S.; Anseel, F.; Duyck, W.; Verbeke, L.P.; De Wulf, R.R. Variability of operator performance in remote-sensing image interpretation: The importance of human and external factors. Int. J. Remote Sens. 2014, 35, 754-778
60. Albrecht, F.; Lang, S.; Hölbling, D. Spatial accuracy assessment of object boundaries for object-based image analysis. In Proceedings of the The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Science, Ghent, Belgium, 29 June-2 July 2010
61. Greenwalt, C.R.; Schultz, M. Principles of Error Theory and Cartographic Applications. Available online: http://earth-info.nga.mil/GandG/publications/tr96.pdf (accessed on 20 June 2017)
62. Tatem, A.J.; Noor, A.M.; Hay, S.I. Assessing the accuracy of satellite derived global and national urban maps in Kenya. Remote Sens. Environ. 2005, 96, 87-97
63. Radoux, J.; Defourny, P. A quantitative assessment of boundaries in automated forest stand delineation using very high resolution imagery. Remote Sens. Environ. 2007, 110, 468-475
64. Radoux, J.; Defourny, P. Quality assessment of segmentation results devoted to object-based classification. In Object-Based Image Analysis; Springer: Berlin, Germany, 2008; pp. 257-271
65. Persello, C.; Bruzzone, L. A novel protocol for accuracy assessment in classification of very high resolution images. IEEE Trans. Geosci. Remote Sens. 2010, 48, 1232-1244