Quality assessment of image classification algorithms for land-cover mapping: A review and a proposal for a cost-based approach

International Journal of Remote Sensing

Volumn 20, Issue 8, 1999, Pages 1461-1486

  Smits, P C ^a   Dellepiane, S G ^a   Schowengerdt, R A ^b  

^a UNIVERSITY OF GENOA   (Italy)

^b University of Arizona   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; IMAGE QUALITY; MAPPING; MAPS;

LAND COVER MAPPING; QUALITY ASSESSMENT;

REMOTE SENSING;

ALGORITHM; IMAGE CLASSIFICATION; LAND COVER; MAPPING; REMOTE SENSING;

EID: 0033586922     PISSN: 01431161     EISSN: 13665901     Source Type: Journal    
DOI: 10.1080/014311699212560     Document Type: Article

References (66)

1. Aronoff, S., 1982, Classification accuracy: a user approach. Photogrammetric Engineering and Remote Sensing, 49, 1299-1307.
2. A ronoff, S., 1982, The map accuracy report: a user’s view. Photogrammetric Engineering and Remote Sensing, 48, 1309-1312.
3. A very, T. E., and Berlin, G. L., 1985, Fundamentals of remote-sensing and airphoto interpretation (New York: Macmillan Publishing Company).
4. Bratko, I., Cestnik, B., and Kononenko, I., 1996, Attribute-based learning. AI Communications, 9, 27-32.
5. Card, D. H., 1982, Using known map category marginal frequencies to improve estimates of thematic map accuracy. Photogrammetric Engineering and Remote Sensing, 48, 431-439.
6. Cohen, J., 1960, A coefficient of agreement for nominal scales. Educational Psychology Measurement, 20, 37-46.
7. Congalton, R. G., 1991, A review of assessing the accuracy of classifications of remotely sensed data. Remote Sensing of Environment, 37, 35-46.
8. Congalton, R. G., and Mead, R. A., 1983, A quantitative method to test for consistency and correctness in photointerpretation. Photogrammetric Engineering and Remote Sensing, 49, 69-74.
9. Congalton, R. G., Oderwald, R., and Mead, R., 1983, Landsat classification accuracy using discrete multivariate analysis statistical techniques. Photogrammetric Engineering and Remote Sensing, 49, 1671-1678.
10. Czaplewski, R. L., 1992, Accuracy assessment of remotely sensed classification with multiphase sampling and the multivariate composite estimator. Proceedings of the 16th International Biometrics Conference, 7-11 December, 1992, pp. 2-22.
11. Duin, R. P. W., 1996, A note on comparing classifiers. Pattern Recognition Letters, 17, 529-536.
12. Estes, J. E., and Mooneyhan, D. W., 1994, Of maps and myths. Photogrammetric Engineering and Remote Sensing, 60, 517-524.
13. Fienberg, S. E., 1970, An iterative procedure for estimation in contingency tables. The Annals of Mathematical Statistics, 41, 907-917.
14. Fienberg, S. E., and Holland, P. W., 1970, Methods for eliminating zero counts in the contingency tables. In Random Counts in Scienti® c W ork, edited by G. P. Patil (University Park, Pennsylvania: Pennsylvania State University Press), pp. 233-260.
15. Fitzpatrick- Lins, K., 1981, Comparison of sampling procedures and data analysis for a land-use and land-cover map. Photogrammetric Engineering and Remote Sensing, 47,343-351.
16. Foody, G. M., and Cox, D. P., 1994, Subpixel land cover composition estimation using a linear mixture model and fuzzy membership functions. International Journal of Remote Sensing, 15, 619-631.
17. Fu, K. S., and Mui, J. K., 1981, A survey on image segmentation. Pattern Recognition, 13, 3-16.
18. Ginevan, M. E., 1979, Testing land-use map accuracy: another look. Photogrammetric Engineering and Remote Sensing, 45, 1371-1377.
19. Gopal, S., and Woodcock, C., 1994, Theory and methods for accuracy assessment of thematic maps using fuzzy sets. Photogrammetric Engineering and Remote Sensing, 60, 181-188.
20. Haralick, R. M., and Shapiro, L. G., 1993, Computer and Robot Vision (New York: Addison-Wesley).
21. Haralick, R. M., and Shapiro, L. G., 1985, Survey: image segmentation techniques. CVGIP, 29, 100-132.
22. Hardin, P. J., 1994, Parametric and nearest-neighbor methods for hybrid classification: a comparison of pixel assignment accuracy. Photogrammetric Engineering and Remote Sensing, 60, 1439-1448.
23. Hay, A. M., 1979, Sampling designs to test land-use map accuracy. Photogrammetric Engineering and Remote Sensing, 45, 529-533.
24. Hord, R. M., and Brooner, W., 1976, Land-use map accuracy criteria. Photogrammetric Engineering and Remote Sensing, 42, 671-677.
25. Hudson, W. D., and R amm, C. W., 1987, Correct formulation of the Kappa Coefficient of agreement. Photogrammetric Engineering and Remote Sensing, 53, 421-422.
26. Jain, R. C., and Binford, T. O., 1991, Ignorance, myopia and naivete in computer vision systems. CVGIP Image Understanding, 53, 112-117.
27. Janssen, L. L. F., and Van der Wel, F. J. M., 1994, Accuracy assessment of satellite derived land-cover data: a review. Photogrammetric Engineering and Remote Sensing, 60, 419-426.
28. Jensen, J. R., 1996, Introductory Digital Image Processing-A Remote Sensing Perspective (Englewood Cliffs, New Jersey: Prentice-Hall).
29. Kanungo, T., Jaisimha, M. Y., Palmer, J., and Haraliok, R. M., 1995, A methodology for quantitative performance evaluation of detection algorithms. IEEE Transactions on Image Processing, 4, 1667-1673.
30. Kanungo, T., and Haraliok, R. M., 1995, Receiver operating characteristic curves and optimal bayesian operating points. proceedings of international conference on image processing, washington, 23-26 oct. 1995, pp. 256-259.
31. Kew, N. R., 1996, Information-theoretic measures for assessment and analysis in image classification. In Soft computing in remote-sensing data analysis, edited by E. Binaghi, P. A. Brivio and A. Rampini (Singapore: World Scientific), pp. 173-180.
32. Kobayashi, T., Xin-Wei, X. U., MaoMahon, H., Metz, C. E., and Doi, K., 1996, Effect of a computer-aided diagnosis scheme on radiologists’ performance in detection of lung nodules on radiographs. Radiology, 199, 843-848.
33. Kononenko, I., and Bratko, I., 1991, Information based evaluation criterion for classifier’s performance. Machine Learning, 6, 67-80.
34. Landgrebe, D. A., and Malaret, E. R., 1986, Noise in remote sensing systems: effect on classification accuracy. IEEE Transactions on Geoscience and Remote Sensing, GE-24, 294-299.
35. Latham, J. P., 1963, Methodology for an instrumented geographic analysis. Ann. Assoc. Amer. Geogr., 53, 194-209.
36. Lee, C., and Chen, L., 1995, High-speed closest codeword search algorithms for vector quantization. IEEE Transactions on Communications, 43, 323-331.
37. Lillesand, T. M., and Kiefer, R. W., 1987, Remote Sensing and Image Interpretation (New York: John Wiley &Sons).
38. Lins, K., 1978, Accuracy and consistency comparisons of land use and land cover maps made from high-altitude photographs and Landsat multispectral image. Journal of Research of the US Geological Survey, February.
39. Lins, K., 1994, Requirements analysis results for land cover and land use data. US Geological Survey, March.
40. Lins, K. F., and Kleokner, R. L., 1996, Land use and land cover mapping in the United States: an overview and history of the concept. In Gap Analysis, A Landscape Approach to Biodiversity Planning, edited by J. M. Scott, T. H. Tear, and F. W. Davis (Bethesda: American Society for Photogrammetry and Remote Sensing), pp. 57-67.
41. Lunetta, R. S., Congalton, R. G., Fenstermaker, L. K., Jenses, J. R., MoGwire, K. C., and Tinney, L. R., 1991, Remote-sensing and geographic information system data integration: error sources and research issues. Photogrammetric Engineering and Remote Sensing, 57, 667-687.
42. Ma, Z., and Redmond, R. L., 1995a, Analysis of classification results of remotely sensed data and evaluation of classification algorithms. Photogrammetric Engineering and Remote Sensing, 61, pp. 427-433.
43. M a, Z., and Redmond, R. L., 1995b, Tau coefficients for accuracy assessment of classification of remote-sensing data. Photogrammetric Engineering and Remote Sensing, 61, 435-439.
44. M aling, D. H., 1989, Measurements from maps: principles and methods of cartometry (Oxford: Pergamon Press), pp. 144-177.
45. M ather, P. M., 1987, Computer processing of remotely-sensed images: an introduction (New York: John Wiley &Sons).
46. MAxim, L. D., and Harrington, L., 1983, The application of pseudo-bayesian estimators to remote-sensing data: ideas and examples. Photogrammetric Engineering and Remote Sensing, 49, 649-658.
47. More, D. s., Whitsitt, S. J., and Landgrebe, D. A., 1976, Variance comparisons for unbiased estimators of probabilities of correct classifications. IEEE Transaction on Information Theory, IT-22, 102-105.
48. Richards, J. A., 1993, Remote-sensing digital image analysis (New York: Springer-Verlag).
49. Richards, J. A., 1996, Classifier performance and map accuracy. Remote Sensing of Environment, 57, 161-166.
50. Rosenfield, G. H., 1981, Analysis of variance of thematic mapping experiment data. Photogrammetric Engineering and Remote Sensing, 47, 1685-1692.
51. Rosenfield, G. H., Fitzpatrick- Lins, K., and Ling, H. S., 1982, Sampling for the thematic map accuracy testing. Photogrammetric Engineering and Remote Sensing, 48, 131-137.
52. Rosenfield, G. H., and Melley, M., 1980, Applications of statistics to thematic mapping. Photogrammetric Engineering and Remote Sensing, 46, 1287-1294.
53. Shahshahani, B. M., and Landgrebe, D. A., 1994, The effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon. IEEE Transactions on Geoscience and Remote Sensing, 32, 1087-1095.
54. Stehman, S. V., 1996, Estimating the Kappa coefficient and its variance under stratified random sampling. Photogrammetric Engineering and Remote Sensing, 62, 401-407.
55. Story, M., and Congalton, R. G., 1986, Accuracy assessment: a user’s perspective. Photogrammetric Engineering and Remote Sensing, 52, 397-399.
56. Strahler, A. H., 1981, Stratification of natural vegetation for forest and rangeland inventory using Landsat digital imagery and collateral data. International Journal of Remote Sensing, 2, 15-41.
57. Swain, P. H., 1978, Fundamentals of pattern recognition in remote-sensing. In Remote-sensing: The Quantitative Approach, edited by P. H. Swain and S. M. Davis (New York: McGraw-Hill), pp. 136-187.
58. THomAs, I. L., and Allcock, G. McK., 1984, Determining the confidence level for a classification. Photogrammetric Engineering and Remote Sensing, 50, 1491-1496.
59. United States Departiment of Agriculture, 1996, USDA 1996 Farm Bill Fact Sheet, April 1996.
60. Van Genderen, J. L., Lock, B. F., and Vass, P. A., 1978, Remote sensing: statistical testing of thematic map accuracy. Remote Sensing of Environment, 7, 3-14.
61. Vossen, P., and M eyer-Roux, J., 1995, Crop monitoring and yield forecasting activities of the MARS Project. In European Land Information Systems for Agro-Environmental Monitoring, edited by D. King, R. J. A. Jones, and A. J. Thomasson, EUR Publication N. 16232/EN: 11-30. More references and information regarding the MARS project can be found via the Space Application Institute webpage: http://www.jrc.it/.
62. Woodcock, C. E., 1996, On roles and goals for map accuracy assessment: a remote-sensing perspective. Proceedings of the Second International Symposium on Spatial Accuracy Assessment in Natural Resources and Environmental Sciences, edited by H. T. Mowrer, R. L. Czaplewski, and R. H. Hamre (Fort Collins, Colorado: USDA Forest Service Rocky Mountain Research Station), pp. 535-540.
63. ZAmPERONl, P., 1996, Plus ca va, moins ca va. Pattern Recognition Letters, 17, 671-677.
64. Zhu, Z., Ohlen, D. O., Czaplewski, R. L., and Burgan, R. E., 1996, Alternative method to validate the seasonal land cover regions of the conterminous United States. In Proceedings of the Second International Symposium on Spatial Accuracy Assessment in Natural Resources and Environmental Sciences, edited by H. T. Mowrer, R. L. Czaplewski and R. H. Hamre (Fort Collins, Colorado: USDA Forest Service Rocky Mountain Research Station), pp. 409-418.
65. Zhuang, Y. J., 1996, A survey on evaluation methods for image segmentation. Pattern Recognition, 29, 1335-1346.
66. Zhuang, X., Engel, B. A., Xiong, X., and Johannsen, C. J., 1995, Analysis of classification results of remotely sensed data and evaluation of classification algorithms. Photogrammetric Engineering and Remote Sensing, 61, 427-433.