Superpixel based land cover classification of VHR satellite image combining multi-scale CNN and scale parameter estimation

Earth Science Informatics

Volumn 12, Issue 3, 2019, Pages 341-363

  Chen, Yangyang ^a   Ming, Dongping ^a   Lv, Xianwei ^a  

^a CHINA UNIVERSITY OF GEOSCIENCES   (China)

Author keywords

Deep learning; High spatial resolution remote sensing image; Land cover classification; OBIA; Scale parameter estimation

Indexed keywords

ACCURACY ASSESSMENT; ALGORITHM; ARTIFICIAL NEURAL NETWORK; IMAGE CLASSIFICATION; IMAGE RESOLUTION; LAND COVER; PARAMETER ESTIMATION; PIXEL; REMOTE SENSING; SATELLITE IMAGERY; SEGMENTATION; SPATIAL RESOLUTION; URBAN AREA;

EID: 85068144209     PISSN: 18650473     EISSN: 18650481     Source Type: Journal    
DOI: 10.1007/s12145-019-00383-2     Document Type: Article

References (113)

1. Abadi M et al (2016) Tensorflow: a system for large-scale machine learning. In: Proceedings of 2016 USENIX Symposium on Operating Systems Design and Implementation 16:265–283
2. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2010) SLIC superpixels. EPFL Technical Report 149300
3. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34:2274–2282
4. Alshehhi R, Marpu PR, Woon WL, Dalla Mura M (2017) Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks. ISPRS J Photogramm Remote Sens 130:139–149
5. Audebert N, Le Saux B, Lefevre S (2016) How useful is region-based classification of remote sensing images in a deep learning framework? In: Geoscience and Remote Sensing Symposium (IGARSS), 2016 IEEE International. IEEE, pp 5091–5094
6. Audebert N, Saux BL, Lefèvre S (2017) Segment-before-Detect: Vehicle Detection and Classification through Semantic Segmentation of Aerial Images. Remote Sens 9:368
7. Benediktsson JA, Chanussot J, Moon WM (2013) Advances in very-high-resolution remote sensing. In: Proceedings of the IEEE 101(3):566–569
8. Bengio Y (2009) Learning deep architectures for AI. Foundations and Trends® in Maching Learning 2(1):1–127
9. Blaschke T (2001) What's wrong with pixels? Some recent developments interfacing remote sensing and GIS. GeoBIT/GIS 6:12–17
10. Blaschke T (2010) Object based image analysis for remote sensing. ISPRS J Photogramm Remote Sens 65:2–16
11. Blaschke T et al (2014) Geographic object-based image analysis–towards a new paradigm. ISPRS J Photogramm Remote Sens 87:180–191
12. Blaschke T, Lang S, Hay G (2008) Object-based image analysis: spatial concepts for knowledge-driven remote sensing applications. IEEE Trans Geosci Remote Sens 65:2–16
13. Cánovas-García F, Alonso-Sarría F (2015a) A local approach to optimize the scale parameter in multiresolution segmentation for multispectral imagery. Geocarto International 30:937–961
14. Cánovas-García F, Alonso-Sarría F (2015b) Optimal combination of classification algorithms and feature ranking methods for object-based classification of submeter resolution Z/I-Imaging DMC imagery. Remote Sens 7:4651–4677
15. Castelluccio M, Poggi G, Sansone C, Verdoliva L (2015) Land use classification in remote sensing images by convolutional neural networks. Acta Ecologica Sinica 28(2):627–635
16. Chavez Jr PS (1984) Digital processing techniques for image mapping with Landsat TM and SPOT simulator data. In: Proceedings of the Eighteenth International Symposium on Remote Sensing of Environment. pp 101–116
17. Chavez P, Berlin GL, Sowers LB (1982) Statistical method for selecting landsat. MSS J Appl Photogr Eng 8:23–30
18. Chen Q, Li L, Xu Q, Yang S, Shi X, Liu X (2017) Multi-Feature Segmentation for High-Resolution Polarimetric SAR Data Based on Fractal Net Evolution Approach. Remote Sens 9:570
19. Chen X, Xiang S, Liu C-L, Pan C-H (2014) Vehicle detection in satellite images by hybrid deep convolutional neural networks. IEEE Geosci Remote Sens Lett 11:1797–1801
20. Chen Y, Jiang H, Li C, Jia X, Ghamisi P (2016) Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Trans Geosci Remote Sens 54:6232–6251
21. Cheng G, Han J, Lu X (2017) Remote sensing image scene classification: benchmark and state of the art. In: Proceedings of the IEEE 105(10):1865–1883
22. Ciregan D, Meier U, Schmidhuber J (2012) Multi-column deep neural networks for image classification. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp 3642–3649
23. Cleve C, Kelly M, Kearns FR, Moritz M (2008) Classification of the wildland–urban interface: A comparison of pixel-and object-based classifications using high-resolution aerial photography Computers. Environment and Urban Systems 32:317–326
24. Csillik O (2017) Fast segmentation and classification of very high resolution remote sensing data using SLIC superpixels. Remote Sens 9:243
25. Dahl GE, Yu D, Deng L, Acero A (2012) Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Trans Audio Speech Lang Process 20:30–42
26. Darwish A, Leukert K, Reinhardt W (2003) Image segmentation for the purpose of object-based classification. In: Geoscience and Remote Sensing Symposium, 2003. IGARSS'03. Proceedings. 2003 IEEE International. IEEE, pp 2039–2041
27. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In: Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, pp 248–255
28. Djerriri K, Karoui MS (2017) Classification of quickbird imagery over urban area using convolutional neural network. In: Urban Remote Sensing Event (JURSE), 2017 Joint. IEEE, pp 1–4
29. Drǎguţ L, Tiede D, Levick SR (2010) ESP: a tool to estimate scale parameter for multiresolution image segmentation of remotely sensed data. Int J Geogr Inf Sci 24:859–871
30. Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88:303–338
31. Fauvel M, Chanussot J, Benediktsson JA (2012) A spatial–spectral kernel-based approach for the classification of remote-sensing images. Pattern Recogn 45:381–392
32. Felzenszwalb PF, Huttenlocher DP (2004) Efficient graph-based image segmentation. Int J Comput Vis 59:167–181
33. Fu G, Liu C, Zhou R, Sun T, Zhang Q (2017) Classification for High Resolution Remote Sensing Imagery Using a Fully Convolutional Network. Remote Sens 9:498
34. Fukushima K, Miyake S (1982) Neocognitron: A self-organizing neural network model for a mechanism of visual pattern recognition. In: Competition and cooperation in neural nets. Springer, pp 267–285
35. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 580–587
36. Gonzalez RC, Woods RE (1992) Digital image processing. Addison-wesley, Boston
37. Gonzalo-Martin C, Garcia-Pedrero A, Lillo-Saavedra M, Menasalvas E (2016) Deep learning for superpixel-based classification of remote sensing images. In: GEOBIA 2016
38. Gonzalo-Martín C, Lillo-Saavedra M, Menasalvas E, Fonseca-Luengo D, García-Pedrero A, Costumero R (2016) Local optimal scale in a hierarchical segmentation method for satellite images. J Intell Inf Syst 46:517–529
39. Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182
40. Hay GJ, Castilla G, Wulder MA, Ruiz JR (2005) An automated object-based approach for the multiscale image segmentation of forest scenes. Int J Appl Earth Obs Geoinf 7:339–359
41. Hinton G et al (2012) Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Process Mag 29:82–97
42. Hu F, Xia G-S, Hu J, Zhang L (2015) Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery. Remote Sens 7:14680–14707
43. Hussain M, Chen D, Cheng A, Wei H, Stanley D (2013) Change detection from remotely sensed images: From pixel-based to object-based approaches. ISPRS J Photogramm Remote Sens 80:91–106
44. Karl JW, Maurer BA (2010) Spatial dependence of predictions from image segmentation: A variogram-based method to determine appropriate scales for producing land-management information. Ecological Informatics 5:194–202
45. Karpathy A, Toderici G, Shetty S, Leung T, Sukthankar R, Fei-Fei L (2014) Large-scale video classification with convolutional neural networks. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. pp 1725–1732
46. Kavzoglu T, Erdemir MY, Tonbul H (2017) Classification of semiurban landscapes from very high-resolution satellite images using a regionalized multiscale segmentation approach. J Appl Remote Sens 11:035016
47. Kavzoglu T, Tonbul H (2017a) A Comparative study of segmentation quality for multi-resolution segmentation and watershed transform. In: Recent Advances in Space Technologies (RAST), 2017 8th International Conference on. IEEE, pp 113–117
48. Kavzoglu T, Tonbul H (2017b) Selecting optimal slic superpixels parameters by using discrepancy measures. In: Asian Conference On Remote Sensing
49. Kavzoglu T, Tonbul H (2018) An experimental comparison of multi-resolution segmentation, SLIC and K-means clustering for object-based classification of VHR imagery. Int J Remote Sens:1–17
50. Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images
51. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems. pp 1097–1105
52. Längkvist M, Kiselev A, Alirezaie M, Loutfi A (2016) Classification and segmentation of satellite orthoimagery using convolutional neural networks. Remote Sens 8:329
53. Larochelle H, Bengio Y, Louradour J, Lamblin P (2009) Exploring strategies for training deep neural networks. J Mach Learn Res 10:1–40
54. LeCun Y, Boser B, Denker JS, Henderson D, Howard RE, Hubbard W, Jackel LD (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1:541–551
55. LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. In: Proceedings of IEEE 86(11):2278–2324
56. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444
57. Levinshtein A, Stere A, Kutulakos KN, Fleet DJ, Dickinson SJ, Siddiqi K (2009) Turbopixels: Fast superpixels using geometric flows. IEEE Trans Pattern Anal Mach Intell 31:2290–2297
58. Li F, Li S, Zhu C, Lan X, Chang H (2017a) Cost-Effective Class-Imbalance Aware CNN for Vehicle Localization and Categorization in High Resolution Aerial Images. Remote Sens 9:494
59. Li W, Fu H, Yu L, Cracknell A (2016) Deep Learning Based Oil Palm Tree Detection and Counting for High-Resolution Remote Sensing Images. Remote Sens 9:22
60. Li X, Shao G (2014) Object-based land-cover mapping with high resolution aerial photography at a county scale in midwestern USA. Remote Sens 6:11372–11390
61. Li Y, Zhang H, Shen Q (2017b) Spectral–Spatial Classification of Hyperspectral Imagery with 3D Convolutional Neural Network. Remote Sens 9:–67
62. Lin H, Shi Z, Zou Z (2017) Maritime Semantic Labeling of Optical Remote Sensing Images with Multi-Scale Fully Convolutional Network. Remote Sens 9:480
63. Liu M-Y, Tuzel O, Ramalingam S, Chellappa R (2011) Entropy rate superpixel segmentation. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, pp 2097–2104
64. Liu Y, Zhang M-H, Xu P, Guo Z-W (2017) SAR ship detection using sea-land segmentation-based convolutional neural network. In: 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP), pp 1–4
65. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp 3431–3440
66. Long Y, Gong Y, Xiao Z, Liu Q (2017) Accurate Object Localization in Remote Sensing Images Based on Convolutional Neural Networks. IEEE Trans Geosci Remote Sens 55:2486–2498
67. Lucieer V (2008) Object-oriented classification of sidescan sonar data for mapping benthic marine habitats. Int J Remote Sens 29:905–921
68. Luus FP, Salmon BP, van den Bergh F, Maharaj BTJ (2015) Multiview deep learning for land-use classification. IEEE Geosci Remote Sens Lett 12:2448–2452
69. Maggiori E, Tarabalka Y, Charpiat G, Alliez P (2016) Fully convolutional neural networks for remote sensing image classification. In: Geoscience and Remote Sensing Symposium (IGARSS), 2016 IEEE International. IEEE, pp 5071–5074
70. Maggiori E, Tarabalka Y, Charpiat G, Alliez P (2017) Convolutional neural networks for large-scale remote-sensing image classification. IEEE Trans Geosci Remote Sens 55:645–657
71. Makantasis K, Karantzalos K, Doulamis A, Doulamis N (2015) Deep supervised learning for hyperspectral data classification through convolutional neural networks. In: Geoscience and Remote Sensing Symposium (IGARSS), 2015 IEEE International. IEEE, pp 4959–4962
72. Marcum RA, Davis CH, Scott GJ, Nivin TW (2017) Rapid broad area search and detection of Chinese surface-to-air missile sites using deep convolutional neural networks. J Appl Remote Sens 11:042614
73. Mash R, Borghetti B, Pecarina J (2016) Improved Aircraft Recognition for Aerial Refueling Through Data Augmentation in Convolutional Neural Networks. In: International Symposium on Visual Computing. Springer, pp 113–122
74. Ming D, Ci T, Cai H, Li L, Qiao C, Du J (2012) Semivariogram-based spatial bandwidth selection for remote sensing image segmentation with mean-shift algorithm. IEEE Geosci Remote Sens Lett 9:813–817
75. Ming D, Li J, Wang J, Zhang M (2015) Scale parameter selection by spatial statistics for GeOBIA: Using mean-shift based multi-scale segmentation as an example. ISPRS J Photogramm Remote Sens 106:28–41
76. Mori G (2005) Guiding model search using segmentation. In: Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on. IEEE, pp 1417–1423
77. Moser G, Serpico SB, Benediktsson JA (2013) Land-cover mapping by Markov modeling of spatial–contextual information in very-high-resolution remote sensing images. Proc IEEE 101:631–651
78. Myint SW, Gober P, Brazel A, Grossman-Clarke S, Weng Q (2011) Per-pixel vs. object-based classification of urban land cover extraction using high spatial resolution imagery. Remote Sensing of Environment 115:1145–1161
79. Neubert P, Protzel P (2012) Superpixel benchmark and comparison. In: Proc. Forum Bildverarbeitung. pp 1–12
80. Nogueira K, Penatti OA, dos Santos JA (2017) Towards better exploiting convolutional neural networks for remote sensing scene classification. Pattern Recogn 61:539–556
81. Pan X, Zhao J, Xu J (2019) An object-based and heterogeneous segment filter convolutional neural network for high-resolution remote sensing image classification. Int J Remote Sens:1–25
82. Pekkarinen A (2002) A method for the segmentation of very high spatial resolution images of forested landscapes. Int J Remote Sens 23:2817–2836
83. Penatti OA, Nogueira K, dos Santos JA (2015) Do deep features generalize from everyday objects to remote sensing and aerial scenes domains? In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. pp 44–51
84. Plaza A, Martínez P, Pérez R, Plaza J (2002) Spatial/spectral endmember extraction by multidimensional morphological operations. IEEE Trans Geosci Remote Sens 40:2025–2041
85. Ren X, Malik J (2003) Learning a classification model for segmentation. In: IEEE International Conference on Computer Vision, pp 10–17
86. Salberg A-B (2015) Detection of seals in remote sensing images using features extracted from deep convolutional neural networks. In: IGARSS. pp 1893–1896
87. Scott GJ, England MR, Starms WA, Marcum RA, Davis CH (2017) Training Deep Convolutional Neural Networks for Land–Cover Classification of High-Resolution Imagery. IEEE Geosci Remote Sens Lett 14:549–553
88. Sertel E, Kaya S, Curran P (2007) The use of geostatistical methods to identify severe earthquake damage in an urban area. In: Urban Remote Sensing Joint Event, 2007. IEEE, pp 1–5
89. Sheng G, Yang W, Xu T, Sun H (2012) High-resolution satellite scene classification using a sparse coding based multiple feature combination. Int J Remote Sens 33:2395–2412
90. Sun X, Shen S, Lin X, Hu Z (2017) Semantic labeling of high-resolution aerial images using an ensemble of fully convolutional networks. J Appl Remote Sens 11:042617
91. Tang J, Deng C, Huang G-B, Zhao B (2015) Compressed-domain ship detection on spaceborne optical image using deep neural network and extreme learning machine. IEEE Trans Geosci Remote Sens 53:1174–1185
92. Van den Bergh M, Boix X, Roig G, de Capitani B, Van Gool L (2012) SEEDS: Superpixels extracted via energy-driven sampling. In: European conference on computer vision. Springer, pp 13–26
93. Van den Bergh M, Boix X, Roig G, Van Gool L (2015) Seeds: Superpixels extracted via energy-driven sampling. Int J Comput Vis 111:298–314
94. Vedaldi A, Soatto S (2008) Quick shift and kernel methods for mode seeking Computer vision. ECCV 2008:705–718
95. Wang H, Wang Y, Zhang Q, Xiang S, Pan C (2017) Gated Convolutional Neural Network for Semantic Segmentation in High-Resolution Images. Remote Sens 9:446
96. Wang Q, Zhang J, Hu X, Wang Y (2016) Automatic detection and classification of oil tanks in optical satellite images based on convolutional neural network. In: International Conference on Image and Signal Processing. Springer, pp 304–313
97. Wei J, Li P, Yang J, Zhang J, Lang F (2014) A New Automatic Ship Detection Method Using $ L $-Band Polarimetric SAR Imagery. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 7:1383–1393
98. Xiao Z, Gong Y, Long Y, Li D, Wang X, Liu H (2017a) Airport detection based on a multiscale fusion feature for optical remote sensing images. IEEE Geosci Remote Sens Lett 14:1469–1473
99. Xiao Z, Liu Q, Tang G, Zhai X (2015) Elliptic Fourier transformation-based histograms of oriented gradients for rotationally invariant object detection in remote-sensing images. Int J Remote Sens 36:618–644
100. Xiao Z, Long Y, Li D, Wei C, Tang G, Liu J (2017b) High-Resolution Remote Sensing Image Retrieval Based on CNNs from a Dimensional Perspective. Remote Sens 9:725
101. Yang Y, Newsam S (2010) Bag-of-visual-words and spatial extensions for land-use classification. In: Proceedings of the 18th SIGSPATIAL international conference on advances in geographic information systems. ACM, pp 270–279
102. Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? In: Advances in neural information processing systems, pp 3320–3328
103. Yu Q, Gong P, Clinton N, Biging G, Kelly M, Schirokauer D (2006) Object-based detailed vegetation classification with airborne high spatial resolution remote sensing imagery. Photogramm Eng Remote Sens 72:799–811
104. Yue J, Zhao W, Mao S, Liu H (2015) Spectral–spatial classification of hyperspectral images using deep convolutional neural networks. Remote Sensing Letters 6:468–477
105. Zhang F, Du B, Zhang L, Xu M (2016) Weakly supervised learning based on coupled convolutional neural networks for aircraft detection. IEEE Trans Geosci Remote Sens 54:5553–5563
106. Zhang P, Niu X, Dou Y, Xia F (2017a) Airport detection on optical satellite images using deep convolutional neural networks. IEEE Geosci Remote Sens Lett 14(8):1183–1187
107. Zhang Y, Wu K, Du B, Zhang L, Hu X (2017b) Hyperspectral Target Detection via Adaptive Joint Sparse Representation and Multi-Task Learning with Locality Information. Remote Sens 9:482
108. Zhao W, Du S (2016a) Learning multiscale and deep representations for classifying remotely sensed imagery. ISPRS J Photogramm Remote Sens 113:155–165
109. Zhao W, Du S (2016b) Spectral–spatial feature extraction for hyperspectral image classification: A dimension reduction and deep learning approach. IEEE Trans Geosci Remote Sens 54:4544–4554
110. Zhao W, Du S, Emery WJ (2017a) Object-based convolutional neural network for high-resolution imagery classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 10(7):3386–3396
111. Zhao W et al (2017b) Superpixel-based multiple local CNN for panchromatic and multispectral image classification. IEEE Trans Geosci Remote Sens 55:4141–4156
112. Zhou W, Huang G, Troy A, Cadenasso M (2009) Object-based land cover classification of shaded areas in high spatial resolution imagery of urban areas: A comparison study. Remote Sens Environ 113:1769–1777
113. Zou Q, Ni L, Zhang T, Wang Q (2015) Deep learning based feature selection for remote sensing scene classification. IEEE Geosci Remote Sens Lett 12:2321–2325