Dense Semantic Labeling of Very-High-Resolution Aerial Imagery and LiDAR with Fully-Convolutional Neural Networks and Higher-Order CRFs

IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops

Volumn 2017-July, Issue , 2017, Pages 1561-1570

  Liu, Yansong ^a   Piramanayagam, Sankaranarayanan ^a   Monteiro, Sildomar T ^a   Saber, Eli ^a  

^a ROCHESTER INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AERIAL PHOTOGRAPHY; COMPUTER VISION; CONVOLUTION; DEEP NEURAL NETWORKS; GEOMETRICAL OPTICS; GRAPHIC METHODS; IMAGE ENHANCEMENT; IMAGE SEGMENTATION; NEURAL NETWORKS; OPTICAL RADAR; PATTERN RECOGNITION; RANDOM PROCESSES; REMOTE SENSING; SEMANTIC WEB; SEMANTICS; SPEECH RECOGNITION;

AIRBORNE REMOTE SENSING; CONDITIONAL RANDOM FIELD; CONTEXTUAL INFORMATION; CONVOLUTIONAL NEURAL NETWORK; DECISION LEVEL FUSION; MULTI-SENSOR FUSION TECHNIQUES; PROBABILISTIC GRAPHICAL MODELS; REMOTE SENSING IMAGES;

IMAGE PROCESSING;

EID: 85030239502     PISSN: 21607508     EISSN: 21607516     Source Type: Conference Proceeding    
DOI: 10.1109/CVPRW.2017.200     Document Type: Conference Paper

References (39)

1. ISPRS 2D semantic labeling contest-Potsdam. http://www2. isprs. org/commissions/comm3/ wg4/2d-sem-label-potsdam. html. Accessed May 12, 2017.
2. R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk. Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell., 34 (11):2274-2282, 2012.
3. M. Alonzo, B. Bookhagen, and D. A. Roberts. Urban tree species mapping using hyperspectral and lidar data fusion. Remote Sens. Environ., 148:70-83, 2014.
4. N. Audebert, B. L. Saux, and S. Lefèvre. Semantic segmentation of earth observation data using multimodal and multiscale deep networks. arXiv preprint arXiv:1609. 06846, 2016.
5. V. Badrinarayanan, A. Kendall, and R. Cipolla. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. arXiv preprint arXiv:1511. 00561, 2015.
6. J. A. Benediktsson, P. H. Swain, and O. K. Ersoy. Neural network approaches versus statistical methods in classification of multisource remote sensing data. IEEE Trans. Geosci. Remote Sens., 28 (4):540-552, 1990.
7. X. Boix, J. M. Gonfaus, J. Van de Weijer, A. D. Bagdanov, J. Serrat, and J. Gonzàlez. Harmony potentials. Int. J. Comput. Vis., 96 (1):83-102, 2012.
8. M. Castelluccio, G. Poggi, C. Sansone, and L. Verdoliva. Land use classification in remote sensing images by convolutional neural networks. arXiv preprint arXiv:1508. 00092, 2015.
9. L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. Yuille. Semantic image segmentation with deep convolutional nets and fully connected crfs. arXiv preprint arXiv:1412. 7062, 2014.
10. X. Chen, S. Xiang, C.-L. Liu, and C.-H. Pan. Vehicle detection in satellite images by hybrid deep convolutional neural networks. IEEE Geosci. Remote Sens. Lett., 11 (10):1797-1801, 2014.
11. L. Cianci, G. Moser, and S. Serpico. Change detection from very highresolution multisensor remote-sensing images by a markovian approach. Proc. of IEEE-GOLD-2012, 2012.
12. C. Debes, A. Merentitis, R. Heremans, J. Hahn, N. Frangiadakis, T. van Kasteren, W. Liao, R. Bellens, A. Pizurica, S. Gautama, et al. Hyperspectral and lidar data fusion: Outcome of the 2013 grss data fusion contest. IEEE J. Sel. Topics Appl. Earth Observ. in Remote Sens., 7 (6):2405-2418, 2014.
13. J. Domke. Learning graphical model parameters with approximate marginal inference. IEEE Trans. Pattern Anal. Mach. Intell., 35 (10):2454-2467, 2013.
14. L. Gómez-Chova, D. Tuia, G. Moser, and G. Camps-Valls. Multimodal classification of remote sensing images: A review and future directions. Proc. IEEE, 103 (9):1560-1584, 2015.
15. S. Kluckner, T. Mauthner, P. Roth, and H. Bischof. Semantic classification in aerial imagery by integrating appearance and height information. In ACCV, pages 477-488. Springer, 2010.
16. P. Kohli and P. H. Torr. Robust higher order potentials for enforcing label consistency. Int. J. Comput. Vis., 82 (3):302-324, 2009.
17. V. Kolmogorov and R. Zabin. What energy functions can be minimized via graph cuts IEEE Trans. Pattern Anal. Mach. Intell., 26 (2):147-159, 2004.
18. V. Koltun. Efficient inference in fully connected crfs with Gaussian edge potentials. Adv. Neural Inf. Process. Syst., 2 (3):4, 2011.
19. L. Ladicky, C. Russell, P. Kohli, and P. H. Torr. Graph cut based inference with co-occurrence statistics. In ECCV, pages 239-253. Springer, 2010.
20. L. Ladickỳ, C. Russell, P. Kohli, and P. H. Torr. Associative hierarchical random fields. IEEE Trans. Pattern Anal. Mach. Intell., 36 (6):1056-1077, 2014.
21. E. Li, J. Femiani, S. Xu, X. Zhang, and P. Wonka. Robust rooftop extraction from visible band images using higher order crf. IEEE Trans. Geosci. Remote Sens., 53 (8):4483-4495, 2015.
22. W. Li, S. Prasad, and J. Fowler. Decision fusion in kernelinduced spaces for hyperspectral image classification. IEEE Geosci. Remote Sens. Lett., 52 (6):3399-3411, 2014.
23. J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In CVPR, pages 3431-3440, 2015.
24. J. Marcello, A. Medina, and F. Eugenio. Evaluation of spatial and spectral effectiveness of pixel-level fusion techniques. IEEE Geosci. Remote Sens. Lett., 10 (3):432-436, 2013.
25. D. Marmanis, M. Datcu, T. Esch, and U. Stilla. Deep learning earth observation classification using imagenet pretrained networks. IEEE Geosci. Remote Sens. Lett., 13 (1):105-109, 2016.
26. D. Marmanis, J. D. Wegner, S. Galliani, K. Schindler, M. Datcu, and U. Stilla. Semantic segmentation of aerial images with an ensemble of cnns. ISPRS Annals Photogramm. Remote Sens., 3:473-480, 2016.
27. V. Mnih and G. Hinton. Learning to detect roads in highresolution aerial images. In ECCV, pages 210-223. Springer, 2010.
28. G. Moser, S. B. Serpico, and J. A. Benediktsson. Land-cover mapping by markov modeling of spatial-contextual information in very-high-resolution remote sensing images. Proc. IEEE, 101 (3):631-651, 2013.
29. S. Paisitkriangkrai, J. Sherrah, P. Janney, V.-D. Hengel, et al. Effective semantic pixel labelling with convolutional networks and conditional random fields. In CVPR Workshops, pages 36-43, 2015.
30. O. Penatti, K. Nogueira, and J. A. dos Santos. Do deep features generalize from everyday objects to remote sensing and aerial scenes domains In CVPR Workshops, pages 44-51, 2015.
31. C. Pohl and J. L. Van Genderen. Review article multisensor image fusion in remote sensing: concepts, methods and applications. Int. J. Remote Sens., 19 (5):823-854, 1998.
32. A. Romero, C. Gatta, and G. Camps-Valls. Unsupervised deep feature extraction for remote sensing image classification. IEEE Trans. Geosci. Remote Sens., 54 (3):1349-1362, 2016.
33. J. Sherrah. Fully convolutional networks for dense semantic labelling of high-resolution aerial imagery. arXiv preprint arXiv:1606. 02585, 2016.
34. A. H. S. Solberg, T. Taxt, and A. K. Jain. A markov random field model for classification of multisource satellite imagery. IEEE Trans. Geosci. Remote Sens., 34 (1):100-113, 1996.
35. L. Ugarriza, E. Saber, S. R. Vantaram, V. Amuso, M. Shaw, and R. Bhaskar. Automatic image segmentation by dynamic region growth and multiresolution merging. IEEE Trans. Image Process., 18 (10):2275-2288, 2009.
36. M. Volpi and D. Tuia. Dense semantic labeling of subdecimeter resolution images with convolutional neural networks. IEEE Trans. Geosci. Remote Sens., 55 (2):881-893, 2017.
37. J. A. Waske, B. and Benediktsson. Fusion of support vector machines for classification of multisensor data. IEEE Trans. Geosci. Remote Sens., 45 (12):3858-3866, 2007.
38. J. D. Wegner, J. A. Montoya-Zegarra, and K. Schindler. A higher-order crf model for road network extraction. In CVPR, pages 1698-1705, 2013.
39. M. Xu, H. Chen, and P. K. Varshney. An image fusion approach based on markov random fields. IEEE Trans. Geosci. Remote Sens., 49 (12):5116-5127, 2011.