Superpixel-based multiple local CNN for panchromatic and multispectral image classification

IEEE Transactions on Geoscience and Remote Sensing

Volumn 55, Issue 7, 2017, Pages 4141-4156

  Zhao, Wei ^a   Jiao, Licheng ^a   Ma, Wenping ^a   Zhao, Jiaqi ^a   Zhao, Jin ^a   Liu, Hongying ^a   Cao, Xianghai ^a   Yang, Shuyuan ^a  

^a XIDIAN UNIVERSITY   (China)

Author keywords

Convolution neural network (CNN); Image classification; Multiple local regions joint representation; Panchromatic and multispectral (MS) images; Superpixel based

Indexed keywords

CLUSTERING ALGORITHMS; IMAGE CLASSIFICATION; IMAGE RECONSTRUCTION; ITERATIVE METHODS; PIXELS; REMOTE SENSING; SEMANTICS;

CLASSIFICATION PERFORMANCE; CLASSIFICATION RESULTS; CONVOLUTION NEURAL NETWORK; ENVIRONMENT INFORMATION; FEATURE REPRESENTATION; MULTISPECTRAL IMAGE CLASSIFICATION; REMOTE SENSING IMAGES; REPRESENTATION METHOD;

CLASSIFICATION (OF INFORMATION);

ALGORITHM; ARTIFICIAL NEURAL NETWORK; IMAGE ANALYSIS; IMAGE CLASSIFICATION; MULTISPECTRAL IMAGE; PANCHROMATIC IMAGE; PIXEL; REMOTE SENSING; SEGMENTATION;

BRITISH COLUMBIA; CANADA; CHINA; SHAANXI; VANCOUVER [BRITISH COLUMBIA]; XIAN;

EID: 85018640168     PISSN: 01962892     EISSN: None     Source Type: Journal    
DOI: 10.1109/TGRS.2017.2689018     Document Type: Article

References (50)

1. C. K. Munechika, J. S. Warnick, C. Salvaggio, and J. R. Schott, "Resolution enhancement of multispectral image data to improve classification accuracy," Photogramm. Eng. Remote Sens., vol. 59, no. 1, pp. 67-72, 1993.
2. A. K. Shackelford and C. H. Davis, "A hierarchical fuzzy classification approach for high-resolution multispectral data over urban areas," IEEE Trans. Geosci. Remote Sens., vol. 41, no. 9, pp. 1920-1932, Sep. 2003.
3. X. Huang, D. Wen, J. Xie, and L. Zhang, "Quality assessment of panchromatic and multispectral image fusion for the ZY-3 satellite: From an information extraction perspective," IEEE Geosci. Remote Sens. Lett., vol. 11, no. 4, pp. 753-757, Apr. 2014.
4. P. P. Shingare, P. M. Hemane, and D. S. Dandekar, "Fusion classification of multispectral and panchromatic image using improved decision tree algorithm," in Proc. Int. Conf. Signal Propag. Comput. Technol., Jul. 2014, pp. 598-603.
5. D. Tuia, F. Ratle, F. Pacifici, M. F. Kanevski, and W. J. Emery, "Active learning methods for remote sensing image classification," IEEE Trans. Geosci. Remote Sens., vol. 47, no. 7, pp. 2218-2232, Jul. 2009.
6. M. T. Pham, G. Mercier, and J. Michel, "Pointwise graph-based local texture characterization for very high resolution multispectral image classification," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 8, no. 5, pp. 1962-1973, May 2015.
7. S. Wei et al., "Textural and local spatial statistics for the objectoriented classification of urban areas using high resolution imagery," Int. J. Remote Sens., vol. 29, no. 11, pp. 3105-3117, 2008.
8. J. Zhong, B. Yang, G. Huang, F. Zhong, and Z. Chen, "Remote sensing image fusion with convolutional neural network," Sens. Imag., vol. 17, no. 1, pp. 1-16, Dec. 2016.
9. G. Masi, D. Cozzolino, L. Verdoliva, and G. Scarpa, "Pansharpening by convolutional neural networks," Remote Sens., vol. 8, no. 7, p. 594, 2016.
10. T. Mao, H. Tang, J. Wu, W. Jiang, S. He, and Y. Shu, "A generalized metaphor of Chinese restaurant franchise to fusing both panchromatic and multispectral images for unsupervised classification," IEEE Trans. Geosci. Remote Sens., vol. 54, no. 8, pp. 4594-4604, Aug. 2016.
11. G. Moser and S. B. Serpico, "Joint classification of panchromatic and multispectral images by multiresolution fusion through Markov random fields and graph cuts," in Proc. Int. Conf. Digit. Signal Process., Jul. 2011, pp. 1-8.
12. J. Zhang, T. Li, X. Lu, and Z. Cheng, "Semantic classification of high-resolution remote-sensing images based on mid-level features," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 9, no. 6, pp. 2343-2353, Jun. 2016.
13. G. Moser, A. De Giorgi, and S. B. Serpico, "Multiresolution supervised classification of panchromatic and multispectral images by Markov random fields and graph cuts," IEEE Trans. Geosci. Remote Sens., vol. 54, no. 9, pp. 5054-5070, Sep. 2016.
14. C. A. Laben and B. V. Brower, "Process for enhancing the spatial resolution of multispectral imagery using pan-sharpening," U.S. Patent 6 011 875 A, Jan. 4, 2000.
15. C. Polh and J. L. Van Genderen, "Multisensor image fusion in remote sensing: Concepts, methods and applications," Int. J. Remote Sens., vol. 19, no. 5, pp. 823-854, 1998.
16. P. S. Chavez, Jr., S. C. Sides, and J. A. Anderson, "Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic," Photogramm. Eng. Remote Sens., vol. 57, no. 3, pp. 295-303, 1991.
17. V. K. Shettigara, "A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set," Photogramm. Eng. Remote Sens., vol. 58, no. 5, pp. 561-567, 1992.
18. G. Hong and Y. Zhang, "Comparison and improvement of wavelet-based image fusion," Int. J. Remote Sens., vol. 29, no. 3, pp. 673-691, 2008.
19. M. Lillo-Saavedra and C. Gonzalo, "Multispectral images fusion by a joint multidirectional and multiresolution representation," Int. J. Remote Sens., vol. 28, no. 18, pp. 4065-4079, 2007.
20. J. Vrabel, "Multispectral imagery advanced band sharpening study," Photogramm. Eng. Remote Sens., vol. 66, no. 1, pp. 73-80, 2000.
21. Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, pp. 436-444, May 2015.
22. L. Zhang, L. Zhang, and B. Du, "Deep learning for remote sensing data: A technical tutorial on the state of the art," IEEE Geosci. Remote Sens. Mag., vol. 4, no. 2, pp. 22-40, Jun. 2016.
23. J. Xie, L. Xu, and E. Chen, "Image denoising and inpainting with deep neural networks," in Proc. Adv. Neural Inf. Process. Syst., vol. 1. 2012, pp. 341-349.
24. W. Huang, L. Xiao, Z. Wei, H. Liu, and S. Tang, "A new pan-sharpening method with deep neural networks," IEEE Geosci. Remote Sens. Lett., vol. 12, no. 5, pp. 1037-1041, May 2015.
25. Z. Zhao, L. Jiao, J. Zhao, J. Gu, and J. Zhao, "Discriminant deep belief network for high-resolution sar image classification," Pattern Recognit., vol. 16, pp. 686-701, Jan. 2017.
26. L. Jiao and F. Liu, "Wishart deep stacking network for fast POLSAR image classification," IEEE Trans. Image Process., vol. 25, no. 7, pp. 3273-3286, Jul. 2016.
27. F. Liu, L. Jiao, B. Hou, and S. Yang, "POL-SAR image classification based on wishart DBN and local spatial information," IEEE Trans. Geosci. Remote Sens., vol 54, no. 6, pp. 3292-3308, Jun. 2016.
28. Y. Chen, X. Zhao, and X. Jia, "Spectral-spatial classification of hyperspectral data based on deep belief network," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 8, no. 6, pp. 2381-2392, Jun. 2015.
29. W. Hu, Y. Huang, L. Wei, F. Zhang, and H. Li, "Deep convolutional neural networks for hyperspectral image classification," J. Sens., vol. 2015, no. 2, 2015, Art. no. 258619.
30. T. Li, J. Zhang, and Y. Zhang, "Classification of hyperspectral image based on deep belief networks," in Proc. IEEE Int. Conf. Image Process., Oct. 2015, pp. 5132-5136.
31. Y. Chen, Z. Lin, X. Zhao, G. Wang, and Y. Gu, "Deep learning-based classification of hyperspectral data," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 7, no. 6, pp. 2094-2107, Jun. 2014.
32. W. Zhao, Z. Guo, J. Yue, X. Zhang, and L. Luo, "On combining multiscale deep learning features for the classification of hyperspectral remote sensing imagery," Int. J. Remote Sens., vol. 36, no. 13, pp. 3368-3379, 2015.
33. X. Chen, S. Xiang, C.-L. Liu, and C.-H. Pan, "Vehicle detection in satellite images by hybrid deep convolutional neural networks," IEEE Geosci. Remote Sens. Lett., vol. 11, no. 10, pp. 1797-1801, Oct. 2014.
34. M. Vakalopoulou, K. Karantzalos, N. Komodakis, and N. Paragios, "Building detection in very high resolution multispectral data with deep learning features," in Proc. IEEE Int. Geosci. Remote Sens. Symp., Jul. 2015, pp. 122-123.
35. L. Zhang, Z. Shi, and J. Wu, "A hierarchical oil tank detector with deep surrounding features for high-resolution optical satellite imagery," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 8, no. 10, pp. 4895-4909, Oct. 2015.
36. J. Tang, C. Deng, G.-B. Huang, and B. Zhao, "Compressed-domain ship detection on spaceborne optical image using deep neural network and extreme learning machine," IEEE Trans. Geosci. Remote Sens., vol. 53, no. 3, pp. 1174-1185, Mar. 2015.
37. W. Diao, X. Sun, F. Dou, M. Yan, H. Wang, and K. Fu, "Object recognition in remote sensing images using sparse deep belief networks," Remote Sens. Lett., vol. 6, no. 10, pp. 745-754, 2015.
38. F. Zhang, B. Du, and L. Zhang, "Scene classification via a gradient boosting random convolutional network framework," IEEE Trans. Geosci. Remote Sens., vol. 54, no. 3, pp. 1793-1802, Mar. 2016.
39. Q. Zou, L. Ni, T. Zhang, and Q. Wang, "Deep learning based feature selection for remote sensing scene classification," IEEE Geosci. Remote Sens. Lett., vol. 12, no. 11, pp. 2321-2325, Nov. 2015.
40. R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk, "SLIC superpixels compared to state-of-The-Art superpixel methods," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 11, pp. 2274-2282, Nov. 2012.
41. S. Gidaris and N. Komodakis, "Object detection via a multi-region and semantic segmentation-Aware CNN model," in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2015, pp. 1134-1142.
42. Y. Zhu, R. Urtasun, R. Salakhutdinov, and S. Fidler, "segDeepM: Exploiting segmentation and context in deep neural networks for object detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., vol. 84. Jun. 2015, pp. 4703-4711
43. V. Nair and G. E. Hinton, "Rectified linear units improve restricted boltzmann machines," in Proc. Int. Conf. Mach. Learn., 2010, pp. 21-24.
44. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," Adv. Neural Inf. Process. Syst., vol. 25, no. 2, p. 2012, 2012.
45. Y. Bengio, "Learning deep architectures for AI," Found. Trends Mach. Learn., vol. 2, no. 1, pp. 1-127, 2009.
46. Y. Jia et al., "Caffe: Convolutional architecture for fast feature embedding," in Proc. ACM Int. Conf. Multimedia, 2014, pp. 675-678.
47. -(2016). IEEE GRSS Data Fusion Contest. [Online]. Available: Http://www.grss-ieee.org/community/technical-committees/data-fusion
48. G. E. Hinton et al., "Improving neural networks by preventing coadaptation of feature detectors," Comput. Sci., vol. 3, no. 4, pp. 212-223, 2012.
49. X. Glorot and Y. Bengio, "Understanding the difficulty of training deep feedforward neural networks," J. Mach. Learn. Res., vol. 9, pp. 249-256, 2010.
50. N. Otsu, "A threshold selection method from gray-level histograms," IEEE Trans. Syst., Man, Cybern., vol. 9, no. 1, pp. 62-66, Jan. 1979.