When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs

IEEE Transactions on Geoscience and Remote Sensing

Volumn 56, Issue 5, 2018, Pages 2811-2821

  Cheng, Gong ^a   Yang, Ceyuan ^a   Yao, Xiwen ^a   Guo, Lei ^a   Han, Junwei ^a  

^a NORTHWESTERN POLYTECHNICAL UNIVERSITY   (China)

Author keywords

Convolutional neural networks (CNNs); deep learning; discriminative CNNs (D CNNs); metric learning; remote sensing image scene classification

Indexed keywords

COMPUTER ARCHITECTURE; CONVOLUTION; FEATURE EXTRACTION; IMAGE CLASSIFICATION; IMAGE ENHANCEMENT; LEARNING SYSTEMS; MEASUREMENT; NEURAL NETWORKS; REMOTE SENSING;

CONVOLUTIONAL NEURAL NETWORK; DISCRIMINATIVE CNNS (D-CNNS); IMAGE COLOR ANALYSIS; METRIC LEARNING; REMOTE SENSING IMAGES;

DEEP LEARNING;

ALGORITHM; ARTIFICIAL NEURAL NETWORK; DATA SET; EXPERIMENTAL DESIGN; IMAGE CLASSIFICATION; NUMERICAL METHOD; NUMERICAL MODEL; REMOTE SENSING;

EID: 85044717760     PISSN: 01962892     EISSN: None     Source Type: Journal    
DOI: 10.1109/TGRS.2017.2783902     Document Type: Article

References (65)

1. G. Cheng, J. Han, and X. Lu, "Remote sensing image scene classification: Benchmark and state of the art," Proc. IEEE, vol. 105, no. 10, pp. 1865-1883, Oct. 2017.
2. G.-S. Xia et al., "AID: A benchmark data set for performance evaluation of aerial scene classification," IEEE Trans. Geosci. Remote Sens., vol. 55, no. 7, pp. 3965-3981, Jul. 2017.
3. G. Cheng and J. Han, "A survey on object detection in optical remote sensing images," ISPRS J. Photogramm. Remote Sens., vol. 117, pp. 11-28, Jul. 2016.
4. J. Han, D. Zhang, G. Cheng, L. Guo, and J. Ren, "Object detection in optical remote sensing images based on weakly supervised learning and high-level feature learning," IEEE Trans. Geosci. Remote Sens., vol. 53, no. 6, pp. 3325-3337, Jun. 2015.
5. J. Han et al., "Efficient, simultaneous detection of multi-class geospatial targets based on visual saliency modeling and discriminative learning of sparse coding," ISPRS J. Photogramm. Remote Sens., vol. 89, pp. 37-48, Mar. 2014.
6. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Proc. Conf. Adv. Neural Inf. Process. Syst., 2012, pp. 1097-1105.
7. S. Chaib, H. Liu, Y. Gu, and H. Yao, "Deep feature fusion for VHR remote sensing scene classification," IEEE Trans. Geosci. Remote Sens., vol. 55, no. 8, pp. 4775-4784, Aug. 2017.
8. E. Othman, Y. Bazi, F. Melgani, H. Alhichri, N. Alajlan, and M. Zuair, "Domain adaptation network for cross-scene classification," IEEE Trans. Geosci. Remote Sens., vol. 55, no. 8, pp. 4441-4456, Aug. 2017.
9. F. Zhang, B. Du, and L. Zhang, "Scene classification via a gradient boosting random convolutional network framework," IEEE Trans. Geosci. Remote Sens., vol. 54, no. 3, pp. 1793-1802, Mar. 2016.
10. G. Cheng, P. Zhou, and J. Han, "Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images," IEEE Trans. Geosci. Remote Sens., vol. 54, no. 12, pp. 7405-7415, Dec. 2016.
11. E. Li, J. Xia, P. Du, C. Lin, and A. Samat, "Integrating multilayer features of convolutional neural networks for remote sensing scene classification," IEEE Trans. Geosci. Remote Sens., vol. 55, no. 10, pp. 5653-5665, Oct. 2017.
12. G. Wang, B. Fan, S. Xiang, and C. Pan, "Aggregating rich hierarchical features for scene classification in remote sensing imagery," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 10, no. 9, pp. 4104-4115, Sep. 2017.
13. G. Cheng, Z. Li, X. Yao, L. Guo, and Z. Wei, "Remote sensing image scene classification using bag of convolutional features," IEEE Geosci. Remote Sens. Lett., vol. 14, no. 10, pp. 1735-1739, Oct. 2017.
14. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," in Proc. Int. Conf. Learn. Represent., 2015, pp. 1-13.
15. C. Szegedy et al., "Going deeper with convolutions," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 1-9.
16. X. Bian, C. Chen, L. Tian, and Q. Du, "Fusing local and global features for high-resolution scene classification," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 10, no. 6, pp. 2889-2901, Jun. 2017.
17. L. Huang, C. Chen, W. Li, and Q. Du, "Remote sensing image scene classification using multi-scale completed local binary patterns and Fisher vectors," Remote Sens., vol. 8, no. 6, p. 483, 2016.
18. C. Chen, B. Zhang, H. Su, W. Li, and L. Wang, "Land-use scene classification using multi-scale completed local binary patterns," Signal, Image Video Process., vol. 10, no. 4, pp. 745-752, 2016.
19. G. Cheng, P. Zhou, J. Han, L. Guo, and J. Han, "Auto-encoder-based shared mid-level visual dictionary learning for scene classification using very high resolution remote sensing images," IET Comput. Vis., vol. 9, no. 5, pp. 639-647, Oct. 2015.
20. G. Cheng, J. Han, L. Guo, Z. Liu, S. Bu, and J. Ren, "Effective and efficient midlevel visual elements-oriented land-use classification using VHR remote sensing images," IEEE Trans. Geosci. Remote Sens., vol. 53, no. 8, pp. 4238-4249, Aug. 2015.
21. G. Cheng, J. Han, P. Zhou, and L. Guo, "Multi-class geospatial object detection and geographic image classification based on collection of part detectors," ISPRS J. Photogramm. Remote Sens., vol. 98, pp. 119-132, Dec. 2014.
22. G. Cheng, J. Han, L. Guo, and T. Liu, "Learning coarse-to-fine sparselets for efficient object detection and scene classification," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 1173-1181.
23. J. Zou, W. Li, C. Chen, and Q. Du, "Scene classification using local and global features with collaborative representation fusion," Inf. Sci., vol. 348, pp. 209-226, Jun. 2016.
24. X. Lu, X. Li, and L. Mou, "Semi-supervised multitask learning for scene recognition," IEEE Trans. Cybern., vol. 45, no. 9, pp. 1967-1976, Sep. 2015.
25. Y. Wang et al., "Learning a discriminative distance metric with label consistency for scene classification," IEEE Trans. Geosci. Remote Sens., vol. 55, no. 8, pp. 4427-4440, Aug. 2017.
26. A. M. Cheriyadat, "Unsupervised feature learning for aerial scene classification," IEEE Trans. Geosci. Remote Sens., vol. 52, no. 1, pp. 439-451, Jan. 2014.
27. G. Sheng, W. Yang, T. Xu, and H. Sun, "High-resolution satellite scene classification using a sparse coding based multiple feature combination," Int. J. Remote Sens., vol. 33, no. 8, pp. 2395-2412, 2012.
28. D. Dai and W. Yang, "Satellite image classification via two-layer sparse coding with biased image representation," IEEE Geosci. Remote Sens. Lett., vol. 8, no. 1, pp. 173-176, Jan. 2011.
29. W. Zhou, Z. Shao, C. Diao, and Q. Cheng, "High-resolution remotesensing imagery retrieval using sparse features by auto-encoder," Remote Sens. Lett., vol. 6, no. 10, pp. 775-783, 2015.
30. Y. Li, C. Tao, Y. Tan, K. Shang, and J. Tian, "Unsupervised multilayer feature learning for satellite image scene classification," IEEE Geosci. Remote Sens. Lett., vol. 13, no. 2, pp. 157-161, Feb. 2016.
31. F. Zhang, B. Du, and L. Zhang, "Saliency-guided unsupervised feature learning for scene classification," IEEE Trans. Geosci. Remote Sens., vol. 53, no. 4, pp. 2175-2184, Apr. 2015.
32. A. Romero, C. Gatta, and G. Camps-Valls, "Unsupervised deep feature extraction for remote sensing image classification," IEEE Trans. Geosci. Remote Sens., vol. 54, no. 3, pp. 1349-1362, Mar. 2016.
33. M. L. Mekhalfi, F. Melgani, Y. Bazi, and N. Alajlan, "Land-use classification with compressive sensing multifeature fusion," IEEE Geosci. Remote Sens. Lett., vol. 12, no. 10, pp. 2155-2159, Oct. 2015.
34. X. Lu, X. Zheng, and Y. Yuan, "Remote sensing scene classification by unsupervised representation learning," IEEE Trans. Geosci. Remote Sens., vol. 55, no. 9, pp. 5148-5157, Sep. 2017.
35. J. Fan, T. Chen, and S. Lu, "Unsupervised feature learning for land-use scene recognition," IEEE Trans. Geosci. Remote Sens., vol. 55, no. 4, pp. 2250-2261, Apr. 2017.
36. Y. Xu, Z. Wu, J. Li, A. Plaza, and Z. Wei, "Anomaly detection in hyperspectral images based on low-rank and sparse representation," IEEE Trans. Geosci. Remote Sens., vol. 54, no. 4, pp. 1990-2000, Apr. 2016.
37. X. Yao, J. Han, G. Cheng, X. Qian, and L. Guo, "Semantic annotation of high-resolution satellite images via weakly supervised learning," IEEE Trans. Geosci. Remote Sens., vol. 54, no. 6, pp. 3660-3671, Jun. 2016.
38. F. Hu, G.-S. Xia, J. Hu, and L. Zhang, "Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery," Remote Sens., vol. 7, no. 11, pp. 14680-14707, 2015.
39. W. Zhao and S. Du, "Scene classification using multi-scale deeply described visual words," Int. J. Remote Sens., vol. 37, no. 17, pp. 4119-4131, 2016.
40. F. P. S. Luus, B. P. Salmon, F. van den Bergh, and B. T. J. Maharaj, "Multiview deep learning for land-use classification," IEEE Geosci. Remote Sens. Lett., vol. 12, no. 12, pp. 2448-2452, Dec. 2015.
41. D. Marmanis, M. Datcu, T. Esch, and U. Stilla, "Deep learning earth observation classification using ImageNet pretrained networks," IEEE Geosci. Remote Sens. Lett., vol. 13, no. 1, pp. 105-109, Jan. 2016.
42. K. Nogueira, O. A. B. Penatti, and J. A. dos Santos, "Towards better exploiting convolutional neural networks for remote sensing scene classification," Pattern Recognit., vol. 61, pp. 539-556, Jan. 2017.
43. L. Zhang, L. Zhang, and B. Du, "Deep learning for remote sensing data: A technical tutorial on the state of the art," IEEE Geosci. Remote Sens. Mag., vol. 4, no. 2, pp. 22-40, Jun. 2016.
44. X. Lu, Y. Yuan, and X. Zheng, "Joint dictionary learning for multispectral change detection," IEEE Trans. Cybern., vol. 47, no. 4, pp. 884-897, Apr. 2017.
45. Q. Zhu, Y. Zhong, B. Zhao, G.-S. Xia, and L. Zhang, "Bag-of-visualwords scene classifier with local and global features for high spatial resolution remote sensing imagery," IEEE Geosci. Remote Sens. Lett., vol. 13, no. 6, pp. 747-751, Jun. 2016.
46. G. Cheng, L. Guo, T. Zhao, J. Han, H. Li, and J. Fang, "Automatic landslide detection from remote-sensing imagery using a scene classification method based on BoVW and pLSA," Int. J. Remote Sens., vol. 34, no. 1, pp. 45-59, 2013.
47. K. Qi, H. Wu, C. Shen, and J. Gong, "Land-use scene classification in high-resolution remote sensing images using improved correlatons," IEEE Geosci. Remote Sens. Lett., vol. 12, no. 12, pp. 2403-2407, Dec. 2015.
48. L.-J. Zhao, P. Tang, and L.-Z. Huo, "Land-use scene classification using a concentric circle-structured multiscale bag-of-visual-words model," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 7, no. 12, pp. 4620-4631, Dec. 2014.
49. S. Chen and Y. Tian, "Pyramid of spatial relatons for scene-level land use classification," IEEE Trans. Geosci. Remote Sens., vol. 53, no. 4, pp. 1947-1957, Apr. 2015.
50. B. Zhao, Y. Zhong, and L. Zhang, "A spectral-structural bag-of-features scene classifier for very high spatial resolution remote sensing imagery," ISPRS J. Photogramm. Remote Sens., vol. 116, pp. 73-85, Jun. 2016.
51. Z. Wu et al., "GPU parallel implementation of spatially adaptive hyperspectral image classification," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., to be published, doi: 10.1109/JSTARS.2017.2755639.
52. Z. Wu, Y. Li, A. Plaza, J. Li, F. Xiao, and Z. Wei, "Parallel and distributed dimensionality reduction of hyperspectral data on cloud computing architectures," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 9, no. 6, pp. 2270-2278, Jun. 2016.
53. R. Hou, C. Chen, and M. Shah, "Tube convolutional neural network (T-CNN) for action detection in videos," in Proc. IEEE Int. Conf. Comput. Vis., Oct. 2017, pp. 5822-5831.
54. X. Yao, J. Han, D. Zhang, and F. Nie, "Revisiting co-saliency detection: A novel approach based on two-stage multi-view spectral rotation co-clustering," IEEE Trans. Image Process., vol. 26, no. 7, pp. 3196-3209, Jul. 2017.
55. X. Lu, Y. Chen, and X. Li, "Hierarchical recurrent neural hashing for image retrieval with hierarchical convolutional features," IEEE Trans. Image Process., vol. 27, no. 1, pp. 106-120, Jan. 2018.
56. K. Li, G. Cheng, S. Bu, and X. You, "Rotation-insensitive and context augmented object detection in remote sensing images," IEEE Trans. Geosci. Remote Sens., vol. 56, no. 4, Apr. 2018, doi: 10.1109/TGRS.2017.2778300.
57. J. Han, H. Chen, N. Liu, C. Yan, and X. Li, "CNNs-based RGB-D saliency detection via cross-view transfer and multiview fusion," IEEE Trans. Cybern., to be published, doi: 10.1109/TCYB.2017.2761775.
58. D. Zhang, J. Han, L. Jiang, S. Ye, and X. Chang, "Revealing event saliency in unconstrained video collection," IEEE Trans. Image Process., vol. 26, no. 4, pp. 1746-1758, Apr. 2017.
59. D. Zhang, D. Meng, and J. Han, "Co-saliency detection via a self-paced multiple-instance learning framework," IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 5, pp. 865-878, May 2017.
60. H. O. Song, Y. Xiang, S. Jegelka, and S. Savarese, "Deep metric learning via lifted structured feature embedding," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2016, pp. 4004-4012.
61. Y. Tian, C. Chen, and M. Shah, "Cross-view image matching for geolocalization in urban environments," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2017, pp. 3608-3616.
62. G. Cheng, P. Zhou, and J. Han, "Duplex metric learning for image set classification," IEEE Trans. Image Process., vol. 27, no. 1, pp. 281-292, Jan. 2018.
63. J. Han, G. Cheng, Z. Li, and D. Zhang, "A unified metric learning-based framework for co-saliency detection," IEEE Trans. Circuits Syst. Video Technol., to be published, doi: 10.1109/TCSVT.2017.2706264.
64. J. Hu, J. Lu, and Y.-P. Tan, "Discriminative deep metric learning for face verification in the wild," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2014, pp. 1875-1882.
65. Y. Yang and S. Newsam, "Bag-of-visual-words and spatial extensions for land-use classification," in Proc. ACM SIGSPATIAL Int. Conf. Adv. Geograph. Inf. Syst., 2010, pp. 270-279.