Spatiotemporal Satellite Image Fusion Using Deep Convolutional Neural Networks

IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing

Volumn 11, Issue 3, 2018, Pages 821-829

  Song, Huihui ^a   Liu, Qingshan ^a   Wang, Guojie ^a   Hang, Renlong ^a   Huang, Bo ^b  

^a NANJING UNIVERSITY OF INFORMATION SCIENCE AND TECHNOLOGY   (China)

^b CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Convolutional neural network (CNN); nonlinear mapping (NLM); spatial resolution; temporal resolution

Indexed keywords

CONVOLUTION; DEEP NEURAL NETWORKS; EARTH (PLANET); IMAGE ENHANCEMENT; IMAGE RESOLUTION; MAPPING; NEURAL NETWORKS; PERSONNEL TRAINING; RADIOMETERS; REMOTE SENSING; SATELLITE IMAGERY; SATELLITES;

CONVOLUTIONAL NEURAL NETWORK; MODIS; NONLINEAR MAPPINGS; SPATIAL RESOLUTION; SPATIOTEMPORAL PHENOMENA; TEMPORAL RESOLUTION;

IMAGE FUSION;

ACCURACY ASSESSMENT; ARTIFICIAL NEURAL NETWORK; IMAGE ANALYSIS; LANDSAT; MAPPING; NONLINEARITY; SATELLITE IMAGERY; SPATIAL RESOLUTION; SPATIOTEMPORAL ANALYSIS;

EID: 85042131669     PISSN: 19391404     EISSN: 21511535     Source Type: Journal    
DOI: 10.1109/JSTARS.2018.2797894     Document Type: Article

References (35)

1. T. Hilker et al., "Generation of dense time series synthetic landsat data through data blending with modis using a spatial and temporal adaptive reflectance fusion model," Remote Sens. Environ., vol. 113, no. 9, pp. 1988-1999, 2009.
2. K. J. Ranson, K. Kovacs, G. Sun, and V. I. Kharuk, "Disturbance recognition in the boreal forest using radar and Landsat-7," Can. J. Remote Sens., vol. 29, no. 2, pp. 271-285, 2003.
3. C. E. Woodcock and M. Ozdogan, "Trends in land cover mapping and monitoring," in Land Change Science: Observing, Monitoring and Understanding Trajectories of Change on the Earth's Surface. New York, NY, USA: Springer-Verlag, 2012.
4. J. G. Masek et al., "North American forest disturbance mapped from a decadal Landsat record," Remote Sens. Environ., vol. 112, no. 6, pp. 2914-2926, 2008.
5. P. Wu, H. Shen, L. Zhang, and F. M. Gottsche, "Integrated fusion of multi-scale polar-orbiting and geostationary satellite observations for the mapping of high spatial and temporal resolution land surface temperature," Remote Sens. Environ., vol. 156, pp. 169-181, 2015.
6. H. Shen, X. Meng, and L. Zhang, "An integrated framework for the spatiotemporal spectral fusion of remote sensing images," IEEE Trans. Geosci. Remote Sens., vol. 54, no. 12, pp. 7135-7148, Dec. 2016.
7. F. Gao, J. Masek, M. Schwaller, and F. Hall, "On the blending of the landsat and modis surface reflectance: Predicting daily landsat surface reflectance," IEEE Trans. Geosci. Remote Sens., vol. 44, no. 8, pp. 2207-2218, Aug. 2006.
8. T. Hilker et al., "A new data fusion model for high spatial-and temporalresolution mapping of forest disturbance based on landsat and modis," Remote Sens. Environ., vol. 113, no. 8, pp. 1613-1627, 2009.
9. X. L. Zhu, C. Jin, G. Feng, X. H. Chen, and J. G. Masek, "An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions.," Remote Sens. Environ., vol. 114, no. 11, pp. 2610-2623, 2010.
10. H. Shen, P.Wu, Y. Liu, T. Ai, Y.Wang, and X. Liu, "Spatial and temporal reflectance fusion model considering sensor observation differences," Int. J. Remote Sens., vol. 34, no. 12, pp. 4367-4383, 2013.
11. Q. Wang, Y. Zhang, A. O. Onojeghuo, X. Zhu, and P. M. Atkinson, "Enhancing spatio-temporal fusion of modis and landsat data by incorporating 250 m modis data," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 10, no. 9, pp. 4116-4123, Sep. 2017.
12. Q. Cheng, H. Liu, H. Shen, P. Wu, and L. Zhang, "A spatial and temporal nonlocal filter-based data fusion method," IEEE Trans. Geosci. Remote Sens., vol. 55, no. 8, pp. 4476-4488, Aug. 2017.
13. R. Zurita-Milla, J. G. P. W. Clevers, and M. E. Schaepman, "Unmixingbased landsat TM and MERIS FR data fusion," IEEE Geosci. Remote Sens. Lett., vol. 5, no. 3, pp. 453-457, Jul. 2008.
14. R. Zuritamilla, G. Kaiser, J. G. P. W. Clevers, W. Schneider, and M. E. Schaepman, "Downscaling time series of MERIS full resolution data to monitor vegetation seasonal dynamics.," Remote Sens. Environ., vol. 113, no. 9, pp. 1874-1885, 2009.
15. B. Huang and H. Song, "Spatiotemporal reflectance fusion via sparse representation," IEEE Trans. Geosci. Remote Sens., vol. 50, no. 10, pp. 3707-3716, Oct. 2012.
16. H. Song and B. Huang, "Spatiotemporal satellite image fusion through one-pair image learning," IEEE Trans. Geosci. Remote Sens., vol. 51, no. 4, pp. 1883-1896, Apr. 2013.
17. M. Guo, H. Zhang, J. Li, L. Zhang, and H. Shen, "An online coupled dictionary learning approach for remote sensing image fusion," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 7, no. 4, pp. 1284-1294, Apr. 2014.
18. C. Jiang, H. Zhang, H. Shen, and L. Zhang, "Two-step sparse coding for the pan-sharpening of remote sensing images," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 7, no. 5, pp. 1792-1805, May 2014.
19. V. Jain and H. S. Seung, "Natural image denoising with convolutional networks," in Proc. Adv. Neural Inf. Process. Syst., 2008, pp. 769-776.
20. C. Dong, C. C. Loy, K. He, and X. Tang, "Image super-resolution using deep convolutional networks," IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 2, pp. 295-307, Feb. 2016.
21. R. Girshick, J. Donahue, T. Darrell, and J.Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation," in Proc. 2014 IEEE Comput. Vis. Pattern Recognit., 2013, pp. 580-587.
22. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Proc. Int. Conf. Neural Inf. Process. Syst., 2012, pp. 1097-1105.
23. V. Nair and G. E. Hinton, "Rectified linear units improve restricted Boltzmann machines," in Proc. Int. Conf. Mach. Learn., 2010, pp. 807-814.
24. S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," in Proc. 32nd Int. Conf. Mach. Learn., 2015, pp. 448-456.
25. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proc. Comput. Vis. Pattern Recognit., 2016, pp. 770-778.
26. K. Zhang,W. Zuo, Y. Chen, D. Meng, and L. Zhang, "Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising.," IEEE Trans. Image Process., vol. 26, no. 7, pp. 3142-3155, Jul. 2017.
27. W. Yang et al., "Deep edge guided recurrent residual learning for image super-resolution," IEEE Trans. Image Process., vol. 26, no. 12, pp. 5895-5907, Dec. 2017.
28. Y. Lcun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
29. C. Thomas, T. Ranchin, L. Wald, and J. Chanussot, "Synthesis of multispectral images to high spatial resolution: A critical review of fusion methods based on remote sensing physics," IEEE Trans. Geosci. Remote Sens., vol. 46, no. 5, pp. 1301-1312, May 2008.
30. I. V. Emelyanova, T. R. Mcvicar, T. G. V. Niel, L. T. Li, and A. I. J. M. V. Dijk, "Assessing the accuracy of blending landsatcmodis surface reflectances in two landscapes with contrasting spatial and temporal dynamics: A framework for algorithm selection," Remote Sens. Environ., vol. 133, no. 12, pp. 193-209, 2013.
31. A. Berk et al., "Modtran4 radiative transfer modeling for atmospheric correction," Proc. SPIE, vol. 3756, pp. 348-353, 1999.
32. F. Li, D. L. B. Jupp, S. Reddy, and L. Lymburner, "An evaluation of the use of atmospheric and BRDF correction to standardize landsat data," IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 3, no. 3, pp. 257-270, Sep. 2010.
33. M. M. Khan, L. Alparone, and J. Chanussot, "Pansharpening quality assessment using the modulation transfer functions of instruments," IEEE Trans. Geosci. Remote Sens., vol. 47, no. 11, pp. 3880-3891, Nov. 2009.
34. R. Yuhas, A. F. H. Goetz, and J. W. Boardman, "Descrimination among semi-arid landscape endmembers using the spectral angle mapper (SAM) algorithm," in Proc. Summaries 3rd Annu. JPL Airborne Geosci. Workshop, 1992, pp. 147-149.
35. Z.Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, Apr. 2004.