Deep learning-based fusion of Landsat-8 and Sentinel-2 images for a harmonized surface reflectance product

Remote Sensing of Environment

Volumn 235, Issue , 2019, Pages

  Shao, Zhenfeng ^a   Cai, Jiajun ^a,b   Fu, Peng ^c   Hu, Leiqiu ^d   Liu, Tao ^e  

^a WUHAN UNIVERSITY   (China)

^b CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

^c UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^d UNIVERSITY OF ALABAMA IN HUNTSVILLE   (United States)

^e OAK RIDGE NATIONAL LABORATORY   (United States)

Author keywords

Continuous monitoring; Data fusion; Deep learning; Landsat 8; Sentinel 2

Indexed keywords

BENCHMARKING; DATA FUSION; IMAGE ENHANCEMENT; IMAGE FUSION; IMAGE QUALITY; IMAGE RESOLUTION; LEARNING ALGORITHMS; MEAN SQUARE ERROR; NEURAL NETWORKS; REFLECTION; SENSOR DATA FUSION;

CONTINUOUS MONITORING; CONVOLUTIONAL NEURAL NETWORK; CORRELATION COEFFICIENT; HIGH SPATIAL RESOLUTION; LANDSAT; QUANTITATIVE ASSESSMENTS; RMSE (ROOT MEAN SQUARE ERROR); SENTINEL-2;

DEEP LEARNING;

BENCHMARKING; DETECTION METHOD; ESTIMATION METHOD; LANDSAT; QUALITATIVE ANALYSIS; SATELLITE DATA; SATELLITE IMAGERY; SENTINEL; SPATIAL RESOLUTION; SURFACE REFLECTANCE; VEGETATION MAPPING;

EID: 85073688762     PISSN: 00344257     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.rse.2019.111425     Document Type: Article

References (53)

1. Acerbi-Junior, F.W., Clevers, J.G.P.W., & Schaepman, M.E. (2006). The assessment of multi-sensor image fusion using wavelet transforms for mapping the Brazilian Savanna. International Journal of Applied Earth Observation and Geoinformation, 8, 278-288
2. Alparone, L., Wald, L., Chanussot, J., Thomas, C., Gamba, P., & Bruce, L.M. (2007). Comparison of pansharpening algorithms: outcome of the 2006 GRS-S data fusion contest. IEEE Transactions on Geoscience and Remote Sensing, 45, 3012-3021
3. Audebert, N., Le Saux, B., & Lefevre, S. (2017). Beyond RGB: Very high resolution urban remote sensing with multimodal deep networks. ISPRS Journal of Photogrammetry and Remote Sensing.
4. Claverie, M., Masek, J. G., Ju, J., & Dungan, J. L. (2017). Harmonized landsat-8 sentinel-2 (HLS) product user's guide. National Aeronautics and Space Administration (NASA): Washington, DC, USA.
5. Claverie, M., Ju, J., Masek, J. G., Dungan, J. L., Vermote, E. F., Roger, J. C., .. & Justice, C. (2018). The Harmonized Landsat and Sentinel-2 surface reflectance data set. Remote Sensing of Environment, 219, 145-161.
6. Das, M., & Ghosh, S. K. (2016). Deep-STEP: A Deep Learning Approach for Spatiotemporal Prediction of Remote Sensing Data. IEEE Geoscience and Remote Sensing Letters, 13(12), 1984-1988.
7. Dong, C., Loy, C.C., He, K., & Tang, X. (2016). Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38, 295-307.
8. Drusch, M., Del Bello, U., Carlier, S., Colin, O., Fernandez, V., Gascon, F., Hoersch, B., Isola, C., Laberinti, P., Martimort, P., Meygret, A., Spoto, F., Sy, O., Marchese, F., & Bargellini, P. (2012). Sentinel-2: ESA's Optical High-Resolution Mission for GMES Operational Services. Remote Sensing of Environment, 120, 25-36.
9. Emelyanova, I. V., McVicar, T. R., Van Niel, T. G., Li, L. T., & van Dijk, A. I. (2013). Assessing the accuracy of blending Landsat-MODIS surface reflectances in two landscapes with contrasting spatial and temporal dynamics: A framework for algorithm selection. Remote Sensing of Environment, 133, 193-209.
10. Fu, P., & Weng, Q. (2016a). A time series analysis of urbanization induced land use and land cover change and its impact on land surface temperature with Landsat imagery. Remote Sensing of Environment, 175, 205-214.
11. Fu, P., & Weng, Q. (2016b). Consistent land surface temperature data generation from irregularly spaced Landsat imagery. Remote Sensing of Environment, 184, 175-187.
12. Gao, F., Masek, J., Schwaller, M., & Hall, F. (2006). On the blending of the Landsat and MODIS surface reflectance: predicting daily Landsat surface reflectance. IEEE Transactions on Geoscience and Remote Sensing, 44, 2207-2218.
13. Gevaert, C. M., & Garcia-Haro, F. J. (2015). A comparison of STARFM and an unmixing-based algorithm for Landsat and MODIS data fusion. Remote sensing of Environment, 156, 34-44.
14. Gorelick, N., Hancher, M., Dixon, M., Ilyushchenko, S., Thau, D., & Moore, R. (2017). Google Earth Engine: Planetary-scale geospatial analysis for everyone. Remote Sensing of Environment, 202, 18-27.
15. He, K., Zhang, X., Ren, S., & Sun, J. (2015). Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37, 1904-1916.
16. Hilker, T., Wulder, M.A., Coops, N.C., Linke, J., McDermid, G., Masek, J.G., Gao, F., & White, J.C. (2009). A new data fusion model for high spatial- and temporal-resolution mapping of forest disturbance based on Landsat and MODIS. Remote Sensing of Environment, 113, 1613-1627.
17. Hirschberg, J., & Manning, C. D. (2015). Advances in natural language processing. Science, 349(6245), 261.
18. Huang, B., & Song, H. (2012). Spatiotemporal reflectance fusion via sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 50(10), 3707-3716.
19. Kim, D.-H., Sexton, J.O., Noojipady, P., Huang, C., Anand, A., Channan, S., Feng, M., & Townshend, J.R. (2014). Global, Landsat-based forest-cover change from 1990 to 2000. Remote Sensing of Environment, 155, 178-193.
20. Krizhevsky, A., Sutskever, I., & Hinton, G.E. (2012). ImageNet classification with deep convolutional neural networks. In, International Conference on Neural Information Processing Systems (pp. 1097-1105).
21. Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based leaning applied to document recognition. Proceedings of the IEEE, 86, 2278-2324.
22. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521, 436.
23. Liu, J. G. (2000). Smoothing Filter-based Intensity Modulation: A spectral preserve image fusion technique for improving spatial details. International Journal of Remote Sensing, 21(18), 3461-3472.
24. Liu, P., Xiao, L. and Tang, S. (2016). A new geometry enforcing variational model for pan-sharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 9(12), 5726-5739.
25. Liu, X., Deng, C., Wang, S., Huang, G. B., Zhao, B., & Lauren, P. (2016). Fast and accurate spatiotemporal fusion based upon extreme learning machine. IEEE Geoscience and Remote Sensing Letters, 13(12), 2039-2043.
26. Malenovsky, Z., Rott, H., Cihlar, J., Schaepman, M.E., Garcia-Santos, G., Fernandes, R., & Berger, M. (2012). Sentinels for science: Potential of Sentinel-1, -2, and -3 missions for scientific observations of ocean, cryosphere, and land. Remote Sensing of Environment, 120, 91-101.
27. Masi, G., Cozzolino, D., Verdoliva, L., & Scarpa G. (2016). Pansharpening by Convolutional Neural Networks. Remote Sensing, 8(7), 594.
28. Melaas, E. K., Friedl, M. A., & Zhu, Z. Detecting interannual variation in deciduous broadleaf forest phenology using Landsat TM/ETM+ data. Remote Sens. Environ., 132, 2013, 176-185.
29. Nair, V., & Hinton, G.E. (2010). Rectified linear units improve restricted boltzmann machines. In, International Conference on Machine Learning (pp. 807-814)
30. Ouyang, W., Wang, X., Zeng, X., Qiu, S., Luo, P., Tian, Y., Li, H., Yang, S., Wang, Z., Loy, C.-C., & Tang, X. (2015). Deepid-net: Deformable deep convolutional neural networks for object detection. In, IEEE Conference on Computer Vision and Pattern Recognition (pp. 2403-2412)
31. Ranchin, T., & Wald, L. (2000). Fusion of high spatial and spectral resolution images: The ARSIS concept and its implementation. Photogrammetric Engineering and Remote Sensing, 66, 49-61.
32. Roy, D. P., Wulder, M. A., Loveland, T. R., C.E, W., Allen, R. G., Anderson, M. C., . . . Zhu, Z. (2014). Landsat-8: Science and product vision for terrestrial global change research. Remote Sensing of Environment, 145, 154-172.
33. Skakun, S., Kussul, N., Shelestov, A., & Kussul, O. (2014). Flood hazard and flood risk assessment using a time series of satellite images: A case study in Namibia. Risk Analysis, 34(8), 1521-1537.
34. Schmidhuber, J. (2015). Deep learning in neural networks: An overview. Neural Networks, 61, 85-117.
35. Senf, C., Pflugmacher, D., Heurich, M., & Krueger, T. (2017). A Bayesian hierarchical model for estimating spatial and temporal variation in vegetation phenology from Landsat time series. Remote Sensing of Environment, 194, 155-160.
36. Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.
37. Shao, Z., & Cai, J. (2018). Remote sensing image fusion with deep convolutional neural network. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 11 (5), 2018, 1656-1669.
38. Sun, Y., Chen, Y., Wang, X., & Tang, X. (2014). Deep learning face representation by joint identification-verification. In, International Conference on Neural Information Processing Systems (pp. 1988-1996)
39. USGS. (2015). Using the USGS Landsat 8 product. [Online]. Available: http://landsat.usgs.gov/Landsat8_Using_Product.php.
40. Veloso, A., Mermoz, S., Bouvet, A., Le Toan, T., Planells, M., Dejoux, J.-F., & Ceschia, E. (2017). Understanding the temporal behavior of crops using Sentinel-1 and Sentinel-2-like data for agricultural applications. Remote Sensing of Environment, 199, 415-426.
41. Verbesselt, J., Zeileis, A., & Herold, M. (2012). Near real-time disturbance detection using satellite image time series. Remote Sensing of Environment, 123, 98-108.
42. Wald, L., Ranchin, T., & Mangolini, M. (1997). Fusion of satellite images of different spatial resolution: Assessing the quality of resulting images. Photogrammetric Engineering and Remote Sensing, 63, 691-699.
43. Wang, Q., Shi, W., Li, Z., & Atkinson, P. M. (2016). Fusion of Sentinel-2 images. Remote Sensing of Environment, 187, 241-252.
44. Wang, Q., Blackburn, G.A., Onojeghuo, A.O., Dash, J., Zhou, L., Zhang, Y., & Atkinson, P.M. (2017). Fusion of Landsat-8 OLI and Sentinel-2 MSI data. IEEE Transactions on Geoscience and Remote Sensing, 55, 3885-3899.
45. Wang, Z., & Bovik, A.C. (2002). A universal image quality index. IEEE Signal Processing Letters, 9, 81-84.
46. Wei, Q., Bioucas-Dias, J., Dobigeon, N., & Tourneret, J. Y. (2015). Hyperspectral and Multispectral Image Fusion Based on a Sparse Representation. IEEE Transactions on Geoscience and Remote Sensing, 53(7), 3658-3668.
47. White, J. C., Wulder, M. A., Hermosilla, T., Coops, N. C., & Hobart, G. W. (2017). A nationwide annual characterization of 25 years of forest disturbance and recovery for Canada using Landsat time series. Remote Sensing of Environment, 194, 303-321.
48. Woodcock, C. E., Allen, R., Anderson, M., Belward, A., Bindschadler, R., Cohen, W., Wynne, R. (2008). Free Access to Landsat Imagery. Science, 320(5879), 1011.
49. Yuan, Q., Wei, Y., Meng, X., Shen, H., & Zhang, L. (2018). A Multiscale and Multidepth Convolutional Neural Network for Remote Sensing Imagery Pan-sharpening. IEEE Journal of Selected Topics in Applied Earth Observation and Remote Sensing, 11(3), 978-989.
50. Zhang, N., Donahue, J., Girshick, R., & Darrel, T. (2014). Part-based RCNNs for fine-grained category detection. In, European Conference on Computer Vision (pp. 834-849)
51. Zhu, X., Chen, J., Gao, F., Chen, X., & Masek, J.G. (2010). An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions. Remote Sensing of Environment, 114, 2610-2623.
52. Zhu, X., Helmer, E. H., Gao, F., Liu, D., Chen, J., & Lefsky, M. A. (2016). A flexible spatiotemporal method for fusing satellite images with different resolutions. Remote Sensing of Environment, 172, 165-177.
53. Zhu, Z., & Woodcock, C. E. (2014). Continuous change detection and classification of land cover using all available Landsat data. Remote Sensing of Environment, 144, 152-171.