Hyperspectral Pansharpening: A Review

IEEE Geoscience and Remote Sensing Magazine

Volumn 3, Issue 3, 2015, Pages 27-46

  Loncan, Laetitia ^a,b   De Almeida, Luis B ^c   Bioucas Dias, Jose M ^c   Briottet, Xavier ^b   Chanussot, Jocelyn ^a   Dobigeon, Nicolas ^d   Fabre, Sophie ^b   Liao, Wenzhi ^e   Licciardi, Giorgio A ^a   Simoes, Miguel ^a   Tourneret, Jean Yves ^d   Veganzones, Miguel Angel ^a   Vivone, Gemine ^f   Wei, Qi ^d   Yokoya, Naoto ^g  

^a GIPSA LAB   (France)

^b ONERA   (France)

^c INSTITUTO SUPERIOR TÉCNICO   (Portugal)

^d IRIT   (France)

^e GHENT UNIVERSITY   (Belgium)

^f North Atlantic Treaty Organization (NATO) Science and Technology Organization (STO) Centre for Maritime Research and Experimentation (CMRE)   (France)

^g UNIVERSITY OF TOKYO   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

FACTORIZATION; SPECTROSCOPY;

COMPONENT SUBSTITUTION; DIFFERENT CLASS; HIGH SPATIAL RESOLUTION; HIGH SPECTRAL RESOLUTION; HYPERSPECTRAL; HYPERSPECTRAL DATA; MULTI-SPECTRAL; MULTI-SPECTRAL DATA; PAN-SHARPENING; STATE-OF-THE-ART METHODS;

SPECTRAL RESOLUTION;

DATA ASSIMILATION; DATA SET; IMAGE ANALYSIS; MULTISPECTRAL IMAGE; SOFTWARE; SPATIAL RESOLUTION; TECHNOLOGICAL DEVELOPMENT;

EID: 84943770379     PISSN: 24732397     EISSN: 21686831     Source Type: Journal    
DOI: 10.1109/MGRS.2015.2440094     Document Type: Article

References (87)

1. [Online]. Available: http://smsc. cnes. fr/PLEIADES/index. htm
2. S. Michel, P. Gamet, and M.-J. Lefevre-Fonollosa, "HYPXIM-A hyperspectral satellite defined for science, security and defence users, " in Proc. 3rd Workshop Hyperspectral Image Signal Processing: Evolution in Remote Sensing), Lisbon, Portugal, June 2011, pp. 1-4.
3. L. Alparone, L. Wald, J. Chanussot, C. Thomas, P. Gamba, and L. M. Bruce, "Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data-fusion contest, " IEEE Trans. Geosci. Remote Sensing, vol. 45, no. 10, pp. 3012-3021, Oct. 2007.
4. G. Vivone, L. Alparone, J. Chanussot, M. D. Mura, Garzelli, and G. Licciardi, "A critical comparison of pansharpening algorithms, " in Proc. IEEE Int. Conf. Geoscience Remote Sensing, July 2014, pp. 191-194.
5. B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, "25 years of pansharpening: A critical review and new developments, " in Signal Image Processing for Remote Sensing, 2nd ed., C. H. Chen, Ed. Boca Raton, FL: CRC Press, 2011, ch. 28, pp. 533-548.
6. C. Thomas, T. Ranchin, L. Wald, and J. Chanussot, "Synthesis of multispectral images to high spatial resolution: A critical review of fusion methods based on remote sensing physics, " IEEE Trans. Geosci. Remote Sensing, vol. 46, no. 5, pp. 1301-1312, May 2008.
7. W. Carper, T. M. Lillesand, and P. W. Kiefer, "The use of Intensity-Hue-Saturation transformations for merging SPOT panchromatic and multispectral image data, " Photogramm. Eng. Remote Sens., vol. 56, no. 4, pp. 459-467, Apr. 1990.
8. T.-M. Tu, S.-C. Su, H.-C. Shyu, and P. S. Huang, "A new look at IHS-like image fusion methods, " Inform. Fusion, vol. 2, no. 3, pp. 117-186, Sept. 2001.
9. P. S. Chavez, Jr., S. C. Sides, and J. A. Anderson, "Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic, " Photogramm. Eng. Remote Sens., vol. 57, no. 3, pp. 295-303, Mar. 1991.
10. P. S. Chavez and A. Y. Kwarteng, "Extracting spectral contrast in Landsat thematic mapper image data using selective principal component analysis, " Photogramm. Eng. Remote Sens., vol. 55, no. 3, pp. 339-348, 1989.
11. V. Shettigara, "A generalized component substitution technique for spatial enhancement of multispectral images using a higher resolution data set, " Photogramm. Eng. Remote Sens., vol. 58, no. 5, pp. 561-567, 1992.
12. V. P. Shah, N. Younan, and R. L. King, "An efficient pan-sharpening method via a combined adaptive PCA approach and contourlets, " IEEE Trans. Geosci. Remote Sensing, vol. 56, no. 5, pp. 1323-1335, May 2008.
13. C. Laben and B. Brower, "Process for enhacing the spatial resolution of multispectral imagery using pan-sharpening, " U. S. Patent 6 011 875, Jan. 4, 2000.
14. S. Mallat, "A theory for multiresolution signal decomposition: The wavelet representation, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 11, no. 7, pp. 674-693, July 1989.
15. G. P. Nason and B. W. Silverman, "The stationary wavelet transform and some statistical applications, " in Wavelets and Statistics, vol. 103, A. Antoniadis and G. Oppenheim, Eds. New York: Springer-Verlag, 1995, pp. 281-299.
16. M. J. Shensa, "The discrete wavelet transform: Wedding the a trous and Mallat algorithms, " IEEE Trans. Signal Processing, vol. 40, no. 10, pp. 2464-2482, Oct. 1992.
17. P. J. Burt and E. H. Adelson, "The Laplacian pyramid as a compact image code, " IEEE Trans. Commun., vol. 31, no. 4, pp. 532-540, Apr. 1983.
18. M. N. Do and M. Vetterli, "The contourlet transform: An efficient directional multiresolution image representation, " IEEE Trans. Image Processing, vol. 14, no. 12, pp. 2091-2106, Dec. 2005.
19. J. F. J.-L. Starck and F. Murtagh, "The undecimated wavelet decomposition and its reconstruction, " IEEE Trans. Image Processing, vol. 16, no. 2, pp. 297-309, Feb. 2007.
20. C. Ballester, V. Caselles, L. Igual, J. Verdera, and B. Rougé, "A variational model for P+XS image fusion, " Int. J. Comput. Vis., vol. 5969, no. 1, pp. 43-58, 2006.
21. F. Palsson, J. Sveinsson, M. Ulfarsson, and J. A. Benediktsson, "A new pansharpening algorithm based on total variation, " IEEE Geosci. Remote Sensing Lett., vol. 11, no. 1, pp. 318-322, Jan. 2013.
22. X. He, L. Condat, J. B. Dias, J. Chanussot, and J. Xia, "A new pansharpening method based on spatial and spectral sparsity priors, " IEEE Trans. Image Processing, vol. 23, no. 9, pp. 4160-4174, Sept. 2014.
23. M. Moeller, T. Wittman, and A. L. Bertozzi, "A variational approach to hyperspectral image fusion, " in Proc. SPIE Defense, Security, Sensing, 2009.
24. A. Garzelli, B. Aiazzi, S. Baronti, M. Selva, and L. Alparone, "Hyperspectral image fusion, " in Proc. Hyperspectral Workshop, 2010, pp. 17-19.
25. L. Alparone, B. Aiazzi, S. Baronti, and A. Garzelli, Remote Sensing Image Fusion (Signal and Image Processing of Earth Observations). Boca Raton, FL: CRC Press, 2015.
26. G. Vivone, L. Alparone, J. Chanussot, M. D. Mura, A. Garzelli, and G. Licciardi, "MultiResolution analysis and component substitution techniques for hyperspectral Pansharpening, " in Proc. IEEE Int. Geoscience Remote Sensing Symp., July 2014, pp. 2649-2652.
27. J. M. B. Dias, A. Plaza, N. Dobigeon, M. Parente, Q. Du, P. Gader, and J. Chanussot, "Hyperspectral unmixing overview: Geometrical, statistical, and sparse regression-based approaches, " IEEE J. Select. Topics Appl. Earth Observ. Remote Sensing, vol. 5, no. 2, pp. 354-379, Apr. 2012.
28. C. Souza, Jr, L. Firestone, L. M. Silva, and D. Roberts, "Mapping forest degradation in the Eastern amazon from SPOT 4 through spectral mixture models, " Remote Sens. Environ., vol. 87, no. 4, pp. 494-506, 2003.
29. A. Mohammadzadeh, A. Tavakoli, and M. J. V. Zoej, "Road extraction based on fuzzy logic and mathematical morphology from pansharpened IKONOS images, " Photogramm. Rec., vol. 21, no. 113, pp. 44-60, Feb. 2006.
30. F. Laporterie-Déjean, H. de Boissezon, G. Flouzat, and M.-J. Lefévre-Fonollosa, "Thematic and statistical evaluations of five panchromatic/multispectral fusion methods on simulated PLEIADES-HR images, " Inform. Fusion, vol. 6, pp. 193-212, 2005.
31. G. A. Licciardi, A. Villa, M. M. Khan, and J. Chanussot, "Image fusion and spectral unmixing of hyperspectral images for spatial improvement of classification maps, " in Proc. IEEE Int. Conf. Geoscience Remote Sensing, 2012, pp. 7290-729.
32. N. Yokoya, T. Yairi, and A. Iwasaki, "Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion, " IEEE Trans. Geosci. Remote Sensing, vol. 50, no. 2, pp. 528-537, Feb. 2012.
33. Q. Wei, N. Dobigeon, and J.-Y. Tourneret, "Bayesian fusion of multi-band images, " IEEE J. Select. Topics Signal Processing, vol. 9, no. 6, pp. 1117-1127, Sept. 2015.
34. Q. Wei, N. Dobigeon, and J.-Y. Tourneret, "Bayesian fusion of hyperspectral and multispectral images, " in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing, Florence, Italy, May 2014, pp. 3176-3180.
35. M. Simões, J. B. Dias, L. Almeida, and J. Chanussot, "A convex formulation for hyperspectral image superresolution via subspace-based regularization, " IEEE Trans. Geosci. Remote Sensing, 2015, to be published.
36. Q. Wei, J. M. B. Dias, N. Dobigeon, and J.-Y. Tourneret, "Hyperspectral and multispectral image fusion based on a sparse representation, " IEEE Trans. Geosci. Remote Sensing, vol. 53, no. 7, pp. 3658-3668, Sept. 2015.
37. B. Aiazzi, S. Baronti, and M. Selva, "Improving component substitution pansharpening through multivariate regression of MS+pan data, " IEEE Trans. Geosci. Remote Sensing, vol. 45, no. 10, pp. 3230-3239, Oct. 2007.
38. S. Baronti, B. Aiazzi, M. Selva, A. Garzelli, and L. Alparone, "A theoretical analysis of the effects of aliasing and misregistration on pansharpened imagery, " IEEE J. Select. Topics Signal Processing, vol. 5, no. 3, pp. 446-453, June 2011.
39. G. Vivone, L. Alparone, J. Chanussot, M. D. Mura, A. Garzelli, and G. Licciardi, "A critical comparison among pansharpening algorithms, " IEEE Trans. Geosci. Remote Sensing, vol. 53, no. 5, pp. 2565-2586, May 2015.
40. T.-M. Tu, P. S. Huang, C.-L. Hung, and C.-P. Chang, "A fast intensity-hue-saturation fusion technique with spectral adjustment for IKONOS imagery, " IEEE Trans. Geosci. Remote Sensing Lett., vol. 1, no. 4, pp. 309-312, 2004.
41. B. Aiazzi, S. Baronti, F. Lotti, and M. Selva, "A comparison between global and context-adaptive pansharpening of multispectral images, " IEEE Trans. Geosci. Remote Sensing Lett., vol. 6, no. 2, pp. 302-306, Apr. 2009.
42. G. Vivone, R. Restaino, M. D. Mura, G. Licciardi, and J. Chanussot, "Contrast and error-based fusion schemes for multispectral image pansharpening, " IEEE Trans. Geosci. Remote Sensing Lett., vol. 11, no. 5, pp. 930-934, May 2014.
43. J. G. Liu, "Smoothing filter based intensity modulation: A spectral preserve image fusion technique for improving spatial details, " Int. J. Remote Sens., vol. 21, no. 18, pp. 3461-3472, Dec. 2000.
44. L. Alparone, B. Aiazzi, S. Baronti, and A. Garzelli, "Sharpening of very high resolution images with spectral distortion minimization, " in Proc. IEEE Int. Conf. Geoscience Remote Sensing, July 2003, pp. 458-460.
45. B. Aiazzi, L. Alparone, S. Baronti, A. Garzelli, and M. Selva, "MTF-tailored multiscale fusion of high-resolution MS and pan imagery, " Photogramm. Eng. Remote Sens., vol. 72, no. 5, pp. 591-596, May 2006.
46. K. He, J. Sun, and X. Tang, "Guided image filtering, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 6, pp. 1397-1409, 2013.
47. X. Kang, J. Li, and J. A. Benediktsson, "Spectral-spatial hyperspectral image classification with edge-preserving filtering, " IEEE Trans. Geosci. Remote Sensing, vol. 52, no. 5, pp. 2666-2677, 2014.
48. W. Liao, X. Huang, F. Coillie, S. Gautama, A. Pizurica, W. Philips, H. Liu, T. Zhu, M. Shimoni, G. Moser, and D. Tuia, "Processing of multiresolution thermal hyperspectral and digital color data: Outcome of the 2014 IEEE GRSS data fusion contest, " IEEE J. Select. Top. Appl. Earth Observ. Remote Sensing, vol. 8, no. 6, pp. 2984-2996, May 2015.
49. W. Liao, B. Goossens, J. Aelterman, H. Luong, A. Pizurica, N. Wouters, W. Saeys, and W. Philips, "Hyperspectral image deblurring with PCA and total variation, " in Proc. IEEE GRSS Workshop Hyperspectral Image Signal Process. : Evolution Remote Sensing, Florida, June 2013, pp. 1-4.
50. R. C. Hardie, M. T. Eismann, and G. L. Wilson, "MAP estimation for hyperspectral image resolution enhancement using an auxiliary sensor, " IEEE Trans. Image Processing, vol. 13, no. 9, pp. 1174-1184, Sept. 2004.
51. Y. Zhang, S. de Backer, and P. Scheunders, "Noise-resistant wavelet-based Bayesian fusion of multispectral and hyperspectral images, " IEEE Trans. Geosci. Remote Sensing, vol. 47, no. 11, pp. 3834-3843, Nov. 2009.
52. M. Joshi and A. Jalobeanu, "MAP estimation for multiresolution fusion in remotely sensed images using an IGMRF prior model, " IEEE Trans. Geosci. Remote Sensing, vol. 48, no. 3, pp. 1245-1255, Mar. 2010.
53. Q. Wei, N. Dobigeon, and J.-Y. Tourneret, "Bayesian fusion of multispectral and hyperspectral images with unknown sensor spectral response, " in Proc. IEEE Int. Conf. Image Processing, Paris, France, Oct. 2014, pp. 698-702.
54. M. Simões, J. B. Dias, L. B. Almeida, and J. Chanussot, "Hyperspectral image superresolution: An edge-preserving convex formulation, " in Proc. IEEE Int. Conf. Image Processing, Paris, France, Oct. 2014, pp. 4166-4170.
55. R. Molina, A. K. Katsaggelos, and J. Mateos, "Bayesian and regularization methods for hyperparameter estimation in image restoration, " IEEE Trans. Image Processing, vol. 8, no. 2, pp. 231-246, 1999.
56. R. Molina, M. Vega, J. Mateos, and A. K. Katsaggelos, "Variational posterior distribution approximation in Bayesian super resolution reconstruction of multispectral images, " Appl. Comput. Harmon. Anal., vol. 24, no. 2, pp. 251-267, 2008.
57. A. K. Gupta and D. K. Nagar, Matrix Variate Distributions (Monographs and surveys in pure and applied mathematics, vol. 104). Boca Raton, FL: Chapman & Hall/CRC, 2000.
58. M. D. Farrell, Jr and R. M. Mersereau, "On the impact of PCA dimension reduction for hyperspectral detection of difficult targets, " IEEE Geosci. Remote Sensing Lett., vol. 2, no. 2, pp. 192-195, 2005.
59. J. M. Nascimento and J. M. B. Dias, "Vertex component analysis: A fast algorithm to unmix hyperspectral data, " IEEE Trans. Geosci. Remote Sensing, vol. 43, no. 4, pp. 898-910, Apr. 2005.
60. Q. Wei, N. Dobigeon, and J.-Y. Tourneret, "Bayesian fusion of multispectral and hyperspectral images using a block coordinate descent method, " in Proc. IEEE GRSS Workshop Hyperspectral Image Signal Processing: Evolution Remote Sensing, Tokyo, Japan, June 2015.
61. Q. Wei, N. Dobigeon, and J.-Y. Tourneret, "Fast fusion of multiband images based on solving a sylvester equation, " IEEE Trans. Image Processing, vol. 24, no. 11, pp. 4109-4121, Nov. 2015.
62. S. Mallat, A Wavelet Tour of Signal Processing. New York: Academic Press, 1999.
63. J.-L. Starck, E. Candes, and D. Donoho, "The curvelet transform for image denoising, " IEEE Trans. Image Processing, vol. 11, no. 6, pp. 670-684, 2002.
64. N. Ahmed, T. Natarajan, and K. Rao, "Discrete cosine transform, " IEEE Trans. Comput., vol. C-23, no. 1, pp. 90-93, 1974.
65. M. Elad and M. Aharon, "Image denoising via sparse and redundant representations over learned dictionaries, " IEEE Trans. Image Processing, vol. 15, no. 12, pp. 3736-3745, 2006.
66. O. G. Guleryuz, "Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising- Part I: Theory, " IEEE Trans. Image Processing, vol. 15, no. 3, pp. 539-554, 2006.
67. A. Gelman, J. B. Carlin, H. S. Stern, D. B. Dunson, A. Vehtari, and D. B. Rubin, Bayesian Data Analysis. 3rd ed. Boca Raton, FL: CRC Press, 2013.
68. X. Bresson and T. Chan, "Fast dual minimization of the vectorial total variation norm and applications to color image processing, " Inverse Problems Image, vol. 2, no. 4, pp. 455-484, 2008.
69. M. Afonso, J. B. Dias, and M. Figueiredo, "An augmented lagrangian approach to the constrained optimization formulation of imaging inverse problems, " IEEE Trans. Image Processing, vol. 20, no. 3, pp. 681-695, 2011.
70. O. Berné, A. Tielens, P. Pilleri, and C. Joblin, "Non-negative matrix factorization pansharpening of hyperspectral data: An application to mid-infrared astronomy, " in Proc. IEEE GRSS Workshop Hyperspectral Image Signal Process. : Evolution Remote Sensing, 2010, pp. 1-4.
71. R. Kawakami, J. Wright, Y. Tai, Y. Matsushita, M. Ben-Ezra, and K. Ikeuchi, "High-resolution hyperspectral imaging via matrix factorization, " in Proc. IEEE Int. Conf. Computer Vision Pattern Recognition, 2011, pp. 2329-2336.
72. A. Charles, B. Olshausen, and C. Rozell, "Learning sparse codes for hyperspectral imagery, " IEEE J. Select. Topics Signal Processing, vol. 5, no. 5, pp. 963-978, 2011.
73. B. Huang, H. Song, H. Cui, J. Peng, and Z. Xu, "Spatial and spectral image fusion using sparse matrix factorization, " IEEE Trans. Geosci. Remote Sensing, vol. 52, no. 3, pp. 1693-1704, 2014.
74. M. Veganzones, M. Simes, G. Licciardi, J. M. B. Dias, and J. Chanussot, "Hyperspectral super-resolution of locally low rank images from complementary multisource data, " in Proc. IEEE Int. Conf. Image Processing, Paris, France, Oct. 2014, pp. 703-707.
75. N. Yokoya, T. Yairi, and A. Iwasaki, "Hyperspectral, multispectral, and panchromatic data fusion based on coupled non-negative matrix factorization, " in Proc. IEEE GRSS Workshop Hyperspectral Image Signal Process. : Evolution Remote Sensing, 2011, pp. 1-4.
76. D. D. Lee and H. S. Seung, "Learning the parts of objects by nonnegative matrix factorization, " Nature, vol. 401, pp. 788-791, Oct. 1999.
77. D. C. Heinz and C.-I. Chang, "Fully constrained least squares linear spectral mixture analysis method for material quantification in hyperspectral imagery, " IEEE Trans. Geosci. Remote Sensing, vol. 29, no. 3, pp. 529-545, Mar. 2001.
78. N. Yokoya, N. Mayumi, and A. Iwasaki, "Cross-calibration for data fusion of EO-1/Hyperion and Terra/ASTER, " IEEE J. Select. Topics Appl. Earth Observ. Remote Sensing, vol. 6, pp. 419-426, Apr. 2013.
79. L. Wald, T. Ranchin, and M. Mangolini, "Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting image, " IEEE Trans. Geosci. Remote Sensing, vol. 43, pp. 1391-1402, 2005.
80. I. Amro, J. Mateos, M. Vega, R. Molina, and A. K. Katsaggelos, "A survey of classical methods and new trends in pansharpening of multispectral images, " EURASIP J. Adv. Signal Processing, vol. 2011, no. 1, pp. 1-22, 2011.
81. Q. Du, N. H. Younan, R. L. King, and V. P. Shah, "On the performance evaluation of pan-sharpening techniques, " IEEE Geosci. Remote Sensing Lett., vol. 4, pp. 518-522, Oct. 2007.
82. Z. Wang and A. C. Bovik, "A universal image quality index, " IEEE Signal Processing Lett., vol. 9, no. 3, pp. 81-84, Mar. 2002.
83. L. Alparone, B. Aiazzi, S. Baronti, A. Garzelli, F. Nencini, and M. Selva, "Multispectral and panchromatic data fusion assessment without reference, " Photogramm. Eng. Remote Sens., vol. 74, no. 2, pp. 193-200, Feb. 2008.
84. G. Piella and H. Heijmans, "A new quality metric for image fusion, " in Proc. IEEE Int. Conf. Image Processing, 2003, vol. 2, pp. 173-176.
85. L. Wald, Data Fusion: Definitions and Architectures-Fusion of Images of Different Spatial Resolutions. Les Presses de l'Ecole des Mines, 2002.
86. L. Loncan, L. B. Almeida, J. M. Bioucas-Dias, X. Briottet, J. Chanussot, N. Dobigeon, S. Fabre, W. Liao, G. Licciardi, M. Simoes, J.-Y. Tourneret, M. Veganzones, G. Vivone, Q. Wei, and N. Yokoya. (2015). Hyperspectral pansharpening: A review-Complementary results and supporting materials. Tech. Rep., [Online]. Available: http://openremotesensing. net
87. Q. Wei, N. Dobigeon, and J.-Y. Tourneret, "FUSE: A fast multiband image fusion algorithm, " in Proc. IEEE Int. Workshop Computational Advances Multi-Sensor Adaptive Processing, Cancun, Mexico, Dec. 2015.