A survey of classical methods and new trends in pansharpening of multispectral images

Eurasip Journal on Advances in Signal Processing

Volumn 2011, Issue 1, 2011, Pages

  Amro, Israa ^a,b   Mateos, Javier ^a   Vega, Miguel ^a   Molina, Rafael ^a   Katsaggelos, Aggelos K ^c  

^a UNIVERSITY OF GRANADA   (Spain)

^b AL QUDS OPEN UNIVERSITY   (Palestine)

^c Northwestern University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLASSICAL METHODS; EARTH OBSERVATIONS; MULTISPECTRAL IMAGES; PANCHROMATIC IMAGES; PANSHARPENED IMAGES; PIXEL LEVEL FUSION; SPATIAL RESOLUTION; SPECTRAL INFORMATION;

ELECTRICAL ENGINEERING;

EID: 85010782847     PISSN: 16876172     EISSN: 16876180     Source Type: Journal    
DOI: 10.1186/1687-6180-2011-79     Document Type: Review

References (119)

1. SPOT Web Page. http://www.spotimage.fr
2. Landsat 7 Web Page. http://geo.arc.nasa.gov/sge/landsat/
3. IKONOS Web Page. http://www.geoeye.com/CorpSite/products-andservices/imagery-sources/
4. QuickBird Web Page. http://www.digitalglobe.com
5. OrbView Web Page. http://www.orbital.com/SatellitesSpace/ImagingDefense/
6. JG Liu, Smoothing filter-based intensity modulation: A spectral preserve image fusion technique for improving spatial details. Int J Remote Sensing. 21, 3461–3472 (2000). doi:10.1080/014311600750037499
7. V Vijayaraj, A quantitative analysis of pansharpened images. Master’s thesis, Mississippi State Univ (2004)
8. RA Schowengerdt, Remote Sensing: Models and Methods for Image Processing, 3rd edn, (Orlando, FL: Academic, 1997)
9. C Pohl, JLV Genderen, Multi-sensor image fusion in remote sensing: Concepts, methods, and applications. Int J Remote Sens. 19(5), 823–854 (1998). doi:10.1080/014311698215748
10. C Thomas, T Ranchin, L Wald, J Chanussot, Synthesis of multispectral images to high spatial resolution: A critical review of fusion methods based on remote sensing physics. IEEE Trans Geosci Remote Sens. 46, 1301–1312 (2008)
11. M Ehlers, S Klonus, P Astrand, Quality assessment for multi-sensor multidate image fusion. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, ser Part B 4. 37, 499–505 (2008)
12. T Bretschneider, O Kao, Image fusion in remote sensing, (Technical University of Clausthal, Germany)
13. T Ranchin, B Aiazzi, L Alparone, S Baronti, L Wald, Image fusion: The ARSIS concept and some successful implementation schemes. ISPRS Journal of Photogrammetry & Remote Sensing 58, 4–18 (2003). doi:10.1016/S0924-2716(03)00013-3
14. Z Wang, D Ziou, C Armenakis, D Li, Q Li, A comparative analysis of image fusion methods. IEEE Trans Geosci Remote Sens. 43(6), 1391–1402 (2005)
15. B Aiazzi, S Baronti, F Lotti, M Selva, A comparison between global and context-adaptive pansharpening of multispectral images. IEEE Geoscience and Remote Sensing Letters 6(2), 302–306 (2009)
16. JA Richards, X Jia, Remote Sensing Digital Image Analysis: An Introduction, 4th edn. (Secaucus, NJ, USA: Springer-Verlag New York, Inc, 2005)
17. CL Parkinson, A Ward, MD King, Eds, Earth Science Reference Handbook A Guide to NASA’s Earth Science Program and Earth Observing Satellite Missions. National Aeronautics and Space Administration (2006)
18. Y Yang, X Gao, Remote sensing image registration via active contour model. Int J Electron Commun. 63, 227–234 (2009). doi:10.1016/j.aeue.2008.01.003
19. B Zitova, J Flusser, Image registration methods: A survey. Image and Vision Computing. 21, 977–1000 (2003). doi:10.1016/S0262-8856(03)00137-9
20. RC Gonzalez, RE Woods, Digital image processing, 3rd edn. (Prentice Hall, 2008)
21. S Takehana, M Kashimura, S Ozawa, Predictive interpolation of remote sensing images using vector quantization. Communications, Computers and Signal Processing, 1993, IEEE Pacific Rim Conference on. 1, 51–54 (1993)
22. KK Teoh, H Ibrahim, SK Bejo, Investigation on several basic interpolation methods for the use in remote sensing application, in Proceedings of the 2008 IEEE Conference on Innovative Technologies (2008)
23. W Dou, Y Chen, An improved IHS image fusion method with high spectral fidelity. The Int Archiv of the Photogramm, Rem Sensing and Spat Inform Sciences. 37, 1253–1256 (2008). part.B7
24. FH Wong, R Orth, Registration of SEASAT/LANDSAT composite images to UTM coordinates. Proceedings of the Sixth Canadian Syinposium on Remote Sensing, 161–164 (1980)
25. P Rebillard, PT Nguyen, An exploration of co-registered SIR-A, SEASAT and Landsat images. Proceeidngs International Symposium on RS of Environment, RS for Exploration Geology, 109–118 (1982)
26. R Simard, Improved spatial and altimetric information from SPOT composite imagery. Proceedings International Symposium of Photogrammetry and Remote Sensing), 434–440 (1982)
27. EP Crist, Comparison of coincident Landsat-4 MSS and TM data over an agricultural region, in Proceedings of the 50th Annual Meeting ASP-ACSM Symposium, ASPRS, pp. 508–517 (1984)
28. G Cliche, F Bonn, P Teillet, Integration of the SPOT panchromatic channel into its multispectral mode for image sharpness enhancement. Photogramm Eng Remote Sens. 51(3), 311–316 (1985)
29. R Welch, M Ehlers, Merging multiresolution SPOT HRV and landsat TM data. Photogramm Eng Remote Sens. 53(3), 301–303 (1987)
30. R Haydn, GW Dalke, J Henkel, Application of the IHS color transform to the processing of multisensor data and image enhancement. lnternational Symposium on Remote Sensing of Arid and Semi-Arid Lands, 599–616 (1982)
31. PS Chavez, JA Bowell Jr, Comparison of the spectral information content of Landsat Thematic Mapper and SPOT for three different sites in the Phoenix, Arizona region. Photogramm Eng Remote Sens. 54(12), 1699–1708 (1988)
32. AR Gillespie, AB Kahle, RE Walker, Color enhancement of highly correlated images. II. Channel Ratio and “Chromaticity” Transformation Techniques. Remote Sensing Of Environment. 22, 343–365 (1987). doi:10.1016/0034-4257(87)90088-5
33. PS Chavez, AY Kwarteng, Extracting spectral contrast in Landsat thematic mapper image data using selective principal component analysis. Photogramm Eng Remote Sens. 55(3), 339–348 (1989)
34. P Chavez, S Sides, J Anderson, Comparison of three different methods to merge multiresolution and multispectral data: Landsat TM and SPOT panchromatic. Photogramm Eng Remote Sens. 57(3), 295–303 (1991)
35. JC Price, Combining panchromatic and multispectral imagery from dual resolution satellite instruments. Remote Sensing Of Environment 21, 119–128 (1987). doi:10.1016/0034-4257(87)90049-6
36. J Park, M Kang, Spatially adaptive multi-resolution multispectral image fusion. Int J Remote Sensing 25(23), 5491–5508 (2004). doi:10.1080/01431160412331270830
37. NDA Mascarenhas, GJF Banon, ALB Candeias, Multispectral image data fusion under a Bayesian approach. Int J Remote Sensing 17, 1457–1471 (1996). doi:10.1080/01431169608948717
38. M Vega, J Mateos, R Molina, A Katsaggelos, Super-resolution of multispectral images. The Computer Journal 1, 1–15 (2008)
39. B Aiazzi, L Alparone, S Baronti, A Garzelli, Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis. IEEE Trans Geosc Remote Sens. 40(10), 2300–2312 (2002). doi:10.1109/TGRS.2002.803623
40. A Garzelli, F Nencini, Interband structure modeling for pan-sharpening of very high-resolution multispectral images. Information Fusion 6, 213–224 (2005). doi:10.1016/j.inffus.2004.06.008
41. SG Mallat, A theory for multiresolution signal decomposition: The wavelet representation. IEEE Transactions On Pattern Analysis And Machine Intelligence 11(7), 674–693 (1989). doi:10.1109/34.192463
42. J Zhou, DL Civco, JA Silander, A wavelet transform method to merge Landsat TM and SPOT panchromatic data. Int J Remote Sens. 19(4), 743–757 (1998). doi:10.1080/014311698215973
43. AL da Cunha, J Zhou, MN Do, The nonsubsampled contourlet transform: Theory, design, and applications. IEEE Trans. Image Process. 15(10), 3089–3101 (2006)
44. M Gonzalo, C Lillo-Saavedra, Multispectral images fusion by a joint multidirectional and multiresolution representation. Int J Remote Sens. 28(18), 4065–4079 (2007). doi:10.1080/01431160601105884
45. I Amro, J Mateos, Multispectral image pansharpening based on the contourlet transform, in Journal of Physics Conference Series. 206(1), 1–3 (2010)
46. T Ranchln, L Wald, Fusion of high spatial and spectral resolution images: The ARSIS concept and its implementation. Photogramm Eng Remote Sens. 66, 49–61 (2000)
47. TM Tu, SC Su, HC Shyu, PS Huang, A new look at IHS-like image fusion methods. Inf Fusion 2(3), 177–186 (2001). doi:10.1016/S1566-2535(01)00036-7
48. VK Sheftigara, A generalized component substitution technique for spatial enhancement of multispectral lmages using a higher resolution data set. Photogramm Eng Remote Sens. 58(5), 561–567 (1992)
49. W Dou, Y Chen, X Li, DZ Sui, A general framework for component substitution image fusion: An implementation using the fast image fusion method. Computers & Geosciences 33, 219–228 (2007). doi:10.1016/j.cageo.2006.06.008
50. B Aiazzi, S Baronti, M Selva, Improving component substitution pansharpening through multivariate regression of MS + Pan data. IEEE Trans Geosc Remote Sens. 45(10), 3230–3239 (2007)
51. L Alparone, B Aiazzi, SBA Garzelli, F Nencini, A critical review of fusion methods for true colour display of very high resolution images of urban areas. 1st EARSeL Workshop of the SIG Urban Remote Sensing, Humboldt- Universität zu Berlin (2006)
52. S Rahmani, M Strait, D Merkurjev, M Moeller, T Wittman, An adaptive HIS pan-sharpening method. IEEE Geoscience And Remote Sensing Letters 7(3), 746–750 (2010)
53. VP Shah, NH Younan, RL King, An efficient pan-sharpening method via a combined adaptive PCA approach and contourlets. IEEE Trans Geosc Remote Sens. 46(5), 1323–1335 (2008)
54. B Aiazzi, S Baronti, M Selva, L Alparone, Enhanced Gram-Schmidt spectral sharpening based on multivariate regression of MS and pan data. IEEE International Conference on Geoscience and Remote Sensing Symposium, IGARSS 2006, 3806–3809 (2006)
55. WJ Carper, TM Lillesand, RW Kiefer, The use of Intensity-Hue-Saturation transform for merging SPOT panchromatic and multispectral image data. Photogramm Eng Remote Sens. 56(4), 459–467 (1990)
56. TM Tu, PS Huang, CL Hung, CP Chang, A fast intensity-hue-saturation fusion technique with spectral adjustment for IKONOS imagery. IEEE Geoscience And Remote Sensing Letters. 1(4), 309–312 (2004). doi:10.1109/LGRS.2004.834804
57. CA Laben, BV Brower, Process for enhancing the spatial resolution of multispectral imagery using pansharpening. US Patent 6 011 875 (2000)
58. RW Farebrother, Gram-schmidt regression. Applied Statistics 23, 470–476 (1974). doi:10.2307/2347151
59. SPOT Users Handbook, Centre National Etude Spatiale (CNES) and SPOT Image, Toulouse, France, (1988)
60. C Ballester, V Caselles, L Igual, J Verdera, A variational model for P+XS image fusion. International Journal of Computer Vision 69, 43–58 (2006). doi:10.1007/s11263-006-6852-x
61. G Cliche, F Bonn, P Teillet, Integration of the SPOT panchromatic channel into its multispectral mode for image sharpness enhancement. Photogrammetric Engineering & Remote Sensing 51(3), 311–316 (1985)
62. V Vijayaraj, CG O’Hara, NH Younan, Quality analysis of pansharpened images. Proc IEEE Int Geosc Remote Sens Symp IGARSS ‘04. 1, 20–24 (2004)
63. L Alparone, L Facheris, S Baronti, A Garzelli, F Nencini, Fusion of multispectral and SAR images by intensity modulation, in Proceedings of the 7th International Conference on Information Fusion, pp. 637–643 (2004)
64. RA Schowengerdt, Reconstruction of multispatial, multispectral image data using spatial frequency contents. Photogrammetric Engineering & Remote Sensing 46(10), 1325–1334 (1980)
65. PS Chavez Jr, Digital merging of Landsat TM and digitized NHAP data for 1:24,000 scale image mapping. Photogramm Eng Remote Sens. 52(10), 1637–1646 (1986)
66. VJD Tsai, Frequency-based fusion of multiresolution images, in 2003 IEEE International Geoscience and Remote Sensing Symposium IGARSS 2003. 6, 3665–3667 (2003)
67. L Wald, T Ranchin, M Mangolini, Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images. Photogramm Eng Remote Sens. 63, 691–699 (1997)
68. M González-Audícana, J Saleta, R García Catalán, R García, Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition. IEEE Trans Geosc Remote Sens. 42(6), 1291–1298 (2004)
69. MM Khan, L Alparone, J Chanussot, Pansharpening quality assessment using the modulation transfer functions of instruments. IEEE Trans Geosc Remote Sens. 47(11), 3880–3891 (2009)
70. C Thomas, L Wald, A MTF-based distance for the assessment of geometrical quality of fused products, in 2006 9th International Conference on Information Fusion, pp. 1–7 (2006)
71. JC Price, Combining multispectral data of differing spatial resolution. IEEE Trans Geosc Remote Sens. 37(3), 1199–1203 (May 1999). doi:10.1109/36.763272
72. O Punska, Bayesian approaches to multi-sensor data fusion, (Master’s thesis, University of Cambridge, 1999)
73. D Fasbender, J Radoux, P Bogaert, Bayesian data fusion for adaptable image pansharpening. IEEE Transactions On Geoscience And Remote Sensing 46, 1847–1857 (2008)
74. RC Hardie, MT Eismann, GL Wilson, MAP estimation for hyperspectral image resolution enhancement using an auxiliary sensor. IEEE Trans Image Process. 13(9), 1174–1184 (2004). doi:10.1109/TIP.2004.829779
75. R Molina, M Vega, J Mateos, AK Katsaggelos, Variational posterior distribution approximation in Bayesian super resolution reconstruction of multispectral images. Applied And Computational Harmonic Analysis 12, 1–27 (2007)
76. M Mateos, J Vega, R Molina, A Katsaggelos, Super resolution of multispectral images using TV image models, in 2th Int Conf on Knowledge- Based and Intelligent Information & Eng Sys, pp. 408–415 (2008)
77. HG Kitaw, Image pan-sharpening with Markov random field and simulated annealing. PhD dissertation, International Institute for Geo-information Science and Earth Observation, NL (2007)
78. MT Eismann, RC Hardie, Hyperspectral resolution enhancement using highresolution multispectral imagery with arbitrary response functions. IEEE Transactions On Geoscience And Remote Sensing 43(3), 455–465 (2005)
79. M Vega, J Mateos, R Molina, A Katsaggelos, Super resolution of multispectral images using l1 image models and interband correlations. 2009 IEEE International Workshop on Machine Learning for Signal Processing, 1–6 (2009)
80. MT Eismann, RC Hardie, Application of the stochastic mixing model to hyperspectral resolution enhancement. IEEE Transactions On Geoscience And Remote Sensing. 42(9), 1924–1933 (2004)
81. A Katsaggelos, R Molina, J Mateos, Super Resolution Of Images And Video, Synthesis Lectures on Image, Video, and Multimedia Processing, Morgan & Claypool, (2007)
82. R Molina, J Mateos, AK Katsaggelos, RZ Milla, A new super resolution Bayesian method for pansharpening Landsat ETM+ imagery, in 9th International Symposium on Physical Measurements and Signatures in Remote Sensing (ISPMSRS), pp. 280–283 (2005)
83. GZ Rong, W Bin, ZL Ming, Remote sensing image fusion based on Bayesian linear estimation. Science in China Series F: Information Sciences. 50(2), 227–240 (2007). doi:10.1007/s11432-007-0008-7
84. W Niu, J Zhu, W Gu, J Chu, Four statistical approaches for multisensor data fusion under non-Gaussian noise, in IITA International Conference on Control, Automation and Systems Engineering (2009)
85. L Brandenburg, H Meadows, Shaping filter representation of nonstationary colored noise. IEEE Transactions on Information Theory 17, 26–31 (1971). doi:10.1109/TIT.1971.1054585
86. PJ Burt, EH Adelson, The Laplacian pyramid as a compact image code. IEEE Transactions On Communications. COM-3l(4), 532–540 (1983)
87. J Zhang, Multi-source remote sensing data fusion: Status and trends. International Journal of Image and Data Fusion 1(1), 5–24 (2010). doi:10.1080/19479830903561035
88. L Alparone, S Baronti, A Garzelli, Assessment of image fusion algorithms based on noncritically-decimated pyramids and wavelets, in Proc IEEE 2001 International Geoscience and Remote Sensing Symposium IGARSS ‘01. 2, 852–854 (2001)
89. MG Kim, I Dinstein, L Shaw, A prototype filter design approach to pyramid generation. IEEE Trans on Pattern Anal and Machine Intell. 15(12), 1233–1240 (1993). doi:10.1109/34.250842
90. B Aiazzi, L Alparone, S Baronti, A Garzelli, Spectral information extraction by means of MS+PAN fusion. Proceedings of ESA-EUSC 2004 - Theory and Applications of Knowledge-Driven Image Information Mining with Focus on Earth Observation, 20.1 (2004)
91. M González-Audícana, X Otazu, Comparison between Mallat’s and the a’trous discrete wavelet transform based algorithms for the fusion of multispectral and panchromatic images. Int J Remote Sens. 26(3), 595–614 (2005). doi:10.1080/01431160512331314056
92. B Garguet-Duport, J Girel, JM Chassery, G Pautou, The use of multiresolution analysis and wavelets transform for merging SPOT panchromatic and multispectral image data. Photogramm Eng Remote Sens. 62(9), 1057–1066 (1996)
93. DA Yocky, Image merging and data fusion by means of the discrete twodimensional wavelet transform. Optical Society of America. 12(9), 1834–1841 (September 1995). doi:10.1364/JOSAA.12.001834
94. J Nuñez, X Otazu, O Fors, A Prades, V Pala, R Arbiol, Multiresolution-based image fusion with additive wavelet decomposition. IEEE Trans Geosci Remote Sens. 37(3), 1204–1211 (1999). doi:10.1109/36.763274
95. Y Chibani, A Houacine, The joint use of IHS transform and redundant wavelet decomposition for fusing multispectral and panchromatic image. Int J Remote Sensing 23(18), 3821–3833 (2002). doi:10.1080/01431160110107626
96. K Amolins, Y Zhang, P Dare, Wavelet based image fusion techniques - an introduction, review and comparison. ISPRS J Photogramm 249–263 (2007)
97. PS Pradhan, RL King, NH Younan, DW Holcomb, Estimation of the number of decomposition levels for a wavelet-based multiresolution multisensor image fusion. IEEE Trans Geosc Remote Sens. 44(12), 3674–3686 (2006)
98. M Song, X Chen, P Guo, A fusion method for multispectral and panchromatic images based on HSI and contourlet transformation, in Proc 10th Workshop on Image Analysis for Multimedia Interactive Services WIAMIS ‘09, pp. 77–80 (2009)
99. AM ALEjaily, IAE Rube, MA Mangoud, Fusion of remote sensing images using contourlet transform. Springer Science, 213–218 (2008)
100. S Yang, M Wang, YX Lu, W Qi, L Jiao, Fusion of multiparametric SAR images based on SW-nonsubsampled contourlet and PCNN. Signal Processing 89, 2596–2608 (2009). doi:10.1016/j.sigpro.2009.04.027
101. J Wu, H Huang, J Liu, J Tian, Remote sensing image data fusion based on HIS and local deviation of wavelet transformation, in Proc IEEE Int Conf on Robotics and Biomimetics ROBIO 2004, pp. 564–568 (2004)
102. X Otazu, M González-Audícana, O Fors, J Núñez, Introduction of sensor spectral response into image fusion methods: Application to wavelet-based methods. IEEE Trans Geosci Remote Sens. 43(10), 2376–2385 (2005)
103. I Amro, J Mateos, M Vega, General contourlet pansharpening method using Bayesian inference, in 2010 European Signal Processing Conference (EUSIPCO- 2010), pp. 294–298 (2010)
104. Y Zhang, SD Backer, P Scheunders, Bayesian fusion of multispectral and hyperspectral image in wavelet domain. IEEE International Geoscience and Remote Sensing Symposium, IGARSS. 5, 69–72 (2008)
105. V Meenakshisundaram, Quality assessment of IKONOS and Quickbird fused images for urban mapping, (Master’s thesis, University of Calgary, 2005)
106. L Wald, Data Fusion Definitions and Architectures: Fusion of Images of Different Spatial Resolutions (Les Presses de l’Ecole des Mines, Paris, 2002)
107. F Nencini, A Garzelli, S Baronti, L Alparone, Remote sensing image fusion using the curvelet transform. Information Fusion. 8, 143–156 (2007). doi:10.1016/j.inffus.2006.02.001
108. V Vijayaraj, NH Younan, CG O’Hara, Quantitative analysis of pansharpened images. Optical Engineering 45(4), 046202 (2006). doi:10.1117/1.2195987
109. Z Wang, AC Bovik, HR Sheikh, EP Simoncelli, Image quality assessment: From error visibility to structural similarity. IEEE Trans Image Process. 13, 600–612 (2004). doi:10.1109/TIP.2003.819861
110. Z Wang, AC Bovik, A universal image quality index. IEEE Signal Process Lett. 9(3), 81–84 (2002). doi:10.1109/97.995823
111. L Wald, Quality of high resolution synthesized images: Is there a simple criterion?, in Proc Int Conf Fusion of Earth Data. 1, 99–105 (2000)
112. Q Du, NH Younan, R King, VP Shah, On the performance evaluation of pansharpening techniques. IEEE Trans Geosc Remote Sens L. 4(4), 518–522 (2007)
113. L Alparone, S Baronti, A Garzelli, F Nencini, A global quality measurement of pan-sharpened multispectral imagery. IEEE Geoscience And Remote Sensing Letters 1(4), 313–317 (2004). doi:10.1109/LGRS.2004.836784
114. C Thomas, L Wald, Comparing distances for quality assessment of fused images, in Proceedings of the 26th EARSeL Symposium “New Strategies for European Remote Sensing”, pp. 101–111 (2007)
115. M Lillo-Saavedra, C Gonzalo, A Arquero, E Martinez, Fusion of multispectral and panchromatic satellite sensor imagery based on tailored filtering in the fourier domain. Int J Remote Sens. 26, 1263–1268 (2005). doi:10.1080/01431160412331330239
116. L Alparone, L Wald, J Chanussot, P Gamba, LM Bruce, Comparison of pansharpening algorithms: Outcome of the 2006 GRS-S data-fusion contest. IEEE Trans Geosc Remote Sens. 45(10), 3012–3021 (2007)
117. Y Jinghui, Z Jixian, L Haitao, S Yushan, P Pengxian, Pixel level fusion methods for remote sensing images: A current review in ISPRS TC VII Symposium 100 Years ISPRS, ed. by W Wagner, B Szkelys, 680–686 (2010)
118. L Alparone, B Aiazzi, S Baronti, A Garzelli, F Nencini, M Selva, Multispectral and panchromatic data fusion assessment without reference. Photogrammetric Engineering & Remote Sensing 74, 193–200 (2008)
119. L Alparone, B Aiazzi, S Baronti, A Garzelli, F Nencini, A new method for MS + Pan image fusion assessment without reference, in Proc IEEE Int Conf Geoscience and Remote Sensing Symp IGARSS 2006, pp. 3802–3805 (2006)