Sparse representation based multi-sensor image fusion for multi-focus and multi-modality images: A review

Information Fusion

Volumn 40, Issue , 2018, Pages 57-75

  Zhang, Qiang ^a   Liu, Yi ^a   Blum, Rick S ^b   Han, Jungong ^c   Tao, Dacheng ^d  

^a XIDIAN UNIVERSITY   (China)

^b Lehigh University   (United States)

^c LANCASTER UNIVERSITY   (United Kingdom)

^d UNIVERSITY OF SYDNEY   (Australia)

Author keywords

Activity level; Dictionary learning; Image fusion; Sparse representation

Indexed keywords

IMAGE PROCESSING;

ACTIVITY LEVELS; DICTIONARY LEARNING; MULTI MODALITY IMAGE; MULTI-SCALE TRANSFORMS; MULTIFOCUS IMAGE FUSION; OVER-COMPLETE DICTIONARIES; SPARSE REPRESENTATION; THEORETICAL INVESTIGATIONS;

IMAGE FUSION;

EID: 85020791523     PISSN: 15662535     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.inffus.2017.05.006     Document Type: Article

References (85)

1. Liu, Y., Liu, S., Wang, Z., Multi-focus image fusion with dense SIFT. Inf. Fusion 23 (2015), 139–155.
2. Li, S., Kang, X., Fang, L., Hu, J., Yin, H., Pixel-level image fusion: a survey of the state of the art. Inf. Fusion 33 (2017), 100–112.
3. Zhang, Q., Wang, Y., Levine, M.D., Yuan, X., Wang, L., Multisensor video fusion based on higher order singular value decomposition. Inf. Fusion 24 (2015), 54–71.
4. Pertuz, S., Puig, D., Garcia, M.A., Fusiello, A., Generation of all-in-focus images by noise-robust selective fusion of limited depth-of-field images. IEEE Trans. Image Process. 22:3 (2013), 1242–1251.
5. James, A.P., Dasarathy, B., Medical image fusion: a survey of the state of the art. Inf. Fusion 19 (2014), 4–19.
6. Li, S., Kang, X., Hu, J., Image fusion with guided filtering. IEEE Trans. Image Process. 22:7 (2013), 2864–2875.
7. Li, S., Yang, B., Hu, J., Performance comparison of different multi-resolution transforms for image fusion. Inf. Fusion 12:2 (2011), 74–84.
8. Pajares, G., Cruz, J.M.D.L., A wavelet-based image fusion tutorial. Pattern Recognit. 37:9 (2004), 1855–1872.
9. Zhang, Z., Blum, R.S., A categorization of multiscale-decomposition-based image fusion schemes with a performance study for a digital camera application. Proc. IEEE 87:8 (1999), 1315–1326.
10. Zhang, Q., Ma, Z., Wang, L., Multimodality image fusion by using both phase and magnitude information. Pattern Recognit. Lett. 34:2 (2013), 185–193.
11. Liu, Y., Jin, J., Wang, Q., Shen, Y., Dong, X., Region level based multi-focus image fusion using quaternion wavelet and normalized cut. Signal Process. 97:7 (2014), 9–30.
12. Guo, L., Dai, M., Zhu, M., Multifocus color image fusion based on quaternion curvelet transform. Opt. Express 20:17 (2012), 18846–18860.
13. Wang, L., Li, B., Tian, L., Multi-modal medical image fusion using the inter-scale and intra-scale dependencies between image shift-invariant shearlet coefficients. Inf. Fusion 19:1 (2014), 20–28.
14. Upla, K.P., Joshi, M.V., Gajjar, P.P., An edge preserving multiresolution fusion: use of contourlet transform and MRF prior. IEEE Trans. Geosci. Remote Sens. 53:6 (2015), 3210–3220.
15. Wright, J., Yang, A.Y., Ganesh, A., Sastry, S., Ma, Y., Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31:2 (2009), 210–227.
16. Guha, T., Ward, R.K., Learning sparse representations for human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 34:8 (2012), 1576–1588.
17. Yuan, X., Liu, X., Yan, S., Visual classification with multitask joint sparse representation. IEEE Trans. Image Process. 21:10 (2012), 4349–4360.
18. Yang, B., Li, S., Multifocus image fusion and restoration with sparse representation. IEEE Trans. Instrum. Meas. 59:4 (2010), 884–892.
19. Liu, Y., Liu, S., Wang, Z., A general framework for image fusion based on multi-scale transform and sparse representation. Inf. Fusion 24 (2015), 147–164.
20. Zhang, Q., Maldague, X., An adaptive fusion approach for infrared and visible images based on NSCT and compressed sensing. Infrared Phys. Technol. 74 (2016), 11–20.
21. Wei, Q., Bioucas-Dias, J., Dobigeon, N., Tourneret, J.-Y., Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Trans. Geosci. Remote Sens. 53:7 (2015), 3658–3668.
22. Liu, Y., Wang, Z., Simultaneous image fusion and denoising with adaptive sparse representation. IET Image Process. 9:5 (2015), 347–357.
23. Nejati, M., Samavi, S., Shirani, S., Multi-focus image fusion using dictionary-based sparse representation. Inf. Fusion 25 (2015), 72–84.
24. Aharon, M., Elad, M., Bruckstein, A., K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54:11 (2006), 4311–4322.
25. Yin, H., Li, S., Fang, L., Simultaneous image fusion and super-resolution using sparse representation. Inf. Fusion 14:3 (2013), 229–240.
26. Guo, M., Zhang, H., Li, J., Zhang, L., Shen, H., An online coupled dictionary learning approach for remote sensing image fusion. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 7:4 (2014), 1284–1294.
27. Wang, J., Peng, J., Feng, X., He, G., Fan, J., Fusion method for infrared and visible images by using non-negative sparse representation. Infrared Phys. Technol. 67 (2014), 477–489.
28. Zhang, Q., Levine, M.D., Robust multi-focus image fusion using multi-task sparse representation and spatial context. IEEE Trans. Image Process. 25:5 (2016), 2045–2058.
29. Li, S., Yin, H., Fang, L., Group-sparse representation with dictionary learning for medical image denoising and fusion. IEEE Trans. Biomed. Eng. 59:12 (2012), 3450–3459.
30. Zhang, Q., Fu, Y., Li, H., Zou, J., Dictionary learning method for joint sparse representation-based image fusion. Opt. Eng. 52:5 (2013), 1–11.
31. Kim, M., Han, D.K., Ko, H., Joint patch clustering-based dictionary learning for multimodal image fusion. Inf. Fusion 27 (2016), 198–214.
32. Vivone, G., Alparone, L., Chanussot, J., Mura, M.D., Garzelli, A., Licciardi, G., Restaino, R., Wald, L., A critical comparison among pansharpening algorithms. IEEE Trans. Geosci. Remote Sens. 53:5 (2015), 2565–2586.
33. Loncan, L., de Almeida, L.B., Bioucas-Dias, J.M., Briottet, X., Chanussot, J., Dobigeon, N., Fabre, S., Liao, W., Licciardi, G.A., Simoes, M., et al. Hyperspectral pansharpening: a review. IEEE Geosci. Remote Sens. Mag. 3:3 (2015), 27–46.
34. Wright, J., Ma, Y., Mairal, J., Sapiro, G., Huang, T.S., Yan, S., Sparse representation for computer vision and pattern recognition. Proc. IEEE 98:6 (2010), 1031–1044.
35. Zhang, Z., Xu, Y., Yang, J., Li, X., Zhang, D., A survey of sparse representation: algorithms and applications. IEEE Access 3 (2015), 490–530.
36. Mallat, S., Zhang, Z., Matching pursuits with time-frequency dictionaries. IEEE Trans. Signal Process. 41:12 (1993), 3397–3415.
37. Bruckstein, A.M., Donoho, D.L., Elad, M., From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev. 51:1 (2009), 34–81.
38. Tropp, J.A., Gilbert, A.C., Strauss, M.J., Algorithms for simultaneous sparse approximation: part I: greedy pursuit. Signal Process. 86:3 (2006), 572–588.
39. Candes, E.J., Li, X., Ma, Y., Wright, J., Robust principal component analysis?. J. ACM 58:3 (2011), 1–37.
40. 1 -norm minimization problems when the solution may be sparse. IEEE Trans. Inf. Theory 54:11 (2008), 4789–4812.
41. Lee, D., Seung, H.S., Learning the parts of objects by non-negative matrix factorization. Nature 401:6755 (1999), 788–791.
42. Dong, W., Fu, F., Shi, G., Cao, X., Wu, J., Li, G., Li, X., Hyperspectral image super-resolution via non-negative structured sparse representation. IEEE Trans. Image Process. 25:5 (2016), 2337–2352.
43. Baron, D., Wakin, M.B., Duarte, M.F., Sarvotham, S., Baraniuk, R.G., Distributed compressed sensing. IEEE Trans. Inf. Theory 52:12 (2006), 5406–5425.
44. Yin, H., Li, S., Multimodal image fusion with joint sparsity model. Opt. Eng. 50:6 (2011), 1–10.
45. Yu, N., Qiu, T., Bi, F., Wang, A., Image features extraction and fusion based on joint sparse representation. IEEE J. Sel. Top. Signal Process. 5:5 (2011), 1074–1082.
46. Majumdar, A., Ward, R.K., Fast group sparse classification. Can. J. Electr. Comput. Eng. 34:4 (2009), 136–144.
47. Lin, Z., Liu, R., Su, Z., Linearized alternating direction method with adaptive penalty for low-rank representation. Advances in Neural Information Processing Systems, 2011, 612–620.
48. Zhang, Y., Jiang, Z., Davis, L.S., Learning structured low-rank representations for image classification. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2013, 676–683.
49. Lang, C., Liu, G., Yu, J., Yan, S., Saliency detection by multitask sparsity pursuit. IEEE Trans. Image Process. 21:3 (2012), 1327–1338.
50. Yang, B., Li, S., Pixel-level image fusion with simultaneous orthogonal matching pursuit. Inf. Fusion 13:1 (2012), 10–19.
51. Landweber, L., An iteration formula for Fredholm integral equations of the first kind. Am. J. Math. 73:3 (1951), 615–624.
52. Horn, R.A., Johnson, C.R., Matrix Analysis. 1985, Cambridge University Press, Cambridge, UK.
53. Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A., Non-local sparse models for image restoration. Proceedings of IEEE International Conference on Computer Vision, 2009, 2272–2279.
54. Chatterjee, P., Milanfar, P., Clustering-based denoising with locally learned dictionaries. IEEE Trans. Image Process. 18:7 (2009), 1438–1451.
55. Dong, W., Li, X., Zhang, L., Shi, G., Sparsity-based image denoising via dictionary learning and structural clustering. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2011, 457–464.
56. Yang, J., Wright, J., Huang, T.S., Ma, Y., Image super-resolution via sparse representation. IEEE Trans. Image Process. 19:11 (2010), 2861–2873.
57. Iqbal, M.T., Chen, J., Unification of image fusion and super-resolution using jointly trained dictionaries and local information contents. IET Image Process. 6:9 (2012), 1299–1310.
58. Ren, K., Xu, F., Super-resolution images fusion via compressed sensing and low-rank matrix decomposition. Infrared Phys. Technol. 68 (2015), 61–68.
59. Zhu, X.X., Bamler, R., A sparse image fusion algorithm with application to pan-sharpening. IEEE Trans. Geosci. Remote Sens. 51:5 (2013), 2827–2836.
60. Ibrahim, R.Y., Alirezaie, J., Babyn, P., Pixel level jointed sparse representation with RPCA image fusion algorithm. Proceedings of International Conference on Telecommunications and Signal Processing, 2015, 592–595.
61. Zhang, Y., Chen, Y., A new image-fusion technique based on blocked sparse representation. Proceedings of International Conference on Computer Science and Information Technology, 2014, 53–60.
62. Yao, Y., Xin, X., Guo, P., OMP or BP ? A comparison study of image fusion based on joint sparse representation. Proceedings of International Conference on Neural Information Processing, 2012, 75–82.
63. Yao, Y., Guo, P., Xin, X., Jiang, Z., Image fusion by hierarchical joint sparse representation. Cogn. Comput. 6:3 (2014), 281–292.
64. Yang, B., Luo, J., Guo, L., Cheng, F., Simultaneous image fusion and demosaicing via compressive sensing. Inf. Process. Lett. 116:7 (2016), 447–454.
65. Lu, X., Zhang, B., Zhao, Y., Liu, H., Pei, H., The infrared and visible image fusion algorithm based on target separation and sparse representation. Infrared Phys. Technol. 67 (2014), 397–407.
66. Zhang, Q., Wang, L., Li, H., Ma, Z., Similarity-based multimodality image fusion with shiftable complex directional pyramid. Pattern Recognit. Lett. 32:13 (2011), 1544–1553.
67. Ding, M., Wei, L., Wang, B., Research on fusion method for infrared and visible images via compressive sensing. Infrared Phys. Technol. 57 (2013), 56–67.
68. Yin, H., Sparse representation with learned multiscale dictionary for image fusion. Neurocomputing 148 (2015), 600–610.
69. Liu, Y., Liu, S., Wang, Z., Medical image fusion by combining nonsubsampled contourlet transform and sparse representation. Chapter in Communications in Computer and Information Science, 484, 2014, 372–381.
70. Yang, B., Li, S., Visual attention guided image fusion with sparse representation. Optik 125:17 (2014), 4881–4888.
71. Vicinanza, M.R., Restaino, R., Vivone, G., Mura, M.D., Chanussot, J., A pansharpening method based on the sparse representation of injected details. IEEE Geosci. Remote Sens. Lett. 12:1 (2015), 180–184.
72. Jiang, C., Zhang, H., Shen, H., Zhang, L., Two-step sparse coding for the pan-sharpening of remote sensing images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 7:5 (2014), 1792–1805.
73. Li, S., Yin, H., Fang, L., Remote sensing image fusion via sparse representations over learned dictionaries. IEEE Trans. Geosci. Remote Sens. 51:9 (2013), 4779–4789.
74. Yin, H., Sparse representation based pansharpening with details injection model. Signal Process. 113:C (2015), 218–227.
75. Li, S., Yang, B., A new pan-sharpening method using a compressed sensing technique. IEEE Trans. Geosci. Remote Sens. 49:2 (2011), 738–746.
76. Zhu, X.X., Grohnfeldt, C., Bamler, R., Exploiting joint sparsity for pansharpening: the J-SparseFI algorithm. IEEE Trans. Geosci. Remote Sens. 54:5 (2016), 2664–2681.
77. Han, C., Zhang, H., Gao, C., Jiang, C., Sang, N., Zhang, L., A remote sensing image fusion method based on the analysis sparse model. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 9:1 (2016), 1–15.
78. Nezhad, Z.H., Karami, A., Heylen, R., Scheunders, P., Fusion of hyperspectral and multispectral images using spectral unmixing and sparse coding. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 9:6 (2016), 1–13.
79. Wang, J., Liu, H., He, N., Exposure fusion based on sparse representation using approximate K-SVD. Neurocomputing 135 (2014), 145–154.
80. Qu, G., Zhang, D., Yan, P.F., Information measure for performance of image fusion. Electron. Lett. 38:7 (2002), 313–315.
81. Xydeas, C.S., Petrovic, V., Objective image fusion performance measure. Electron. Lett. 36:4 (2000), 308–309.
82. Piella, G., Heijmans, H., A new quality metric for image fusion. Proceedings of IEEE International Conference on Image Processing, 2003, 173–176.
83. Liu, Z., Forsyth, D.S., Laganiere, R., A feature-based metric for the quantitative evaluation of pixel-level image fusion. Comput. Vis. Image Understanding 109:1 (2008), 56–68.
84. Zhao, H., Shang, Z., Tang, Y.Y., Fang, B., Multi-focus image fusion based on the neighbor distance. Pattern Recognit. 46:3 (2013), 1002–1011.
85. Liu, Y., Chen, X., Ward, R., Wang, Z.J., Image fusion with convolutional sparse representation. IEEE Signal Process. Lett. 23:12 (2016), 1882–1886.