An overview of multi-modal medical image fusion

Neurocomputing

Volumn 215, Issue , 2016, Pages 3-20

  Du, Jiao ^a   Li, Weisheng ^a   Lu, Ke ^b   Xiao, Bin ^a  

^a CHONGQING UNIVERSITY OF POSTS AND TELECOMMUNICATIONS   (China)

^b UNIVERSITY OF CHINESE ACADEMY OF SCIENCES   (China)

Author keywords

Image decomposition; Image fusion; Image fusion rules; Image quality assessment; Image reconstruction; Multi modal

Indexed keywords

ARTIFICIAL INTELLIGENCE; BENCHMARKING; COMPUTER VISION; IMAGE FUSION; IMAGE PROCESSING; IMAGE QUALITY; IMAGE RECONSTRUCTION; LEARNING SYSTEMS; MEDICAL IMAGING; PATTERN RECOGNITION;

BENCHMARK DATASETS; HOT TOPICS; IMAGE DECOMPOSITION; IMAGE QUALITY ASSESSMENT; IMAGING QUALITY; MULTI-MODAL; MULTIPLE IMAGE; MULTIPLE IMAGING MODALITY;

MEDICAL IMAGE PROCESSING;

ARTICLE; BENCHMARKING; COMPUTER ASSISTED TOMOGRAPHY; FUZZY LOGIC; HUMAN; IMAGE ANALYSIS; IMAGE FUSION; IMAGE PROCESSING; IMAGE QUALITY; IMAGE RECONSTRUCTION; MULTIMODAL IMAGING; NUCLEAR MAGNETIC RESONANCE IMAGING; POSITRON EMISSION TOMOGRAPHY; PRIORITY JOURNAL; SINGLE PHOTON EMISSION COMPUTER TOMOGRAPHY; STATISTICS; VISUAL SYSTEM; WAVELET ANALYSIS;

EID: 84977557211     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2015.07.160     Document Type: Article

References (76)

1. [1] James, A.P., Dasarathy, B.V., Medical image fusion: a survey of the state of the art. Inf. Fusion 19 (2014), 4–19.
2. [2] Shen, R., Cheng, I., Basu, A., Cross-scale coefficient selection for volumetric medical image fusion. IEEE Trans. Biomed. Eng. 60:4 (2013), 1069–1079.
3. [3] He, C., Liu, Q., Li, H., et al. Multimodal medical image fusion based on IHS and PCA. Proc. Eng. 7 (2010), 280–285.
4. [4] Daneshvar, S., Ghassemian, H., MRI and PET image fusion by combining IHS and retina-inspired models. Inf. Fusion 11:2 (2010), 114–123.
5. [5] Matsopoulos, G.K., Marshall, S., Brunt, J.N.H. 1994. Multiresolution morphological fusion of MR and CT images of the human brain, IEE Proceedings of Visual Image Signal Processing, vol. 141 (3), pp. 137–142.
6. [6] Patil, U., Mudengudi, U. 2011. Image fusion using hierarchical PCA, Proceedings of Internal Conference Image Information Processing, pp. 1–6.
7. [7] Guihong, Q., Dali, Z., Pingfan, Y., Medical image fusion by wavelet transform modulus maxima. Opt. Express 9:4 (2001), 184–190.
8. [8] Singh, R., Khare, A., Fusion of multimodal medical images using Daubechies complex wavelet transform–A multiresolution approach. Inf. Fusion, 2012.
9. [9] Zheng, Y., Essock, A.E., Hansen, B.C., et al. A new metric based on extended spatial frequency and its application to DWT based fusion algorithms. Inf. Fusion 8:2 (2007), 177–192.
10. [10] Singh, R., Vatsa, M., Noore, A. 2009. Multimodal medical image fusion using redundant discrete wavelet transform, In: IEEE Seventh International Conference on Advances in Pattern Recognition, 2009, pp. 232–235.
11. [11] Liu, Y., Yang, J., Sun, J. 2010. PET/CT medical image fusion algorithm based on multiwavelet transform, In: IEEE 2nd International Conference on Advanced Computer Control, 2010, pp. 264–268.
12. [12] Xue-jun, W., Ying, M. 2010. A medical image fusion algorithm based on lifting wavelet transform, In: IEEE International Conference on Artificial Intelligence and Computational Intelligence, 2010 (3), pp. 474–476.
13. [13] Yang, L., Guo, B.L., Ni, W., Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform. Neurocomputing 72:1 (2008), 203–211.
14. [14] Bhatnagar, G., Wu, Q.M., Liu, Z., Directive contrast based multimodal medical image fusion in NSCT domain. IEEE Trans. Multimed. 15:5 (2013), 1014–1024.
15. [15] Miao, Q., Shi, C., Xu, P., et al. A novel algorithm of image fusion using shearlets. Opt. Commun. 284:6 (2011), 1540–1547.
16. [16] Wang, L., Li, B., Tian, L., Multi-modal medical volumetric data fusion using 3D discrete shearlet transform and global-to-local rule. IEEE Trans. Biomed. Eng., 2014.
17. [17] Yin, H., Li, S., Fang, L., Simultaneous image fusion and super-resolution using sparse representation. Inf. Fusion 14:3 (2013), 229–240.
18. [18] Yang, B., Li, S., Pixel-level image fusion with simultaneous orthogonal matching pursuit. Inf. Fusion 13:1 (2012), 10–19.
19. [19] Yin, H., Li, S., Multimodal image fusion with joint sparsity model. Opt. Eng. 50:6 (2011), 7–10.
20. [20] Yu, N., Qiu, T., Bi, F., et al. Image features extraction and fusion based on joint sparse representation. IEEE J. Sel. Top. Signal Process. 5:5 (2011), 1074–1082.
21. [21] Li, S., Yin, H., Fang, L., Group-sparse representation with dictionary learning for medical image denoising and fusion. IEEE Trans. Biomed. Eng. 59:12 (2012), 3450–3459.
22. [22] Hu J, J., Li, S., The multiscale directional bilateral filter and its application to multisensor image fusion. Inf. Fusion 13:3 (2012), 196–206.
23. [23] Li, S., Kang X, X., Hu, J., Image fusion with guided filtering. IEEE Trans. Image Process. 22:7 (2013), 2864–2875.
24. [24] Xu, Z., Medical image fusion using multi-level local extrema. Inf. Fusion 19 (2014), 38–48.
25. [25] Barra, V., Boire, J.Y., A general framework for the fusion of anatomical and functional medical images. NeuroImage 13:3 (2001), 410–424.
26. [26] Wang, P.Y., Dang, J.W., Li, Q. et al. 2007. Multimodal medical image fusion using fuzzy radial basis function neural networks, In: IEEE International Conference on Wavelet Analysis and Pattern Recognition 2007 (2), pp. 778–782.
27. [27] Javed, U., Riaz, M.M., Ghafoor, A., et al. Mri and pet image fusion using fuzzy logic and image local features. Sci. World J., 2014.
28. [28] W. Hao-quan, X. Hao, Multi-mode medical image fusion algorithm based on principal component analysis, In: IEEE International Symposium on Computer Network and Multimedia Technology, 2009, pp. 1–4.
29. [29] H. Chen, A multiresolution image fusion based on principle component analysis, In: IEEE Fourth International Conference on Image and Graphics, 2007, pp. 737–741.
30. [30] Vijayarajan, R., Muttan, S., Iterative block level principal component averaging medical image fusion. Opt. Int. J. Light. Electron Opt. 125:17 (2014), 4751–4757.
31. [31] Wang, L., Li, B., Tian, L.F., EGGDD: an explicit dependency model for multi-modal medical image fusion in shift-invariant shearlet transform domain. Inf. Fusion 19 (2014), 29–37.
32. [32] Wang, L., Li, B., Tian, L., Multi-modal medical image fusion using the inter-scale and intra-scale dependencies between image shift-invariant shearlet coefficients. Inf. Fusion 19 (2014), 20–28.
33. [33] Bhatnagar, G., Wu, Q.M.J., Liu, Z., Human visual system inspired multi-modal medical image fusion framework. Expert. Syst. Appl. 40:5 (2013), 1708–1720.
34. [34] Zhang, Q.P., Tang, W.J., Lai, L.L., et al. Medical diagnostic image data fusion based on wavelet transformation and self-organising features mapping neural networks. IEEE Proc. Int. Conf. Mach. Learn. Cybern. 2004 5 (2004), 2708–2712.
35. [35] Liu, Z., Yin, H., Chai, Y., et al. A novel approach for multimodal medical image fusion. Expert. Syst. Appl. 41:16 (2014), 7425–7435.
36. [36] Li, W., Zhu, X.F., A new image fusion algorithm based on wavelet packet analysis and PCNN. IEEE Proc. Int. Conf. Mach. Learn. Cybern. 2005 9 (2005), 5297–5301.
37. [37] Wang, N., Ma, Y., Zhan, K., et al. Multimodal medical image fusion framework based on simplified PCNN in nonsubsampled contourlet transform domain. J. Multimed. 8:3 (2013), 270–276.
38. [38] Daneshvar, S., Ghassemian, H., MRI and PET images fusion based on human retina model. J. Zhejiang Univ. Sci. A 8:10 (2007), 1624–1632.
39. [39] Jang, J.H., Bae, Y., Ra, J.B., Contrast-enhanced fusion of multisensor images using subband-decomposed multiscale retinex. IEEE Trans. Image Process. 21:8 (2012), 3479–3490.
40. [40] Zheng, Y., Essock, E.A., Hansen, B.C., et al. A new metric based on extended spatial frequency and its application to DWT based fusion algorithms. Inf. Fusion 8:2 (2007), 177–192.
41. [41] Z. Wencang, C. Lin, Medical image fusion method based on wavelet multi-resolution and entropy, IEEE International Conference on Automation and Logistics, 2008, pp. 2329–2333).
42. [42] Bhatnagar, G., Wu, Q.M.J., Liu, Z., Directive contrast based multimodal medical image fusion in NSCT domain. IEEE Trans. Multimed. 15:5 (2013), 1014–1024.
43. [43] Garg, S., Ushah Kiran, K., Mohan, R. et al. 2006. Multilevel medical image fusion using segmented image by level set evolution with region competition, In: IEEE Engineering 27th Annual International Conference of Medicine and Biology Society, 2005, pp. 7680–7683.
44. [44] Li, X., Tian, X., Sun, Y., et al. Medical image fusion by multi-resolution analysis of wavelets transform. Wavel. Anal. Appl., 2007, 389–396.
45. [45] Sheikh, H.R., Bovik, A.C., Image information and visual quality. IEEE Trans. Image Process. 15:2 (2006), 430–444.
46. [46] Yang, Y., Wang, X., Guan, T., et al. A multi-dimensional image quality prediction model for user-generated images in social networks. Inf. Sci. 281 (2014), 601–610.
47. [47] Yang, Y., Wang, X., Liu, Q., et al. User models of subjective image quality assessment on virtual viewpoint in free-viewpoint video system. Multimed. Tools Appl., 2014, 1–21.
48. [48] Yang, Y., Dai, Q., Contourlet-based image quality assessment for synthesised virtual image. Electron. Lett. 46:7 (2010), 492–494.
49. [49] Wang, Z., Lu, L., Bovik, A.C., Video quality assessment based on structural distortion measurement. Signal Process. Image Commun. 19:2 (2004), 121–132.
50. [50] Hore, A., Ziou, D., Image quality metrics: PSNR vs. SSIM. ICPR 34 (2010), 2366–2369.
51. [51] Zheng, S., Shi, W.Z., Liu, J., et al. Multisource image fusion method using support value transform. IEEE Trans. Image Process. 16:7 (2007), 1831–1839.
52. [52] Zhao, H., Shang, Z., Tang, Y.Y., et al. Multi-focus image fusion based on the neighbor distance. Pattern Recognit. 46:3 (2013), 1002–1011.
53. [53] Xiao-Bo, Q., Jing-Wen, Y., Hong-Zhi, X., et al. Image fusion algorithm based on spatial frequency-motivated pulse coupled neural networks in nonsubsampled contourlet transform domain. Acta Autom. Sin. 34:12 (2008), 1508–1514.
54. [54] Buades, A., Coll, B. Morel, J.M. 2005. A non-local algorithm for image denoising, In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005 (2), pp. 60–65.
55. [55] Petrović, V., Subjective tests for image fusion evaluation and objective metric validation. Inf. Fusion 8:2 (2007), 208–216.
56. [56] Sheikh, H.R., Sabir, M.F., Bovik, A.C., A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process. 15:11 (2006), 3440–3451.
57. [57] Wang, Z., Bovik, A.C., Sheikh, H.R., et al. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13:4 (2004), 600–612.
58. [58] Hossny, M., Nahavandi, S., Creighton, D., Comments on ‘Information measure for performance of image fusion’. Electron. Lett. 44:18 (2008), 1066–1067.
59. [59] Y. Horibe, Entropy and correlation, IEEE Trans. System, Man, Cybern, SMC-15 (1985), pp. 641–642.
60. [60] Wang, Z., Bovik, A.C., A universal image quality index. IEEE Trans. Signal Process. Lett. 9:3 (2002), 81–84.
61. [61] Cvejic, N., Loza, A., Bull, D., et al. A similarity metric for assessment of image fusion algorithms. Int. J. Signal Process. 2:3 (2005), 178–182.
62. [62] Zhao, J., Laganiere, R., Liu, Z., Performance assessment of combinative pixel-level image fusion based on an absolute feature measurement. Int. J. Innov. Comput. Inf. Control. 3:6 (2007), 1433–1447.
63. [63] Piella, G., Heijmans, H., A new quality metric for image fusion. IEEE Int. Conf. Proc. Image Process. 2003, 3(2), 2003 (173-6).
64. [64] Xydeas, C.S., Petrovic, V.S., Objective pixel-level image fusion performance measure. Int. Soc. Opt. Photon. 2000, 2000, 89–98.
65. [65] Li, S., Yang, B., Hu, J., Performance comparison of different multi-resolution transforms for image fusion. Inf. Fusion 12:2 (2011), 74–84.
66. [66] Xydeas, C.S., Petrović, V., Objective image fusion performance measure. Electron. Lett. 36:4 (2000), 308–309.
67. [67] Mittal, A., Soundararajan, R., Bovik, A.C., Making a “completely blind” image quality analyzer. IEEE Trans. Signal Process. Lett. 20:3 (2013), 209–212.
68. [68] Johnson, K.A., Becker, J.A. 2016. The Whole Brain Altas [Online], (Available): 〈 http://www.med.harvard.edu/aanlib/〉.
69. [69] Li, S., Yang, B., Multifocus image fusion by combining curvelet and wavelet transform. Pattern Recognit. Lett. 29:9 (2008), 1295–1301.
70. [70] Zhang, Q., Guo, B., Multifocus image fusion using the nonsubsampled contourlet transform. Signal Process. 89:7 (2009), 1334–1346.
71. [71] Nencini, F., Garzelli, A., Baronti, S., et al. Remote sensing image fusion using the curvelet transform. Inf. Fusion 8:2 (2007), 143–156.
72. [72] Subr, K., Soler, C., Durand, F., Edge-preserving multiscale image decomposition based on local extrema. ACM Trans. Graph., 28(5), 2009, 147.
73. [73] He, K., Sun, J., Tang, X., Guided Image filtering. IEEE Trans. Pattern Anal. Mach. Intell. 35:6 (2013), 1397–1409.
74. [74] Durand, F., Dorsey, J., Fast bilateral filtering for the display of high-dynamic-range images. ACM Trans. Graph. 21:3 (2002), 257–266.
75. [75] Nill, N., A visual model weighted cosine transform for image compression and quality assessment IEEE Trans. Communications 33:6 (1985), 551–557.
76. [76] Li, M., Cai, W., Tan, Z., A region-based multi-sensor image fusion scheme using pulse-coupled neural network. Pattern Recognit. Lett. 27:16 (2006), 1948–1956.