Image Segmentation-Based Multi-Focus Image Fusion Through Multi-Scale Convolutional Neural Network

IEEE Access

Volumn 5, Issue , 2017, Pages 15750-15761

  Du, Chaoben ^a   Gao, Shesheng ^a  

^a NORTHWESTERN POLYTECHNICAL UNIVERSITY   (China)

Author keywords

Convolutional neural network; decision map; image fusion; multi focus image

Indexed keywords

CONVOLUTION; IMAGE ANALYSIS; IMAGE FUSION; MATHEMATICAL MORPHOLOGY; MATHEMATICAL TRANSFORMATIONS; NEURAL NETWORKS;

ALGORITHM DESIGN AND ANALYSIS; CONVOLUTIONAL NEURAL NETWORK; INITIAL SEGMENTATION; MORPHOLOGICAL OPERATIONS; MULTI SCALE ANALYSIS; MULTIFOCUS IMAGE FUSION; MULTIFOCUS IMAGES; QUANTITATIVE EVALUATION;

IMAGE SEGMENTATION;

EID: 85028919325     PISSN: None     EISSN: 21693536     Source Type: Journal    
DOI: 10.1109/ACCESS.2017.2735019     Document Type: Article

References (46)

1. T. Stathaki, Image Fusion: Algorithms and Applications. San Diego, CA, USA: Academic, 2008.
2. P. J. Burt and E. H. Adelson, "The Laplacian pyramid as a compact image code, " IEEE Trans. Commun., vol. COM-31, no. 4, pp. 532-540, Apr. 1983.
3. A. Toet, "A morphological pyramidal image decomposition, " Pattern Recognit. Lett., vol. 9, no. 4, pp. 255-261, May 1989.
4. H. Li, B. S. Manjunath, S. K. Mitra, "Multisensor image fusion using the wavelet transform, " Graph. Models Image Process., vol. 57, no. 3, pp. 235-245, May 1995.
5. J. J. Lewis, R. O'Callaghan, S. G. Nikolov, D. R. Bull, N. Canagarajah, "Pixel-A nd region-based image fusion with complex wavelets, " Inf. Fusion, vol. 8, no. 2, pp. 119-130, Apr. 2007.
6. Q. Zhang and B.-L. Guo, "Multifocus image fusion using the nonsubsampled contourlet transform, " Signal Process., vol. 89, no. 7, pp. 1334-1346, 2009.
7. G. Piella, "A general framework for multiresolution image fusion: From pixels to regions, " Inf. Fusion, vol. 4, no. 4, pp. 259-280, Dec. 2003.
8. X. Li, H. Li, Z. Yu, Y. Kong, "Multifocus image fusion scheme based on the multiscale curvature in nonsubsampled contourlet transform domain, " Opt. Eng, vol. 54, no. 7, p. 073115, Jul. 2015.
9. Y. Liu, S. Liu, Z.Wang, "A general framework for image fusion based on multi-scale transform and sparse representation, " Inf. Fusion, vol. 24, no. 1, pp. 147-164, Jul. 2015.
10. N. Mitianoudis and T. Stathaki, "Pixel-based and region-based image fusion schemes using ICA bases, " Inf. Fusion, vol. 8, no. 2, pp. 131-142, Apr. 2007.
11. B. Yang and S. Li, "Multifocus image fusion and restoration with sparse representation, " IEEE Trans. Instrum. Meas., vol. 59, no. 4, pp. 884-892, Apr. 2010.
12. J. Liang, Y. He, D. Liu, X. Zeng, "Image fusion using higher order singular value decomposition, " IEEE Trans. Image Process., vol. 21, no. 5, pp. 2898-2909, May 2012.
13. Y. Jiang and M. Wang, "Image fusion with morphological component analysis, " Inf. Fusion, vol. 18, no. 1, pp. 107-118, Jul. 2014.
14. Z. Liu, Y. Chai, H. Yin, J. Zhou, Z. Zhu, "A novel multi-focus image fusion approach based on image decomposition, " Inf. Fusion, vol. 35, pp. 102-116, May 2017.
15. W. Huang and Z. Jing, "Multi-focus image fusion using pulse coupled neural network, " Pattern Recognit. Lett., vol. 28, no. 9, pp. 1123-1132, Jul. 2007.
16. W. Huang and Z. Jing, "Evaluation of focus measures in multi-focus image fusion, " Pattern Recognit. Lett., vol. 28, no. 4, pp. 493-500, Mar. 2007.
17. S. Li, J. T. Kwok, Y. Wang, "Combination of images with diverse focuses using the spatial frequency, " Inf. Fusion, vol. 2, no. 3, pp. 169-176, Sep. 2001.
18. S. Li, J. T. Kwok, Y. Wang, "Multifocus image fusion using artificial neural networks, " Pattern Recognit. Lett., vol. 23, no. 8, pp. 985-997, Jun. 2002.
19. V. Aslantas and R. Kurban, "Fusion of multi-focus images using differential evolution algorithm, " Expert Syst. Appl., vol. 37, pp. 8861-8870, Dec. 2010.
20. I. De and B. Chanda, "Multi-focus image fusion using a morphologybased focus measure in a quad-tree structure, " Inf. Fusion, vol. 14, no. 2, pp. 136-146, Apr. 2013.
21. X. Bai, Y. Zhang, F. Zhou, B. Xue, "Quadtree-based multi-focus image fusion using a weighted focus-measure, " Inf. Fusion, vol. 22, no. 1, pp. 105-118, Mar. 2015.
22. M. Li, W. Cai, Z. Tan, "A region-based multi-sensor image fusion scheme using pulse-coupled neural network, " Pattern Recognit. Lett., vol. 27, no. 16, pp. 1948-1956, 2006.
23. X. Bai, M. Liu, Z. Chen, P.Wang, andY. Zhang, "Multi-focus image fusion through gradient-based decision map construction and mathematical morphology, " IEEE Access, vol. 4, pp. 4749-4760, 2016.
24. S. Li, X. Kang, J. Hu, "Image fusion with guided filtering, " IEEE Trans. Image Process., vol. 22, no. 7, pp. 2864-2875, Jul. 2013.
25. Y. Liu, S. Liu, Z.Wang, "Multi-focus image fusion with dense SIFT, " Inf. Fusion, vol. 23, pp. 139-155, May 2015.
26. Z. Zhou, S. Li, B.Wang, "Multi-scale weighted gradient-based fusion for multi-focus images, " Inf. Fusion, vol. 20, pp. 60-72, Nov. 2014.
27. Z.-H. Zhao, S.-P. Yang, Z.-Q. Ma, "License plate character recognition based on convolutional neural network LeNet-5, " J. Syst. Simul., vol. 22, no. 3, pp. 638-641, 2010.
28. S. Ji, W. Xu, M. Yang, K. Yu, "3D convolutional neural networks for human action recognition, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 1, pp. 221-231, Jan. 2013.
29. O. Abdel-Hamid, A.-R. Mohamed, H. Jiang, L. Deng, G. Penn, D. Yu, "Convolutional neural networks for speech recognition, " IEEE/ACM Trans. Audio, Speech, Language Process., vol. 22, no. 10, pp. 1533-1545, Oct. 2014.
30. E. A. Smirnov, D. M. Timoshenko, S. N. Andrianov, "Comparison of regularization methods for imagenet classification with deep convolutional neural networks, " AASRI Procedia, vol. 6, no. 1, pp. 89-94, 2014.
31. Accessed on Jan. 1, 2017. [Online]. Available: Https://en.wikipedia.org/wiki/Deepfilearning
32. Y. Kim and T. Moon, "Human detection and activity classification based on micro-Doppler signatures using deep convolutional neural networks, " IEEE Geosci. Remote Sens. Lett., vol. 13, no. 1, pp. 8-12, Jan. 2016.
33. S. S. Farfade, M. J. Saberian, L.-J. Li, "Multi-view face detection using deep convolutional neural networks, " in Proc. 5th ACM Int. Conf. Multimedia Retr., 2015, pp. 643-650.
34. J. Long, E. Shelhamer, T. Darrell, "Fully convolutional networks for semantic segmentation, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 3431-3440.
35. Y. Jia, et al., "Caffe: Convolutional architecture for fast feature embedding, " in Proc. ACM Int. Conf. Multimedia, 2014, pp. 675-678.
36. X. Glorot and Y. Bengio, "Understanding the difficulty of training deep feedforward neural networks, " in Proc. Int. Conf. Artif. Intell. Statist., 2010, pp. 249-256.
37. Y. Liu, X. Chen, H. Peng, Z. Wang, "Multi-focus image fusion with a deep convolutional neural network, " Inf. Fusion, vol. 36, pp. 191-207, Jul. 2017.
38. Y. Zhang, X. Bai, T. Wang, "Boundary finding based multi-focus image fusion through multi-scale morphological focus-measure, " Inf. Fusion, vol. 35, pp. 81-101, May 2017.
39. Z. Wang, S. Wang, Y. Zhu, "Multi-focus image fusion based on the improved PCNN and guided filter, " Neural Process. Lett., vol. 45, no. 1, pp. 75-94, Feb. 2017.
40. Y. Yang, S. Tong, S. Huang, P. Lin, "Multifocus image fusion based on NSCT and focused area detection, " IEEE Sensors J., vol. 15, no. 5, pp. 2824-2838, May 2015.
41. G. Piella and H. Heijmans, "A new quality metric for image fusion, " in Proc. IEEE Int. Conf. Image Process., Amsterdam, The Netherlands, Sep. 2003, pp. 173-176.
42. G. Bhatnagar, Q. M. J. Wu, Z. Liu, "Directive contrast based multimodal medical image fusion in NSCT domain, " IEEE Trans. Multimedia, vol. 15, no. 5, pp. 1014-1024, Aug. 2013.
43. B. Yang and S. Li, "Pixel-level image fusion with simultaneous orthogonal matching pursuit, " Inf. Fusion, vol. 13, no. 1, pp. 10-19, Jan. 2012.
44. D. Guo, J. Yan, X. Qu, "High quality multi-focus image fusion using self-similarity and depth information, " Opt. Commun., vol. 338, pp. 138-144, Mar. 2015.
45. C. Yang, J.-Q. Zhang, X.-R. Wang, X. Liu, "A novel similarity based quality metric for image fusion, " Inf. Fusion, vol. 9, no. 2, pp. 156-160, Apr. 2008.
46. Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity, " IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, Apr. 2004.