Using Deep Neural Networks for Inverse Problems in Imaging: Beyond Analytical Methods

IEEE Signal Processing Magazine

Volumn 35, Issue 1, 2018, Pages 20-36

  Lucas, Alice ^a   Iliadis, Michael ^b,c   Molina, Rafael ^d   Katsaggelos, Aggelos K ^a  

^a Northwestern University   (United States)

^b UNIVERSITY OF BATH   (United Kingdom)

^c US Research Center   (United States)

^d UNIVERSITY OF GRANADA   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

DEEP LEARNING; DEEP NEURAL NETWORKS; DIFFERENTIAL EQUATIONS; IMAGE RECONSTRUCTION; NETWORK ARCHITECTURE; NEURAL NETWORKS; PROBLEM SOLVING;

ANALYTICAL APPROACH; ANALYTICAL METHOD; CURRENT LIMITATION; DOMAIN KNOWLEDGE; IMAGING PROBLEMS; LEARNING TECHNIQUES; PRIOR KNOWLEDGE; SUPERRESOLUTION;

INVERSE PROBLEMS;

EID: 85040644946     PISSN: 10535888     EISSN: None     Source Type: Journal    
DOI: 10.1109/MSP.2017.2760358     Document Type: Article

References (55)

1. A. C. Bovik, The Essential Guide to Image Processing. New York: Academic, 2009.
2. A. C. Bovik, The Essential Guide to Video Processing 2nd ed. New York:Academic, 2009.
3. A. K. Katsaggelos. (2015). Fundamentals of Digital Image and VideoProcessing. [Online]. Available: Https://www.coursera.org/learn/digital
4. A. K. Katsaggelos, R. Molina, J. Mateos, Super Resolution of Images andVideo: Synthesis Lectures on Image, Video, Multimedia Processing. SanRafael, CA: Morgan & Claypool, 2007.
5. Z. Chen, S. D. Babacan, R. Molina, A. K. Katsaggelos, "Variational Bayesianmethods for multimedia problems," IEEE Trans. Multimedia, vol. 16, no. 4, pp. 1000-10017, 2014.
6. I. Goodfellow, Y. Bengio, A. Courville, Deep Learning. Cambridge, MA: MITPress, 2016.
7. S. Zhang and E. Salari, "Image denoising using a neural network based non-linearfilter in wavelet domain," in Proc. IEEE Int. Conf. Acoustics, Speech, SignalProcessing, Mar. 2005, pp. 989-992.
8. H. C. Burger, C. J. Schuler, S. Harmeling, "Image denoising: Can plain neuralnetworks compete with BM3D" in Proc. IEEE Conf. Computer Vision PatternRecognition, 2012, pp. 2392-2399.
9. C. J. Schuler, H. C. Burger, S. Harmeling, B. Scholkopf, "A machine learningapproach for non-blind image deconvolution," in Proc. IEEE Conf. Computer VisionPattern Recognition, 2013, pp. 1067-1074.
10. J. Xie, L. Xu, E. Chen, "Image denoising and inpainting with deep neural networks,"Adv. Neural Inform. Process. Syst., vol. 25, pp. 341-349 Dec. 2012.
11. F. Agostinelli, M. R. Anderson, H. Lee, "Adaptive multicolumn deep neuralnetworks with application to robust image denoising," in Proc. Neural InformationProcessing Systems Conf., 2013, pp. 1-9.
12. K. Hornik, M. Stinchcombe, H. White, "Universal approximation of anunknown mapping and its derivatives using multilayer feedforward networks," NeuralNetw., vol. 3, no. 5, pp. 551-560, 1990.
13. V. Papyan, Y. Romano, M. Elad, "Convolutional neural networks analyzed viaconvolutional sparse coding," arXiv preprint, arXiv:1607.08194, 2016.
14. V. Jain and H. Seung, "Natural image denoising with convolutional networks," inProc. Neural Information Processing Systems Conf., 2008, pp. 1-8.
15. D. Eigen, D. Krishnan, R. Fergus, "Restoring an image taken through a windowcovered with dirt or rain," in Proc. IEEE Int. Conf. Computer Vision, 2013, pp.633-640.
16. C. Dong, C. C. Loy, K. He, X. Tang, "Image super-resolution using deep convolutionalnetworks," IEEE Trans. Pattern Anal. Machine Intell., vol. 38, no. 2, pp.295-307, Feb. 2016.
17. A. Kappeler, S. Yoo, Q. Dai, A. K. Katsaggelos, "Video super-resolution withconvolutional neural networks," IEEE Trans. Computational Imaging, vol. 2, no. 2, pp.109-122, 2016.
18. K. Kulkarni, S. Lohit, P. Turaga, R. Kerviche, A. Ashok, "Reconnet: Noniterativereconstruction of images from compressively sensed measurements," in Proc.IEEE Conf. Computer Vision Pattern Recognition, June 2016, pp. 449-458.
19. M. Hradiš, J. Kotera, P. Zemcík, F. Šroubek, "Convolutional neural networksfor direct text deblurring," in Proc. British Machine Vision Conf., 2015, vol. 10, pp.1-13.
20. V. Nair and G. E. Hinton, "Rectified linear units improve restricted Boltzmannmachines," in Proc. 27th Int. Conf. Machine Learning, 2010, pp. 807-814.
21. K. He, X. Zhang, S. Ren, J. Sun, "Delving deep into rectifiers: Surpassinghuman-level performance on imagenet classification," in Proc. IEEE Int. Conf.Computer Vision, Washington, DC, 2015, pp. 1026-1034.
22. S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network trainingby reducing internal covariate shift," in Proc. Int. Conf. Machine Learning, 2015, pp.448-456.
23. K. He, X. Zhang, S. Ren, J. Sun, "Deep residual learning for image recognition,"in Proc. IEEE Conf. Computer Vision Pattern Recognition, June 2016, pp. 770-778.
24. M. Sajjadi, B. Schölkopf, M. Hirsch, "Enhancenet: Single image super-resolutionthrough automated texture synthesis," arXiv preprint, arXiv:1612.07919, 2016.
25. C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Aitken, A. Tejani, J. Totz, Z.Wang, W. Shi, "Photo-realistic single image super-resolution using a generativeadversarial network," arXiv preprint, arXiv:1609.04802, 2016.
26. J. Kim, J. K. Lee, K. M. Lee, "Accurate image super-resolution using verydeep convolutional networks," in Proc. IEEE Conf. Computer Vision PatternRecognition, 2016, pp. 1646-1654.
27. H. Yao, F. Dai, D. Zhang, Y. Ma, S. Zhang, Y. Zhang, "Dr2-net: Deep residualreconstruction network for image compressive sensing," arXiv preprint,arXiv:1702.05743, 2017.
28. K. Zhang, W. Zuo, Y. Chen, D. Meng, L. Zhang, "Beyond a Gaussian denoiser:Residual learning of deep CNN for image denoising," IEEE Trans. ImageProcessing, vol. 26, no. 2, pp. 1-13, Feb. 2017.
29. O. Ronneberger, P. Fischer, T. Brox, "U-Net: Convolutional networks for biomedicalimage segmentation," in Proc. Int. Conf. Medical Image Computing andComputer-Assisted Intervention, 2015, pp. 234-241.
30. X. J. Mao, C. Shen, Y. Yang, "Image denoising using very deep fully convolutionalencoder-decoder networks with symmetric skip connections," arXiv preprint,arXiv:abs/1606.08921, 2016.
31. D. Pathak, P. Krhenbhl, J. Donahue, T. Darrell, A. A. Efros, "Context encoders:Feature learning by inpainting," in Proc. IEEE Conf. Computer Vision PatternRecognition, June 2016, pp. 2536-2544.
32. P. Fischer, A. Dosovitskiy, E. Ilg, P. Hausser, C. Hazirbas, V. Golkov, P. v. d.Smagt, D. Cremers, T. Brox, "Flownet: Learning optical flow with convolutionalnetworks," in Proc. IEEE Int. Conf. Computer Vision, 2015, pp. 2758-2766.
33. K. Jin, M. McCann, E. Froustey, M. Unser, "Deep convolutional neural networkfor inverse problems in imaging," IEEE Trans. Image Processing, vol. 26, no. 9,pp. 4509-4522, 2017.
34. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, P.-A. Manzagol, "Stackeddenoising autoencoders: Learning useful representations in a deep network with a localdenoising criterion," J. Machine Learning Res., vol. 11, pp. 3371-3408, 2010.
35. Z. Cui, H. Chang, S. Shan, B. Zhong, X. Chen, "Deep network cascade forimage super-resolution," in Proc. European Conf. Computer Vision, 2016, pp. 49-64.
36. J. Zhao, M. Mathieu, Y. LeCun, "Energy-based generative adversarial network,"arXiv preprint, arXiv:1609.03126, 2016.
37. K. Zeng, J. Yu, R. Wang, C. Li, D. Tao, "Coupled deep autoencoder for singleimage super-resolution," IEEE Trans. Cybern., vol. 47, no. 1, pp. 27-37, 2017.
38. M. Iliadis, L. Spinoulas, A. K. Katsaggelos, "Deepbinarymask: Learning abinary mask for video compressive sensing," arXiv preprint, arXiv:1607.03343, 2016.
39. A. Mousavi, A. B. Patel, R. G. Baraniuk, "A deep learning approach to structuredsignal recovery," in Proc. Communication, Control Computing (Allerton) 53rdAnnu. Allerton Conf., 2015, vol. abs/1508.04065, pp. 1336-1343.
40. A. Bora, A. Jalal, E. Price, A. G. Dimakis, "Compressed sensing using generativemodels," arXiv preprint, arXiv:1703.03208, 2017.
41. R. Borhani, J. Watt, A. K. Katsaggelos, Machine Learning Refined:Foundations, Algorithms, Applications. Cambridge, U.K.: Cambridge Univ.Press, 2016.
42. J. Johnson, A. Alahi, L. Fei-Fei, "Perceptual losses for real-time style transferand super-resolution," in Proc. European Conf. Computer Vision, 2016, pp. 694-711.
43. J. Bruna, P. Sprechmann, Y. LeCun, "Super-resolution with deep convolutionalsufficient statistics," in Proc. Int. Conf. Learning Representations, 2016, pp. 1-17.
44. Y. Chen and T. Pock, "Trainable nonlinear reaction diffusion: A flexible frameworkfor fast and effective image restoration," IEEE Trans. Pattern Anal. MachineIntell., vol. 39, no. 6, pp. 1256-1272, 2015.
45. L. Xu, J. S. Ren, C. Liu, J. Jia, "Deep convolutional neural network for imagedeconvolution," Adv. Neural Inform. Process. Syst., vol. 27, pp. 1790-1798, Dec. 2014.
46. Z. Wang, D. Liu, J. Yang, W. Han, T. Huang, "Deep networks for imagesuper-resolution with sparse prior," in Proc. IEEE Int. Conf. Computer Vision, 2015,pp. 370-378.
47. K. Gregor and Y. LeCun, "Learning fast approximations of sparse coding," inProc. 27th Int. Conf. Machine Learning, 2010, pp. 399-406.
48. J.R. Hershey, J. L. Roux, F. Weninger, "Deep unfolding: Model-based inspirationof novel deep architectures," arXiv preprint, arXiv:1409.2574, 2014.
49. Y. Yang, J. Sun, H. Li, Z. Xu, "Deep ADMM-net for compressive sensingMRI," in Proc. Neural Information Processing Systems Conf., 2016, pp. 10-18.
50. M. V. Afonso, J. M. Bioucas-Dias, M. A. T. Figueiredo, "Fast image recoveryusing variable splitting and constrained optimization," IEEE Trans. Image Processing,vol. 19, no. 3, pp. 2345-2356, 2010.
51. C. J. Schuler, M. Hirsch, S. Harmeling, B. Schlkopf, "Learning to deblur,"IEEE Trans. Pattern Anal. Machine Intell., vol. 38, no. 7, pp. 1439-1451, 2016.
52. K. Zhang, W. Zuo, S. Gu, L. Zhang, "Learning deep CNN denoiser prior forimage restoration," arXiv preprint, arXiv:1704.03264, 2017.
53. J. H. R. Chang, C.-L. Li, B. Poczos, V. K. Vijaya Kumar, A. C.Sankaranarayanan, "One network to solve them all: Solving linear inverse problemsusing deep projection models," arXiv preprint, arXiv:1703.09912, 2017.
54. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A.Courville, Y. Bengio, "Generative adversarial nets," Adv. Neural Inform. Process.Syst., vol. 26, pp. 2672-2680, Dec. 2014.
55. C. K. Sonderby, J. Caballero, L. Theis, W. Shi, F. Huszár, "Amortised MAPinference for image super-resolution," in Proc. Int. Conf. Learning Representations,2017, pp. 1-17.