Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 39, Issue 6, 2017, Pages 1256-1272

  Chen, Yunjin ^a   Pock, Thomas ^a,b  

^a GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

^b AIT AUSTRIAN INSTITUTE OF TECHNOLOGY   (Austria)

Author keywords

image denoising; image super resolution; JPEG deblocking; loss specific training; Nonlinear reaction diffusion

Indexed keywords

DIFFUSION IN LIQUIDS; IMAGE DENOISING; IMAGE RECONSTRUCTION; METHOD OF MOMENTS; OPTICAL RESOLVING POWER; PROGRAM PROCESSORS; RESTORATION;

DEBLOCKING; IMAGE RESTORATION PROBLEM; IMAGE SUPER RESOLUTIONS; INFLUENCE FUNCTIONS; NONLINEAR DIFFUSION MODEL; NONLINEAR REACTION; PARALLEL COMPUTATION; TIME DEPENDENT PARAMETER;

DIFFUSION;

EID: 85019248552     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2016.2596743     Document Type: Article

References (60)

1. S. T. Acton, D. Prasad Mukherjee, J. P. Havlicek, and A. C. Bovik, "Oriented texture completion by AM-FM reaction-diffusion," IEEE Trans. Image Process., vol. 10, no. 6, pp. 885-896, Jun. 2001.
2. F. Agostinelli, M. Hoffman, P. Sadowski, and P. Baldi, "Learning activation functions to improve deep neural networks," Int. Conf. Learn. Represent. (ICLR), arXiv:1412.6830, 2015.
3. A. Barbu, "Training an active random field for real-time image denoising," IEEE Trans. Image Process., vol. 18, no. 11, pp. 2451-2462, Nov. 2009.
4. C. A. Z. Barcelos, M. Boaventura, and E. C. Silva, Jr, "A wellbalanced flow equation for noise removal and edge detection," IEEE Trans. Image Process., vol. 12, no. 7, pp. 751-763, Jul. 2003.
5. Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, "Greedy layer-wise training of deep networks," in Proc. Advances Neural Inf. Process. Syst., 2006, pp. 153-160.
6. Y. Bengio, P. Simard, and P. Frasconi, "Learning long-term dependencies with gradient descent is difficult," IEEE Trans. Neural Netw., vol. 5, no. 2, pp. 157-166, Mar. 1994.
7. M. Black, G. Sapiro, D. Marimont, and D. Heeger, "Robust anisotropic diffusion and sharpening of scalar and vector images," in Proc. IEEE Int. Conf. Image Process., 1997, pp. 263-266.
8. K. Bredies and M. Holler, "Artifact-free decompression and zooming of JPEG compressed images with total generalized variation," in Proc. 7th Int. Joint Conf. Comput. Vis., Imaging Comput. Graph Theory Appl., 2013, pp. 242-258.
9. H. Chang, M. K. Ng, and T. Zeng, "Reducing artifact in JPEG decompression via a learned dictionary," IEEE Trans. Sig. Process., vol. 62, no. 3, pp. 718-728, Feb. 2014.
10. Y. Chen, T. Pock, R. Ranftl, and H. Bischof, "Revisiting lossspecific training of filter-based MRFs for image restoration," in Proc. 35th German Conf. Pattern Recog., 2013, pp. 271-281.
11. Y. Chen, R. Ranftl, and T. Pock, "Insights into analysis operator learning: From patch-based sparse models to higher order MRFs," IEEE Trans. Image Process., vol. 23, no. 3, pp. 1060-1072, Mar. 2014.
12. Y. Chen, W. Yu, and T. Pock, "On learning optimized reaction diffusion processes for effective image restoration," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recog., 2015, pp. 5261-5269.
13. G.-H. Cottet and L. Germain, "Image processing through reaction combined with nonlinear diffusion," in Proc. Mathematics Comput., 1993, pp. 659-673.
14. D. Cozzolino, S. Parrilli, G. Scarpa, G. Poggi, and L. Verdoliva, "Fast adaptive nonlocal SAR despeckling," IEEE Trans. Geosci. Remote Sensing Lett., vol. 11, no. 2, pp. 524-528, Feb. 2014.
15. K. Dabov, A. Foi, V. Katkovnik, and K. O. Egiazarian, "Image denoising by sparse 3-d transform-domain collaborative filtering," IEEE Trans. Image Process., vol. 16, no. 8, pp. 2080-2095, Aug. 2007.
16. S. Didas, J. Weickert, and B. Burgeth, "Stability and local feature enhancement of higher order nonlinear diffusion filtering," in Proc. 27thDAGMSymp. German Conf. Pattern Recog., 2005, pp. 451-458.
17. S. Didas, J. Weickert, and B. Burgeth, "Properties of higher order nonlinear diffusion filtering," J. Math. Imaging Vis., vol. 35, no. 3, pp. 208-226, 2009.
18. J. Domke, "Generic methods for optimization-based modeling," in Proc. Int. Conf. Artificial Intell. Statistics, 2012, pp. 318-326.
19. C. Dong, C. C. Loy, K. He, and X. Tang, "Learning a deep convolutional network for image super-resolution," in Proc. 13th Eur. Conf. Comput. Vis., 2014, pp. 184-199.
20. W. Dong, L. Zhang, G. Shi, and X. Li, "Nonlocally centralized sparse representation for image restoration," IEEE Trans. Image Process., vol. 22, no. 4, pp. 1620-1630, Apr. 2013.
21. J. Esclarín and L. Alvarez, "Image quantization using reaction-diffusion equations," SIAM J. Appl. Mathematics, vol. 57, no. 1, pp. 153-175, 1997.
22. A. Foi, V. Katkovnik, and K. Egiazarian, "Pointwise shape-adaptive DCT for high-quality denoising and deblocking of grayscale and color images," IEEE Trans. Image Process., vol. 16, no. 5, pp. 1395-1411, May 2007.
23. G. Gilboa, N. Sochen, and Y. Y. Zeevi, "Forward-and-backward diffusion processes for adaptive image enhancement and denoising," IEEE Trans. Image Process., vol. 11, no. 7, pp. 689-703, Jul. 2002.
24. R. Giryes and M. Elad, "Sparsity-based Poisson denoising with dictionary learning," IEEE Trans. Image Process., vol. 23, no. 12, pp. 5057-5069, Dec. 2014.
25. A. Graves and J. Schmidhuber, "Offline handwriting recognition with multidimensional recurrent neural networks," in Proc. Advances Neural Inf. Process. Syst., 2009, pp. 545-552.
26. S. Gu, L. Zhang, W. Zuo, and X. Feng, "Weighted nuclear norm minimization with application to image denoising," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recog., 2014, pp. 2862-2869.
27. P. Guidotti and K. Longo, "Two enhanced fourth order diffusion models for image denoising," J. Math. Imaging Vis., vol. 40, no. 2, pp. 188-198, 2011.
28. M. R. Hajiaboli, "An anisotropic fourth-order diffusion filter for image noise removal," Int. J. Comput. Vis., vol. 92, no. 2, pp. 177-191, 2011.
29. K. He, X. Zhang, S. Ren, and J. Sun, "Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification," IEEE Int. Conf. Comput. Vis., (ICCV) 2015, Santiago, Chile, pp. 1026-1034, Dec. 2015, Doi: 10.1109/ICCV.2015.123.
30. N. Heess, C. K. I. Williams, and G. E. Hinton, "Learning generative texture models with extended Fields-of-Experts," presented at the British Machine Vis. Conf., London, U.K., 2009.
31. Y. H. Hu and J.-N. Hwang, Handbook of Neural Network Signal Processing. Boca Raton, FL, USA: CRC Press, 2010.
32. V. Jain and S. Seung, "Natural image denoising with convolutional networks," in Proc. Advances Neural Inf. Process. Syst., 2009, pp. 769-776.
33. J. Jancsary, S. Nowozin, and C. Rother, "Loss-specific training of non-parametric image restoration models: A new state of the art," in Proc. 12th Eur. Conf. Comput. Vis., 2012, pp. 112-125.
34. K. Krajsek and H. Scharr, "Diffusion filtering without parameter tuning: Models and inference tools," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recog., 2010, pp. 2536-2543.
35. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
36. A. Levin and B. Nadler, "Natural image denoising: Optimality and inherent bounds," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recog., 2011, pp. 2833-2840.
37. P.-L. Lions and B. Mercier, "Splitting algorithms for the sum of two nonlinear operators," SIAM J. Numerical Anal., vol. 16, no. 6, pp. 964-979, 1979.
38. D. C. Liu and J. Nocedal, "On the limited memory BFGS method for large scale optimization," Math. Programming, vol. 45, no. 1-3, pp. 503-528, 1989.
39. R. Liu, Z. Lin, W. Zhang, and Z. Su, "Learning PDEs for image restoration via optimal control," in Proc. 11th Eur. Conf. Comput. Vis., 2010, pp. 115-128.
40. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, "Non-local sparse models for image restoration," in Proc. IEEE 12th Int. Conf. Comput. Vis., 2009, pp. 2272-2279.
41. P. Milanfar, "A tour of modern image filtering: New insights and methods, both practical and theoretical," IEEE Sig. Process. Mag., vol. 30, no. 1, pp. 106-128, Jan. 2013.
42. V. Nair and G. E. Hinton, "Rectified linear units improve restricted Boltzmann machines," in Proc. IEEE 27th Int. Conf. Mach. Learn., 2010, pp. 807-814.
43. K. N. Nordström, "Biased anisotropic diffusion: A unified regularization and diffusion approach to edge detection," Image Vision Comput., vol. 8, no. 4, pp. 318-327, 1990.
44. P. Ochs, Y. Chen, T. Brox, and T. Pock. "iPiano: Inertial proximal algorithm for nonconvex optimization," SIAM J. Imag. Sci., vol. 7, no. 2, pp. 1388-1419, 2014.
45. P. Perona and J. Malik, "Scale-space and edge detection using anisotropic diffusion," IEEE Trans. Pattern Anal. Mach. Intell., vol. 12, no. 7, pp. 629-639, Jul. 1990.
46. A. Rajwade, A. Rangarajan, and A. Banerjee, "Image denoising using the higher order singular value decomposition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 4, pp. 849-862, Apr. 2013.
47. Y. Romano, M. Protter, and M. Elad, "Single image interpolation via adaptive nonlocal sparsity-based modeling," IEEE Trans. Image Process., vol. 23, no. 7, pp. 3085-3098, Jul. 2014.
48. S. Roth and M. J. Black, "Fields of Experts," Int. J. Comput. Vis., vol. 82, no. 2, pp. 205-229, 2009.
49. L. I. Rudin, S. Osher, and E. Fatemi, "Nonlinear total variation based noise removal algorithms," Physica D: Nonlinear Phenomena, vol. 60, no. 1-4, pp. 259-268, 1992.
50. H. Scharr, M. J. Black, and H. W. Haussecker, "Image statistics and anisotropic diffusion," in Proc. 9th IEEE Int. Conf. Comput. Vis., 2003, pp. 840-847.
51. O. Scherzer and J. Weickert, "Relations between regularization and diffusion filtering," J. Math. Imaging Vis., vol. 12, no. 1, pp. 43-63, 2000.
52. U. Schmidt and S. Roth, "Shrinkage fields for effective image restoration," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recog., 2014, pp. 2774-2781.
53. S. Schulter, C. Leistner, and H. Bischof, "Fast and accurate image upscaling with super-resolution forests," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recog., 2015, pp. 3791-3799.
54. R. Timofte, V. De, and L. V. Gool, "Anchored neighborhood regression for fast example-based super-resolution," in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 1920-1927.
55. J. Weickert, Anisotropic Diffusion in Image Processing. Vaihingen, Germany: Teubner Stuttgart, 1998.
56. M. Welk, G. Gilboa, and J. Weickert, "Theoretical foundations for discrete forward-and-backward diffusion filtering," in Proc. 2nd Int. Conf. Scale Space Variational Methods Comput. Vis., 2009, pp. 527-538.
57. L. Xu, Q. Yan, Y. Xia, and J. Jia, "Structure extraction from texture via relative total variation," ACM Trans. Graphics, vol. 31, no. 6, pp. 139:1-139:10, 2012.
58. R. Zeyde, M. Elad, and M. Protter, "On single image scale-up using sparse-representations," in Proc. 7th Int. Conf. Curves Surfaces, 2012, pp. 711-730.
59. S. C. Zhu and D. Mumford, "Prior learning and Gibbs reactiondiffusion," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 11, pp. 1236-1250, Nov. 1997.
60. D. Zoran and Y. Weiss, "From learning models of natural image patches to whole image restoration," in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 479-486.