EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis

Proceedings of the IEEE International Conference on Computer Vision

Volumn 2017-October, Issue , 2017, Pages 4501-4510

  Sajjadi, Mehdi S M ^a   Scholkopf, Bernhard ^a   Hirsch, Michael ^a  

^a MAX PLANCK INSTITUTE FOR INTELLIGENT SYSTEMS   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER GRAPHICS; COMPUTER VISION; IMAGE ENHANCEMENT; IMAGE PROCESSING; IMAGE QUALITY; NEURAL NETWORKS; OPTICAL RESOLVING POWER; SIGNAL TO NOISE RATIO;

CONVOLUTIONAL NEURAL NETWORK; HIGH FREQUENCY HF; HIGH MAGNIFICATIONS; HIGH RESOLUTION IMAGE; NOVEL APPLICATIONS; PEAK SIGNAL TO NOISE RATIO; PERFORMANCE OF ALGORITHM; TEXTURE SYNTHESIS;

IMAGE TEXTURE;

EID: 85041907399     PISSN: 15505499     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICCV.2017.481     Document Type: Conference Paper

References (66)

1. VGG19 and VGG16 on tensorflow. https://github. com/machrisaa/tensorflow-vgg, 2016. (visited on June 6, 2016)
2. M. Abadi et. al. TensorFlow: Large-scale machine learning on heterogeneous systems, 2015. Software available from tensorflow.org
3. M. Bevilacqua, A. Roumy, C. Guillemot, and M.-L. A. Morel. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In BMVC, 2012
4. J. Bruna, P. Sprechmann, and Y. LeCun. Super-resolution with deep convolutional sufficient statistics. In ICLR, 2016
5. H. Chang, D.-Y. Yeung, and Y. Xiong. Super-resolution through neighbor embedding. In CVPR, 2004
6. R. Dahl. ResNet in tensorflow. https://github.com/ ry/tensorflow-resnet, 2016. (visited on November 10, 2016)
7. E. L. Denton, S. Chintala, R. Fergus, et al. Deep generative image models using a laplacian pyramid of adversarial networks. In NIPS, 2015
8. C. Dong, C. C. Loy, K. He, and X. Tang. Learning a deep convolutional network for image super-resolution. In ECCV, 2014
9. C. Dong, C. C. Loy, and X. Tang. Accelerating the superresolution convolutional neural network. In ECCV, 2016
10. A. Dosovitskiy and T. Brox. Generating images with perceptual similarity metrics based on deep networks. In NIPS, 2016
11. C. E. Duchon. Lanczos filtering in one and two dimensions. Journal of Applied Meteorology, 18(8):1016-1022, 1979
12. C. Fookes, F. Lin, V. Chandran, and S. Sridharan. Evaluation of image resolution and super-resolution on face recognition performance. JVCIR, 23(1):75-93, 2012
13. G. Freedman and R. Fattal. Image and video upscaling from local self-examples. ACM TOG, 30(2):12, 2011
14. W. T. Freeman, T. R. Jones, and E. C. Pasztor. Examplebased super-resolution. IEEE CG and A, 22(2):56-65, 2002
15. L. Gatys, A. S. Ecker, and M. Bethge. Texture synthesis using convolutional neural networks. In NIPS, 2015
16. L. A. Gatys, A. S. Ecker, and M. Bethge. Image style transfer using convolutional neural networks. In CVPR, 2016
17. D. Glasner, S. Bagon, and M. Irani. Super-resolution from a single image. In ICCV, 2009
18. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D.Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In NIPS, 2014
19. Y. HaCohen, R. Fattal, and D. Lischinski. Image upsampling via texture hallucination. In ICCP, 2010
20. K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, 2016
21. M. Hradis, J. Kotera, P. Zemcik, and F. Sroubek. Convolutional neural networks for direct text deblurring. In BMVC, 2015
22. J.-B. Huang, A. Singh, and N. Ahuja. Single image superresolution from transformed self-exemplars. In CVPR, 2015
23. M. Irani and S. Peleg. Improving resolution by image registration. CVGIP: Graphical models and image processing, 53(3):231-239, 1991
24. J. Johnson, A. Alahi, and L. Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In ECCV, 2016
25. J. Kim, J. Kwon Lee, and K. Mu Lee. Accurate image superresolution using very deep convolutional networks. In CVPR, 2016
26. J. Kim, J. Kwon Lee, and K. Mu Lee. Deeply-recursive convolutional network for image super-resolution. In CVPR, 2016
27. K. I. Kim and Y. Kwon. Single-image super-resolution using sparse regression and natural image prior. IEEE TPAMI, 32(6):1127-1133, 2010
28. V. Laparra, J. Balle, A. Berardino, and E. P. Simoncelli. Perceptual image quality assessment using a normalized laplacian pyramid. Electronic Imaging, 2016(16):1-6, 2016
29. C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Aitken, A. Tejani, J. Totz, Z. Wang, and W. Shi. Photo-realistic single image super-resolution using a generative adversarial network. arXiv:1609.04802v3, 2016
30. F. Lin, C. Fookes, V. Chandran, and S. Sridharan. Superresolved faces for improved face recognition from surveillance video. In ICB, 2007
31. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollar, and C. L. Zitnick. Microsoft COCO: Common objects in context. In ECCV, 2014
32. J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In CVPR, 2015
33. X. Lu, H. Yuan, P. Yan, Y. Yuan, and X. Li. Geometry constrained sparse coding for single image super-resolution. In CVPR, 2012
34. A. L. Maas, A. Y. Hannun, and A. Y. Ng. Rectifier nonlinearities improve neural network acoustic models. In ICML, 2013
35. P. Milanfar. Super-resolution Imaging. CRC Press, 2010
36. V. P. Namboodiri, V. Smet, and L. Van Gool. Systematic evaluation of super-resolution using classification. In VCIP, pages 1-4. IEEE, 2011
37. K. Nasrollahi and T. B. Moeslund. Super-resolution: A comprehensive survey. Machine Vision and Applications, 25(6):1423-1468, 2014
38. A. Odena, V. Dumoulin, and C. Olah. Deconvolution and checkerboard artifacts. http://distill.pub/2016/deconvcheckerboard/, 2016
39. D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros. Context encoders: Feature learning by inpainting. In CVPR, 2016
40. E. Perez-Pellitero, J. Salvador, J. Ruiz-Hidalgo, and B. Rosenhahn. PSyCo: Manifold span reduction for super resolution. In CVPR, 2016
41. A. Radford, L. Metz, and S. Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. In ICLR, 2016
42. Y. Romano, J. Isidoro, and P. Milanfar. RAISR: Rapid and accurate image super resolution. IEEE TCI, 2016
43. L. I. Rudin, S. Osher, and E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 60(1):259-268, 1992
44. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. IJCV, 115(3):211-252, 2015
45. S. Schulter, C. Leistner, and H. Bischof. Fast and accu-rate image upscaling with super-resolution forests. In CVPR, 2015
46. H. R. Sheikh, A. C. Bovik, and G. Veciana. An information fidelity criterion for image quality assessment using natural scene statistics. IEEE TIP, 2005
47. W. Shi, J. Caballero, F. Huszar, J. Totz, A. P. Aitken, R. Bishop, D. Rueckert, and Z. Wang. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In CVPR, 2016
48. Y. Shi, K. Wang, L. Xu, and L. Lin. Local-and holisticstructure preserving image super resolution via deep joint component learning. In ICME, 2016
49. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015
50. Y.-W. Tai, S. Liu, M. S. Brown, and S. Lin. Super resolution using edge prior and single image detail synthesis. In CVPR, 2010
51. R. Timofte, V. Smet, and L. Van Gool. A+: Adjusted anchored neighborhood regression for fast super-resolution. In ACCV, 2014
52. R. Timofte, R. Rothe, and L. Van Gool. Seven ways to improve example-based single image super resolution. In CVPR, 2016
53. D. Ulyanov, V. Lebedev, A. Vedaldi, and V. Lempitsky. Texture networks: Feed-forward synthesis of textures and stylized images. In ICML, 2016
54. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE TIP, 13(4):600-612, 2004
55. L. Xiao, J. Wang, W. Heidrich, and M. Hirsch. Learning high-order filters for efficient blind deconvolution of document photographs. In ECCV, 2016
56. C.-Y. Yang, J.-B. Huang, and M.-H. Yang. Exploiting selfsimilarities for single frame super-resolution. In ACCV, 2010
57. C.-Y. Yang, C. Ma, and M.-H. Yang. Single-image superresolution: a benchmark. In ECCV, 2014
58. J. Yang, Z. Lin, and S. Cohen. Fast image super-resolution based on in-place example regression. In CVPR, 2013
59. J. Yang, Z. Wang, Z. Lin, S. Cohen, and T. Huang. Coupled dictionary training for image super-resolution. IEEE TIP, 21(8):3467-3478, 2012
60. J. Yang, J. Wright, T. Huang, and Y. Ma. Image superresolution as sparse representation of raw image patches. In CVPR, 2008
61. J. Yang, J. Wright, T. S. Huang, and Y. Ma. Image super-resolution via sparse representation. IEEE TIP, 19(11):2861-2873, 2010
62. X. Yu and F. Porikli. Ultra-resolving face images by discriminative generative networks. In ECCV, 2016
63. H. Yue, X. Sun, J. Yang, and F. Wu. Landmark image super-resolution by retrieving web images. IEEE TIP, 22(12):4865-4878, 2013
64. K. Zhang, X. Gao, D. Tao, and X. Li. Multi-scale dictionary for single image super-resolution. In CVPR, 2012
65. R. Zhang, P. Isola, and A. A. Efros. Colorful image colorization. In ECCV, 2016
66. J.-Y. Zhu, P. Krahenbuhl, E. Shechtman, and A. A. Efros. Generative visual manipulation on the natural image manifold. In ECCV, 2016