Image Super-Resolution Using Deep Convolutional Networks

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 38, Issue 2, 2016, Pages 295-307

  Dong, Chao ^a   Loy, Chen Change ^a   He, Kaiming ^b   Tang, Xiaoou ^a  

^a CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

^b MICROSOFT RESEARCH ASIA   (China)

Author keywords

deep convolutional neural networks; sparse coding; Super resolution

Indexed keywords

CODES (SYMBOLS); ECONOMIC AND SOCIAL EFFECTS; FACE RECOGNITION; MAPPING; NEURAL NETWORKS; OPTICAL RESOLVING POWER;

CONVOLUTIONAL NETWORKS; CONVOLUTIONAL NEURAL NETWORK; IMAGE SUPER RESOLUTIONS; LOW RESOLUTION IMAGES; NETWORK STRUCTURES; RECONSTRUCTION QUALITY; SPARSE CODING; SUPER RESOLUTION;

CONVOLUTION;

EID: 84962128851     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2015.2439281     Document Type: Article

References (50)

1. M. Aharon, M. Elad, and A. Bruckstein, "K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation," IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311-4322, Nov. 2006.
2. M. Bevilacqua, A. Roumy, C. Guillemot, and M. L. A. Morel, "Low-complexity single-image super-resolution based on nonnegative neighbor embedding," in Proc. Brit. Mach. Vis. Conf., 2012, pp. 1-10.
3. H. C. Burger, C. J. Schuler, and S. Harmeling, "Image denoising: Can plain neural networks compete with BM3D" in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2012, pp. 2392-2399.
4. H. Chang, D. Y. Yeung, and Y. Xiong, "Super-resolution through neighbor embedding," presented at the IEEE Conf. Comput. Vis. Pattern Recog., Washington, DC, USA, 2004.
5. Z. Cui, H. Chang, S. Shan, B. Zhong, and X. Chen, "Deep network cascade for image super-resolution," in Proc. Eur. Conf. Comput. Vis., 2014, pp. 49-64.
6. D. Dai, R. Timofte, and L. Van Gool, "Jointly optimized regressors for image super-resolution," Eurographics, vol. 7, p. 8, 2015.
7. S. Dai, M. Han, W. Xu, Y. Wu, Y. Gong, and A. K. Katsaggelos, "Softcuts: A soft edge smoothness prior for color image superresolution," IEEE Trans. Image Process., vol. 18, no. 11, pp. 969-981, May 2009.
8. N. Damera-Venkata, T. D. Kite, W. S. Geisler, B. L. Evans, and A. C. Bovik, "Image quality assessment based on a degradation model," IEEE Trans. Image Process., vol. 9, no. 11, pp. 636-650, Apr. 2000.
9. J. Deng, W. Dong, R. Socher, L. J. Li, K. Li, and L. Fei-Fei, "ImageNet: A large-scale hierarchical image database," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2009, pp. 248-255.
10. E. Denton, W. Zaremba, J. Bruna, Y. LeCun, and R. Fergus, "Exploiting linear structure within convolutional networks for efficient evaluation," in Proc. Adv. Neural Inf. Process. Syst., 2014, pp. 1269-1277.
11. C. Dong, C. C. Loy, K. He, and X. Tang, "Learning a deep convolutional network for image super-resolution," in Proc. Eur. Conf. Comput. Vis., 2014, pp. 184-199.
12. D. Eigen, D. Krishnan, and R. Fergus, "Restoring an image taken through a window covered with dirt or rain," in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 633-640.
13. G. Freedman and R. Fattal, "Image and video upscaling from local self-examples," ACM Trans. Graph., vol. 30, no. 11, p. 12, 2011.
14. W. T. Freeman, T. R. Jones, and E. C. Pasztor, "Example-based super-resolution," Comput. Graph. Appl., vol. 22, no. 11, pp. 56-65, 2002.
15. W. T. Freeman, E. C. Pasztor, and O. T. Carmichael, "Learning low-level vision," Int. J. Comput. Vis., vol. 40, no. 11, pp. 25-47, 2000.
16. D. Glasner, S. Bagon, and M. Irani, "Super-resolution from a single image," in Proc. IEEE Int. Conf. Comput. Vis., 2009, pp. 349-356.
17. K. He and J. Sun, "Convolutional neural networks at constrained time cost," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2015, pp. 3791-3799.
18. K. He, X. Zhang, S. Ren, and J. Sun, "Spatial pyramid pooling in deep convolutional networks for visual recognition," in Proc. Eur. Conf. Comput. Vis., 2014, pp. 346-361.
19. J. B. Huang, A. Singh, and N. Ahuja, "Single image super-resolution from transformed self-exemplars," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2015, pp. 5197-5206.
20. M. Irani and S. Peleg, "Improving resolution by image registration," Graph. Models Image Process., vol. 53, no. 11, pp. 231-239, 1991.
21. V. Jain and S. Seung, "Natural image denoising with convolutional networks," in Proc. Adv. Neural Inf. Process. Syst., 2008, pp. 769-776.
22. K. Jia, X. Wang, and X. Tang, "Image transformation based on learning dictionaries across image spaces," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 11, pp. 367-380, Feb. 2013.
23. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell, "Caffe: Convolutional architecture for fast feature embedding," in Proc. ACM Int. Conf. Multimedia, 2014, pp. 675-678.
24. K. I. Kim and Y. Kwon, "Single-image super-resolution using sparse regression and natural image prior," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 6, pp. 1127-1133, Jun. 2010.
25. A. Krizhevsky, I. Sutskever, and G. Hinton, "ImageNet classification with deep convolutional neural networks," in Proc. Adv. Neural Inf. Process. Syst., 2012, pp. 1097-1105.
26. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, "Backpropagation applied to handwritten zip code recognition," Neural Comput., vol. 1, no. 4, pp. 541-551, 1989.
27. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
28. H. Lee, A. Battle, R. Raina, and A. Y. Ng, "Efficient sparse coding algorithms," in Proc. Adv. Neural Inf. Process. Syst., 2006, pp. 801-808.
29. C. Liu, H. Y. Shum, and W. T. Freeman, "Face hallucination: Theory and practice," Int. J. Comput. Vis., vol. 75, no. 11, pp. 115-134, 2007.
30. F. Mamalet and C. Garcia, "Simplifying convNets for fast learning," in Proc. Int. Conf. Artif. Neural Netw., 2012, pp. 58-65.
31. D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," in Proc. IEEE Int. Conf. Comput. Vis., 2001, vol. 2, pp. 416-423.
32. V. Nair and G. E. Hinton, "Rectified linear units improve restricted Boltzmann machines," in Proc. Int. Conf. Mach. Learn., 2010, pp. 807-814.
33. W. Ouyang, X. Wang, X. Zeng, S. Qiu, P. Luo, Y. Tian, H. Li, S. Yang, Z. Wang, C.-C. Loy, and X. Tang, "Deepid-net: Deformable deep convolutional neural networks for object detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recogn., 2015, pp. 2403-2412.
34. W. Ouyang and X. Wang, "Joint deep learning for pedestrian detection," in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 2056-2063.
35. C. J. Schuler, H. C. Burger, S. Harmeling, and B. Scholkopf, "A machine learning approach for non-blind image deconvolution," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2013, pp. 1067-1074.
36. S. Schulter, C. Leistner, and H. Bischof, "Fast and accurate image upscaling with super-resolution forests," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2015, pp. 3791-3799.
37. H. R. Sheikh, A. C. Bovik, and G. De Veciana, "An information fidelity criterion for image quality assessment using natural scene statistics," IEEE Trans. Image Process., vol. 14, no. 11, pp. 2117-2128, Dec. 2005.
38. Y. Sun, Y. Chen, X. Wang, and X. Tang, "Deep learning face representation by joint identification-verification," in Proc. Adv. Neural Inf. Process. Syst., 2014, pp. 1988-1996.
39. R. Timofte, V. De Smet, and L. Van Gool, "Anchored neighborhood regression for fast example-based super-resolution," in Proc. IEEE Int. Conf. Comput. Vis., 2013, pp. 1920-1927.
40. R. Timofte, V. De Smet, and L. Van Gool, "A+: Adjusted anchored neighborhood regression for fast super-resolution," in Proc. IEEE Asian Conf. Comput. Vis., 2014, pp. 111-126.
41. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Process., vol. 13, no. 11, pp. 600-612, Apr. 2004.
42. Z. Wang, E. P. Simoncelli, and A. C. Bovik, "Multiscale structural similarity for image quality assessment," in Proc. IEEE Conf. Rec. 37th Asilomar Conf. Signals, Syst. Comput., 2003, vol. 2, pp. 1398-1402.
43. C. Y. Yang, J. B. Huang, and M. H. Yang, "Exploiting self-similarities for single frame super-resolution," in Proc. IEEE Asian Conf. Comput. Vis., 2010, pp. 497-510.
44. C. Y. Yang, C. Ma, andM. H. Yang, "Single-image super-resolution: Abenchmark," in Proc. Eur. Conf. Comput. Vis., 2014, pp. 372-386.
45. J. Yang, Z. Lin, and S. Cohen, "Fast image super-resolution based on in-place example regression," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2013, pp. 1059-1066.
46. J. Yang, Z. Wang, Z. Lin, S. Cohen, and T. Huang, "Coupled dictionary training for image super-resolution," IEEE Trans. Image Process., vol. 21, no. 11, pp. 3467-3478, Aug. 2012.
47. J. Yang, J. Wright, T. Huang, and Y. Ma, "Image super-resolution as sparse representation of raw image patches," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2008, pp. 1-8.
48. J. Yang, J. Wright, T. S. Huang, and Y. Ma, "Image super-resolution via sparse representation," IEEE Trans. Image Process., vol. 19, no. 11, pp. 2861-2873, Nov. 2010.
49. R. Zeyde, M. Elad, and M. Protter, "On single image scale-up using sparse-representations," in Proc. 7th Int. Conf. Curves Surfaces, 2012, pp. 711-730.
50. N. Zhang, J. Donahue, R. Girshick, and T. Darrell, "Part-based RCNNs for fine-grained category detection," in Proc. Eur. Conf. Comput. Vis., 2014, pp. 834-849.