Robust Single Image Super-Resolution via Deep Networks with Sparse Prior

IEEE Transactions on Image Processing

Volumn 25, Issue 7, 2016, Pages 3194-3207

  Liu, Ding ^a   Wang, Zhaowen ^b   Wen, Bihan ^a   Yang, Jianchao ^c   Han, Wei ^a   Huang, Thomas S ^a  

^a University of Illinois at Urbana Champaign   (United States)

^b ADOBE RESEARCH   (United States)

^c SNAP INC   (United States)

Author keywords

deep neural networks; image super resolution; sparse coding

Indexed keywords

CASCADE CONTROL SYSTEMS; CODES (SYMBOLS); OPTICAL RESOLVING POWER;

DEEP NEURAL NETWORKS; HIGH RESOLUTION IMAGE; IMAGE SUPER RESOLUTIONS; SPARSE CODING; SPARSE REPRESENTATION; STATE-OF-THE-ART METHODS; SUBJECTIVE ASSESSMENTS; TRAINING AND TESTING;

IMAGE CODING;

EID: 84971529522     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2016.2564643     Document Type: Article

References (50)

1. S. Baker and T. Kanade, "Limits on super-resolution and how to break them, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 9, pp. 1167-1183, Sep. 2002.
2. Z. Lin and H.-Y. Shum, "Fundamental limits of reconstruction-based superresolution algorithms under local translation, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 1, pp. 83-97, Jan. 2004.
3. R. Fattal, "Image upsampling via imposed edge statistics, " ACM Trans. Graph., vol. 26, no. 3, 2007, Art. no. 95.
4. H. A. Aly and E. Dubois, "Image up-sampling using total-variation regularization with a new observation model, " IEEE Trans. Image Process., vol. 14, no. 10, pp. 1647-1659, Oct. 2005.
5. J. Yang, J. Wright, T. S. Huang, and Y. Ma, "Image super-resolution via sparse representation, " IEEE Trans. Image Process., vol. 19, no. 11, pp. 2861-2873, Nov. 2010.
6. J. Yang, Z. Wang, Z. Lin, S. Cohen, and T. Huang, "Coupled dictionary training for image super-resolution, " IEEE Trans. Image Process., vol. 21, no. 8, pp. 3467-3478, Aug. 2012.
7. X. Gao, K. Zhang, D. Tao, and X. Li, "Image super-resolution with sparse neighbor embedding, " IEEE Trans. Image Process., vol. 21, no. 7, pp. 3194-3205, Jul. 2012.
8. D. Glasner, S. Bagon, and M. Irani, "Super-resolution from a single image, " in Proc. ICCV, Sep./Oct. 2009, pp. 349-356.
9. G. Freedman and R. Fattal, "Image and video upscaling from local self-examples, " ACM Trans. Graph., vol. 30, no. 2, 2011, Art. no. 12.
10. K. I. Kim and Y. Kwon, "Single-image super-resolution using sparse regression and natural image prior, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 6, pp. 1127-1133, Jun. 2010.
11. C. Deng, J. Xu, K. Zhang, D. Tao, X. Gao, and X. Li, "Similarity constraints-based structured output regression machine: An approach to image super-resolution, " IEEE Trans. Neural Netw. Learn. Syst., to be published.
12. C.-Y. Yang, C. Ma, and M.-H. Yang, "Single-image super-resolution: A benchmark, " in Proc. ECCV, 2014, pp. 372-386.
13. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks, " in Proc. NIPS, 2012, pp. 1097-1105.
14. Z. Cui, H. Chang, S. Shan, B. Zhong, and X. Chen, "Deep network cascade for image super-resolution, " in Proc. ECCV, 2014, pp. 49-64.
15. Z. Wang et al., "Self-tuned deep super resolution, " in Proc. IEEE Conf. CVPR Workshops, Jun. 2015, pp. 1-8.
16. C. Dong, C. C. Loy, K. He, and X. Tang, "Learning a deep convolutional network for image super-resolution, " in Proc. ECCV, 2014, pp. 184-199.
17. C. Osendorfer, H. Soyer, and P. van der Smagt, "Image super-resolution with fast approximate convolutional sparse coding, " in Neural Information Processing. New York, NY, USA: Springer, 2014, pp. 250-257.
18. K. Gregor and Y. LeCun, "Learning fast approximations of sparse coding, " in Proc. ICML, 2010, pp. 399-406.
19. B. Wen, S. Ravishankar, and Y. Bresler, "Structured overcomplete sparsi-fying transform learning with convergence guarantees and applications, " Int. J. Comput. Vis., vol. 114, no. 2, pp. 137-167, 2015.
20. Z. Wang et al., Sparse Coding and Its Applications in Computer Vision. Singapore: World Scientific, 2015.
21. Z. Wang, D. Liu, J. Yang, W. Han, and T. Huang, "Deep networks for image super-resolution with sparse prior, " in Proc. CVPR, Dec. 2015, pp. 370-378.
22. K. Kavukcuoglu, M. Ranzato, and Y. LeCun. (2010). "Fast inference in sparse coding algorithms with applications to object recognition." [Online]. Available: http://arxiv.org/abs/1010.3467
23. I. Daubechies, M. Defrise, and C. De Mol, "An iterative thresholding algorithm for linear inverse problems with a sparsity constraint, " Commun. Pure Appl. Math., vol. 57, no. 11, pp. 1413-1457, Nov. 2004.
24. C. J. Rozell, D. H. Johnson, R. G. Baraniuk, and B. A. Olshausen, "Sparse coding via thresholding and local competition in neural circuits, " Neural Comput., vol. 20, no. 10, pp. 2526-2563, 2008.
25. M. Lin, Q. Chen, and S. Yan. (2013). "Network in network." [Online]. Available: http://arxiv.org/abs/1312.4400
26. V. Nair and G. E. Hinton, "Rectified linear units improve restricted Boltzmann machines, " in Proc. ICML, 2010, pp. 807-814.
27. S. Chang, W. Han, J. Tang, G.-J. Qi, C. C. Aggarwal, and T. S. Huang, "Heterogeneous network embedding via deep architectures, " in Proc. ACM SIGKDD, 2015, pp. 119-128.
28. R. Timofte, V. De, and L. Van Gool, "Anchored neighborhood regression for fast example-based super-resolution, " in Proc. ICCV, Dec. 2013, pp. 1920-1927.
29. Q. V. Le, "Building high-level features using large scale unsupervised learning, " in Proc. IEEE ICASSP, May 2013, pp. 8595-8598.
30. M. Oquab, L. Bottou, I. Laptev, and J. Sivic, "Learning and transferring mid-level image representations using convolutional neural networks, " in Proc. CVPR, Jun. 2014, pp. 1717-1724.
31. A. Buades, B. Coll, and J.-M. Morel, "A non-local algorithm for image denoising, " in Proc. CVPR, Jun. 2005, pp. 60-65.
32. M. Aharon, M. Elad, and A. Bruckstein, "K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation, " IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311-4322, Nov. 2006.
33. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, "Image denoising by sparse 3D transform-domain collaborative filtering, " IEEE Trans. Image Process., vol. 16, no. 8, pp. 2080-2095, Aug. 2007.
34. B. Wen, S. Ravishankar, and Y. Bresler, "Video denoising by online 3D sparsifying transform learning, " in Proc. IEEE ICIP, Sep. 2015, pp. 118-122.
35. S. Ravishankar, B. Wen, and Y. Bresler, "Online sparsifying transform learning-Part I: Algorithms, " IEEE J. Sel. Topics Signal Process., vol. 9, no. 4, pp. 625-636, Jun. 2015.
36. Z. Wang, Y. Yang, Z. Wang, S. Chang, J. Yang, and T. S. Huang, "Learning super-resolution jointly from external and internal examples, " IEEE Trans. Image Process., vol. 24, no. 11, pp. 4359-4371, Nov. 2015.
37. J. Yang, Z. Lin, and S. Cohen, "Fast image super-resolution based on in-place example regression, " in Proc. CVPR, Jun. 2013, pp. 1059-1066.
38. R. A. Bradley and M. E. Terry, "Rank analysis of incomplete block designs: I. The method of paired comparisons, " Biometrika, vol. 39, nos. 3-4, pp. 324-345, 1952.
39. M. Bevilacqua, A. Roumy, C. Guillemot, and M.-L. Alberi-Morel, "Low-complexity single-image super-resolution based on nonnegative neighbor embedding, " in Proc. BMVC, 2012, pp. 135.1-135.10.
40. R. Zeyde, M. Elad, and M. Protter, "On single image scale-up using sparse-representations, " in Curves and Surfaces. Heidelberg, Germany: Springer, 2012, pp. 711-730.
41. D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics, " in Proc. ICCV, vol. 2. Jul. 2001, pp. 416-423.
42. C. Dong, C. C. Loy, K. He, and X. Tang, "Image super-resolution using deep convolutional networks, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 2, pp. 295-307, Feb. 2015.
43. R. Timofte, V. De Smet, and L. Van Gool, "A+: Adjusted anchored neighborhood regression for fast super-resolution, " in Proc. ACCV, 2014, pp. 111-126.
44. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "ImageNet: A large-scale hierarchical image database, " in Proc. CVPR, Jun. 2009, pp. 248-255.
45. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity, " IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, Apr. 2004.
46. J.-B. Huang, A. Singh, and N. Ahuja, "Single image super-resolution from transformed self-exemplars, " in Proc. CVPR, Jun. 2015, pp. 5197-5206.
47. W. Dong, L. Zhang, and G. Shi, "Centralized sparse representation for image restoration, " in Proc. ICCV, Nov. 2011, pp. 1259-1266.
48. K. Zhang, X. Gao, D. Tao, and X. Li, "Single image super-resolution with non-local means and steering kernel regression, " IEEE Trans. Image Process., vol. 21, no. 11, pp. 4544-4556, Nov. 2012.
49. X. Lu, H. Yuan, P. Yan, Y. Yuan, and X. Li, "Geometry constrained sparse coding for single image super-resolution, " in Proc. CVPR, Jun. 2012, pp. 1648-1655.
50. Z. Wang, S. Chang, Y. Yang, D. Liu, and T. S. Huang, "Studying very low resolution recognition using deep networks, " in Proc. CVPR, 2016, pp. 1-9.