Sparse representation-based multiple frame video super-resolution

IEEE Transactions on Image Processing

Volumn 26, Issue 2, 2017, Pages 765-781

  Dai, Qiqin ^a   Yoo, Seunghwan ^a   Kappeler, Armin ^a   Katsaggelos, Aggelos K ^a  

^a Northwestern University   (United States)

Author keywords

dictionary learning; motion estimation; optical flow; sparse coding; Video super resolution

Indexed keywords

IMAGE REGISTRATION; OPTICAL FLOWS; OPTICAL RESOLVING POWER;

DICTIONARY LEARNING; EXPERIMENTAL COMPARISON; SPARSE CODING; SPARSE REPRESENTATION; STATE OF THE ART; SUBPIXEL ACCURACY; SUPER RESOLUTION; VIDEO SUPER-RESOLUTION;

MOTION ESTIMATION;

EID: 85015240385     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2016.2631339     Document Type: Article

References (48)

1. S. Borman and R. L. Stevenson, " Super-resolution from image sequences-A review, " in Proc. IEEE Comput. Soc. Circuits Syst., Midwest Symp., Aug. 1998, pp. 374-378.
2. S. C. Park, M. K. Park, and M. G. Kang, " Super-resolution image reconstruction: A technical overview, " IEEE Signal Process. Mag., vol. 20, no. 3, pp. 21-36, May 2003.
3. A. K. Katsaggelos, R. Molina, and J. Mateos, " Super resolution of images and video, " Synth. Lect. Image, Video, Multimedia Process., vol. 1, no. 1, pp. 1-134, 2007.
4. K. Nasrollahi and T. B. Moeslund, " Super-resolution: A comprehensive survey, " Mach. Vis. Appl., vol. 25, no. 6, pp. 1423-1468, 2014.
5. R. Y. Tsai and T. S. Huang, " Multiframe image restoration and registration, " Adv. Comput. Vis. Image Process., vol. 1, no. 2, pp. 317-339, 1984.
6. C. Liu and D. Sun, " A Bayesian approach to adaptive video super resolution, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2011, pp. 209-216.
7. C. A. Segall, R. Molina, and A. K. Katsaggelos, " High-resolution images from low-resolution compressed video, " IEEE Signal Process. Mag., vol. 20, no. 3, pp. 37-48, May 2003.
8. C. A. Segall, A. K. Katsaggelos, R. Molina, and J. Mateos, " Bayesian resolution enhancement of compressed video, " IEEE Trans. Image Process., vol. 13, no. 7, pp. 898-911, Jul. 2004.
9. S. P. Belekos, N. P. Galatsanos, and A. K. Katsaggelos, " Maximum a posteriori video super-resolution using a new multichannel image prior, " IEEE Trans. Image Process., vol. 19, no. 6, pp. 1451-1464, Jun. 2010.
10. Q. Shan, Z. Li, J. Jia, and C.-K. Tang, " Fast image/video upsampling, " ACM Trans. Graph., vol. 27, no. 5, Dec. 2008, Art. no. 153.
11. H. Takeda, P. Milanfar, M. Protter, and M. Elad, " Super-resolution without explicit subpixel motion estimation, " IEEE Trans. Image Process., vol. 18, no. 9, pp. 1958-1975, Sep. 2009.
12. E. M. Hung, R. L. D. Queiroz, F. Brandi, K. F. D. Oliveira, and D. Mukherjee, " Video super-resolution using codebooks derived from key-frames, " IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 9, pp. 1321-1331, Sep. 2012.
13. F. Zhou, W. Yang, and Q. Liao, " Interpolation-based image superresolution using multisurface fitting, " IEEE Trans. Image Process., vol. 21, no. 7, pp. 3312-3318, Jul. 2012.
14. F. Zhou, S.-T. Xia, and Q. Liao, " Nonlocal pixel selection for multisurface fitting-based super-resolution, " IEEE Trans. Circuits Syst. Video Technol., vol. 24, no. 12, pp. 2013-2017, Dec. 2014.
15. C. Liu and D. Sun, " On Bayesian adaptive video super resolution, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 2, pp. 346-360, Feb. 2014.
16. Z. Ma, R. Liao, X. Tao, L. Xu, J. Jia, and E. Wu, " Handling motion blur in multi-frame super-resolution, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 5224-5232.
17. R. Liao, X. Tao, R. Li, Z. Ma, and J. Jia, " Video super-resolution via deep draft-ensemble learning, " in Proc. IEEE Int. Conf. Comput. Vis., Dec. 2015, pp. 531-539.
18. M. Elad and Y. Hel-Or, " A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur, " IEEE Trans. Image Process., vol. 10, no. 8, pp. 1187-1193, Aug. 2001.
19. J. Yang, J. Wright, T. Huang, and Y. Ma, " Image super-resolution as sparse representation of raw image patches, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2008, pp. 1-8.
20. J. Yang, J. Wright, T. S. Huang, and Y. Ma, " Image super-resolution via sparse representation, " IEEE Trans. Image Process., vol. 19, no. 11, pp. 2861-2873, Nov. 2010.
21. B. C. Song, S.-C. Jeong, and Y. Choi, " Video super-resolution algorithm using bi-directional overlapped block motion compensation and onthe-fly dictionary training, " IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 3, pp. 274-285, Mar. 2011.
22. J. Yang, Z. Wang, Z. Lin, X. Shu, and T. Huang, " Bilevel sparse coding for coupled feature spaces, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2012, pp. 2360-2367.
23. J. Yang, Z. Wang, Z. Lin, S. Cohen, and T. Huang, " Coupled dictionary training for image super-resolution, " IEEE Trans. Image Process., vol. 21, no. 8, pp. 3467-3478, Aug. 2012.
24. S. Wang, D. Zhang, Y. Liang, and Q. Pan, " Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2012, pp. 2216-2223.
25. R. Nakagaki and A. K. Katsaggelos, " A VQ-based blind image restoration algorithm, " IEEE Trans. Image Process., vol. 12, no. 9, pp. 1044-1053, Sep. 2003.
26. M. Protter and M. Elad, " Image sequence denoising via sparse and redundant representations, " IEEE Trans. Image Process., vol. 18, no. 1, pp. 27-35, Jan. 2009.
27. M. Elad and M. Aharon, " Image denoising via sparse and redundant representations over learned dictionaries, " IEEE Trans. Image Process., vol. 15, no. 12, pp. 3736-3745, Dec. 2006.
28. M. Liang, J. Du, and L. Li, " Learning-based video superresolution reconstruction using spatiotemporal nonlocal similarity, " Math. Problems Eng., vol. 2015, 2015, Art. no. 687074.
29. X. Gao, Q. Wang, X. Li, D. Tao, and K. Zhang, " Zernike-moment-based image super resolution, " IEEE Trans. Image Process., vol. 20, no. 10, pp. 2738-2747, Oct. 2011.
30. W. Dong, L. Zhang, G. Shi, and X. Wu, " Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization, " IEEE Trans. Image Process., vol. 20, no. 7, pp. 1838-1857, Jul. 2011.
31. R. Zeyde, M. Elad, and M. Protter, " On single image scale-up using sparse-representations, " in Proc. Int. Conf. Curves Surf., 2012, pp. 711-730.
32. I. Ramirez, P. Sprechmann, and G. Sapiro, " Classification and clustering via dictionary learning with structured incoherence and shared features, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2010, pp. 3501-3508.
33. B. K. P. Horn and B. G. Schunck, " Determining optical flow, " Artif. Intell., vol. 17, nos. 1-3, p. 185-203, Aug. 1981.
34. S. Baker, D. Scharstein, J. P. Lewis, S. Roth, M. J. Black, and R. Szeliski, " A database and evaluation methodology for optical flow, " Int. J. Comput. Vis., vol. 92, no. 1, pp. 1-31, 2011.
35. D. Sun, S. Roth, and M. J. Black, " Secrets of optical flow estimation and their principles, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2010, pp. 2432-2439.
36. M. Drulea and S. Nedevschi, " Total variation regularization of localglobal optical flow, " in Proc. 14th Int. IEEE Conf. Intell. Transp. Syst. (ITSC), Oct. 2011, pp. 318-323.
37. M. Bevilacqua, A. Roumy, C. Guillemot, and M. L. Alberi-Morel, " Low-complexity single-image super-resolution based on nonnegative neighbor embedding, " in Proc. 23rd Brit. Mach. Vis. Conf. (BMVC), 2012, pp. 135.1-135.10.
38. H. Chang, D.-Y. Yeung, and Y. Xiong, " Super-resolution through neighbor embedding, " in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (CVPR), vol. 1. Jun. 2004, pp. I-I.
39. R. Timofte, V. De, and L. Van Gool, " Anchored neighborhood regression for fast example-based super-resolution, " in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Dec. 2013, pp. 1920-1927.
40. C. Dong, C. C. Loy, K. He, and X. Tang, " Learning a deep convolutional network for image super-resolution, " in Proc. Eur. Conf. Comput. Vis., 2014, pp. 184-199.
41. Infognition. (2010). Video Enhancer. [Online]. Available: http://www.infognition.com/videoenhancer/
42. Q. Dai, S. Yoo, A. Kappeler, and A. K. Katsaggelos, " Dictionary-based multiple frame video super-resolution, " in Proc. IEEE Int. Conf. Image Process. (ICIP), Sep. 2015, pp. 83-87.
43. M. K. Park, A. K. Katsaggelos, and M. G. Kang, " Regularized highresolution image reconstruction considering inaccurate motion information, " Opt. Eng., vol. 46, no. 11, p. 117004, Nov. 2007.
44. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
45. H. Lee, A. Battle, R. Raina, and A. Y. Ng, " Efficient sparse coding algorithms, " in Proc. Adv. Neural Inf. Process. Syst., Dec. 2006, pp. 801-808.
46. H. Inc. (2014). Harmonic 4k Footage. [Online]. Available: http:// www.harmonicinc.com/resources/videos/4k-video-clip-center
47. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, " Image quality assessment: From error visibility to structural similarity, " IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, Apr. 2004.
48. W. Dong, X. Li, L. Zhang, and G. Shi, " Sparsity-based image denoising via dictionary learning and structural clustering, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2011, pp. 457-464.