Video super-resolution using an adaptive superpixel-guided auto-regressive model

Pattern Recognition

Volumn 51, Issue , 2016, Pages 59-71

  Li, Kun ^a   Zhu, Yanming ^a   Yang, Jingyu ^a   Jiang, Jianmin ^b  

^a TIANJIN UNIVERSITY   (China)

^b SHENZHEN UNIVERSITY   (China)

Author keywords

Auto regressive model; Spatio temporal correlation; Superpixel; Video super resolution

Indexed keywords

ADAPTIVE OPTICS; IMAGE SEGMENTATION; OPTICAL CORRELATION; OPTICAL FLOWS; OPTICAL RESOLVING POWER; PIXELS; REGRESSION ANALYSIS; SIGNAL ENCODING;

AUTO REGRESSIVE MODELS; OPTICAL FLOW METHODS; PEAK SIGNAL TO NOISE RATIO; SPATIOTEMPORAL CORRELATION; STATE-OF-THE-ART METHODS; SUPER PIXELS; TEMPORAL CORRELATIONS; VIDEO SUPER-RESOLUTION;

SIGNAL TO NOISE RATIO;

EID: 84955630405     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2015.08.008     Document Type: Article

References (58)

1. A. Akyol, and M. Gökmen Super-resolution reconstruction of faces by enhanced global models of shape and texture Pattern Recognit. 45 12 2012 4103 4116
2. R. He, and Z. Zhang Locally affine patch mapping and global refinement for image super-resolution Pattern Recognit. 44 9 2011 2210 2219
3. H. Huang, H. He, X. Fan, and J. Zhang Super-resolution of human face image using canonical correlation analysis Pattern Recognit. 43 7 2010 2532 2543
4. H. Su, L. Tang, D. Tretter, J. Zhou, A practical and adaptive framework for super-resolution, in: Proceedings of IEEE International Conference on Image Processing, 2008, pp. 1236-1239.
5. M. Protter, M. Elad, H. Takeda, and P. Milanfar Generalizing the nonlocal-means to super-resolution reconstruction IEEE Trans. Image Process. 18 1 2009 36 51
6. H. Takeda, P. Milanfar, M. Protter, and M. Elad Super-resolution without explicit subpixel motion estimation IEEE Trans. Image Process. 18 9 2009 1958 1975
7. F. Brandi, R. Queiroz, D. Mukherjee, Super-resolution of video using key frames and motion estimation, in: Proceedings of IEEE International Conference on Image Processing, 2008, pp. 321-324.
8. E.M. Hung, R.L. de Queiroz, and F. Brandi Video super-resolution using codebooks derived from key-frames IEEE Trans. Circuits Syst. Video Technol. 22 9 2012 1321 1331
9. W.T. Freeman, T.R. Jones, and E.C. Pasztor Example-based super-resolution IEEE Comput. Graph. Appl. 22 2 2002 56 65
10. F. Brandi, R. de Queiroz, D. Mukherjee, Super resolution of video using key frames, in: Proceedings of IEEE International Symposium on Circuits and Systems, 2008, pp. 1608-1611.
11. B.C. Song, S.-C. Jeong, and Y. Choi Video super-resolution algorithm using bi-directional overlapped block motion compensation and on-the-fly dictionary training IEEE Trans. Circuits Syst. Video Technol. 21 3 2011 274 285
12. S. Belekos, N.P. Galatsanos, and A.K. Katsaggelos Maximum a posteriori video super-resolution using a new multichannel image prior IEEE Trans. Image Process. 19 6 2010 1451 1464
13. Y. Altunbasak, A.J. Patti, and R.M. Mersereau Super-resolution still and video reconstruction from MPEG-coded video IEEE Trans. Circuits Syst. Video Technol. 12 4 2002 217 226
14. C.A. Segall, A.K. Katsaggelos, R. Molina, and J. Mateos Super-resolution from compressed video Kluwer Int. Ser. Eng. Comput. Sci. 2001 211 242
15. M.F. Tappen, B.C. Russell, W.T. Freeman, Exploiting the sparse derivative prior for super-resolution and image demosaicing, in: Proceedings of IEEE Workshop on Statistical and Computational Theories of Vision, 2003.
16. B.C. Tom, and A.K. Katsaggelos Resolution enhancement of monochrome and color video using motion compensation IEEE Trans. Image Process. 10 2 2001 278 287
17. Z. Jiang, T.-T. Wong, H. Bao, Practical super-resolution from dynamic video sequences, in: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, 2003, pp. 549-554.
18. H. How, and H. Andrews Cubic spline for image interpolation and digital filtering IEEE Trans. Acoust. Speech Signal Process. 26 5 1978 508 517
19. R.G. Keys Cubic convolution interpolation for digital image processing IEEE Trans. Acoust. Speech Signal Process. 29 6 1981 1153 1160
20. X. Li, and M.T. Orchard New edge-directed interpolation IEEE Trans. Image Process. 10 10 2001 1521 1527
21. Q. Wang, and R.K. Ward A new orientation-adaptive interpolation method IEEE Trans. Image Process. 16 4 2007 889 900
22. H. Ji, and C. Fermuller Robust wavelet-based super-resolution reconstruction: theory and algorithm IEEE Trans. Pattern Anal. Mach. Intell. 31 4 2009 649 660
23. X. Gao, K. Zhang, D. Tao, and X. Li Image super-resolution with sparse neighbor embedding IEEE Trans. Image Process. 21 7 2012 3194 3205
24. K. Zhang, X. Gao, D. Tao, X. Li, Image super-resolution via non-local steering kernel regression regularization, in: Proceedings of IEEE International Conference on Image Processing, 2013, pp. 943-947.
25. H. Su, Y. Wu, and J. Zhou Super-resolution without dense flow IEEE Trans. Image Process. 21 4 2012 1782 1795
26. C. Liu, and D. Sun On Bayesian adaptive video super resolution IEEE Trans. Pattern Anal. Mach. Intell. 36 2 2013 346 360
27. J. Lu, H. Zhang, and Y. Sun Video super resolution based on non-local regularization and reliable motion estimation Signal Process.: Image Commun. 29 4 2014 514 529
28. Y. Liao, M. Tian, and L. Guo A novel video super-resolution algorithm based on non-local normalized convolution Unifying Electr. Eng. Electron. Eng. 238 2013 1141 1150
29. J. Yang, J. Wright, T.S. Huang, Y. Ma, Image super-resolution via sparse representation, IEEE Trans. Image Process. 19 (11) (2010) 2861-2873.
30. G. Mu, X. Gao, K. Zhang, X. Li, D. Tao, Single image super resolution with high resolution dictionary, in: Proceedings of IEEE Conference on Image Processing, 2011, pp. 1141-1144.
31. K. Jia, X. Wang, and X. Tang Image transformation based on learning dictionaries across image spaces IEEE Trans. Pattern Anal. Mach. Intell. 35 2 2013 367 380
32. K. Zhang, X. Gao, D. Tao, and X. Li Single image super-resolution with multiscale similarity learning IEEE Trans. Neural Netw. Learn. Syst. 24 10 2013 1648 1659
33. K.I. Kim, and Y. Kwon Single-image super-resolution using sparse regression and natural image prior IEEE Trans. Pattern Anal. Mach. Intell. 32 6 2010 1127 1133
34. H. Kawano, N. Suetake, B. Cha, T. Aso, Super-resolution via matching from self-decomposed codebook with local distance measure incorporating pixel correlation, in: Proceedings of Biennial Conference of the Australian Pattern Recognition Society on Digital Image Computing Techniques and Applications, 2007, pp. 317-323.
35. K.F. de Oliveira, F. Brandi, E.M. Hung, R.L. de Queiroz, D. Mukherjee, Bipredictive video super-resolution using key-frames, in: Proceedings of SPIE Symposium on Electronic Imaging, Visual Information Processing and Communication, 2010.
36. ISO, Information technology-coding of moving pictures and associated audio for digital storage media at up to about 1.5 mbit/s, Technical Report, Motion Pictures Experts Group, 1993.
37. I.T.T.C. Community, Video codec for audiovisual services at px64 kb/s, Technical Report, ITU-T Recommendation H.261, 1993.
38. F. Brandi, R. de Queiroz, D. Mukherjee, Super-resolution of video using key frames and motion estimation, in: Proceedings of IEEE International Conference on Image Processing, 2008, pp. 321-324.
39. J.Y. Tham, S. Ranganath, M. Ranganath, and A.A. Kassim A novel unrestricted center-biased diamond search algorithm for block motion estimation IEEE Trans. Circuits Syst. Video Technol. 12 10 2002 934 947
40. B.-D. Choi, J.-W. Han, C.-S. Kim, and S.-J. Ko Motion-compensated frame interpolation using bilateral motion estimation and adaptive overlapped block motion compensation IEEE Trans. Circuits Syst. Video Technol. 17 4 2007 407 416
41. X. Li, and M.T. Orchard New edge-directed interpolation IEEE Trans. Image Process. 10 10 2001 1521 1527
42. M.V. Joshi, S. Chaudhuri, and R. Panuganti A learning-based method for image super-resolution from zoomed observations IEEE Trans. Syst. Man Cybern. Part B: Cybern. 35 3 2005 527 537
43. X. Zhang, S. Ma, Y. Zhang, L. Zhang, W. Gao, Nonlocal edge-directed interpolation, in: Advances in Multimedia Information Processing-PCM, 2009, pp. 1197-1207.
44. X. Zhang, and X. Wu Image interpolation by adaptive 2-D autoregressive modeling and soft-decision estimation IEEE Trans. Image Process. 17 6 2008 887 896
45. X. Gao, J. Zhang, F. Jiang, X. Fan, S. Ma, D. Zhao, High-quality image interpolation via local autoregressive and nonlocal 3-D sparse regularization, in: Proceedings of IEEE Visual Communications and Image Processing, 2012, pp. 1-6.
46. J. Yang, X. Ye, K. Li, C. Hou, Depth recovery using an adaptive color-guided auto-regressive model, in: Proceedings of European Conference on Computer Vision, 2012, pp. 158-171.
47. J. Yang, X. Ye, K. Li, C. Hou, and Y. Wang Color-guided depth recovery from RGB-D data using an adaptive autoregressive model IEEE Trans. Image Process. 23 8 2014 3443 3458
48. W. Dong, L. Zhang, R. Lukac, and G. Shi Sparse representation based image interpolation with nonlocal autoregressive modeling IEEE Trans. Image Process. 22 4 2013 1382 1394
49. L.L. Rakêt, L. Roholm, A. Bruhn, J. Weickert, Motion compensated frame interpolation with a symmetric optical flow constraint, in: Advances in Visual Computing, Springer, 2012, pp. 447-457.
50. K. Chen, and D.A. Lorenz Image sequence interpolation based on optical flow, segmentation, and optimal control IEEE Trans. Image Process. 21 3 2012 1020 1030
51. A. Ayvaci, M. Raptis, and S. Soatto Sparse occlusion detection with optical flow Int. J. Comput. Vis. 97 3 2012 322 338
52. S. Yang, L. An, B. Bhanu, N. Thakoor, Improving action units recognition using dense flow-based face registration in video, in: Proceedings of IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 2013, pp. 1-8.
53. M. Eisemann, B. De Decker, M. Magnor, P. Bekaert, E. De Aguiar, N. Ahmed, C. Theobalt, A. Sellent, Floating textures, in: Proceedings of Computer Graphics Forum, vol. 27, 2008, pp. 409-418.
54. R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Susstrunk SLIC superpixels compared to state-of-the-art superpixel methods IEEE Trans. Pattern Anal. Mach. Intell. 34 11 2012 2274 2282
55. S. Farsiu, M.D. Robinson, M. Elad, and P. Milanfar Fast and robust multiframe super resolution IEEE Trans. Image process. 13 10 2004 1327 1344
56. W. Fan, D.Y. Yeung, Image hallucination using neighbor embedding over visual primitive manifolds, in: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2007, pp. 1-7.
57. Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli Image quality assessment: from error visibility to structural similarity IEEE Trans. Image Process. 13 4 2004 600 612
58. L. Zhang, D. Zhang, and X. Mou Fsim: a feature similarity index for image quality assessment IEEE Trans. Image Process. 20 8 2011 2378 2386