Video super-resolution using codebooks derived from key-frames

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 22, Issue 9, 2012, Pages 1321-1331

  Hung, Edson Mintsu ^a   De Queiroz, Ricardo L ^a   Brandi, Fernanda ^b   De Oliveira, Karen França ^c   Mukherjee, Debargha ^d  

^a UNIVERSITY OF BRASILIA   (Brazil)

^b TECHNICAL UNIVERSITY OF MUNICH   (Germany)

^c Brazilian Court of Audit ^*  (Brazil)

^d GOOGLE INC   (United States)

Author keywords

Example based super resolution (SR); video processing

Indexed keywords

BAYESIAN APPROACHES; CODEBOOK CONSTRUCTIONS; CODEBOOKS; EXAMPLE-BASED SUPER-RESOLUTION; HIGH FREQUENCY HF; HIGH RESOLUTION IMAGE; HIGHER RESOLUTION; KEY-FRAMES; LOW FREQUENCY; LOWER RESOLUTION; MULTI-RESOLUTION APPROACH; REFERENCE BLOCK; VIDEO PROCESSING; VIDEO SEQUENCES; VIDEO SUPER-RESOLUTION;

BAYESIAN NETWORKS; IMAGE RESOLUTION;

VIDEO SIGNAL PROCESSING;

EID: 84866005025     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2012.2201669     Document Type: Article

References (49)

1. S. Chaudhuri, Super-Resolution Imaging. Norwell, MA: Kluwer, 2001.
2. S. C. Park, M. K. Park, and M. G. Kang, "Super-resolution image reconstruction: A technical overview," IEEE Signal Process. Mag., vol. 20, no. 3, pp. 21-36, May 2003.
3. A. K. Katsaggelos, R. Molina, and J. Mateos, "Super resolution of images and video," Synthesis Lectures on Image, Video, and Multimedia Processing. San Mateo, CA: Morgan and Claypool, 2007.
4. C. A. Segall, A. K. Katsaggelos, R. Molina, and J. Mateos, "Bayesian resolution enhancement of compressed video," IEEE Trans. Image Process., vol. 13, no. 7, pp. 898-911, Jul. 2004.
5. M. E. Tipping and C. M. Bishop, "Bayesian image super-resolution, " Adv. Neural Inform. Process. Syst., vol. 15, pp. 1303-1310, Jan. 2002.
6. M. Irani and S. Peleg, "Motion analysis for image enhancement: Resolution, occlusion and transparency," J. Vis. Commun. Image Representat., vol. 4, no. 4, pp. 324-335, Dec. 1993.
7. S. Dai, M. Han, Y. Wu, and Y. Gong, "Bilateral back-projection for single image super resolution," in Proc. IEEE Int. Conf. Multimedia Expo., Jul. 2007, pp. 1039-1042.
8. H. Ji and C. Fermuller, "Robust wavelet-based super-resolution reconstruction: Theory and algorithm," IEEE Trans. Patt. Anal. Mach. Intell., vol. 31, no. 4, pp. 649-660, Apr. 2009.
9. H. Stark and P. Oskoui, "High-resolution image recovery from imageplane arrays using convex projections," J. Optic. Soc. America, Series A, vol. 6, pp. 1715-1726, Nov. 1989
10. F. W. Wheeler, R. T. Hoctor, and E. B. Barrett, "Super-resolution image synthesis using projections onto convex sets in the frequency domain," in Proc. IS&T/SPIE Symp. Electron. Imag., Conf. Computat. Imag., vol. 5674. Jan. 2005, pp. 479-490.
11. J. Yang, J. Wright, T. S. Huang, and Y. Ma, "Image super-resolution as sparse representation of raw image patches," in Proc. IEEE CVPR, Jun. 2008, pp. 1-8.
12. M. Protter and M. Elad, "Image sequence denoising via sparse and redundant representations," IEEE Trans. Image Process., vol. 18, no. 1, pp. 27-36, Jan. 2009.
13. R. Zeyde, M. Elad, and M. Protter, "On single image scale-up using sparse-representations," in Proc. Curves Surfaces, Jun. 2010, pp. 711- 730.
14. J. Yang, J. Wright, T. S. Huang, and Y. Ma, "Image super-resolution via sparse representation," IEEE Trans. Image Process., vol. 19, no. 11, pp. 2861-2873, Nov. 2010.
15. H. Ur and D. Gross, "Improved resolution from sub-pixel shifted pictures," CVGIP: Graph. Models Image Process., vol. 54, pp. 181-186, Mar. 1992.
16. M. Elad and A. Feuer, "Restoration of single super-resolution image from several blurred, noisy and down-sampled measured images," IEEE Trans. Image Process., vol. 6, no. 12, pp. 1646-1658, Dec. 1997.
17. S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe super resolution," IEEE Trans. Image Process., vol. 13, no. 10, pp. 1327-1344, Oct. 2004.
18. M. Protter, M. Elad, H. Takeda, and P. Milanfar, "Generalizing the non-local-means to super-resolution reconstruction," IEEE Trans. Image Process., vol. 18, no. 1, pp. 36-51, Jan. 2009.
19. M. Protter and M. Elad, "Super-resolution with probabilistic motion estimation," IEEE Trans. Image Process., vol. 18, no. 8, pp. 1899-1904, Aug. 2009.
20. R. R. Schultz and R. L. Stevenson, "Extraction of high-resolution frames from video sequences," IEEE Trans. Image Process., vol. 5, no. 6, pp. 996-1011, Jun. 1996.
21. H. Shen, L. Zhang, B. Huang, and P. Li, "A MAP approach for joint motion estimation, segmentation, and super resolution," IEEE Trans. Image Process., vol. 16, no. 2, pp. 479-490, Feb. 2007.
22. S. P. Belekos, N. P. Galatsanos, and A. K. Katsaggelos, "Maximum a posteriori video super-resolution with a new multichannel image prior," in Proc. EUSIPCO, Aug. 2008, pp. 25-29.
23. S. Belekos, N. Galatsanos, S. D. Babacan, and A. K. Katsaggelos, "Maximum a posteriori super-resolution of compressed video using a new multichannel image prior," in Proc. IEEE Int. Conf. Image Process., Nov. 2009, pp. 2797-2800.
24. S. P. Belekos, N. P. Galatsanos, and A. K. Katsaggelos, "Maximum a posteriori video super-resolution using a new multichannel image prior," IEEE Trans. Image Process., vol. 19, no. 6, pp. 1451-1464, Jun. 2010.
25. S. P. Belekos, N. P. Galatsanos, and A. K. Katsaggelos, "Maximum a posteriori super-resolution of compressed video with a novel multichannel image prior and a new observation model," in Proc. EUSIPCO, Aug. 2011, pp. 293-297.
26. S. P. Belekos, J. Jeon, J. Lee, J. Paik, and A. K. Katsaggelos, "Region-based super-resolution reconstruction using parallel programming," in Proc. ITC-CSCC, Jun. 2011, pp. 95-96.
27. R. Y. Tsai and T. S. Huang, "Multiframe image restoration and registration," Adv. Comput. Vis. Image Process., vol. 1, no. 1, pp. 317- 339, 1984.
28. R. Chan, T. Chan, L. Shen, and Z. Shen, "Wavelet deblurring algorithms for spatially varying blur from high-resolution image reconstruction," in Proc. Linear Algebra Its Applicat., 2003, pp. 139-155.
29. R. Chan, S. Riemenschneider, L. Shen, and Z. Shen, "High-resolution image reconstruction with displacement errors: A framelet approach," Int. J. Imag. Syst. Technol., vol. 14, no. 3, pp. 91-104, 2004.
30. R. Chan, S. Riemenschneider, L. Shen, and Z. Shen, "Tight frame: An efficient way for high-resolution image reconstruction," Appl. Computat. Harmonic Anal., vol. 17, no. 1, pp. 91-115, 2004.
31. P. Vandewalle, S. E. Süsstrunk, and M. Vetterli, "A frequency domain approach to registration of aliased images with application to superresolution," EURASIP J. Appl. Signal Process., vol. 2006, pp. 1-14, Jan. 2006.
32. W. T. Freeman, T. R. Jones, and E. C. Pasztor, "Example-based superresolution," IEEE Comput. Graphics Applicat., vol. 22, no. 2, pp. 56-65, Mar. 2002.
33. F. Brandi, R. de Queiroz, and D. Mukherjee, "Super resolution of video using key-frames," in Proc. IEEE Int. Symp. Circuits Syst., May 2008, pp. 1608-1611.
34. F. Brandi, R. L. de Queiroz, and D. Mukherjee, "Super-resolution of video using key-frames and motion estimation," in Proc. IEEE ICIP, Oct. 2008, pp. 321-324.
35. K. F. Oliveira, F. Brandi, E. M. Hung, R. L. de Queiroz, and D. Mukherjee, "Bipredictive video super-resolution using key-frames," in Proc. IS&T/SPIE Symp. Electron. Imag., Vis. Inform. Process. Commun., vol. 7543. Jan. 2010, pp. 1-5.
36. B. C. Song, S. C. Jeong, and Y. Choi, "Video super-resolution algorithm using bi-directional overlapped block motion compensation and on-the-fly dictionary training," IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 3, pp. 274-285, Mar. 2011.
37. T. Wiegand, G. Sullivan, G. Bjøntegaard, and A. Luthra, "Overview of the H.264 video coding standard," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 560-576, Jul. 2003.
38. I. E. Richardson, H.264 and MPEG-4 Video Compression: Video Coding for Next Generation Multimedia. New York: Wiley, 2003.
39. E. M. Hung and R. L. de Queiroz, "Blocking-effect reduction in a reversed-complexity video codec based on a mixed-quality framework," in Proc. Int. Telecommun. Symp., Sep. 2010, pp. 1-6.
40. S. Nogaki and M. Ohta, "An overlapped block motion compensation for high quality motion picture coding," in Proc. IEEE Int. Symp. Circuits Syst., vol. 1. May 1992, pp. 184-187.
41. J. Zhang, M. O. Ahmad, and M. N. S. Swamy, "Overlapped variable size block motion compensation," in Proc. IEEE ICIP, Oct. 1997, pp. 164-167.
42. R. Li, B. Zeng, and M. L. Liou, "A new three-step search algorithm for block motion estimation," IEEE Trans. Circuits Syst. Video Technol., vol. 4, no. 4, pp. 438-42, Aug. 1994.
43. S. Zhu and K. K. Ma, "A new diamond search algorithm for fast block matching motion estimation," in Proc. Int. Conf. Inform., Commun. Signal Process., vol. 1. 1997, pp. 292-296.
44. J. Y. Tham, S. Ranganath, M. Ranganath, and A. A. Kassim, "A novel unrestricted center-biased diamond search algorithm for block motion estimation," IEEE Trans. Circuits Syst. Video Technol., vol. 8, no. 4, pp. 369-377, Aug. 1998.
45. A. Tourapis, O. C. Au, and M. L. Liou, "Highly efficient predictive zonal algorithm for fast block-matching motion estimation," IEEE Trans. Circuits Syst. Video Technol., vol. 12, no. 10, pp. 934-947, Oct. 2002.
46. D. Mukherjee, "A robust reversed complexity Wyner-Ziv video codec introducing sign-modulated codes," HP Labs., Palo Alto, CA, Tech. Rep. HPL-2006-80, May 2006.
47. D. Mukherjee, B. Macchiavello, and R. L. de Queiroz, "A simple reversed complexity Wyner-Ziv video coding mode based on a spatial reduction framework," Proc. SPIE VCIP, vol. 6508, pp. 1-12, Jan. 2007.
48. C. Yeo, W. T. Tan, and D. Mukherjee, "Receiver error concealment using acknowledge preview (RECAP): An approach to resilient video streaming," in Proc. Int. Conf. Acou., Speech Signal Process., Apr. 2009, pp. 785-788.
49. G. Bjøntegaard, "Calculation of average PSNR differences between RD curves," document ITU-T SC16/Q6, 13th VCEG Meeting, Austin, TX, Apr. 2001.