A survey on super-resolution imaging

Signal, Image and Video Processing

Volumn 5, Issue 3, 2011, Pages 329-342

  Tian, Jing ^a   Ma, Kai Kuang ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

Regularization; Resolution enhancement; Super resolution imaging

Indexed keywords

ACQUISITION PROCESS; ILL POSED; IMAGE FRAMES; IMAGE SEQUENCE; IMAGE-BASED; KEY OBJECTIVE; MULTI-VIEW IMAGE; REGULARIZATION; RESEARCH CHALLENGES; RESEARCH DIRECTIONS; RESOLUTION ENHANCEMENT; SCENE RECOGNITION; SUPER RESOLUTION IMAGING; SUPERRESOLUTION; TEMPORAL DOMAIN; VIDEO RECONSTRUCTION;

OPTICAL CHARACTER RECOGNITION; VISUALIZATION;

OPTICAL RESOLVING POWER;

EID: 80051947769     PISSN: 18631703     EISSN: 18631711     Source Type: Journal    
DOI: 10.1007/s11760-010-0204-6     Document Type: Review

References (119)

1. Chaudhuri S.: Super-Resolution Imaging. Kluwer Academic Publishers, Boston (2001).
2. Katsaggelos A. K., Molina R., Mateos J.: Super Resolution of Images and Video. Morgan & Claypool, San Rafael (2007).
3. Bannore V.: Iterative-Interpolation Super-Resolution Image Reconstruction: A Computationally Efficient Technique. Springer, Berlin (2009).
4. Milanfar P.: Super-Resolution Imaging. CRC Press, Boca Raton (2010).
5. Kang M. G., Chaudhuri S.: Super-resolution image reconstruction. IEEE Signal Process. Mag. 20, 19-20 (2003).
6. Bose N. K., Chan R. H., Ng M. K.: Special issue on high resolution image reconstruction. Int. J. Imaging Syst. Technol. 14(2-3), 35-145 (2004).
7. Ng M. K., Chan T., Kang M. G., Milanfar P.: Special issue on super-resolution imaging: analysis, algorithms, and applications. EURASIP J. Appl. Signal Process. 2006, 1-2 (2006).
8. Ng M. K., Lam E., Tong C. S.: Special issue on superresolution imaging: theory, algorithms and applications. Multidimens. Syst. Signal Process. 18, 55-188 (2007).
9. Hardie R. C., Schultz R. R., Barner K. E.: Special issue on super-resolution enhancement of digital video. EURASIP J. Appl. Signal Process. 2007, 1-3 (2007).
10. Katsaggelos A. K., Molina R.: Special issue on super resolution. Comput. J. 52, 1-167 (2008).
11. Hunt B. R.: Superresolution of images: algorithms, principles, performance. Int. J. Imaging Syst. Technol. 6(4), 297-304 (1995).
12. Borman, S., Stevenson R. L.: Spatial resolution enhancement of low-resolution image sequences: a comprehensive review with directions for future research. University of Notre Dame, Technical Report (1998).
13. Borman, S., Stevenson, R. L.: Super-resolution from image sequences-a review. In: Proceedings of the IEEE Midwest Symposium on Circuits and Systems, pp. 374-378. Notre Dame, IN (1998).
14. Park S. C., Park M. K., Kang M. G.: Super-resolution image reconstruction: a technical overview. IEEE Signal Process. Mag. 20, 21-36 (2003).
15. Ng M. K., Bose N. K.: Mathematical analysis of super-resolution methodology. IEEE Signal Process. Mag. 20, 62-74 (2003).
16. Capel D., Zisserman A.: Computer vision applied to super resolution. IEEE Signal Process. Mag. 20, 75-86 (2003).
17. Choi E., Choi J., Kang M. G.: Super-resolution approach to overcome physical limitations of imaging sensors: an overview. Int. J. Imaging Syst. Technol. 14, 36-46 (2004).
18. Farsiu S., Robinson D., Elad M., Milanfar P.: Advances and challenges in super-resolution. Int. J. Imaging Syst. Technol. 14, 47-57 (2004).
19. Ouwerkerk J. D.: Image super-resolution survey. Image Vis. Comput. 24, 1039-1052 (2006).
20. Cristobal, G., Gila, E., Sroubek, F., Flusser, J., Miravet, C., Rodriguez, F. B.: Superresolution imaging: a survey of current techniques. In: Proceedings of the SPIE Conference on Advanced Signal Processing Algorithms, Architectures, and Implementations, pp. 70740C. 1-70740C. 18 (2008).
21. Tsai R. Y., Huang T. S.: Multiframe image restoration and registration. In: Huang, T. S. (ed.) Advances in Computer Vision and Image Processing, JAI Press Inc., London (1984).
22. Max B., Wolf E.: Principles of Optics. Cambridge University Press, Cambridge (1997).
23. Dropkin, H., Ly, C.: Superresolution for scanning antenna. In: Proceedings of the IEEE International Radar Conference, pp. 306-30. (1997).
24. Caner, G., Tekalp, A. M., Heinzelman, W.: Performance evaluation of super-resolution reconstruction from video. In: Proceedings of the IS& T/SPIE's Annual Symposium on Electronic Imaging, pp. 177-185. Santa Clara, CA (2003).
25. Tanaka, M., Okutomi, M.: Theoretical analysis on reconstruction-based super-resolution for an arbitrary PSF. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 947-954. San Diego, CA (2005).
26. Rajagopalan A. N., Kiran V. P.: Motion-free superresolution and the role of relative blur. Opt. Soc. Am. J. A Opt. Image Sci. 20, 2022-2032 (2003).
27. Pham, T. Q., van Vliet, L. J., Schutte, K.: Influence of signal-to-noise ratio and point spread function on limits of superresolution. In: Proceedings of the SPIE Conference on Image Processing: Algorithms and Systems IV, pp. 169-180. San Jose, CA (2005).
28. Wang Z.-Z., Qi F.-H.: Analysis of multiframe super-resolution reconstruction for image anti-aliasing and deblurring. Image Vis. Comput. 23, 393-404 (2005).
29. Eekeren A. W. M., Schutte K., Oudegeest O. R., Vliet L. J.: Performance evaluation of super-resolution reconstruction methods on real-world data. EURASIP J. Appl. Signal Process. 2007(43953), 1-11 (2007).
30. Baker S., Kanade T.: Limits on super-resolution and how to break them. IEEE Trans. Pattern Anal. Mach. Intell. 24, 1167-1183 (2002).
31. Lin Z., Shum H.-Y.: Fundamental limits of reconstruction-based superresolution algorithms under local translation. IEEE Trans. Pattern Anal. Mach. Intell. 26, 83-97 (2004).
32. Lin, Z.-C., He, J.-F., Tang, X.-O., Tang, C.-K.: Limits of learning-based superresolution algorithms. In: Proceedings of the IEEE International Conference on Computer Vision. Rio de Janeiro, Brazil (2007).
33. Robinson D., Milanfar P.: Statistical performance analysis of super-resolution. IEEE Trans. Image Process. 15, 1413-1428 (2006).
34. Chiang M.-C., Boult T. E.: Efficient super-resolution via image warping. Image Vis. Comput. 28, 761-771 (2000).
35. Patti A., Altunbasak Y.: Artifact reduction for set theoretic super resolution image reconstruction with edge adaptive constraints and higher-order interpolants. IEEE Trans. Image Process. 10, 179-186 (2001).
36. Lertrattanapanich S., Bose N. K.: High resolution image formation from low resolution frames using Delaunay triangulation. IEEE Trans. Image Process. 11, 1427-1441 (2002).
37. Wang Z.-Z., Qi F.-H.: On ambiguities in super-resolution modeling. IEEE Signal Process. Lett. 11, 678-681 (2004).
38. Nguyen N., Milanfar P.: A wavelet-based interpolation-restoration method for superresolution. Circuits Syst. and Signal Process. 19, 321-338 (2000).
39. Ei-Khamy S. E., Hadhoud M. M., Dessouky M. I., Salam B. M., Ei-Samie F. E.: Regularized super-resolution reconstruction of images using wavelet fusion. Opt. Eng. 44, 097001. 1-097001. 10 (2005).
40. Ei-Khamy S. E., Hadhoud M. M., Dessouky M. I., Salam B. M., Ei-Samie F. E.: Wavelet fusion: a tool to break the limits on LMMSE image super-resolution. Int. J. Wavel. Multiresolut. Inf. Process. 4, 105-118 (2006).
41. Ji, H., Fermuller, C.: Wavelet-based super-resolution reconstruction: theory and algorithm. In: Proceedings of the European Conference on Computer Vision, pp. 295-307. Graz, Austria (2006).
42. Ji H., Fermuller C.: Robust wavelet-based super-resolution reconstruction: theory and algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 31, 649-660 (2009).
43. Chappalli M. B., Bose N. K.: Simultaneous noise filtering and super-resolution with second-generation wavelets. IEEE Signal Process. Lett. 12, 772-775 (2005).
44. Ur H., Gross D.: Improved resolution from subpixel shifted pictures. CVGIP Graph. Models Image Process. 54, 181-186 (1992).
45. Bose N. K., Ahuja N. A.: Superresolution and noise filtering using moving least squares. IEEE Trans. Image Process. 15, 2239-2248 (2006).
46. Irani M., Peleg S.: Improving resolution by image registration. CVGIP Graph. Models Image Process. 53, 231-239 (1991).
47. Patti A. J., Tekalp A. M.: Super resolution video reconstruction with arbitrary sampling lattices and nonzero aperture time. IEEE Trans. Image Process. 6, 1446-1451 (1997).
48. Tom, B. C., Katsaggelos, A. K.: Reconstruction of a high-resolution image by simultaneous registration, restoration, and interpolation of low-resolution images. In: Proceedings of the IEEE International Conference on Image Processing, pp. 539-542. Washington, DC (1995).
49. Hardie R. C., Barnard K. J., Armstrong E. E.: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Trans. Image Process. 6, 1621-1633 (1997).
50. Tian J., Ma K.-K.: Stochastic super-resolution image reconstruction. J. Vis. Commun. Image Represent. 21, 232-244 (2010).
51. Tian, J., Ma, K.-K.: Edge-adaptive super-resolution image reconstruction using a Markov chain Monte Carlo approach. In: Proceedings of the International Conference on Information, Communications and Signal Processing. Singapore (2007).
52. Schultz R. R., Stevenson R. L.: Extraction of high-resolution frames from video sequences. IEEE Trans. Image Process. 5, 996-1011 (1996).
53. Kong, D., Han, M., Xu, W., Tao, H., Gong Y.: A conditional random field model for video super-resolution. In: Proceedings of the IEEE International Conference on Pattern Recognition, pp. 619-622. Hong Kong (2006).
54. Suresh K. V., Rajagopalan A. N.: Robust and computationally efficient superresolution algorithm. Opt. Soc. Am. J. A Opt. Image Sci. 24, 984-992 (2007).
55. Belekos S., Galatsanos N. P., Katsaggelos A. K.: Maximum a posteriori video super-resolution using a new multichannel image prior. IEEE Trans. Image Process. 19, 1451-1464 (2010).
56. Shen H., Zhang L., Huang B., Li P.: A MAP approach for joint motion estimation, segmentation, and super resolution. IEEE Trans. Image Process. 16, 479-490 (2007).
57. Humblot F., Mohammad-Djafari A.: Super-resolution using hidden Markov model and Bayesian detection estimation framework. EURASIP J. Appl. Signal Process. 2006(36971), 1-16 (2006).
58. Mohammad-Djafari A.: Super-resolution: A short review, a new method based on hidden Markov modeling of hr image and future challenges. Comput. J. 52, 126-141 (2008).
59. Capel, D., Zisserman, A.: Super-resolution enhancement of text image sequences. In: Proceedings of the IEEE International Conference on Pattern Recognition, pp. 600-605. Barcelona, Spain (2000).
60. Chakrabarti A., Rajagopalan A. N., Chellappa R.: Super-resolution of face images using kernel PCA-based prior. IEEE Trans. Multimed. 9, 888-892 (2007).
61. Kim, K. I., Franz, M., Scholkopf, B.: Kernel hebbian algorithm for single-frame super-resolution. In: Proceedings of the ECCV Workshop on Statistical Learning in Computer Vision, pp. 135-149. Prague, Czech Republic (2004).
62. Tipping M. E., Bishop C. M.: Bayesian image super-resolution. In: Becker, S., Thrun, S., Obermeyer, K. (eds) Advances in Neural Information Processing Systems, MIT Press, Cambridge (2002).
63. Pickup L. C., Capel D., Roberts S. J., Zisserman A.: Bayesian image super-resolution, continued. In: Scholkopf, B., Platt, J., Hoffman, T. (eds) Advances in Neural Information Processing Systems, pp. 1089-1096. MIT Press, Cambridge (2006).
64. He H., Kondi L. P.: Resolution enhancement of video sequences with simultaneous estimation of the regularization parameter. J. Electron. Imaging 13, 586-596 (2004).
65. Zibetti M., Bazan F., Mayer J.: Determining the regularization parameters for super-resolution problems. Signal Process. 88, 2890-2901 (2008).
66. Zibettia M. V. W., Baznb F. S. V., Mayer J.: Estimation of the parameters in regularized simultaneous super-resolution. Pattern Recognit. Lett. 32, 69-78 (2011).
67. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., Salesin, D. H.: Image analogies. In: Proceedings of the SIGGRAPH, pp. 327-340. Los Angeles (2001).
68. Chang, H., Yeung, D. Y., Xiong, Y.: Super-resolution through neighbor embedding. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 275-282. Washington, D. C. (2004).
69. Freeman W. T., Pasztor E. C., Carmichael O. T.: Learning low-level vision. Int. J. Comput. Vis. 40, 25-47 (2000).
70. Pickup L. C., Roberts S. J., Zisserman A.: A sampled texture prior for image super-resolution. In: Thrun, S., Saul, L., Scholkopf, B. (eds) Advances in Neural Information Processing Systems, MIT Press, Cambridge (2003).
71. Datsenko D., Elad M.: Example-based single document image super-resolution: a global MAP approach with outlier rejection. Multidimens. Syst. Signal Process. 18, 103-121 (2007).
72. Wang J., Zhu S., Gong Y.: Resolution enhancement based on learning the sparse association of image patches. Pattern Recognit. Lett. 31, 1-10 (2010).
73. Kim K. I., Kwon Y.: Single-image super-resolution using sparse regression and natural image prior. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1127-1133 (2010).
74. Yang J., Wright J., Huang T., Ma Y.: Image super-resolution via sparse representation. IEEE Trans. Image Process. 19, 2861-2873 (2010).
75. Rhee S., Kang M. G.: Discrete cosine transform based regularized high-resolution image reconstruction algorithm. Opt. Eng. 38, 1348-1356 (1999).
76. Woods N. A., Galatsanos N. P., Katsaggelos A. K.: Stochastic methods for joint registration, restoration, and interpolation of multiple undersampled images. IEEE Trans. Image Process. 15, 201-213 (2006).
77. Dempster P., Laird N. M., Rubin D. B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. B 39(1), 1-38 (1977).
78. Bertero M., Boccacci P.: Introduction to Inverse Problems in Imaging. IOP Publishing Ltd, Philadelphia (1998).
79. Li S. Z.: Markov Random Field Modeling in Computer Vision. Springer, New York (1995).
80. Rue H.: Gaussian Markov Random Fields: Theory and Applications. Chapman & Hall, Boca Raton (2005).
81. Bose, N. K., Lertrattanapanich, S., Koo, J.: Advances in superresolution using L-curve. In: Proceedings of the IEEE International Symposium on Circuits and Systems, pp. 433-436. Sydney, Australia (2001).
82. Wu C. F. J.: On the convergence properties of the EM algorithm. Ann. Stat. 11, 95-103 (1983).
83. Metropolis N., Rosenbluth M. N., Rosenbluth A. W., Teller A. H., Teller E.: Equations of state calculations by fast computing machines. J. Chem. Phys. 21, 1087-1092 (1953).
84. Schultz R. R., Li M., Stevenson R. L.: Subpixel motion estimation for super-resolution image sequence enhancement. J. Vis. Commun. Image Represent. 9, 38-50 (1998).
85. Protter M., Elad M.: Super resolution with probabilistic motion estimation. IEEE Trans. Image Process. 18, 1899-1904 (2009).
86. Barreto D., Alvarez L. D., Abad J.: Motion estimation techniques in super-resolution image reconstruction: a performance evaluation. In: Tsvetkov, M., Golev, V., Murtagh, F., Molina, R. (eds) Virtual Observatory. Plate Content Digitalization, Archive Mining and Image Sequence Processing, pp. 254-268. Sofia, Bulgary (2005).
87. Callico G., Lopez S., Sosa O., Lopez J. F., Sarmiento R.: Analysis of fast block matching motion estimation algorithms for video super-resolution systems. IEEE Trans. Consum. Electron. 54, 1430-1438 (2008).
88. Baker, S., Kanade, T.: Super resolution optical flow. The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, Technical Report, Oct (1999).
89. Zhao, W., Sawhney, H.: Is super-resolution with optical flow feasible? In: Proceedings of the European Conference on Computer Vision, pp. 599-613. Copenhagen, Denamrk (2002).
90. Lin, F., Fookes, C., Chandran, V., Sridharan, S.: Investigation into optical flow super-resolution for surveillance applications. In: Proceedings of the APRS Workshop on Digital Image Computing, pp. 73-78 (2005).
91. Le H. V., Seetharaman G.: A super-resolution imaging method based on dense subpixel-accurate motion fields. J. VLSI Signal Process. 42, 79-89 (2006).
92. Fransens R., Strecha C., Gool L. V.: Optical flow based super-resolution: a probabilistic approach. Comput. Vis. Image Underst. 106, 106-115 (2007).
93. Suresh K. V., Kumar G. M., Rajagopalan A. N.: Superresolution of license plates in real traffic videos. IEEE Trans. Intell. Trans. Syst. 8, 321-331 (2007).
94. Narayanan B., Hardie R. C., Barner K. E., Shao M.: A computationally efficient super-resolution algorithm for video processing using partition filters. IEEE Trans. Circuits Syst. Video Technol. 17, 621-634 (2007).
95. Ng M. K., Shen H.-F., Zhang L.-P., Lam E.: A total variation regularization based super-resolution reconstruction algorithm for digital video. EURASIP J. Appl. Signal Process. 2007(74585), 1-16 (2007).
96. Patanavijit V., Jitapunkul S.: A Lorentzian stochastic estimation for a robust iterative multiframe super-resolution reconstruction with Lorentzian-Tikhonov regularization. EURASIP J. Appl. Signal Process. 2007(34821), 1-21 (2007).
97. Borman, S., Stevenson, R. L.: Simultaneous multi-frame MAP super-resolution video enhancement using spatio-temporal priors. In: Proceedings of the IEEE International Conference on Image Processing, pp. 469-473. Kobe, Japan (1999).
98. Zibetti M., Mayer J.: A robust and computationally efficient simultaneous super-resolution scheme for image sequences. IEEE Trans. Circuits Syst. Video Technol. 17, 1288-1300 (2007).
99. Alvarez, L. D., Molina, R., Katsaggelos, A. K.: Multi-channel reconstruction of video sequences from low-resolution and compressed observations. In: The 8th Iberoamerican Congress on Pattern Recognition, pp. 46-53. Havana, Cuba (2003).
100. Elad M., Feuer A.: Superresolution restoration of an image sequence: adaptive filtering approach. IEEE Trans. Image Process. 8, 387-395 (1999).
101. Elad M., Feuer A.: Super-resolution reconstruction of image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 21, 817-834 (1999).
102. Farsiu S., Elad M., Milanfar P.: Video-to-video dynamic superresolution for grayscale and color sequences. EURASIP J. Appl. Signal Process. 2006(61859), 1-15 (2006).
103. Costa G. H., Bermudez J. C. M.: Statistical analysis of the LMS algorithm applied to super-resolution image reconstruction. IEEE Trans. Signal Process. 55, 2084-2095 (2007).
104. Tian J., Ma K.-K.: A state-space super-resolution approach for video reconstruction. Signal Image Video Process. 3, 217-240 (2009).
105. Bishop, C. M., Blake, A., Marthi, B.: Super-resolution enhancement of video. In: Proceedings of the International Conference on Artificial Intelligence and Statistics. Key West, FL (2003).
106. Dedeoglu, G., Kanade, T., August, J.: High-zoom video hallucination by exploiting spatio-temporal regularities. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 151-158. Washington, DC (2004).
107. Kong, D., Han, M., Xu, W., Tao, H., Gong, Y.: Video superresolution with scene-specific priors. In: Proceedings of the British Machine Vision Association. Edinburgh (2006).
108. Kimura, K., Nagai, T., Nagayoshi, H., Sako, H.: Simultaneous estimation of super-resolved image and 3D information using multiple stereo-pair images. In: Proceedings of the IEEE International Conference on Image Processing, pp. 417-420. (2007).
109. Bhavsar, A. V., Rajagopalan, A. N.: Resolution enhancement for binocular stereo. In: Proceedings of the International Conference on Pattern Recognition, pp. 1-4. (2008).
110. Bhavsar A. V., Rajagopalan A. N.: Resolution enhancement in multi-image stereo. IEEE Trans. Pattern Anal. Mach. Intell. 99, 1721-1728 (2010).
111. Robertson M. A., Stevenson R. L.: Temporal resolution enhancement in compressed video sequences. EURASIP J. Appl. Signal Process. 2001, 230-238 (2001).
112. Shechtman E., Caspi Y., Irani M.: Space-time super-resolution. IEEE Trans. Pattern Anal. Mach. Intell. 27, 531-545 (2005).
113. Watanabe, K., Iwai, Y., Nagahara, H., Yachnida, M., Suzuki, T.: Video synthesis with high spatio-temporal resolution using motion compensation and image fusion in wavelet domain. In: Proceedings of the Asian Conference on Computer Vision, pp. 480-489. Hyderabad, India (2006).
114. Maor N., Feuer A., Goodwin G. C.: Compression at the source for digital camcorders. EURASIP J. Image Video Process. 2007(41516), 1-11 (2007).
115. Ishikawa, H., Geiger, D.: Rethinking the prior model for stereo. In: Proceedings of the European Conference on Computer Vision, pp. 526-537. (2006).
116. Szeliski, R., Veksler, O., Agarwala, A., Rother, C.: A comparative study of energy minimization methods for Markov random fields. In: Proceedings of the European Conference on Computer Vision, pp. 16-29. (2006).
117. Krishnamurthy R., Woods J., Moulin P.: Frame interpolation and bidirectional prediction of video using compactly encoded optical flow fields and label fields. IEEE Trans. Circuits Syst. Video Technol. 9, 713-726 (1999).
118. Zhang Y., Zhao D., Ji X., Wang R., Gao W.: A spatio-temporal auto regressive model for frame rate upconversion. IEEE Trans. Circuits Syst. Video Technol. 19, 1289-1301 (2009).
119. Zhang Y., Zhao D., Ma S., Wang R., Gao W.: A motion-aligned auto-regressive model for frame rate up conversion. IEEE Trans. Image Process. 19, 1248-1258 (2010).