A two-step neural-network based algorithm for fast image super-resolution

Image and Vision Computing

Volumn 25, Issue 9, 2007, Pages 1449-1473

  Miravet, Carlos ^a,b   Rodríguez, Francisco B ^a  

^a UNIVERSIDAD AUTÓNOMA DE MADRID   (Spain)

^b Aerospace Division   (Spain)

Author keywords

Image restoration; Multi layer perceptron; Probabilistic neural network; Sequence processing; Super resolution

Indexed keywords

COMPUTATIONAL METHODS; DIGITAL FILTERS; IMAGE QUALITY; IMAGE RECONSTRUCTION; INTERPOLATION; MULTILAYER NEURAL NETWORKS;

IMAGE SEQUENCES; PROBABILISTIC NEURAL NETWORKS; SEQUENCE PROCESSING; WEIGHTED INTERPOLATION;

ALGORITHMS;

EID: 34250214166     PISSN: 02628856     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.imavis.2006.12.016     Document Type: Article

References (71)

1. R.Y. Tsai, T.S. Huang (Eds.), Multiframe image restoration and registration, in: Advances in Computer Vision and Image Processing, vol. 1, Wiley, 1984, pp. 317-339.
2. Borman S., and Stevenson R.L. Super-resolution from image sequences - a review. Proc. Midwest Symp. Circ. Syst. (1998)
3. Park S.C., Park M.K., and Kang M.G. Super-resolution image reconstruction: a technical overview. IEEE Sig. Process. Mag. 20 3 (2003) 21-36
4. Kim S.P., Bose N.K., and Valenzuela H.M. Recursive reconstruction of high resolution image from noisy undersampled multiframes. IEEE Trans. Acoust. Speech Sig. Process. 38 6 (1990) 1013-1027
5. Stark H., and Oskoui P. High resolution image recovery from image-plane arrays, using convex projections. J. Opt. Soc. Am. A 6 (1989) 1715-1726
6. Patti A.J., Sezan M.I., and Tekalp A.M. Superresolution video reconstruction with arbitrary sampling lattices and non-zero aperture time. IEEE Trans. Img. Process. 6 8 (1997) 1064-1076
7. Granrath D., and Lersch J. Fusion of images on affine sampling grids. J. Opt. Soc. Am. A 15 4 (1998) 791-801
8. Patti A.J., and Altunbasak Y. Artifact reduction for set theoretic super resolution image reconstruction with edge adaptive constraints and higher-order interpolants. IEEE Trans. Img. Process. 10 1 (2001) 179-186
9. Tom B.C., and Katsaggelos A.K. Reconstruction of a high-resolution image by simultaneous registration, restoration, and interpolation of low-resolution images. Proc. IEEE Int. Conf. Img. Process. 2 (1995) 539-542
10. Schultz R.R., and Stevenson R.L. Extraction of high-resolution frames from video sequences. IEEE Trans. Img. Process. 5 6 (1996) 996-1011
11. Hardie R.C., Barnard K.J., Bognar J.G., Armstrong E.E., and Watson E.A. High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system. Opt. Eng. 37 1 (1998) 247-260
12. Lu Y., and Imamura M. Pyramid-based super-resolution of the undersampled and subpixel image sequence. Int. J. Img. Syst. Tech. 12 6 (2003) 254-263
13. Ur H., and Gross D. Improved resolution from subpixel shifted pictures. CVGIP: Graph. Model. Img. Process. 54 2 (1992) 181-186
14. Papoulis A. Generalized sampling expansion. IEEE Trans. Circ. Syst. 24 11 (1977) 652-654
15. Chiang M.C., and Boult T.E. Efficient super-resolution via image warping. Image Vis. Comput. 18 10 (2000) 761-771
16. Elad M., and Hel-Or Y. A fast super-resolution reconstruction algorithm for pure translational motion and common space-invariant blur. IEEE Trans. Img. Process. 10 8 (2001) 1187-1193
17. Gillette J.C., Stadtmiller T.M., and Hardie R.C. Aliasing reduction in staring infrared imagers utilizing subpixel techniques. Opt. Eng. 34 11 (1995) 3130-3137
18. Alam M.S., Bognar J.G., Hardie R.C., and Yasuda B.J. Infrared image registration and high-resolution reconstruction using multiple translationally shifted aliased video frames. IEEE Trans. Instr. Meas. 49 5 (2000) 915-923
19. Komatsu T., Aizawa K., Igarashi T., and Saito T. Signal-processing based method for acquiring very high resolution images with multiple cameras and its theoretical analysis. IEE Proc. I, Commun. Speech Vis. 140 1 (1993) 19-24
20. Shah N.R., and Zakhor A. Resolution enhancement of color video sequences. IEEE Trans. Img. Process. 8 6 (1999) 879-885
21. Nguyen N., and Milanfar P. A wavelet-based interpolation-restoration method for superresolution. Circ. Syst. Sig. Process. 19 4 (2000) 321-338
22. Candocia F.M., and Principe J.C. Super-resolution of images based on local correlations. IEEE Trans. Neural Netw. 10 2 (1999) 372-380
23. Freeman W.T., Jones T.R., and Pasztor E.C. Example-based super-resolution. IEEE Comput. Graph. Appl. 22 2 (2002) 56-65
24. Hertzmann A., Jacobs C.E., Oliver N., Curless B., and Salesin D.H. Image analogies. Proc. 28th Conf. Comput. Graph. Interact. Tech. (Proc. Siggraph 2001) (2001), ACM press 327-340
25. Bishop C.M., Blake A., and Marthi B. Super-resolution enhancement of video. Proc. Artif. Intell. Stat. (2003)
26. Kim K.I., Franz M.O., and Schölkopf B. Iterative kernel principal component analysis for image modeling. IEEE Trans. Pattern Anal. Mach. Intell. 27 9 (2005) 1351-1366
27. Jiji C.V., Joshi M.V., and Chaudhuri S. Single-frame image super-resolution using learned wavelet coefficients. Int. J. Img. Syst. Tech. 14 3 (2004) 105-112
28. Plaziac N. Image interpolation using neural networks. IEEE Trans. Img. Process. 8 11 (1999) 1647-1651
29. Gustafson S.C., and Meyer G.J. Spline-based neural networks for digital image interpolation. Proc. SPIE - Appl. Artif. Neural Netw. Img. Process. VI 4305 (2001) 39-44
30. C. Staelin, D. Greig, M. Fischer, R. Maurer, Neural Network Image Scaling Using spatial Errors, HP tech. rep. HPL-2003-26R1 (2003), 1-11.
31. Davila C.A., and Hunt B.R. Training of a neural network for image superresolution based on a nonlinear interpolative vector quantizer. Appl. Opt. 39 20 (2000) 3473-3485
32. Baker S., and Kanade T. Limits on super-resolution and how to break them. IEEE Trans. Pattern Anal. Mach. Intell. 24 9 (2002) 1167-1183
33. Su C., Zhuang Y., Huang L., and Wu F. Steerable pyramid-based face hallucination. Pattern Recognit. 38 6 (2005) 813-824
34. Capel D., and Zisserman A. Super-resolution from multiple views using learnt image models. Proc. IEEE Int. Conf. Compt. Vis. Pattern Recognit. II (2001) 627-634
35. Szu H., and Kopriva I. Artificial neural networks for noisy image super-resolution. Opt. Commun. 198 (2001) 71-81
36. Salari E., and Zhang S. Integrated recurrent neural network for image resolution enhancement from multiple image frames. IEE Proc.-Vis. Img. Sig. Process. 150 5 (2003) 299-305
37. Irani M., and Peleg S. Improving resolution by image registration. CVGIP: Graph. Model. Img. Process. 53 3 (1991) 231-239
38. Lucas B.D., and Kanade T. An iterative image registration technique with an application to stereo vision. Int. J. Conf. Artif. Intell. (1981) 674-679
39. Gonzalez R.C., and Wintz P. Digital Image Processing. second ed. (1987), Addison-Wesley, Reading, Massachusetts
40. In: Rosenfeld A. (Ed). Multiresolution Image Processing and Analysis (1984), Springer-Verlag, Berlin, New York
41. Keys R. Cubic convolution interpolation for digital image processing. IEEE Trans. Acoust. Speech Sig. Process. 29 6 (1981) 1153-1160
42. Specht D.F. Probabilistic neural networks. Neural Netw. 3 1 (1990) 109-118
43. Bishop C.M. Neural Networks for Pattern Recognition (1995), Oxford University Press
44. Duda R., and Hart P. Pattern Classification and Scene Analysis (1973), Wiley-Interscience, New York
45. R. Hecht-Nielsen, Theory of the back-propagation neural network, in: Proc. Int. J. Conf. Neural Netw., 1, Washington DC, 1989, 593-605.
46. Hornik K., Stinchcombe M., and White H. Multilayer feedforward networks are universal approximators. Neural Netw. 2 5 (1989) 359-366
47. Hestenes M.R. Conjugate Direction Methods in Optimization (1980), Springer Verlag, New York
48. Rumelhart D.E., and McClelland J.L. Parallel Distributed Processing: Explorations in the Microstructure of Cognition (1987), MIT Press
49. Holst G.C. Electro-Optical Imaging System Performance. fourth ed. (2006), SPIE Press, JCD Publishing, Washington
50. Press W.H., Teukolsky S.A., Vetterling W.T., and Flannery B.P. Numerical Recipes in C: The Art of Scientific Computing. second ed. (1992), Cambridge University Press
51. Storey J.C. Landsat 7 on-orbit modulation transfer function estimation. Proc. SPIE Sens. Syst. Next Gen. Sat. V 4540 (2001) 50-61
52. P.W. Scott, QuickBird on-orbit spatial image quality assessment, http://www.digitalglobe.com/downloads/asprs/DG%20ASPRS%20On-Orbit%20Spatial%20Image%20Quality%20Assessment.pdf.
53. Available from: http://www.digitalglobe.com/.
54. Kirkpatrick S., Gelatt C.D., and Vecchi M.P. Optimization by simulated annealing. Science 220 4598 (1983) 671-680
55. Fausset L.V. Fundamentals of Neural Networks: Architectures, Algorithms and Applications (1994), Prentice Hall, Upper Saddle River, NJ
56. C. Miravet, F.B. Rodriguez, A hybrid MLP-PNN architecture for fast image superresolution, in: O. Kaynak, E. Alpaydin, E. Oja, L. Xu (Eds.), Lect. Not. Comput. Sci. 2714 (2003), pp. 417-425.
57. Borgefors G. Distance transformations in digital images. Comput. Vis. Graph. Img. Process. 34 3 (1986) 344-371
58. Haykin S. Neural Networks: A Comprehensive Foundation. second ed. (1998), Prentice Hall PTR, Upper Saddle River, NJ
59. Jain A.K. Fundamentals of Digital Image Processing (1989), Prentice Hall ISSS, Upper Saddle River, NJ
60. Hummel R.A., Kimia B., and Zucker S.W. Deblurring gaussian blur. Comput. Vis. Graph. Img. Process. 38 1 (1987) 66-80
61. Capel D., and Zisserman A. Computer vision applied to superresolution. IEEE Sig. Process. Mag. 20 3 (2003) 75-86
62. Huber P.J. Robust Statistics (1981), Wiley Interscience, New York
63. Farsiu S., Robinson M.D., Elad M., and Milanfar P. Fast and robust multiframe super resolution. IEEE Trans. Img. Process. 13 10 (2004) 1327-1344
64. Pickup L.C., Capel D.P., Roberts S.J., and Zisserman A. Bayesian image super-resolution continued. Adv. Neural Informat. Process. Syst. (2006)
65. Nguyen N., Milanfar P., and Golub G. A computationally efficient supperresolution image reconstruction algorithm. IEEE Trans. Img. Process. 10 4 (2001) 573-583
66. Chan R., and Strang G. Toeplitz equations by conjugate gradients with circulant preconditioner. SIAM J. Sci. Stat. Comput. 10 1 (1989) 104-119
67. Hanke M., and Nagy J. Toeplitz approximate inverse preconditioner for banded Toeplitz matrices. Numer. Algorithms 7 2 (1994) 183-199
68. Available from: http://www.soe.ucsc.edu/~milanfar/MDSP/superres.htm.
69. Available from: http://www.robots.ox.ac.uk/~vgg/data4.html.
70. Capel D., and Zisserman A. Automated mosaicing with super-resolution zoom. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Santa Barbara (1998)
71. Available from: http://sipi.usc.edu/database/database.cgi?volume=sequences.