A MAP approach for joint motion estimation, segmentation, and super resolution

IEEE Transactions on Image Processing

Volumn 16, Issue 2, 2007, Pages 479-490

  Shen, Huangfeng ^a   Zhang, Liangpei ^a   Huang, Bo ^b   Li, Pingxiang ^a  

^a WUHAN UNIVERSITY   (China)

^b CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Joint estimation; Maximum a posterior (MAP); Motion estimation; Segmentation; Super resolution

Indexed keywords

GRADIENT METHODS; IMAGE ENHANCEMENT; IMAGE RESOLUTION; IMAGE SEGMENTATION; MOTION ESTIMATION;

IMAGE SEQUENCE; JOINT ESTIMATION; MAXIMUM A POSTERIORI; SUPER RESOLUTION;

IMAGE RECONSTRUCTION;

ALGORITHM; ARTICLE; ARTIFACT; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; COMPUTER SIMULATION; EVALUATION; IMAGE ENHANCEMENT; IMAGE SUBTRACTION; INFORMATION RETRIEVAL; MATHEMATICAL COMPUTING; METHODOLOGY; MOTION; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; SIGNAL PROCESSING; STATISTICAL MODEL; VIDEORECORDING;

ALGORITHMS; ARTIFACTS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; INFORMATION STORAGE AND RETRIEVAL; MODELS, STATISTICAL; MOTION; NUMERICAL ANALYSIS, COMPUTER-ASSISTED; PATTERN RECOGNITION, AUTOMATED; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; SIGNAL PROCESSING, COMPUTER-ASSISTED; SUBTRACTION TECHNIQUE; VIDEO RECORDING;

EID: 33847755152     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2006.888334     Document Type: Article

References (45)

1. R. Y. Tsai and T. S. Huang, "Multi-frame image restoration and registration," Adv. Comput. Vis. Image Process., vol. 1, pp. 317-339, 1984.
2. S. P. Kim, N. K. Bose, and H. M. Valenzuela, "Recursive reconstruction of high resolution image from noisy undersampled multiframes," IEEE Trans. Acoust., Speech, Signal Process., vol. 38, no. 2, pp. 1013-1027, Jun. 1990.
3. S. P. Kim and W. Y. Su, "Recursive high-resolution reconstruction of blurred multiframe images," IEEE Trans. Image Process., vol. 2, no. 4, pp. 534-539, Apr. 1993.
4. S. Rhee and M. G. Kang, "Discrete cosine transform based regularized high-resolution image reconstruction algorithm," Opt. Eng., vol. 38, pp. 1348-1356, 1999.
5. S. C. Park, M. K. Park, and M. G. Kang, "Super-resolution image reconstruction: A technical overview," IEEE Signal Process. Mag., vol. 20, no. 3, pp. 21-36, May 2003.
6. H. Ur and D. Gross, "Improved resolution from sub-pixel shifted pictures," CVGIP: Graph. Models Image Process., vol. 54, pp. 181-186, 1992.
7. A. Papoulis, "Generalized sampling expansion," IEEE Trans. Circuits Syst., vol. CAS-24, no. 11, pp. 652-654, Nov. 1977.
8. L. J. Yen, "On nonuniform sampling of bandwidth limited signals," IRE Trans. Circuits Theory, vol. 3, pp. 251-257, 1956.
9. M. Irani and S. Peleg, "Improving resolution by image registration," CVGIP: Graph. Models Image Process., vol. 53, pp. 231-239, 1991.
10. H. Stark and P. Oskoui, "High-resolution image recovery from image plane arrays, using convex projections," J. Opt. Soc. Amer. A: Opt. Image Sci., Vis., vol. 6, pp. 1715-1726, 1989.
11. A. M. Tekalp, M. K. Ozkan, and M. I. Sezan, "High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration," presented at the IEEE Int. Conf. Acoustics, Speech, Signal Processing, San Francisco, CA, 1992.
12. A. J. Patti, M. I. Sezan, and A. M. Tekalp, "High-resolution image reconstruction from a low-resolution image sequence in the presence of time-varying motion blur," presented at the IEEE Int. Conf. Image Processing, Austin, TX, 1994.
13. A. J. Patti, M. I. Sezan, and A. M. Tekalp, "Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time," IEEE Trans. Image Process., vol. 6, no. 8, pp. 1064-1076, Aug. 1997.
14. B. C. Tom and A. K. Katsaggelos, "Reconstruction of a high-resolution image from multiple-degraded misregistered low-resolution images," Proc. SPIE, 1994.
15. R. R. Schultz and R. L. Stevenson, "Extraction of high-resolution frames from video sequences," IEEE Trans. Image Process., vol. 5, no. 6, pp. 996-1011, Jun. 1996.
16. R. C. Hardie, T. R. Tuinstra, J. Bognar, K. J. Barnard, and E. Armstrong, "High resolution image reconstruction from digital video with global and non-global scene motion," presented at the IEEE Int. Conf. Image Processing, Santa Barbara, CA, 1997.
17. M. Elad and A. Feuer, "Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images," IEEE Trans. Image Process., vol. 6, no. 12, pp. 1646-1658, Dec. 1997.
18. M. Elad and A. Feuer, "Superresolution restoration of an image sequence: Adaptive filtering approach," IEEE Trans. Image Process., vol. 8, no. 3, pp. 387-395, Mar. 1999.
19. N. K. Bose, H. C. Kim, and H. M. Valenzuela, "Recursive implementation of total least squares algorithm for image reconstruction from noisy, undersampled multiframes," presented at the IEEE Int. Conf. Acoustics, Speech and Signal Processing, Minneapolis, MN, 1993.
20. M. K. Ng and N. K. Bose, "Analysis of displacement errors in high-resolution image reconstruction with multisensors," IEEE Trans. Circuits Syst. I, Fundam. Theory Appl., vol. 49, no. 6, pp. 806-813, Jul. 2002.
21. E. S. Lee and M. G. Kang, "Regularized adaptive high-resolution image reconstruction considering inaccurate subpixel registration," IEEE Trans. Image Process., vol. 12, no. 7, pp. 826-837, Jul. 2003.
22. N. R. Shah and A. Zakhor, "Resolution enhancement of color video sequences," IEEE Trans. Image Process., vol. 8, no. 8, pp. 879-885, Aug. 1999.
23. B. C. Tom and A. K. Katsaggelos, "Reconstruction of a high-resolution image by simultaneous registration, restoration, and interpolation of low-resolution images," presented at the IEEE Int. Conf. Image Processing, Washington, DC, 1995.
24. R. C. Hardie, K. J. Barnard, and E. E. Armstrong, "Joint MAP registration and high-resolution image estimation using a sequence of undersampled images," IEEE Trans. Image Process., vol. 6, no. 12, pp. 1621-1633, Dec. 1997.
25. C. A. Segall, R. Molina, and A. K. Katsaggelos, "High-resolution images from low-resolution compressed video," IEEE Signal Process. Mag., vol. 20, no. 3, pp. 37-48, May 2003.
26. C. A. Segall, A. K. Katsaggelos, R. Molina, and J. Mateos, "Bayesian resolution enhancement of compressed video," IEEE Trans. Image Process., vol. 13, no. 7, pp. 898-910, Jul. 2004.
27. N. A. Woods, N. P. Galatsanos, and A. K. Katsaggelos, "Stochastic methods for joint registration, restoration, and interpolation of multiple undersampled images," IEEE Trans. Image Process., vol. 15, no. 1, pp. 201-213, Jan. 2006.
28. J. Chung, E. Haber, and J. Nagy, "Numerical methods for coupled super resolution," Inv. Probl., to be published.
29. S. Borman and R. Stevenson, Spatial Resolution Enhancement of Low-Resolution Image Sequences: A Comprehensive Review with Directions for Future Research Lab. Image and Signal Analysis (LISA), Univ. Notre Dame, Notre Dame, IN, 1998.
30. M. Irani and S. Peleg, "Motion analysis for image enhancement. Resolution, occlusion, and transparency," J. Vis. Commun. Image Represent., vol. 4, p. 324, 1993.
31. P. E. Eren, M. I. Sezan, and A. M. Tekalp, "Robust, object-based high-resolution image reconstruction from low-resolution video," IEEE Trans. Image Process., vol. 6, no. 10, pp. 1446-1451, Oct. 1997.
32. L. D. Alvarez, J. Mateos, R. Molina, and A. K. Katsaggelos, "High-resolution images from compressed low-resolution video: Motion estimation and observable pixels," Int. J. Imag. Syst. Technol., vol. 14, pp.58-66, 2004.
33. A. M. Tekalp, Digital Video Processing. Englewood Cliffs, NJ: Prentice-Hall, 1995.
34. M. M. Chang, A. M. Tekalp, and M. I. Sezan, "Simultaneous motion estimation and segmentation," IEEE Trans. Image Process., vol. 6, no. 9, pp. 1326-1333, Sep. 1997.
35. N. Nguyen, P. Milanfar, and G. Golub, "Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement," IEEE Trans. Image Process., vol. 10, no. 9, pp. 1299-1308, Sep. 2001.
36. S. Hsu, P. Anandan, and S. Peleg, "Accurate computation of optical flow by using layered motion representations," presented at the Int. Conf. Pattern Recognition, Jerusalem, Israel, 1994.
37. A. J. Tabatabai, R. S. Jasinschi, and T. Naveen, "Motion estimation methods for video compression - A review," J. Franklin Inst. Eng. Appl. Math. B, vol. 335, pp. 1411-1441, 1998.
38. J. Besag, "On the statistical analysis of dirty pictures," J. Roy. Statist. Soc. B, pp. 259-302, 1986.
39. M. M. Chang, M. I. Sezan, and A. M. Tekalp, "An algorithm for simultaneous motion estimation and scene segmentation," presented at the IEEE Int. Conf. Acoustics, Speech, Signal Processing, 1994.
40. T. N. Pappas, "An adaptive clustering algorithm for image segmentation," IEEE Trans. Signal Process., vol. 40, no. 4, pp. 901-914, Apr. 1992.
41. J. K. Fwu and P. M. Djuric, "Unsupervised vector image segmentation by a tree structure - ICM algorithm," IEEE Trans. Med. Imag., vol. 15, no. 6, pp. 871-880, Dec. 1996.
42. J. Y. A. Wang and E. H. Adelson, "Representing moving images with layers," IEEE Trans. Image Process., vol. 3, no. 5, pp. 625-638, Sep. 1994.
43. T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, "High accuracy optical flow estimation based on a theory for warping," presented at the 8th Eur. Conf. Computer Vision, Prague, Czech Republic, 2004.
44. S. Farsiu, M. D. Robinson, M. Elad, and P. Milanfar, "Fast and robust multiframe super resolution," IEEE Trans. Image Process., vol. 13, no. 10, pp. 1327-1344, Oct. 2004.
45. S. Farsiu, M. Elad, and P. Milanfar, "Multiframe demosaicing and super-resolution of color images," IEEE Trans. Image Process., vol. 15, no. 1, pp. 141-159, Jan. 2006.